The ICDAR2007 Handwriting Segmentation Contest will be based on the evaluation strategies of previous document segmentation contests also organized by us ( ICDAR2001 Newspaper Page Segmentation Contest, ICDAR2003 Page Segmentation Competition and ICDAR2005 Page Segmentation Competition).

• The performance evaluation method will be based on counting the number of matches between the areas detected by the algorithm and the areas in the ground truth. We will use a MatchScore table whose values will be calculated according to the intersection of the ON pixel sets of the result and the ground truth.

• Let G_i the set of all points of the i ground truth region, R_j the set of all points of the j result region, Τ(s) a function that counts the elements of set s. Table MatchScore(i,j) represents the matching results of i ground truth region and the j result region as follows: MatchScore(i,j)=T(G_i R_j)/T(G_i  R_j). For example:

for a given ground truth:

and a segmentation result:

MatchScore is calculated:

e.g. MatchScore (1,5) = T(G₁ R₅)/T(G₁   R₅)=11/14=0.79.

•  Detection Rate and Recognition Accuracy are calculated from the MatchScore table following the steps of [1].

[1] I. Phillips and A. Chhabra, "Empirical Performance Evaluation of Graphics Recognition Systems," IEEE Transaction of Pattern Analysis and Machine Intelligence, Vol. 21, No. 9, pp. 849-870, September 1999.