Evaluation:
                                   
The ICFHR 2010 Handwriting Segmentation Contest will be based on the evaluation strategies of previous document segmentation contests (ICDAR2001 Newspaper Page Segmentation Contest, ICDAR2003 Page Segmentation CompetitionICDAR2005 Page Segmentation Competition, ICDAR2007 Page Segmentation CompetitionICDAR2007 Handwriting Segmentation Contest and ICDAR2009 Handwriting Segmentation Contest).

  • The performance evaluation method will be based on counting the number of matches between the areas detected by the algorithm and the areas in the ground truth. We will use a MatchScore table whose values will be calculated according to the intersection of the ON pixel sets of the result and the ground truth.
  • Let Gi the set of all points of the i ground truth region, Rj the set of all points of the j result region, Τ(s) a function that counts the elements of set s. Table MatchScore(i,j) represents the matching results of i ground truth region and the j result region as follows: MatchScore(i,j)=T(Gi Rj)/T(Gi Rj). For example:
For a given ground truth:


and a segmentation result:



MatchScore is calculated as follows:



e.g. MatchScore (1,5) = T(G1 R5)/T(G1 R5) = 11/14 = 0.79

  • Detection Rate and Recognition Accuracy are calculated from the MatchScore table1.

1 I. Phillips and A. Chhabra, "Empirical Performance Evaluation of Graphics Recognition Systems," IEEE Transaction of Pattern Analysis and Machine Intelligence, Vol. 21, No. 9, pp. 849-870, September 1999.




Site maintained by B. Gatos, email: bgat<at>iit<dot>demokritos<dot>gr
Last update: 9/3/2010