| Evaluation:
The
ICFHR 2010 Handwriting Segmentation Contest will be based on the
evaluation strategies of previous document segmentation contests (ICDAR2001 Newspaper Page Segmentation Contest, ICDAR2003 Page Segmentation Competition, ICDAR2005 Page Segmentation Competition, ICDAR2007 Page Segmentation Competition, ICDAR2007 Handwriting Segmentation Contest and ICDAR2009 Handwriting Segmentation Contest).
- The
performance evaluation method will be based on counting the number of
matches between the areas detected by the algorithm and the areas in
the ground truth. We will use a MatchScore table whose values will be
calculated according to the intersection of the ON pixel sets of the
result and the ground truth.
- Let Gi the set of all points of the
i ground truth region, Rj the set of all points of the j result region,
Τ(s) a function that counts the elements of set s. Table
MatchScore(i,j) represents the matching results of i ground truth
region and the j result region as follows: MatchScore(i,j)=T(Gi
 Rj)/T(Gi  Rj). For example:
For a given ground truth:
and a segmentation result: MatchScore is calculated as follows: e.g. MatchScore (1,5) = T(G1 R5)/T(G1 R5) = 11/14 = 0.79
- Detection Rate and Recognition Accuracy are calculated from the MatchScore table1.
1 I.
Phillips and A. Chhabra, "Empirical Performance Evaluation of Graphics
Recognition Systems," IEEE Transaction of Pattern Analysis and Machine
Intelligence, Vol. 21, No. 9, pp. 849-870, September 1999. | |
|
|
