Critical discussion of evaluation parameters for inter-observer variability in target definition for radiation therapy

I. Fotina, C. Lütgendorf-Caucig, M. Stock, R. Pötter, D. Georg

Research output: Contribution to journalArticle

73 Citations (Scopus)


Background and purpose. Inter-observer studies represent a valid method for the evaluation of target definition uncertainties and contouring guidelines. However, data from the literature do not yet give clear guidelines for reporting contouring variability. Thus, the purpose of this work was to compare and discuss various methods to determine variability on the basis of clinical cases and a literature review. Patients and methods. In this study, 7 prostate and 8 lung cases were contoured on CT images by 8 experienced observers. Analysis of variability included descriptive statistics, calculation of overlap measures, and statistical measures of agreement. Cross tables with ratios and correlations were established for overlap parameters. Results. It was shown that the minimal set of parameters to be reported should include at least one of three volume overlap measures (i.e., generalized conformity index, Jaccard coefficient, or conformation number). High correlation between these parameters and scatter of the results was observed. Conclusion. A combination of descriptive statistics, overlap measure, and statistical measure of agreement or reliability analysis is required to fully report the interrater variability in delineation.

Original languageEnglish
Pages (from-to)160-167
Number of pages8
JournalStrahlentherapie und Onkologie
Issue number2
Publication statusPublished - Feb 2012
Externally publishedYes



  • Conformity index
  • Inter-observer variability
  • Radiotherapy
  • Similarity metrics
  • Target volume delineation

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging
  • Oncology

Cite this