Critical discussion of evaluation parameters for inter-observer variability in target definition for radiation therapy

I. Fotina, C. Lütgendorf-Caucig, M. Stock, R. Pötter, D. Georg

Research output: Contribution to journalArticle

70 Citations (Scopus)

Abstract

Background and purpose. Inter-observer studies represent a valid method for the evaluation of target definition uncertainties and contouring guidelines. However, data from the literature do not yet give clear guidelines for reporting contouring variability. Thus, the purpose of this work was to compare and discuss various methods to determine variability on the basis of clinical cases and a literature review. Patients and methods. In this study, 7 prostate and 8 lung cases were contoured on CT images by 8 experienced observers. Analysis of variability included descriptive statistics, calculation of overlap measures, and statistical measures of agreement. Cross tables with ratios and correlations were established for overlap parameters. Results. It was shown that the minimal set of parameters to be reported should include at least one of three volume overlap measures (i.e., generalized conformity index, Jaccard coefficient, or conformation number). High correlation between these parameters and scatter of the results was observed. Conclusion. A combination of descriptive statistics, overlap measure, and statistical measure of agreement or reliability analysis is required to fully report the interrater variability in delineation.

Original languageEnglish
Pages (from-to)160-167
Number of pages8
JournalStrahlentherapie und Onkologie
Volume188
Issue number2
DOIs
Publication statusPublished - Feb 2012
Externally publishedYes

Fingerprint

Observer Variation
Radiotherapy
Guidelines
Uncertainty
Prostate
Lung

Keywords

  • Conformity index
  • Inter-observer variability
  • Radiotherapy
  • Similarity metrics
  • Target volume delineation

ASJC Scopus subject areas

  • Radiology Nuclear Medicine and imaging
  • Oncology

Cite this

Critical discussion of evaluation parameters for inter-observer variability in target definition for radiation therapy. / Fotina, I.; Lütgendorf-Caucig, C.; Stock, M.; Pötter, R.; Georg, D.

In: Strahlentherapie und Onkologie, Vol. 188, No. 2, 02.2012, p. 160-167.

Research output: Contribution to journalArticle

Fotina, I. ; Lütgendorf-Caucig, C. ; Stock, M. ; Pötter, R. ; Georg, D. / Critical discussion of evaluation parameters for inter-observer variability in target definition for radiation therapy. In: Strahlentherapie und Onkologie. 2012 ; Vol. 188, No. 2. pp. 160-167.
@article{1e5cc1ecd3c141829201fc51667ea3b7,
title = "Critical discussion of evaluation parameters for inter-observer variability in target definition for radiation therapy",
abstract = "Background and purpose. Inter-observer studies represent a valid method for the evaluation of target definition uncertainties and contouring guidelines. However, data from the literature do not yet give clear guidelines for reporting contouring variability. Thus, the purpose of this work was to compare and discuss various methods to determine variability on the basis of clinical cases and a literature review. Patients and methods. In this study, 7 prostate and 8 lung cases were contoured on CT images by 8 experienced observers. Analysis of variability included descriptive statistics, calculation of overlap measures, and statistical measures of agreement. Cross tables with ratios and correlations were established for overlap parameters. Results. It was shown that the minimal set of parameters to be reported should include at least one of three volume overlap measures (i.e., generalized conformity index, Jaccard coefficient, or conformation number). High correlation between these parameters and scatter of the results was observed. Conclusion. A combination of descriptive statistics, overlap measure, and statistical measure of agreement or reliability analysis is required to fully report the interrater variability in delineation.",
keywords = "Conformity index, Inter-observer variability, Radiotherapy, Similarity metrics, Target volume delineation",
author = "I. Fotina and C. L{\"u}tgendorf-Caucig and M. Stock and R. P{\"o}tter and D. Georg",
year = "2012",
month = "2",
doi = "10.1007/s00066-011-0027-6",
language = "English",
volume = "188",
pages = "160--167",
journal = "Strahlentherapie und Onkologie",
issn = "0179-7158",
publisher = "Urban und Vogel",
number = "2",

}

TY - JOUR

T1 - Critical discussion of evaluation parameters for inter-observer variability in target definition for radiation therapy

AU - Fotina, I.

AU - Lütgendorf-Caucig, C.

AU - Stock, M.

AU - Pötter, R.

AU - Georg, D.

PY - 2012/2

Y1 - 2012/2

N2 - Background and purpose. Inter-observer studies represent a valid method for the evaluation of target definition uncertainties and contouring guidelines. However, data from the literature do not yet give clear guidelines for reporting contouring variability. Thus, the purpose of this work was to compare and discuss various methods to determine variability on the basis of clinical cases and a literature review. Patients and methods. In this study, 7 prostate and 8 lung cases were contoured on CT images by 8 experienced observers. Analysis of variability included descriptive statistics, calculation of overlap measures, and statistical measures of agreement. Cross tables with ratios and correlations were established for overlap parameters. Results. It was shown that the minimal set of parameters to be reported should include at least one of three volume overlap measures (i.e., generalized conformity index, Jaccard coefficient, or conformation number). High correlation between these parameters and scatter of the results was observed. Conclusion. A combination of descriptive statistics, overlap measure, and statistical measure of agreement or reliability analysis is required to fully report the interrater variability in delineation.

AB - Background and purpose. Inter-observer studies represent a valid method for the evaluation of target definition uncertainties and contouring guidelines. However, data from the literature do not yet give clear guidelines for reporting contouring variability. Thus, the purpose of this work was to compare and discuss various methods to determine variability on the basis of clinical cases and a literature review. Patients and methods. In this study, 7 prostate and 8 lung cases were contoured on CT images by 8 experienced observers. Analysis of variability included descriptive statistics, calculation of overlap measures, and statistical measures of agreement. Cross tables with ratios and correlations were established for overlap parameters. Results. It was shown that the minimal set of parameters to be reported should include at least one of three volume overlap measures (i.e., generalized conformity index, Jaccard coefficient, or conformation number). High correlation between these parameters and scatter of the results was observed. Conclusion. A combination of descriptive statistics, overlap measure, and statistical measure of agreement or reliability analysis is required to fully report the interrater variability in delineation.

KW - Conformity index

KW - Inter-observer variability

KW - Radiotherapy

KW - Similarity metrics

KW - Target volume delineation

UR - http://www.scopus.com/inward/record.url?scp=84858032106&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=84858032106&partnerID=8YFLogxK

U2 - 10.1007/s00066-011-0027-6

DO - 10.1007/s00066-011-0027-6

M3 - Article

VL - 188

SP - 160

EP - 167

JO - Strahlentherapie und Onkologie

JF - Strahlentherapie und Onkologie

SN - 0179-7158

IS - 2

ER -