Performance of five research-domain automated WM lesion segmentation methods in a multi-center MS study.
- de Sitter A Department of Radiology and Nuclear Medicine, VUmc, Amsterdam, The Netherlands. Electronic address: a.desitter@vumc.nl.
- Steenwijk MD Department of Anatomy and Neuroscience, VUmc, Amsterdam, The Netherlands.
- Ruet A Department of Neurology, CHU-Bordeaux, Bordeaux, France; University of Bordeaux, Bordeaux, France; Inserm U-1215 Magendie Neurocenter-Pathophysiology of Neural Plasticity, CHU-Bordeaux, Bordeaux, France.
- Versteeg A Department of Radiology and Nuclear Medicine, VUmc, Amsterdam, The Netherlands.
- Liu Y Department of Radiology and Nuclear Medicine, VUmc, Amsterdam, The Netherlands.
- van Schijndel RA Department of Radiology and Nuclear Medicine, VUmc, Amsterdam, The Netherlands.
- Pouwels PJW Department of Radiology and Nuclear Medicine, VUmc, Amsterdam, The Netherlands.
- Kilsdonk ID Department of Radiology and Nuclear Medicine, VUmc, Amsterdam, The Netherlands.
- Cover KS Department of Radiology and Nuclear Medicine, VUmc, Amsterdam, The Netherlands.
- van Dijk BW Department of Anatomy and Neuroscience, VUmc, Amsterdam, The Netherlands.
- Ropele S Department of Neurology, Medical University of Graz, Graz, Austria.
- Rocca MA Neuroimaging Research Unit, Institute of Experimental Neurology, Division of Neuroscience, San Raffaele Scientific Institute, UniSR, Milan, Italy.
- Yiannakas M Department of Neuroinflammation, Institute of Neurology, UCL, London, UK.
- Wattjes MP Department of Radiology and Nuclear Medicine, VUmc, Amsterdam, The Netherlands.
- Damangir S Department of Neurobiology, Care Sciences and Society, Karolinska Institute, Stockholm, Sweden.
- Frisoni GB Laboratory of Epidemiology, Neuroimaging and Telemedicine, IRCCS Centro "S. Giovanni di Dio-F.B.F.", Brescia, Italy; Memory Clinic and LANVIE - Laboratory of Neuroimaging of Aging, HUG, Geneva, Switzerland.
- Sastre-Garriga J Centre d'Esclerosi Múltiple de Catalunya (Cemcat), Department of Neurology/Neuroimmunology, VHIR, Barcelona, Spain.
- Rovira A Magnetic Resonance Unit, Department of Radiology (IDI), VHIR, Barcelona, Spain.
- Enzinger C Department of Neurology, Medical University of Graz, Graz, Austria; Division of Neuroradiology, Vascular and Interventional Radiology, Department of Radiology, Medical University of Graz, Graz, Austria.
- Filippi M Neuroimaging Research Unit, Institute of Experimental Neurology, Division of Neuroscience, San Raffaele Scientific Institute, UniSR, Milan, Italy.
- Frederiksen J Department of Neurology, Glostrup University Hospital, Copenhagen, Denmark.
- Ciccarelli O UK/NIHR UCL-UCLH Biomedical Research Centre, Institute of Neurology, UCL, London, UK.
- Kappos L Neurologic Clinic and Policlinic, University Hospital, University of Basel, Switzerland.
- Barkhof F Department of Radiology and Nuclear Medicine, VUmc, Amsterdam, The Netherlands; Institutes of Neurology & Healthcare Engineering, UCL, London, UK.
- Vrenken H Department of Radiology and Nuclear Medicine, VUmc, Amsterdam, The Netherlands; Department of Anatomy and Neuroscience, VUmc, Amsterdam, The Netherlands.
- 2017-09-14
Published in:
- NeuroImage. - 2017
Automated methods segmentation
MRI
Multiple sclerosis
White matter lesion
Adult
Female
Humans
Image Interpretation, Computer-Assisted
Magnetic Resonance Imaging
Male
Multiple Sclerosis
White Matter
English
BACKGROUND AND PURPOSE
In vivoidentification of white matter lesions plays a key-role in evaluation of patients with multiple sclerosis (MS). Automated lesion segmentation methods have been developed to substitute manual outlining, but evidence of their performance in multi-center investigations is lacking. In this work, five research-domain automated segmentation methods were evaluated using a multi-center MS dataset.
METHODS
70 MS patients (median EDSS of 2.0 [range 0.0-6.5]) were included from a six-center dataset of the MAGNIMS Study Group (www.magnims.eu) which included 2D FLAIR and 3D T1 images with manual lesion segmentation as a reference. Automated lesion segmentations were produced using five algorithms: Cascade; Lesion Segmentation Toolbox (LST) with both the Lesion growth algorithm (LGA) and the Lesion prediction algorithm (LPA); Lesion-Topology preserving Anatomical Segmentation (Lesion-TOADS); and k-Nearest Neighbor with Tissue Type Priors (kNN-TTP). Main software parameters were optimized using a training set (N = 18), and formal testing was performed on the remaining patients (N = 52). To evaluate volumetric agreement with the reference segmentations, intraclass correlation coefficient (ICC) as well as mean difference in lesion volumes between the automated and reference segmentations were calculated. The Similarity Index (SI), False Positive (FP) volumes and False Negative (FN) volumes were used to examine spatial agreement. All analyses were repeated using a leave-one-center-out design to exclude the center of interest from the training phase to evaluate the performance of the method on 'unseen' center.
RESULTS
Compared to the reference mean lesion volume (4.85 ± 7.29 mL), the methods displayed a mean difference of 1.60 ± 4.83 (Cascade), 2.31 ± 7.66 (LGA), 0.44 ± 4.68 (LPA), 1.76 ± 4.17 (Lesion-TOADS) and -1.39 ± 4.10 mL (kNN-TTP). The ICCs were 0.755, 0.713, 0.851, 0.806 and 0.723, respectively. Spatial agreement with reference segmentations was higher for LPA (SI = 0.37 ± 0.23), Lesion-TOADS (SI = 0.35 ± 0.18) and kNN-TTP (SI = 0.44 ± 0.14) than for Cascade (SI = 0.26 ± 0.17) or LGA (SI = 0.31 ± 0.23). All methods showed highly similar results when used on data from a center not used in software parameter optimization.
CONCLUSION
The performance of the methods in this multi-center MS dataset was moderate, but appeared to be robust even with new datasets from centers not included in training the automated methods.
In vivoidentification of white matter lesions plays a key-role in evaluation of patients with multiple sclerosis (MS). Automated lesion segmentation methods have been developed to substitute manual outlining, but evidence of their performance in multi-center investigations is lacking. In this work, five research-domain automated segmentation methods were evaluated using a multi-center MS dataset.
METHODS
70 MS patients (median EDSS of 2.0 [range 0.0-6.5]) were included from a six-center dataset of the MAGNIMS Study Group (www.magnims.eu) which included 2D FLAIR and 3D T1 images with manual lesion segmentation as a reference. Automated lesion segmentations were produced using five algorithms: Cascade; Lesion Segmentation Toolbox (LST) with both the Lesion growth algorithm (LGA) and the Lesion prediction algorithm (LPA); Lesion-Topology preserving Anatomical Segmentation (Lesion-TOADS); and k-Nearest Neighbor with Tissue Type Priors (kNN-TTP). Main software parameters were optimized using a training set (N = 18), and formal testing was performed on the remaining patients (N = 52). To evaluate volumetric agreement with the reference segmentations, intraclass correlation coefficient (ICC) as well as mean difference in lesion volumes between the automated and reference segmentations were calculated. The Similarity Index (SI), False Positive (FP) volumes and False Negative (FN) volumes were used to examine spatial agreement. All analyses were repeated using a leave-one-center-out design to exclude the center of interest from the training phase to evaluate the performance of the method on 'unseen' center.
RESULTS
Compared to the reference mean lesion volume (4.85 ± 7.29 mL), the methods displayed a mean difference of 1.60 ± 4.83 (Cascade), 2.31 ± 7.66 (LGA), 0.44 ± 4.68 (LPA), 1.76 ± 4.17 (Lesion-TOADS) and -1.39 ± 4.10 mL (kNN-TTP). The ICCs were 0.755, 0.713, 0.851, 0.806 and 0.723, respectively. Spatial agreement with reference segmentations was higher for LPA (SI = 0.37 ± 0.23), Lesion-TOADS (SI = 0.35 ± 0.18) and kNN-TTP (SI = 0.44 ± 0.14) than for Cascade (SI = 0.26 ± 0.17) or LGA (SI = 0.31 ± 0.23). All methods showed highly similar results when used on data from a center not used in software parameter optimization.
CONCLUSION
The performance of the methods in this multi-center MS dataset was moderate, but appeared to be robust even with new datasets from centers not included in training the automated methods.
- Language
-
- English
- Open access status
- green
- Identifiers
-
- DOI 10.1016/j.neuroimage.2017.09.011
- PMID 28899746
- Persistent URL
- https://sonar.ch/global/documents/46439
Statistics
Document views: 96
File downloads:
- Full-text: 0