Automatic estimation of brain parenchymal fraction in patients with multple sclerosis: a comparison between synthetic MRI and an established automated brain segmentation software based on FSL

Multiple sclerosis (MS), which is considered the most common chronic inflammatory central nervous system disease in young people [1], leads to demyelination in the white and grey matter, which secondarily induces neurodegeneration and resulting macroscopic atrophy of the brain [2]. Previous research has shown that brain volume loss due to the degenerative component of the disease, evolves in all stages of MS starting at clinically isolated syndromes, and progresses in association with the patients’ disability status [3‐5]. The clinical importance of brain atrophy is underlined by the inclusion of whole brain quantification in the current No Evidence of Disease Activity-4 criteria (brain atrophy rates < 0.4% per year, besides changes in lesion load) and that new therapeutic concepts in MS increasingly target neuroprotection [6, 7]. Magnetic resonance imaging (MRI) can be used to quantify brain atrophy [8, 9]. However, manual or semi-automatic volumetric methods are time-consuming and require a great deal of expertise. Automated software applications for brain segmentation such as FSL-SIENAX, CAT12, or FreeSurfer not only help to reduce the time required, but are also more reproducible, objective, and comparable than manual methods [10]. Those methods are mostly based on isotropic 3D T1-weighted (T1w) MRI series and have been used in a broad spectrum of research applications. However, such techniques require advanced post-processing strategies, posing significant hurdles for clinical implementation.

Synthetic-MRI (SyMRI) is another fully automatic quantification tool that is based on a dedicated Multi-Dynamic Multi-Echo (MDME) MRI sequence. This single sequence is typically implemented as a 2D axial acquisition with a 4-mm slice thickness at an in-plane resolution of 1 mm. It is processed by the associated software SyMRI (Synthetic MR, Linköping, Sweden) to instantaneously and fully automatically determine brain volumes and the Brain Parenchymal Fraction (BPF; fraction of total brain volume relative to intra-cranial volume). SyMRI can be integrated and operated within the commonly used radiological picture archiving and communication system (PACS), which allows for use in clinical infrastructures. SyMRI has been applied in different research topics, such as paediatric brain development [11], hippocampal sclerosis [12], aging [13], Alzheimer’s disease [14], and MS [15], to study volume changes in the brain.

In the present study, we aimed to compare the fully automated brain segmentation software applications SyMRI and FSL-SIENAX (SIENAX) in regard to their ability to detect brain atrophy in a large sample of patients with MS. SIENAX was used as the gold standard because it has been used similarly for the establishment of a cut-off value for pathologic brain atrophy [16] and in many other studies on MS [17‐20]. We addressed the consistency and differences between BPF estimations with a focus on the methods’ sensitivity to detect disease-related differences in brain volumes between patient groups as well as on their associations with disability levels. Furthermore, we performed a longitudinal analysis of BPF measurements in patients with MS comparing SyMRI and FSL-SIENA (SIENA) for a two-timepoint analysis of percentage brain volume changes (PBVC). We predicted that brain quantification using SyMRI would show good agreement and comparable sensitivity with the established SIENAX method.

Images and segmentation maps were visually inspected for image quality after analysis with SyMRI and SIENAX. We excluded 14 patients due to artefacts or incorrect tissue segmentation, which resulted in a total of 244 patients and control participants. In the patients, both MS subtypes were represented: 110 cases of relapsing–remitting MS (RRMS) and 106 of primary or secondary progressive MS (PMS, including N = 83 SPMS and N = 23 PPMS). The demographic data of the final cohort are summarised in Table 1. Details of the SPMS and PPMS subgroups are shown in the electronic supplement (Table S1). Patients in the PMS group were older and had significantly higher EDSS scores and longer disease duration than patients with RRMS.

In the present study, we validated the estimation of BPF from SyMRI in a large sample of patients with MS via comparison with the widely used SIENA(X) pipelines of FSL as the gold standard. In addition, we performed a longitudinal analysis of BPF measurement in patients with MS using SyMRI in a real-world clinical setting. High correlations between both methods suggested that they yield robust and comparable results. Furthermore good agreement highlighted the consistency of SyMRI compared with SIENAX segmentation. However, a systematic trend of higher BPF values estimated by SyMRI than SIENAX was seen, suggesting that SyMRI overestimated BPF, as well as GMV, relative to SIENAX. In contrast, SyMRI underestimated WMV and CSFV. Such overestimation of BPF and underestimation of CSFV was even more pronounced at higher BPF and lower CSFV values, reflecting excellent consistency (according to ICCs) in patients with brain atrophy and poor consistency in patients without atrophy. Therefore, it might be considered that the systematic bias of larger BPF estimation in SyMRI was mostly due to segmentation or partial volume effects of CSF, as this trend was less pronounced for WMV or GMV. As a secondary finding, we were able to show that larger GM volumes and smaller WM volumes calculated byn SyMRI compared to Sienax were not strongly dependent on the use of lesion-filled or non-lesion-filled T1w images, but must be due to other systematic differences between the methods.

Interestingly, in line with the present results, such overestimation of SyMRI for whole-brain and GM volumes was also obtained in a study comparing SyMRI with statistical parametric mapping (SPM) segmentation [35]. This study validated SyMRI volumetry in children by comparison with SPM and found consistent segmentation results, except for with CSFV, although a systematic influence of age and brain tissue volumes was observed.

Differences in brain volume measures between both methods could be due to the different segmentation algorithms. Because strong correlations and a good agreement between brain tissue volumes were observed between SyMRI and SIENAX, this systematic bias is likely the main source of the volume differences. The SyMRI segmentation approach is based on classification using relaxation times. The tissue types are related to the R1–R2–PD space using a predefined look-up grid, providing fractions of the tissue classes in each voxel [26]. In contrast, SIENAX uses a statistical segmentation approach in which the intensity and neighbouring sites influence spatial information [34]. Hence, differences in, for example, partial volume effects will possibly have an effect on brain volume estimates. Another reason for differences in brain volume is related to discrepancies in image resolution and geometry. Segmentation in SIENAX is based on an isotropic 3D T1w sequence with a slice thickness of 1 mm, whereas SyMRI segmentation uses a 2D image dataset with 4-mm slices. Thus, segmentation with SyMRI is more strongly affected by partial volume effects, which has been shown to be significant for volumetry in previous studies [26, 37, 38].

Because the main goal of developing automatic brain volume estimation approaches is to quantify disease-specific differences or therapeutic effects, it is of crucial importance to validate the ability to differentiate between disease types and to monitor longitudinal changes. Both aspects have been considered in this study. First, comparable reliability in differentiating between RRMS, PMS, and healthy controls was obtained for SyMRI and SIENAX. This confirmed the ability to quantify disease-specific brain volume loss using SyMRI segmentation, as it showed results consistent with the established SIENAX approach. For both methods, significantly lower tissue volumes were observed for PMS compared to RRMS and CS. Taking into account the duration of disease, which is significantly longer in PMS compared to RRMS, the volume differences were plausible from a pathophysiological point of view. The process of neurodegeneration and subsequent barin volume loss continues progressivly in the PMS group and intermittently in the RRMS group.

Second, in a smaller subset of patients with longitudinal data with relatively short follow-up times (median, 15 months), we compared brain atrophy rates assessed by SyMRI and SIENA. SyMRI showed higher atrophy rates than SIENA, approximately twice as high, both, in the whole sample and for each disease subtype. Considering the proposed cut-off value for pathological PBVC in MS of -0.4% [16], the present annualized PBVC value from SIENA of -0.47% for the PMS group reflected relevant brain atrophy within 15 months, slightly exceeding the cut-off, but this was not seen in the small RRMS sample (PBVC, 0.61%), which showed a very high variability of 1.5%. Comparable brain atrophy rates in PMS (about -0.5% / year) compared to RRMS have been reported previously in a study using SIENA for a similar follow-up period of 14 months [20]. Thus our SIENA results seem consistent with previous findings. Additionally, atrophy rates in the RRMS group might differ from PMS because our patients with RRMS had short to medium disease duration, while the disease duration in the PMS group was significantly longer. In contrast to SIENA, SyMRI yielded an averaged PBVC of -0.85% per year for PMS, corresponding to considerable brain atrophy rates compared to the -0.4% threshold, and 0.17% per year for patients with RRMS. These discrepancies between these methods complicate the interpretation and comparison of atrophy results and, thus provide evidence for the need to establish a method-dependent cut-off value for defining clinically significant brain atrophy rates. There are probably different effects that con contribute to higher brain atrophy estimations for SyMRI compared to SIENA especially in the PMS group. First it must be considered that there are methodological differences regarding the estimation of atrophy rates. Thus, SIENA uses a registration-based approach to estimate volume changes, whereas, for SyMRI, we calculated rates based on cross-sectionally obtained volumes from segmentation-based methods at each timepoint. Atrophy assessment using individually segmented MRI data is known to induce higher variabilities because, for example, differences in intensity scales are not taken into account and noise increases because of inconsistent segmentation of the same brain regions. Thus, segmentation-based methods are less reliable for longitudinal atrophy quantification in comparison with registration-based approaches [39]. Second, provided there is substantial brain atrophy during the follow-up period in the PMS group, the increase of the MS-lesion volume that is typically associated with atrophy development can lead to an additional loss of segmented brain volume in the follow-up MRI using SyMRI, as no lesion filling is used with this method and chronical MS lesions can partly be classified as CSF. Furthermore, more brain atrophy in the follow-up MRI and associated widening of sulci or ventricles might result in altered partial volume effects at the CSF-parenchymal boundaries, reflected by an additional loss of the segmented brain volume. Nonetheless, we observed in both methods, SIENA and SyMRI, consistent results regarding group-specific differences, similarly to the cross-sectional results. Thus, both approaches yielded higher atrophy rates for patients with PMS compared to patients with RRMS.

Due to the overestimation of brain volumes in patients with relatively high BPF, which applies particularly to the early phases of the disease, the results obtained with SyMRI have to be interpretated with caution in a clinical setting. Moreover, in future studies, more longitudinal data would be necessary to provide a method-dependent PBVC cut-off for SyMRI segmentation. An improved 3D SyMRI sequence that has recently been released for research purposes could be used to refine segmentation results at a higher resolution [40].

Overall, the automatic brain volume estimation based on SyMRI offers many advantages within a clinical setting. It requires less time than the FSL-SIENA(X) based approaches, not only for MRI acquisition but also for post-processing, as the SyMRI software can be used within the clinical PACS system without the need for image export and tissue volumes are calculated automatically. In contrast, SIENAX and SIENA require more post processing steps with user intervention, which have to be performed on external computers outside the clinical PACS.

In summary, BPF estimations from SyMRI and FSL-based SIENAX or SIENA (for longitudinal analyses) are not interchangeable since an overestimation of volumes and brain atrophy rates, as well as higher variability of SyMRI results, were observed. However, the consistency and correlations between the two methods were satisfactory, and our results showed that SyMRI was similarly to FSL suited to quantifying disease specific atrophy in MS from both cross-sectional and longitudinal MRI. Therefore, our analyses provided evidence that SyMRI volume estimation delivered reliable results, and it can be considered a fast and easy-to-use alternative to tissue segmentation based on 3D T1w images.