Deep transformer-based personalized dosimetry from SPECT/CT images: a hybrid approach for [¹⁷⁷Lu]Lu-DOTATATE radiopharmaceutical therapy

Radiopharmaceutical therapy (RPT) has emerged as a promising approach for managing various cancers, thus enabling selective delivery of high radiation dose to the target while minimizing toxicity to normal tissues [1, 2]. Among RPT techniques, peptide receptor radionuclide therapy (PRRT) with [¹⁷⁷Lu]Lu-DOTATATE has demonstrated efficacy in palliative treatment of patients diagnosed with non-resectable metastatic neuroendocrine tumors (NETs) with significant improvement in their overall survival and progression-free survival [3, 4]. The commonly used standard practice of this treatment employs “one-size-fits-all” empirical protocol, consisting of the delivery of four cycles of 7.4 GBq at 8-week intervals [5, 6]. However, this regimen may not be optimal for individual variations in tumor burden, patient physiology, body size, and overall health condition. Therefore, a personalized dosimetry and treatment planning approach is indispensable to ensure a balance between therapeutic efficacy and patient safety [7].

Implementing personalized dosimetry as a prerequisite for treatment planning in routine clinical setting poses challenges owing to (i) the lack of standardized image quantification and dosimetry protocols [8, 9], (ii) lack or weak dose-effect relationship information [10], (iii) lack of suitable radiobiological models [11], (iv) inherent low-quality and incomplete imaging data at different time points [12], (v) the heterogeneity of tumors and organs at risk (OARs), and (vi) the variability of patients’ physiology. Besides, time and labor resource requirements [11, 13] further complicate the process.

On the other hand, traditional image-based organ-level dosimetry approaches based on MIRD formalism may not suffice for patient-specific purposes as they rely on using tabulated organ-level S-values derived from standard phantoms [14]. These S-values can neither reflect activity heterogeneities within a region of interest nor do they account for inter-patient anatomical differences [15]. Voxel-level image-based dosimetry approaches have gained popularity as an alternative to overcome this limitation. Direct Monte Carlo (MC) is the consensual gold standard approach for voxel-level dose calculations which provides accurate and reliable dose estimation by considering the individual non-homogeneity of both anatomical and activity distribution into account. However, its extensive computational requirements make it impractical for routine clinical use [16‐19].

To address the challenges associated with MC, various voxel-level dosimetry methods have been developed [20‐22]. This includes the single S-value (SSV) approach using the MIRD scheme pre-tabulated S-values, which lacks consideration of anatomical heterogeneities as in these approaches dose calculations occurs within homogenous water medium [21]. Another approach is the multiple voxel S-value (MSV) technique in which instead of using a single-dose kernel calculated in soft tissue, multiple-dose kernels are used according to different tissue densities [22].

Deep learning (DL) has been successfully employed for different computational medical imaging tasks [23‐29]. There have been some attempts to use DL-based voxel-wise internal dosimetry in previous studies [15, 30‐33]. Lee et al. employed a U-net trained by PET and CT image patches as input to generate 3D voxel-level dose rate maps [30]. Kim et al. [33] developed a modified U-net model for voxel-wise [¹⁷⁷Lu]Lu-DOTATATE dosimetry, incorporating CT and time-integrated activity (TIA) patch images as an input followed by a summation of MSV dose map for residual learning and validated their model through comparison with direct MC calculation at both organ and voxel levels. In the study by Li et al. [32], a residual deep convolutional neural network (CNN) trained with virtual patients obtained from PET images instead of SPECT/CT images was used to estimate the dose-rate maps compensated for blurring due to the poor spatial resolution of SPECT images. While the abovementioned studies benefited from utilizing CNNs, the application of transformer architectures remains limited in dosimetry tasks [34]. Transformers benefit from self-attention and can process the input data in parallel which leads to increased efficacy. In addition, they can achieve adequate performance with very limited training data [35‐38].

The main objective of the present work is to evaluate a hybrid transformer-based deep learning network for voxel-level dosimetry of [¹⁷⁷Lu]Lu-DOTATATE RPT in terms of computational costs and accuracy of absorbed dose calculations. To this end, we trained an MSV/DL hybrid model to predict the necessary corrections on MSV dose maps to generate MC dose maps as the ground-truth. Subsequently, DL-based absorbed dose distributions were validated by comparing the results with MC-based absorbed dose distributions. SSV and MSV dose maps were also calculated for further comparison.

Only tumors with volumes >2 ml (mean volume: 38.18 ± 79.4 ml, range: 2.2–947.2 ml) were delineated in this study to be included in the analysis. The mean absorbed dose values (Gy) of tumors and OARs for each patient, calculated from MC simulations for the first treatment cycle, are presented in Supplementary Table 1. Figure 3 displays the voxel-wise relative absolute error maps, highlighting the majority of errors in lung tissue when using the SSV dose calculation approach. Voxel-wise quantitative metrics for three dosimetry approaches with respect to MC are tabulated in Table 2. Our DL approach achieved the lowest errors (e.g., RMSE, ME, MAE, MSE, RE, RAE) and maximum resemblance representatives (e.g., SSIM, PSNR, Gamma pass rates). The distribution of these quantitative metrics is visualized using violin plots in Fig. 4. Additionally, as a result of employing the 5-fold cross-validation approach, these quantitative metrics and their distribution are reported separately for each fold in Supplementary Table 2 and Supplementary Fig. 1, respectively.

We conducted a 3D Gamma evaluation considering MC dose maps as standard of reference. Figure 5 presents an example of Gamma maps with the highest number of rejected voxels (Gamma value >1) in the SSV approach. Our hybrid DL approach, on the other hand, generated dose maps with reduced number of rejected points in this evaluation. Furthermore, we performed the Gamma evaluation within the lesion volumes, and the pass rates were found to be 96.8 ± 6%, 97.2 ± 5.3%, and 97.35 ± 5.1% for SSV, MSV, and hybrid DL approaches, respectively.

Additionally, we reported the organ-level errors for lesions and OARs in Table 3. Our hybrid DL approach showed a reduction in error values for most organs, except for the left kidney. Organ-wise errors for each fold are also reported in Supplementary Table 3. The Bland-Altman plots in Fig. 6 depict the differences in mean absorbed dose calculated by each dosimetry approach with respect to MC for OARs and lesions. The solid line represents the mean difference with MC, whereas the dashed lines indicate the upper and lower limits of agreement (mean difference ± 1.96 times the standard deviation of the differences). An example of the DVH for lesions and various OARs is depicted in Supplementary Figure 2.

Fast and accurate generation of dose maps may help the implementation and adoption of personalized dosimetry in RPT in the clinic. A modified UNETR model combined with MSV was developed to achieve efficient and accurate voxel-wise dosimetry for [¹⁷⁷Lu]Lu-DOTATATE therapy. This study evaluated organ- and voxel-wise errors in comparison to MC dose calculation, considered as standard of reference. While this study does not aim to conduct a direct head-to-head comparison of independent approaches, we included the analysis of the voxel S-value MIRD formalisms to ensure a comprehensive dosimetry investigation. Additionally, our objective is to highlight and demonstrate the potential limitations associated with these approaches. One of the motivations behind employing a DL network for [¹⁷⁷Lu]Lu-DOTATATE voxel-wise dosimetry was the limitation of current voxel-level MIRD formalisms. In the SSV approach, the dose map from a radiopharmaceutical is constructed through the convolution of only one voxel S-value kernel (water) with the TIA calculated from emission tomography images (PET or SPECT). This approach does not account for organ heterogeneities [21]. The alternative MIRD-based dosimetry approach, referred to as multiple S-value (MSV), utilizes multiple kernels, overcoming the limitations of the SSV technique. However, it still presents visible errors in tissue boundaries and interfaces [22].

Monte Carlo simulations were used as the gold standard. However, MC itself is prone to statistical errors and differences with real-world occurrences that may lead to missing dose heterogeneities. We had access to limited computational resources, and hence, only 100 million tracked particles were simulated which means the MC itself is prone to sampling statistical errors even negligible [44]. Besides, the inherent approximations within MC calculations and geometry simplifications may not accurately capture the complexity of clinical scenarios.

In a broader point of view, the distinctions among SSV, MSV, and MC techniques lie in the extent of approximations and assumptions made. SSV assumes uniform radiation absorption across all tissues (assumed as water), whereas MSV introduces tissue-specific kernels, reducing assumptions but increasing the descriptive parameters and calculations. In contrast, MC makes the fewest assumptions but requires more parameters to be defined. This explains the superior performance of MSV over SSV and even better performance of MC over MSV.

Noteworthy, absorbed dose heterogeneity in the context of RPT includes both spatial and temporal heterogeneities. The first refers to uneven distribution of absorbed doses within a VOI, while the second refers to the variation of absorbed dose rates over time due to pharmacokinetics and radioactive decay. Although dynamic imaging or imaging in multiple time points is performed to address the temporal heterogeneity, radiopharmaceutical kinetics is oversimplified in TAC fitting process [51]. The accuracy of TAC derivation depends on the spatial resolution of the imaging modality as well as the frequency of the time points. Low spatial resolution or infrequent time points lead to inadequate capturing of rapid changes in radiotracer distributions, affecting the accuracy of TACs and, consequently, the calculated absorbed doses. However, the initial steps including time-point registration, curve fitting, and TIA calculations were similar for all four dose calculation methods (SSV, MSV, MC, and DL) evaluated in this work.

Most of the errors in voxel-wise dose calculations occur when using the SSV approach, which is evident in Fig. 3. The SSV method overestimates the absorbed dose values inside the low-density tissues, such as the lungs, and underestimates the absorbed dose values in tissues with high density, such as bones. While MSV mitigated these errors, it still showed limitations in high-gradient tissue density regions, i.e., the tissue boundaries (lung/soft tissue and bone/soft tissue). However, our proposed hybrid DL model effectively reduced errors in these areas as confirmed by Gamma maps (Fig. 5). Our method improves the MSV dose map accuracy both in voxel-level and organ-level evaluations as presented in Figs. 4 and 6. Although Gamma evaluation pass rates were reported, displaying Gamma maps allowed observing regions of failure. In addition, lesion-wise Gamma evaluation indicated a slightly higher pass rate for the DL approach. To the best of our knowledge, there is no guideline indicating a specific criterion for Gamma evaluation in targeted radiotherapy. However, the chosen criteria (DTA = 4.79 mm and DD = 1%) were based on the spatial resolution of SPECT images as a determinative factor and external beam radiation therapy standards.

DVHs are shown in supplementary Figure 2, where we can compare the dose distributions visually. The rationale behind using DVHs relies on their ability to provide a comprehensive and quantitative summary of dose distribution. This involves representing the percentage volume of a volume of interest (VOI) that receives a specific dose or higher (V_x (%)) or a specific absorbed dose received by a percentage of volume (D_x (Gy)). This quantitative data is crucial to evaluate the delivered absorbed doses to lesions and OARs, especially for heterogenous dose distributions. DVHs visually depict dose distribution, facilitating straightforward comparisons between different dose distributions. Although DVHs are well-studied for handling heterogeneous dose distributions, they are not able to provide spatial information on where dose inhomogeneities occur within the volume. Furthermore, by simplifying the complex 3D dose distribution into a single metric for each dose level, DVHs may overlook specific geometric patterns or gradients that hold clinical significance. Additionally, the granularity of a DVH depends on the selected dose-volume bin size, and slight changes in bin size can alter the shape of the DVH curve, potentially influencing interpretations [52]. One of the reasons that prevented us from distinguishing the differences among various dosimetry approaches using DVHs is attributed to the absence of spatial information. To address this limitation, we have employed Gamma analysis and visualized the results to identify specific regions showing observed differences.

In this study, a transformer-based UNET-shaped model was utilized. To our knowledge, in dosimetry-related studies, CNNs have been mostly utilized, while the application of transformers in this specific task remains limited [34, 5 4]. Transformer-based models are generally used in Natural Language Processing (NLP) with the capability of highlighting the important features of word sequences due to self-attention mechanism [47, 54]. The computational efficiency and superiority of our model in image domain-related tasks, such as organ segmentation, has been demonstrated compared to state-of-the-art CNNs [34, 47]. While the optimal performance of complex transformers often relies on the availability of a large-scale dataset, our model as a less-complex transformer model does not necessarily perform better using larger-scale dataset. It should be mentioned that the true value of a large-scale dataset lies in its capacity to inject diversity into the training process. The absence of diversity in a large-scale dataset may result in suboptimal performance when applied on external datasets. In our study, despite the limited size of our dataset, we paid attention to diversity by including various patients with personalized injected activities ranging from 1017 to 9657 MBq. While transformer-based models may not consistently outperform other networks in terms of performance metrics, their adaptability to small-size datasets, proficiency in managing diversity, and ability to capture complicated relationships make them well-suited for personalized dosimetry tasks. Nevertheless, to cope with the inherent black-box nature of deep learning methods, including transformers, the development of explainable AI models to enhance the interpretability and trustworthiness of the predicted results is suggested [55].

It is worth mentioning that before opting for the current approach, we tried other strategies involving the use of other networks, such as UNET and U2-NET. However, the results were not as favorable as using the adopted network. The best results we could achieve through other networks or strategies are summarized in Supplementary Table 4.

Noteworthy, the size, shape, and texture variations of organs/lesions, as well as SPECT and CT images, in different regions of interest, can influence the performance of the DL method, justifying distinct errors in various regions of interest.

Despite using limited training data, our hybrid model provided fast and accurate voxel-wise dose estimations. Dose map generation time for an image size of 128 × 128 × 80 voxels is approximately 1 min, which is an impressive improvement compared to MC running time. The model performance was consistent among folds showing robustness. Moreover, the DL approach exhibited a slight advantage over MIRD-based dose calculations, consistent with recent studies on [¹⁷⁷Lu]Lu-DOTATATE dosimetry calculated by DL [33, 53]. In this study, rather than directly generating MC dose maps, we calculated the difference between MC and MSV which resulted in further error reduction. Since MSV provides a good approximation of MC dose maps, and errors only occur in small heterogeneous and boundary regions, therefore, training a model that focuses on correcting these errors helped in performance improvement. The network predicts the difference map based on anatomical information in the CT image. Although generating the difference dose map between MC and MSV requires the MSV dose map calculations to perform the DL-based dosimetry calculation, once the model has been trained, the inference time (~1 min including pre- and post-processing) is negligible compared to the MSV calculation time (~1.2 h). Nevertheless, it is important to highlight the dependence of the proposed approach on the MSV map post-inferencing to reconstruct the MC dose map, as outlined in the “Material and methods” section (predicted MC = DL + MSV). Therefore, the DL model is presented as a hybrid MSV/DL model, and the inference time should be added to MSV calculation time (1.2 h on CPU + 1 min). As described in the “Material and methods” section, the MSV calculations were performed on CPU. We implemented the MSV calculation part on GPU to accelerate the procedure. Indeed, GPU-based computations for MSV would further reduce the calculation time (14 s for each voxel S-value kernel of MSV for a 2-bed SPECT image with a resolution of ~4.8 mm traceable on P#6 of SNMMI dosimetry challenge dataset [13]). Therefore, conducting our hybrid DL approach on GPU would take ~3 min for a two-bed SPECT (MSV with 8 kernels ~2 min, and inference time ~1 min).

Automated tools, such as OAR segmentation, which is the most time-consuming step in the dosimetry workflow, and SPECT-SPECT registration facilitated and accelerated our study. The OAR segmentation model demonstrated a high degree of reliability as reflected by the performance metrics (high Dice coefficients) [45]. This model can be employed in dosimetry workflow of other radionuclides and embedded in treatment planning systems. It should be mentioned that organ segmentation on CT images is a more practical approach, yet susceptible to potential errors arising from the different resolutions of CT and SPECT images, the spill-in/out phenomena in SPECT images, and the likely mismatch between SPECT and CT images. Lesion segmentation was manually carried out by a physician, considering both SPECT and CT images, along with the fused image at all three time points.

The SPECT-SPECT registration for the generation of TIA has an advantage over transformation methods based on the CT of hybrid SPECT/CT, avoiding issues with mis-alignment errors in SPECT-CT co-registered images originating from patient movement and respiratory motion [44, 56]. However, given the time intervals between different acquisitions, several factors introduce challenges to achieving precise alignment even with SPECT-SPECT registration approach. These factors include patient motion, fluctuations in radiopharmaceutical distribution, radioactive decay, inherent image noise and poor spatial resolution, lack of anatomic information, and anatomical-related variables, such as nutritional and voiding status of the patient. The complicated interplay of these dynamic elements necessitates a comprehensive appraisal to ensure accurate and reliable image registration [56]. The incorporation of trapezoidal and mono-exponential TAC fitting in this study allowed modeling both the uptake and washout of the tracer, which is not achievable by using mono-exponential TAC fitting alone [13], particularly in organs at risk, such as the kidneys [41].

This study bears inherently a number of limitations, including the small sample size. Yet, the results demonstrated the effectiveness of the network with a 5-fold cross-validation strategy. Furthermore, the unavailability of diagnostic images for accurate tumor delineation may lead to underestimation of absorbed dose values [13]. Manual lesion segmentation on SPECT/CT images introduces inherent sources of error, particularly in cases where the lesions are distributed across various anatomical regions. The reliance on visually selected image windowing (window width and window level) poses challenges in accurately defining lesion borders. Such misalignments adversely impact the accuracy of TAC calculation, as the delineated lesion may include voxels from surrounding tissues. Conversely, there is a possibility of missing part of the lesion when using this approach.

Another limitation of our study is that users are required to compute the MSV dose maps for reconstructing MC estimate from DL results. Nevertheless, the MSV dose map can serve as a conventional method with a known extent of error, aiding in identifying cases where the DL dose map prediction may be unreliable in an external dataset. The time required for MSV calculation contributes to the safety of our methodology in identifying outliers in an external, unseen dataset. Another limitation of this work is the use of mono-centric instead of multi-institutional datasets to ensure the robustness and generalizability of trained hybrid DL model, and as such, further evaluation on larger multi-centric datasets is still required.

This work aimed at developing a hybrid transformer-based deep learning model incorporating the MSV approach, trained for fast and accurate generation of absorbed dose maps. The results demonstrated superiority of this hybrid network over the MIRD SSV approach, and a slight improvement compared to the MSV approach while significantly outperforming MC in terms of computational efficiency. It can be concluded that our method achieved an accuracy close to the gold standard and surpassed it in terms of time and computational effort, making it more readily applicable in clinical setting once trained using a large-scale and multi-centric dataset. In summary, according to our results, using the MSV method is recommended for organ-level dosimetry, whereas MC or hybrid DL method is suggested for voxel-wise dosimetry, especially for small lesions/organs in regions with high gradient of density.