A deep learning model for radiological measurement of adolescent idiopathic scoliosis using biplanar radiographs

Abstract

Background

Accurate measurement of the spinal alignment parameters is crucial for diagnosing and evaluating adolescent idiopathic scoliosis (AIS). Manual measurement is subjective and time-consuming. The recently developed artificial intelligence models mainly focused on measuring the coronal Cobb angle (CA) and ignored the evaluation of the sagittal plane. We developed a deep-learning model that could automatically measure spinal alignment parameters in biplanar radiographs.

Methods

In this study, our model adopted ResNet34 as the backbone network, mainly consisting of keypoint detection and CA measurement. A total of 600 biplane radiographs were collected from our hospital and randomly divided into train and test sets in a 3:1 ratio. Two senior spinal surgeons independently manually measured and analyzed spinal alignment and recorded the time taken. The reliabilities of automatic measurement were evaluated by comparing them with the gold standard, using mean absolute difference (MAD), intraclass correlation coefficient (ICC), simple linear regression, and Bland-Altman plots. The diagnosis performance of the model was evaluated through the receiver operating characteristic (ROC) curve and area under the curve (AUC). Severity classification and sagittal abnormalities classification were visualized using a confusion matrix.

Results

Our AI model achieved the MAD of coronal and sagittal angle errors was 2.15° and 2.72°, and ICC was 0.985, 0.927. The simple linear regression showed a strong correction between all parameters and the gold standard (p < 0.001, r² ≥ 0.686), the Bland-Altman plots showed that the mean difference of the model was within 2° and the automatic measurement time was 9.1 s. Our model demonstrated excellent diagnostic performance, with an accuracy of 97.2%, a sensitivity of 96.8%, a specificity of 97.6%, and an AUC of 0.972 (0.940–1.000).For severity classification, the overall accuracy was 94.5%. All accuracy of sagittal abnormalities classification was greater than 91.8%.

Conclusions

This deep learning model can accurately and automatically measure spinal alignment parameters with reliable results, significantly reducing diagnostic time, and might provide the potential to assist clinicians.

Introduction

Adolescent Idiopathic Scoliosis (AIS) is a common three-dimensional spinal deformity with an unclear etiology, which may be related to factors such as estrogen, low bone density, and genetic predisposition [1,2,3]. The prevalence of AIS is about 1–3%, with a higher incidence in girls than boys [1]. Without prompt intervention, deformity may worsen and reduce the quality of life. School screening is one of the important methods for early detection of AIS [4], and following a positive screening result, further imaging with anteroposterior(AP)and lateral(LAT) X-rays is usually necessary [3].

Accurate accessment of spinal alignment is crucial for making proper treatment. It is based on radiographic measurements from the biplanar radiographs, including the coronal Cobb angle(CA) and sagittal CA such as the proximal thoracic kyphosis(PTK), mid-thoracic kyphosis(MTK), and thoracolumbar sagittal alignment(TSA).On AP X-rays, the diagnostic gold standard for AIS is a coronal CA exceeding 10°, and AIS is typically classified into normal (CA < 10°), mild (10°–25°), moderate (25°–45°), and severe (> 45°). Clinicians select treatment strategies based on the severity, including regular monitoring, brace, and surgical correction [1, 5]. For LAT X-rays, the measurement of the sagittal CA is used to assess the sagittal plane balance of the spine, which is crucial for surgical planning. However, the diagnosis and evaluation of AIS rely mainly on manual measurements by clinicians, which are time-consuming, labor-intensive, and subjective, leading to considerable measurement errors [6, 7]. Thus, accurately and rapidly measuring and analyzing AIS parameters remains a challenge.

In recent years, artificial intelligence (AI), particularly deep learning, has emerged as a promising tool for image processing and is gradually being applied in the field of scoliosis [8,9,10,11]. AI-based algorithms can automatically detect key points and segment vertebrae from spinal images, enabling the automatic measurement of radiological parameters. This can significantly streamline clinical workflows and enhance diagnostic efficiency. However, current research mainly focuses on the automatic measurement of the coronal plane [8,9,10,11], with limited studies on the automatic measurement of the sagittal plane [12]. Furthermore, there is a lack of comprehensive diagnostic and assessment research based on biplanar radiographs, which restricts the application of AI in treatment planning and follow-up assessments.

Therefore, this study aims to develop and validate a deep learning-based model that can automatically, accurately, and rapidly measure parameters from biplanar X-rays, and perform diagnosis and assessment of AIS, thereby assisting clinicians in AIS practice.

Materials and methods

Patients

This study was approved by the Ethical Review Board of the Xijing Hospital of the Air Force Military Medical University. As this study was retrospective, the ethics committee waived the requirement for informed consent. All radiographs were taken by a digital X-ray machine (Definium 6000 DR, General Electric Company, United States). When taking an AP radiograph, the patient should stand naturally, barefoot, with feet shoulder-width apart. The knees should be naturally extended, arms should hang loosely by the sides. If the leg length discrepancy is greater than 2 cm, the shorter side should be elevated to keep the pelvis level. For the LAT radiograph, the patient should stand in the same manner, with arms bent at the elbows at a 90°and raised horizontally in front of the chest. All radiographic examinations are used for clinical measurement to diagnose or monitor scoliosis progression.

Inclusion criteria: (1)Includes complete biplane X-ray images; (2)Suspected or diagnosed as AIS; (3)10–18 years old; (4)The interval between two examinations for the same patient exceeds 6 months; (5) no significant rotation or tilt for radiographs.

Exclusion criteria:(1) Poor image quality; (2) Severe pelvic tilt; (3) Previous history of spinal surgery; (4) Congenital vertebral abnormalities or other musculoskeletal disorders,; (5) Wearing braces.

Based on the inclusion and exclusion criteria mentioned above, we collected a total of 600 AP and LAT full spinal radiographs taken in the radiology department from October 2021 to July 2023. A dataset was constructed, including 276 males and 324 females with a mean age of 15.3 ± 6.1 years. The dataset consists of radiographic images of normal individuals and patients with scoliosis. We used the random number table method to divide the dataset into train and test sets in a ratio of 3:1. The demographic information of the dataset was presented in Table 1.

Table 1 Demographics in data set

Definition of parameters and classification of severity

For coronal parameters, CA is defined as the angle between the upper endplate of the upper end vertebra(UEV) and the lower endplate of the lower end vertebra(LEV) [1, 6, 13], and end vertebra(EV) refers to the most inclined vertebra in the spine. According to the clinical diagnostic standard, a CA greater than 10° can diagnose scoliosis [1]. For the severity classification of deformity, if the coronal Cobb angle is 0–10°, it is considered normal; 10–25° is classified as mild, 25–45° as moderate, and greater than 45° as severe. For sagittal parameters, we calculated PTK, MTK, and TSA, which are widely used in Lenke classification [14]. The previously reported normal range of PTK, MTK, and TSA were 0–20$^\circ$ [14], 10 –40° [14, 15], and 0–10  [16], respectively.

Model construction

Image annotation

Two junior orthopedics doctors used the image annotation software "Labelme" to manually label T1-L5 in the AP and LAT radiographs. Specifically,454 X-ray images in the train set are randomly assigned to those two doctors for annotation. Before independent annotation, both of them had undergone rigorous training on the standardization of scoliosis radiographic diagnosis and annotation software usage, unifying the definition and methods of annotation. We invited two senior spinal surgeons to review the annotated images one by one. Any inaccurate annotations will be corrected. Finally, the annotated images were saved in JSON format.

Network structure

For the input spinal AP and LAT images, we adjusted the image size by uniformly resizing all images to 1024 × 512. Subsequently, image grayscale, denoising, normalization, and other processing were carried out to obtain higher-quality images while reducing subsequent computational complexity; Finally, to enhance the model's generalization ability and adaptability to variations in imaging data. data augmentation techniques including random flipping, translation, and scaling are employed.

In this study, we adopted ResNet34 [17] as the basic framework and improved it by integrating the network characteristics of U-net (Fig. 1). This design effectively solves the common problem of gradient vanishing in deep network training, while utilizing ResNet34's deep residual learning mechanism to endow the network with powerful feature extraction capabilities. In addition, skip connections enable the network to integrate deep high-level semantic features and shallow detail features at different levels, thereby simultaneously utilizing global contextual information and local fine information in image analysis [18, 19]. This combination not only improves the network's understanding of image content but also enhances its localization accuracy in complex medical image processing tasks. The residual network performs multiple convolutions and pooling operations on the image to extract high-level semantic feature maps at different levels of the image. Then, skip connections are used to combine deep and shallow features. We constructed a heatmap, center offset, and corner offset in the output layer [19].

Fig. 1

Model structure and diagnostic process. After inputting the coronal and sagittal X-rays of the full spine, our model will automatically perform landmark detection and CA measurement, output diagnostic results, and then evaluate the severity of scoliosis and sagittal plane abnormalities based on the range of parameter values. CA: Cobb angle

The heatmap is used to locate the center of the vertebra and can be represented by a Gaussian disk, whose calculation method is the same as Yi et al. [19]. The center offset is used to map the reduced feature map after extracting the center point back to the original image, while the corner offset is the vector pointing from the center of the vertebrate to the vertebral corner, used to locate the four vertebra corners. In this deep learning training process, the supervision module includes heatmap loss, center offset loss, and corner offset loss. Finally, our model identified the top left, top right, bottom left, and bottom right corners of each vertebral body based on the coordinates obtained from vertebra corner detection (Fig. 1).

CA measurement

On the AP radiograph, the coordinates of the upper left, upper right, lower right, and lower left corners of the vertebra in the plane rectangular coordinate system are denoted as A(x₁, y₁), B(x₂, y₂), C(x₃, y₃), and D(x₄, y₄) respectively. Based on the detected four vertebral corners, identify the midpoint coordinates P (x₁ + x₃/2, y₁ + y₃/2) and Q (x₂ + x₄/2, y₂ + y₄/2) on the left and right sides of each vertebra. The straight line connecting the two represents the inclination of the vertebra and is denoted as $\overrightarrow{PQ}$. Thus, a total of 17 vectors, from T1 to L5, are obtained, The angle θ between any two lines i and j can be calculated using the following formula:

$$\theta ={\text{cos}}^{-1}\frac{\underset{{P}_{j}{Q}_{j}}{\to }\underset{{P}_{k}{Q}_{k}}{\to }}{\left|\overrightarrow{{P}_{j}{Q}_{j}}\right|\left|\overrightarrow{{P}_{k}{Q}_{k}}\right|}$$

Consistent with Sun et al. [11], our model first determines the CA of the major curve in a recursive manner, and then detects compensatory curves based on the type of scoliosis. Specifically, if the major curve is a thoracic curve, the model automatically continues to detect compensatory curves above and below the major curve in a recursive Method and calculates their CAs. Similarly, if the major curve is a thoracolumbar/lumbar curve, the model automatically detects the compensatory curve above the major curve. For the sagittal curve, our model initially locates the vertebral sequence from T1 to L5, and then automatically detects the four corners of each vertebra. Based on this, our model calculates the PTK, MTK, and TSA respectively.

Manual measurement

Two spinal surgeons measured alignment parameters through MicroDicom viewer (MicroDicom Ltd., Sofia, Bulgaria), then diagnosed and classified the severity of all radiographs in the test set, and recorded the time taken. The test set radiographs were divided into a normal group (84 cases, Cobb angle range 0–10°) and a patient group (62 cases, Cobb angle range 10–77.7°). The CA in the normal group was relatively small, even some curves were difficult to distinguish. Therefore, we only measured the CA of the major and minor curves in the patient group, while the sagittal CAs of both groups were measured. A retest was conducted to evaluate the reliability of the intro-observer and inter-observer after a four-week interval. The average of all measurements was used as the gold standard (GS) for this study. We tested the inter-observer consistency of parameter measurement between two experts and found a MAD of 1–4°(mean 2.5°±1.6°)and the agreements were excellent with an ICC of 0.969.

Statistical analysis

The evaluation of the model was based on the test set. For parameters measurement, the reliability of the model was calculated through the ICC and its 95% confidence interval, where ICC < 0.50, 0.50–0.75, 0.75–0.90, and > 0.90 are considered poor, moderate, good, and excellent, respectively [20]. The validity of the model was evaluated through Simple Linear Regression and Bland-Atman plots. The ROC curve and AUC were used to evaluate the diagnostic performance of the model. The accuracy, sensitivity, and specificity of the model classification were calculated using a confusion matrix. Use paired sample t-test to compare the time difference between the model and expert diagnosis.

The above data processing and statistical analysis were conducted using GraphPad Prism (Version 9.5.1, GraphPad; San Diego, United States) software. P < 0.05 indicates statistical significance.

Results

Comparison of model predictive results with the gold standard

To evaluate the accuracy of the model in measuring coronal CA, we calculated the MAD between the model and the GS (Table 2). The MAD(± SD) of the PT, MT, and TL/L were 1.87° (± 1.43°), 2.23° (± 1.80°), and 2.33° (± 1.76°), respectively. Additionally, We tested the consistency of the model in automatically identifying the end vertebrae and found that, compared to the gold standard, the mean errors in identifying the UEV and LEV were approximately 0.57 and 0.53 vertebrae, respectively. The consistency was nearly perfect. For the sagittal alignment parameters, PTK, MTK, and TSA, they were 2.00 $^\circ$(± 2.00°), 3.20° (± 2.05°), and 2.11° (± 1.75°), respectively. Overall, coronal prediction is more accurate than sagittal prediction, with a smaller MAD(2.15 $^\circ \pm 1.69^\circ$). The ICC of all parameters between AI and GS was between 0.985 and 0.927 coronally and sagittally, and Table 2 shows that its reliability performance in coronal TL/L and sagittal MTK is more satisfactory. Overall, the reliability of coronal measurements is higher.

Table 2 Inter-rater reliability of the Cobb angle measurements and end vertebra: Gold standard vs AI

Simple linear regression showed a strong correlation between the model and the GS(r² ≥ 0.686, p < 0.001, Fig. 2). The slopes measured in the coronal and sagittal parameters were 0.992 and 0.907, respectively, both close to the perfect line, indicating that the measured values are completely consistent with the GS. The Bland–Altman plots showed the differences between the model and the GS (Fig. 3), where the 95% limits of agreement(95%LoA) for PT, MT, and TL/L were − 4.484° − 4.774° (Bias = − 0.1450°), − 5.120° − 5.913° (Bias = − 0.3967°), and − 4.538° − 6.312° (Bias = 0.8871°), respectively. The 95% LoA for total CA was − 4.736° − 5.697° (Bias = 0.4803°). The 95% LoA for PTK, MTK, and TSA were − 4.851° − 7.693° (Bias = 1.421°), − 4.610° − 8.385° (Bias = − 0.3967°), and − 4.975° − 5.761° (Bias = 0.0363°), respectively. The 95% LoA for sagittal CA is − 4.947° − 7.395° (Bias = 1.224°).

Fig. 2

Linear regression analysis for the coronal and sagittal parameters. A-D showed the regression results of the coronal parameters, including PT, MT, TL/L and total CAs. E–H showed the regression results of the sagittal parameters, including PTK, MTK, TSA, and total sagittal CAs

Fig. 3

Comparison of automatic and manual measurement. A-D showed the Bland-Altman plot of the coronal parameters, including PT, MT, TL/L, and total CAs. E–H showed the Bland-Altman plot of the sagittal parameters, including PTK, MTK, TSA, and total sagittal CAs. All the Bland-Altman plots showed perfect agreements between the model and GS

Performance of diagnosis and classification of model

The ROC curve demonstrates the performance of AI in diagnosing scoliosis (Fig. 4), AUC is 0.972 (0.940–1.000), with an accuracy of 97.2% (142/146), sensitivity of 96.8% (60/62), and specificity of 97.6% (82/84). Among the 84 images diagnosed as "normal" by experts, 2 were diagnosed as "patients" by the model. The confusion matrix shows the results of the model in severity classification and sagittal alignment evaluation (Fig. 5).

Fig. 4

Model diagnostic performance. The ROC curve demonstrated the excellent diagnostic performance of AI

Fig. 5

Confusion matrices for the severity classification and sagittal abnormality evaluation. A showed the confusion matrices for severity. B-D showed the confusion matrices for PTK, MTK, and TSA, respectively

For severity classification, the overall accuracy is 94.5%. From the classification results of subclasses, the model shows low sensitivity (88.2%) for identifying severe cases, but good specificity (100.0%). For sagittal alignment, the accuracy, sensitivity, and specificity of distinguishing normal or abnormal PTK were 98.6%, 99.3%, and 87.5%, respectively. MTK was 93.8%, 95.8%, 81.5%, TSA was 91.8%, 93.6%, 86.5%.

Automatic measurement time

In this study, the automatic measurement time involved loading image data into software or MicroDicom viewer, identifying the upper and lower end vertebra, drawing the upper and lower endlines, and obtaining measurements of CA at different locations. We only compared the differences in automatic measurement time among the patient groups, as in some images of the healthy group, there was no scoliosis and it was difficult to measure CA. We aimed to observe the automatic measurement efficiency of AI. The average model time was 9.1 s(± 1.4 s), which was significantly lower than the manual time of 185.1 s(± 37.7s), and the t-test showed that the difference was statistically significant (P < 0.001) (Fig. 6).

Fig. 6

Comparison of diagnostic time between clinicians and AI model

Discussion

In this study, we developed a deep learning model based on biplane radiographs. It can automatically measure coronal and sagittal alignment parameters. We tested the validity of the model, including CA measurement values, severity classification, and evaluation of sagittal anomalies. The results indicate that our model has a good generalization ability and robustness, and can accurately detect keypoint in the coronal and sagittal radiographs.

CA is an important parameter for quantifying the severity of scoliosis. Compared with the GS, our model showed that the MAD of total CAs in the coronal plane was 2.15°, and the ICC was 0.985. It also achieved excellent results in sagittal parameters measurement, with a MAD of 2.72° and ICC of 0.927. Recently, methods based on vertebral segmentation and landmark detection have been widely used in scoliosis and have achieved good accuracy. Previous studies reported the mean difference as 2–8° [9, 11, 21,22,23]. Some studies have reported that the error in automatically measuring MTK is about 7° [12], but there is currently no measurement for PTK and TSA. Our automatic measurement errors on PTK, MTK, and TSA were 2.86°, 3.20°, and 2.11°, respectively. We excluded images such as clothing or shoulder joint obstruction, which may be the reason for the improved accuracy. However, the unclear vertebral features remain one of the challenges faced by lateral automated measurement. In addition, the results of simple linear regression indicated a strong correlation between our model and the GS (r² ≥ 0.686, p < 0.001). The Bland-Altman plots show that the mean difference of the measured values in the coronal and sagittal planes is 0.4803° and 1.224°, respectively. The above results indicated that our model has better reliability and validity, which may be related to using ResNet34 as the network backbone. It has an excellent performance in keypoint detection, can accurately identify vertebral corners, and has excellent image generalization ability.

However, the radiograph may be non-standard when the spine rotates or tilts during the image capture process. The non-standard AP or lateral radiograph can indeed lead to errors in measurement of spine parameters [24, 25]. A solution to this problem is that the AI model should assess the quality of spine radiographs before measuring them to eliminate the potential errors. Some parameters should be designed to evaluate the rotation and tilt of the lateral radiograph [26]. Another solution is to correct non-standard radiograph to standard radiograph [27]. However, this correction algorithm needs a powerful calculation engine and geometric theory supports. But there has been no breakthrough in the correction algorithm until now. However, this correction algorithm is promising in the future. The current AI model can only achieve intelligent radiographic measurements and cannot evaluate whether the radiographs are standard or non-standard.

Accurate severity classification of AIS can help clinicians choose appropriate treatment plans [1, 23]. In this study, the model diagnosed established radiographs and classified them into normal, mild, moderate, and severe based on the CA range, with higher accuracy than previous studies [28], reaching up to 98.6%. However, the model showed low sensitivity (88.2%) in severe scoliosis. In such spinal images, severe vertebral rotation, tilt [11], and unclear endplate morphology may affect vertebral corner recognition. Moreover, our model distinguished normal or abnormal sagittal parameters for the sagittal plane to assist clinicians in assessing sagittal alignment.

Most previous studies focused on the major curve CA in the coronal plane [22, 29,30,31] but neglected the evaluation of the alignment parameters of the minor curves and sagittal plane. In clinical practice, Each spine may have multiple curves, and the evaluation of the sagittal plane is equally important. In this study, our model calculated at least two CAs in the coronal plane and three kyphosis angles in the sagittal plane, respectively. More importantly, it also has high accuracy in measuring the minor curve CAs, indicating that our model has certain potential in screening for scoliosis. Identifying multiple curves and determining their structural characteristics is crucial for Lenke classification [14], which contributes surgeons to making a surgery plan. Combining Bending radiographs in the future can be conducive to Lenke classification.

There are some limitations. First, the number of test sets is relatively small. This was primarily due to the limited availability of data during the initial phase of our research. Despite this constraint, we ensured that the selected test sets were representative of the variability observed in the broader population, encompassing it relevant to the study. Previous studies have effectively tested models with smaller datasets [10, 11]. Second, this study was a retrospective and was conducted at a single center. Due to differences in image size and quality, our model may experience poor performance when applied to other medical centers. Therefore, future prospective studies with multiple centers will be considered. Third, all of the radiographs were selected from AIS and normal individuals, and the image recognition performance for patients with congenital scoliosis, Marfan syndrome, and other conditions still needs to be validated. Fourth, our study did not account for the potential impact of flexible or rigid curves on the model's automated measurements. This is because assessing spinal curve flexibility requires both anteroposterior (AP) and bending radiographs, while bending radiographs are not routinely required for AIS measurements. Further research is needed to explore this aspect in the future.

Conclusion

In conclusion, This deep learning model can accurately and automatically measure spinal alignment parameters with reliable results, significantly reducing diagnostic time. The model can also evaluate the severity of AIS and sagittal abnormalities. In the future, it might provide the potential to assist clinicians.