Application of a multimodal model based on radiomics and 3D deep learning in predicting severe acute pancreatitis

Xianglin DING; Xin CHEN; Meiyu CHEN; Yiping SHEN; Yu WANG; Minyue YIN; Kai ZHAO; Jinzhou ZHU

doi:10.12449/JCH251022

Volume 41 Issue 10

Oct. 2025

Turn off MathJax

Article Contents

Article Navigation > Journal of Clinical Hepatology > 2025 > 41(10): 2110-2117

PDF( 1541 KB)

Application of a multimodal model based on radiomics and 3D deep learning in predicting severe acute pancreatitis

DOI: 10.12449/JCH251022

Xianglin DING¹,
Xin CHEN¹,
Meiyu CHEN²,
Yiping SHEN²,
Yu WANG³,
Minyue YIN⁵,
Kai ZHAO^{4
,
,
,},
Jinzhou ZHU^{2
,
,
,}

1.
Department of Gastroenterology，Suzhou Yongding Hospital，Suzhou，Jiangsu 215200，China
2.
Department of Gastroenterology，The First Affiliated Hospital of Soochow University，Suzhou，Jiangsu 215006，China
3.
Department of Hepatobiliary Surgery，Jintan Hospital Affiliated to Jiangsu University， Changzhou， Jiangsu 213200， China
4.
Department of Gastroenterology， Jintan Hospital Affiliated to Jiangsu University， Changzhou， Jiangsu 213200， China
5.
Department of Gastroenterology，Beijing Friendship Hospital，Capital Medical University，National Clinical Research Center for Digestive Diseases，Beijing 100050，China

Research funding:

National Natural Science Foundation of China (82000540);

Suzhou Clinical Center of Digestive Diseases (Szlcyxzx202101);

Education and Science for Health Development Program, Suzhou Health Committee (KJXW2019001);

The Open Fund of Key Laboratory of Hepatosplenic Surgery,Ministry of Education (GPKF202304);

Frontier Technologies of Science and Technology Projects of Changzhou Municipal Health Commission (QY202309);

Changzhou Municipal Health Commission Science and Technology Project (CJ20230002);

Education and Science for Health Development Program, Wujiang Health Committee (WWK202513)

More Information

Corresponding author: ZHAO Kai， zk@jtrmyy.com （ORCID： 0000-0003-0866-6504）; ZHU Jinzhou， jzzhu@zju.edu.cn （ORCID： 0000-0003-0544-9248）
Received Date: 2025-05-27
Accepted Date: 2025-06-10
Published Date: 2025-10-25

Abstract

Abstract

Objective To investigate the application value of a multimodal model integrating radiomics features， deep learning features， and clinical structured data in predicting severe acute pancreatitis （SAP）， and to provide more accurate tools for the early identification of SAP in clinical practice. Methods The patients with acute pancreatitis （AP） who attended The First Affiliated Hospital of Soochow University， Jintan Hospital Affiliated to Jiangsu University， and Suzhou Yongding Hospital from January 1， 2017 to December 31， 2023 were included. Related data were collected， including demographic information， previous medical history， etiology， laboratory test data， and systemic inflammatory response syndrome （SIRS） within 24 hours after admission， as well as imaging data within 72 hours after admission， while related scores were calculated， including Ranson score， modified CT severity index （MCTSI）， bedside index for severity in acute pancreatitis （BISAP）， and systemic inflammatory response syndrome， albumin， blood urea nitrogen and pleural effusion （SABP） score. The model was constructed in the following process：（1） three-dimensional CT images were used to extract and identify radiomics features， and a radiomics classification model was established based on the extreme gradient Boost （XGBoost） algorithm；（2） U-Net is used to perform semantic segmentation of three-dimensional CT images， and then the results of segmentation were imported into 3D ResNet50 to construct a deep learning classification model；（3） the predicted values of the above two models were integrated with clinical structured data to establish a multimodal model based on the XGBoost algorithm. The variable importance plot and local interpretability plot were used to perform visual interpretation of the model. The independent samples t-test was used for comparison of normally distributed continuous data between groups， and the Mann-Whitney U test was used for comparison of non-normally distributed continuous data between groups； the chi-square test or Fisher’s exact test was used for comparison of categorical data between groups. The receiver operating characteristic （ROC） curve was plotted for each model and existing scoring systems， and the area under the ROC curve （AUC） was calculated to assess their performance； the Delong test was used for comparison of AUC. Results A total of 609 patients who met the criteria were included， among whom 114 （18.7%） developed SAP. In this study， the data of 426 patients from The First Affiliated Hospital of Soochow University was used as the training set， and the data of 183 patients from Jintan Hospital Affiliated to Jiangsu University and Suzhou Yongding Hospital were used as the independent test set. The multimodal model had an AUC of 0.914 in the test set， which was significantly higher than the AUC of traditional scoring systems such as MCTSI （AUC=0.827）， Ranson score （AUC=0.675）， BISAP （AUC=0.791）， and SABP score （AUC=0.648）； in addition， the multimodal model showed a significant improvement in performance compared with the radiomics classification model （AUC=0.739） and the deep learning classification model （AUC=0.685）（the Delong test： Z=-3.23， -4.83， -3.48， -4.92， -4.31， and -4.59， all P <0.01）. The top 10 variables in terms of importance in the multimodal model were pleural effusion， predicted value of the deep learning model， predicted value of the radiomics model， triglycerides， calcium ions， SIRS， white blood cell count， age， platelets， and C-reactive protein， suggesting that the above variables had significant contributions to the performance of the model in predicting SAP. Conclusion Based on structured data， radiomic features， and deep learning features， this study constructs a multicenter prediction model for SAP based on the XGBoost algorithm， which has a better predictive performance than existing traditional scoring systems and unimodal models.
- Pancreatitis， Acute Necrotizing,
- XGBoost Algorithm,
- Imaging Genomics,
- Multimodal Imaging,
- Deep Learning

FullText(HTML)

References(22)

References

[1]	PETROV MS, YADAV D. Global epidemiology and holistic prevention of pancreatitis[J]. Nat Rev Gastroenterol Hepatol, 2019, 16( 3): 175- 184. DOI: 10.1038/s41575-018-0087-5.
[2]	TRIKUDANATHAN G, YAZICI C, EVANS PHILLIPS A, et al. Diagnosis and management of acute pancreatitis[J]. Gastroenterology, 2024, 167( 4): 673- 688. DOI: 10.1053/j.gastro.2024.02.052.
[3]	MEDEROS MA, REBER HA, GIRGIS MD. Acute pancreatitis[J]. Jama, 2021, 325( 4): 382. DOI: 10.1001/jama.2020.20317.
[4]	YIN MY, ZHU JD, LIU L, et al. Research advances in machine learning models for acute pancreatitis[J]. J Clin Hepatol, 2023, 39( 12): 2978- 2984. DOI: 10.3969/j.issn.1001-5256.2023.12.034. 殷民月, 朱锦舟, 刘璐, 等. 急性胰腺炎机器学习模型的研究进展[J]. 临床肝胆病杂志, 2023, 39( 12): 2978- 2984. DOI: 10.3969/j.issn.1001-5256.2023.12.034.
[5]	XIE F, ZHANG KQ, LI F, et al. Diagnostic accuracy of convolutional neural network-based endoscopic image analysis in diagnosing gastric cancer and predicting its invasion depth: A systematic review and meta-analysis[J]. Gastrointest Endosc, 2022, 95( 4): 599- 609. DOI: 10.1016/j.gie.2021.12.021.
[6]	HAGGENMÜLLER S, MARON RC, HEKLER A, et al. Skin cancer classification via convolutional neural networks: Systematic review of studies involving human experts[J]. Eur J Cancer, 2021, 156: 202- 216. DOI: 10.1016/j.ejca.2021.06.049.
[7]	LI GW, LIU J, CAO H, et al. Research review of deep learning in colon polyp image segmentation[J]. J Front Comput Sci Technoly, 2025, 19( 5): 1198- 1216. DOI: 10.3778/j.issn.1673-9418.2408012. 李国威, 刘静, 曹慧, 等. 深度学习在结肠息肉图像分割中的研究综述[J]. 计算机科学与探索, 2025, 19( 5): 1198- 1216. DOI: 10.3778/j.issn.1673-9418.2408012.
[8]	PACAL I, KARABOGA D. A robust real-time deep learning based automatic polyp detection system[J]. Comput Biol Med, 2021, 134: 104519. DOI: 10.1016/j.compbiomed.2021.104519.
[9]	YU T, LIN N, ZHANG X, et al. An end-to-end tracking method for polyp detectors in colonoscopy videos[J]. Artif Intell Med, 2022, 131: 102363. DOI: 10.1016/j.artmed.2022.102363.
[10]	LIU H, ZHUANG YZ, SONG EM, et al. A 3D boundary-guided hybrid network with convolutions and Transformers for lung tumor segmentation in CT images[J]. Comput Biol Med, 2024, 180: 109009. DOI: 10.1016/j.compbiomed.2024.109009.
[11]	BANKS PA, BOLLEN TL, DERVENIS C, et al. Classification of acute pancreatitis: 2012: Revision of the Atlanta classification and definitions by international consensus[J]. Gut, 2013, 62( 1): 102- 111. DOI: 10.1136/gutjnl-2012-302779.
[12]	KLINE A, WANG HY, LI YK, et al. Multimodal machine learning in precision health: A scoping review[J]. NPJ Digit Med, 2022, 5( 1): 171. DOI: 10.1038/s41746-022-00712-8.
[13]	LIPKOVA J, CHEN RJ, CHEN BW, et al. Artificial intelligence for multimodal data integration in oncology[J]. Cancer Cell, 2022, 40( 10): 1095- 1110. DOI: 10.1016/j.ccell.2022.09.012.
[14]	MOHSEN F, ALI H, HAJJ N EL, et al. Artificial intelligence-based methods for fusion of electronic health records and imaging data[J]. Sci Rep, 2022, 12( 1): 17981. DOI: 10.1038/s41598-022-22514-4.
[15]	SHAO J, MA JC, ZHANG Q, et al. Predicting gene mutation status via artificial intelligence technologies based on multimodal integration(MMI) to advance precision oncology[J]. Semin Cancer Biol, 2023, 91: 1- 15. DOI: 10.1016/j.semcancer.2023.02.006.
[16]	HUANG SC, PAREEK A, SEYYEDI S, et al. Fusion of medical imaging and electronic health records using deep learning: A systematic review and implementation guidelines[J]. NPJ Digit Med, 2020, 3: 136. DOI: 10.1038/s41746-020-00341-z.
[17]	GLIEM N, AMMER-HERRMENAU C, ELLENRIEDER V, et al. Management of severe acute pancreatitis: An update[J]. Digestion, 2021, 102( 4): 503- 507. DOI: 10.1159/000506830.
[18]	GAO X, LIN JX, WU AR, et al. Application of machine learning model based on XGBoost algorithm in early prediction of patients with acute severe pancreatitis[J]. Chin Crit Care Med, 2023, 35( 4): 421- 426. DOI: 10.3760/cma.j.cn121430-20221019-00930. 高欣, 林嘉希, 吴爱荣, 等. 基于XGBoost算法的机器学习模型在早期预测重症急性胰腺炎中的应用[J]. 中华危重病急救医学, 2023, 35( 4): 421- 426. DOI: 10.3760/cma.j.cn121430-20221019-00930.
[19]	CHEN ZY, WANG Y, ZHANG HL, et al. Deep learning models for severity prediction of acute pancreatitis in the early phase from abdominal nonenhanced computed tomography images[J]. Pancreas, 2023, 52( 1): e45- e53. DOI: 10.1097/MPA.0000000000002216.
[20]	YIN MY, LIN JX, WANG Y, et al. Development and validation of a multimodal model in predicting severe acute pancreatitis based on radiomics and deep learning[J]. Int J Med Inform, 2024, 184: 105341. DOI: 10.1016/j.ijmedinf.2024.105341.
[21]	KAMNITSAS K, LEDIG C, NEWCOMBE VFJ, et al. Efficient multi-scale 3D CNN with fully connected CRF for accurate brain lesion segmentation[J]. Med Image Anal, 2017, 36: 61- 78. DOI: 10.1016/j.media.2016.10.004.
[22]	LITJENS G, KOOI T, BEJNORDI BE, et al. A survey on deep learning in medical image analysis[J]. Med Image Anal, 2017, 42: 60- 88. DOI: 10.1016/j.media.2017.07.005.

Relative Articles

Supplements(0)

Cited By

Proportional views

Proportional views

通讯作者: 陈斌, bchen63@163.com

1.
沈阳化工大学材料科学与工程学院沈阳 110142

Figures(3) / Tables(2)

Get Citation

PDF

XML

Article Metrics

Article views (56) PDF downloads(9)

Application of a multimodal model based on radiomics and 3D deep learning in predicting severe acute pancreatitis

DOI: 10.12449/JCH251022

Abstract

References

Proportional views

Catalog

通讯作者: 陈斌, bchen63@163.com

Article Metrics

Proportional views

Related

About Journal

Submission Guideline

Journal Online

Hepatobiliary College

Guidelines

Export File

Citation

Format

Content