多模态法医病理学智能化鉴定的应用展望

doi:10.3969/j.issn.1671-2072.2024.06.004

摘要/Abstract

摘要： 传统法医病理学鉴定主要依赖于现场勘查、案情分析、尸体解剖和组织病理学检验等方法，这一过程高度依赖鉴定人的个人经验和主观判断。历经数百年的鉴定实践，该模式一直是法医病理学领域的基石。然而，随着人工智能（artifical intelligence，AI）技术的突飞猛进，特别是生成式AI技术的兴起，AI在法医病理学中的应用已经实现了从单一模态数据分析到多模态数据综合处理的转变条件。深入探讨了多模态AI技术及其数据融合策略在法医病理学鉴定中的应用展望，尤其是潜在应用场景和发展前景。同时，还审视了多模态AI技术在法医病理鉴定实践中可能遇到的挑战和伦理问题，并对如何克服这些“瓶颈”问题提出了见解。

关键词: 法医病理学, 人工智能, 多模态, 数据融合

Abstract: Traditional forensic pathological identification has primarily relied on methods such as on-site inspections, case analysis, autopsies, and histopathological examinations, all of which highly depend on the personal experience and subjective judgement of the examiners. After centuries of practice, this mode of identification has been the cornerstone of the forensic pathology. However, with the rapid advancement of artificial intelligence （AI） technologies, particularly the emergence of generative AI technologies, the application of AI in forensic pathology has evolved from analyzing single-modal data to processing multimodal data. This paper aims to delve into multimodal AI technologies and their data integration strategies in forensic pathological identification, especially the potential application scenarios and developmental prospects in forensic pathological identification. Additionally, this paper discusses the challenges and ethical issues that might arise with the implementation of multimodal AI technologies in forensic pathological identification practice and offers insights on how to overcome these obstacles.

Key words: forensic pathology, artificial intelligence （AI）, multimodal, data fusion

中图分类号:

DF793.1
TP18

黄平. 多模态法医病理学智能化鉴定的应用展望[J]. 中国司法鉴定, 2024(6): 36-43.

HUANG Ping. The Prospects of Multimodal Artificial Intelligence Identification in Forensic Pathology[J]. Chinese Journal of Forensic Sciences, 2024(6): 36-43.

参考文献

[ 1 ] GIUSEPPE L R，ANTONINA A，MASSIMO M，et al. Radiology in forensic medicine[M]. Berlin，Germany：Springer，2019：3-14.
[ 2 ] CASTIGLIONE V，MODENA M，AIMO A，et al. Molecular autopsy of sudden cardiac death in the genomics era[J]. Diagnostics（Basel），2021，11（8）：1378.
[ 3 ] DAWIDOWSKA J，KRZYŻANOWSKA M，MARKU-SZEWSKI M J，et al. The application of metabolomics in forensic science with focus on forensic toxicology and time-of-death estimation[J]. Metabolites，2021，11（12）：801.
[ 4 ] DUONG V A，PARK J M，LIM H J，et al. Proteomics in forensic analysis：Applications for human samples[J]. Applied Sciences，2021，11（8）：3393.
[ 5 ] HAAS C，NEUBAUER J，SALZMANN A P，et al. Forensic transcriptome analysis using massively parallel sequencing[J]. Forensic Science International：Genetics，2021，52：102486.
[ 6 ] ROS A C，BACCI S，LUNA A，et al. Forensic impact of the omics science involved in the wound：A systematic review[J]. Frontiers in Medicine，2022，6（8）：786798.
[ 7 ] SZEREMETA M，PIETROWSKA K，NIEMCUNOWI-CZ-JANICA A，et al. Applications of metabolomics in forensic toxicology and forensic medicine [J]. International Journal of Molecular Sciences，2021，22（6）：3010.
[ 8 ] ACOSTA J N，FALCONE G J，RAJPURKAR P，et al. Multimodal biomedical AI[J].Nature Medicine，2022，28（9）：1773-1784.
[ 9 ] GAO J，LI P，CHEN Z K，et al. A survey on deep learning for multimodal data fusion[J]. Neural Computation，2020，32（5）：829-864.
[10] BALTRUSAITIS T，AHUJA C，MORENCY L P. Multimodal machine learning：A survey and taxonomy[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2019，41（2）：423-443.
[11] SHARMA K，GIANNAKOS M. Multimodal data capabilities for learning：What can multimodal data tell us about learning?[J]. British Journal of Educational Technology，2020，51（5）：1450-1484.
[12] BORNIK A, URSCHLER M, SCHMALSTIEG D, et al. Integrated computer-aided forensic case analysis, presentation, and documentation based on multimodal 3D data[J]. Forensic Science International, 2018, 287:12-24.
[13] STAHLSCHMIDT S R，ULFENBORG B，SYNNERG-REN J. Multimodal deep learning for biomedical data fusion：A review[J]. Briefings in Bioinformatics，2022，23（2）：bbab569.
[14] 黄平，张吉，邹冬华，等. 数字化与智能化技术在法医病理学鉴定中的研究现状与挑战[J]. 中国司法鉴定，2023（2）：35-47.
[15] 赵亮，马文静，赵旭舒，等. 人工智能技术在法医学中的应用研究进展[J]. 刑事技术，2024，49（2）：185-189.
[16] 刘蓬然，霍彤彤，陆林，等. 人工智能在医学中的应用现状与展望[J]. 中华医学杂志，2021，101（44）：3677-3683.
[17] GORE J C. Artificial intelligence in medical imaging[J]. Magnetic Resonance Imaging，2020，68：A1-A4.
[18] JIANG T，GRADUS J L，ROSELLINI A J. Supervised machine learning：A brief primer[J]. Behavior Therapy，2020，51（5）：675-687.
[19] SEN P C，HAJRA M，GHOSH M. Supervised classification algorithms in machine learning：A survey and review[C]//Proceedings of IEM Graph 2018. Berlin，Germany：Springer，2020：99-111.
[20] THURZO A，KOSNÁČOVÁ H S，KURILOVÁ V，et al. Use of advanced artificial intelligence in forensic medicine，forensic anthropology and clinical anatomy[J].Healthcare，2021，9（11）：1545.
[21] GARLAND J，ONDRUSCHKA B，STABLES S，et al. Identifying fatal head injuries on postmortem computed tomography using convolutional neural network/deep learning：A feasibility study[J]. Journal of Forensic Sciences，2020，65（6）：2019-2022.
[22] FANNI S C，FEBI M，AGHAKHANYAN G，et al. Natural language processing[C]//Proceedings of the Introduction to Artificial Intelligence. Berlin，Germany：Springer，2023：87-99.
[23] PAOLANTI M，FRONTONI E. Multidisciplinary pattern recognition applications：A review[J]. Computer Science Review，2020，37：100276.
[24] XU P，ZHU X，CLIFTON D A. Multimodal learning with transformers：A survey[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence，2023，45（10）：12113-12132.
[25] RAMESH A，PAVLOV M，GOH G，et al. Zero-shot text-to-image generation[EB/OL].[2024-05-02].https：//arxiv.org/pdf/2102.12092.
[26] NICHOL A，DHARIWAL P，RAMESH A，et al. GLIDE：Towards photorealistic image generation and editing with text-guided diffusion models[J]. International Conference on Machine Learning，2021，2112：10741.
[27] WANG J M，LI Z D，YING F，et al. Reconstruction of a real-world car-to-pedestrian collision using geomatics techniques and numerical simulations[J]. Journal of Forensic and Legal Medicine，2022，91：102433.
[28] BAYLY P V，ALSHAREEF A，KNUTSEN A K，et al. MR imaging of human brain mechanics in vivo：New measurements to facilitate the development of computational models of brain injury[J]. Annals of Biomedical Engineering，2021，49（10）：2677-2692.
[29] OURA P，JUNNO A，JUNNO J A. Deep learning in forensic gunshot wound interpretation—a proof-of-concept study[J]. International Journal of Legal Medicine，2021，135（5）：2101-2106.
[30] ZIMMERMANN N，SIEBERTH T，DOBAY A. Automated wound segmentation and classification of seven common injuries in forensic medicine[J]. Forensic Science，Medicine and Pathology，2024，20（2）：443-451.
[31] CHEN Y J. Current state and progress of research on forensic biomechanics in China[J]. Forensic Sciences Research，2021，6（1）：1-12.
[32] ALBEEDAN M，KOLIVAND H，HAMMADY R，et al. Seamless crime scene reconstruction in mixed reality for investigation training：A design and evaluation study[J]. IEEE Transactions on Learning Technologies，2024，17：856-873.
[33] VILLA C，LYNNERUP N，JACOBSEN C. A virtual，3D multimodal approach to victim and crime scene reconstruction[J]. Diagnostics，2023，13（17）：2764.
[34] LIU R N，GU Y X，SHEN M W，et al. Predicting postmortem interval based on microbial community sequences and machine learning algorithms[J]. Environmental Microbiology，2020，22（6）：2273-2291.
[35] MADEA B. Estimation of the time since death[M]. 4th ed. Boca Raton，USA：CRC Press，2023：187-281.
[36] ZISSLER A，STOIBER W，GEISSENBERGER J，et al. Influencing factors on postmortem protein degradation for PMI estimation：A systematic review[J]. Diagnostics，2021，11（7）：1146.
[37] GUEBELIN D L C，DOBAY A，EBERT L，et al. Correlation of age，sex and season with the state of human decomposition as quantified by postmortem computed tomography[J]. Forensic Science，Medicine and Pathology，2021，17（2）：185-191.
[38] BUONASERA T，EERKENS J，DE FLAMINGH A，et al. A comparison of proteomic，genomic，and osteological methods of archaeological sex estimation[J]. Scientific Reports，2020，10（1）：11897.
[39] LI Y，WANG J，LIANG W B，et al. CR-GAN：Automatic craniofacial reconstruction for personal identification[J]. Pattern Recognition，2022，124（1/2/3）：108400.
[40] REMYA S，DWAZ C G，PILLAI P G，et al. A novel framework for forensic bone age estimation using CNN architecture and multimodal fusion[C]//Proceedings of the 7th International Conference on Electronics，Materials Engineering & Nano-Technology （IEMENTech）. Washington D.C.，USA：IEEE Press，2023：1-6.
[41] TESI C，RICCI S，CREZZINI J，et al. Wounded to death. Holistic，multimodal reconstruction of the dynamics in a case of multiple perimortem cranial injuries from a medieval site in northern Italy[J]. Journal of Archaeological Science，2022，46：103643.
[42] BYAHATTI P，SHETTAR M S. Fusion strategies for multimodal biometric system using face and voice cues[C]// Proceedings of Computational Engineering and Material Science （ICCEMS-2020）.[s.l.]：IOP Publishing，2020：012031.
[43] BALA N，GUPTA R，KUMAR A. Multimodal biometric system based on fusion techniques：A review[J]. Information Security Journal：A Global Perspective，2022，31（3）：289-337.
[44] TALREJA V，VALENTI M C，NASRABADI N M. Deep hashing for secure multimodal biometrics[J]. IEEE Transactions on Information Forensics and Security，2020，16：1306-1321.
[45] RAJASEKAR V，SARACEVIC M，HASSABALLAH M，et al. Efficient multimodal biometric recognition for secure authentication based on deep learning approach[J]. International Journal on Artificial Intelligence Tools，2023，32（3）：2340017.
[46] SIDDIQUI M F H，JAVAID A Y. A multimodal facial emotion recognition framework through the fusion of speech with visible and infrared images[J]. Multimodal Technologies and Interaction，2020，4（3）：46.
[47] DERAKHSHAN A，MIKAEILI M，GEDEON T，et al. Identifying the optimal features in multimodal deception detection[J]. Multimodal Technologies and Interaction，2020，4（2）：1-14.
[48] BONCI A，FIORI S，HIGASHI H，et al. An introductory tutorial on brain-computer interfaces and their applications[J]. Electronics，2021，10（5）：560.
[49] BERISHA V，KRANTSEVICH C，HAHN P R，et al. Digital medicine and the curse of dimensionality[J]. NPJ Digital Medicine，2021，4（1）：1-8.
[50] LIANG P P，ZADEH A，MORENCY L P. Foundations and trends in multimodal machine learning：Principles，challenges，and open questions[J]. ACM Computing Surveys，2024，56（10）：1-42.
[51] ESTEVA M，XU W，SIMONE N，et al. Modeling data curation to scientific inquiry：A case study for multimodal data integration[C]//Proceedings of the ACM/IEEE Joint Conference on Digital Libraries in 2020. New York，USA：Association for Computing Machinery，2020：235-242.
[52] ONDRUSCHKA B，SEIFERT D，ROTERMUND N，et al. A critical review of “artificial intelligence in the practice of forensic medicine：A scoping review”[J]. International Journal of Legal Medicine，2024，138（4）：1667-1668.

[1]	王亚辉, 刘宁国, 王飞翔, 张吉, 董贺文, 张慧, 于笑天, 夏文涛. 通过研究和创新实现科学与正义——第26届国际法医学大会综述[J]. 中国司法鉴定, 2024, 0(6): 44-49.
[2]	张栋, 陈修勇. 人工智能参与生成鉴定意见的程序法机理及方案[J]. 中国司法鉴定, 2024, 0(5): 1-10.
[3]	张颖. 人工智能浪潮下司法鉴定的机遇与挑战[J]. 中国司法鉴定, 2024, 0(5): 11-17.
[4]	冯浩, 吴何坚, 陈丰, 刘嘉鲯. 多模态信息融合在智能网联汽车交通事故鉴定中的应用探索[J]. 中国司法鉴定, 2024, 0(5): 18-26.
[5]	韦铭菲, 吕晓瑜, 马秀伟, 史俊展, 秦豪杰, 李凡. 血痕时序性研究进展[J]. 中国司法鉴定, 2024, 0(5): 50-58.
[6]	贾宇晴, 姚承谕, 毛瑞明, 王天琦, 张孟周, 刘技辉, 朱宝利, 曹志鹏. 创伤性肺脂肪栓塞的法医病理学研究进展及其鉴定应用[J]. 中国司法鉴定, 2024, 0(4): 49-53.
[7]	张立为, 王甜, 余代静, 张君, 严江伟. 基于机器学习的法医微生物研究现状[J]. 中国司法鉴定, 2024, 0(3): 32-40.
[8]	李娜, 张书芮, 孙俊红. 人工智能技术驱动的法医病理时间类问题解决策略[J]. 中国司法鉴定, 2024, 0(3): 41-48.
[9]	马文静, 刘凡, 赵亮, 刘华, 裴京哲, 张睿, 施维. 股骨颈骨折影像人工智能快速诊断方法研究[J]. 中国司法鉴定, 2024, 0(3): 49-54.
[10]	孙大明. 英国死因裁判制度的功能、特色与镜鉴[J]. 中国司法鉴定, 2024, 0(2): 25-32.
[11]	李立亮. 精神障碍者不明原因猝死现状及鉴识进展[J]. 中国司法鉴定, 2024, 0(2): 47-54.
[12]	周慧明, 李丹阳, 万雷, 李成涛, 汪茂文, 王亚辉. 人工智能技术在法医学骨龄和牙龄评估中的研究前沿及展望[J]. 中国司法鉴定, 2023, 0(5): 31-38.
[13]	张平, 徐辉, 高觉, 蒋士浩, 张志湘, 施晓玲. 心源性猝死医疗纠纷鉴定78例分析[J]. 中国司法鉴定, 2023, 0(5): 57-63.
[14]	黄平, 张吉, 邹冬华, 刘宁国, 李正东, 王金明, 秦志强, 张建华. 数字化与智能化技术在法医病理学鉴定中的研究现状与挑战[J]. 中国司法鉴定, 2023, 0(2): 35-47.
[15]	叶伟权, 吴赵斌, 蔡留新, 杨博帆, 乔智. 182例广州市南沙区交通事故死亡案例分析[J]. 中国司法鉴定, 2022, 0(6): 61-66.