In a recent study published in the journal Radiology, researchers found that the use of large language models (LLMs) like GPT-4 and Google Gemini in breast imaging classification could potentially have negative implications for patient management.

Large language models, a type of artificial intelligence, have been increasingly utilized in various fields including radiology. While these models have shown promise in tasks such as processing radiology request forms and providing imaging recommendations, they may fall short in more complex tasks requiring higher levels of medical reasoning.

The study, conducted by researchers from the Imaging Institute of Southern Switzerland, Memorial Sloan Kettering Cancer Center in New York City, and the Netherlands Cancer Institute in Amsterdam, focused on the agreement between human readers and LLMs in assigning Breast Imaging Reporting and Data System (BI-RADS) categories.

Using 2,400 breast imaging reports in English, Italian, and Dutch, the researchers compared the performance of LLMs like GPT-4 and Google Gemini with that of board-certified breast radiologists. While the agreement among human readers was nearly perfect, the agreement between LLMs and human readers was not as strong.

Lead author of the study, Dr. Andrea Cozzi, emphasized the importance of evaluating the capabilities of generic LLMs, especially in scenarios where medical reasoning plays a critical role. Dr. Cozzi highlighted that patients and non-radiologist physicians may not always be aware of the limitations of these AI tools and could potentially misuse them for medical advice.

The findings of this study underscore the need for better regulation of LLMs in medical settings where accurate and precise classification of imaging reports is crucial for patient care. As the use of AI in radiology continues to evolve, ensuring the proper training and understanding of these models becomes paramount for improving patient outcomes.