Foundation Models for Imaging: Are We Entering the Era of ‘Generalist’ Radiology AI?

Introduction

Medical imaging has long relied on the ability to identify subtle visual patterns that may not be immediately apparent to the human eye. For decades, radiologists have served as highly trained experts responsible for interpreting these patterns across a wide range of imaging modalities, including radiography, computed tomography, magnetic resonance imaging, ultrasound, and nuclear medicine studies. Their role requires not only technical expertise in image interpretation but also the integration of clinical context, disease progression, and diagnostic probability into complex decision making processes. As imaging volumes continue to increase globally, artificial intelligence has emerged as a promising tool to support radiologists in improving diagnostic accuracy, efficiency, and workflow optimization.

The earliest applications of artificial intelligence in radiology were primarily task specific systems developed to perform narrowly defined functions. These conventional models were trained using supervised learning approaches on highly curated datasets designed for a single clinical objective. One algorithm might be optimized to detect pulmonary nodules on chest imaging, another to quantify cardiac ejection fraction, and another to identify fractures on musculoskeletal radiographs. While these systems demonstrated impressive performance within their intended domains, they remained limited by their lack of adaptability. Each application required separate development pipelines, independent validation processes, dedicated maintenance, and retraining whenever imaging protocols or clinical environments changed.

This fragmented approach presented practical challenges for large scale implementation in clinical radiology. Modern imaging departments manage diverse imaging modalities and address a broad spectrum of clinical questions throughout the day. A radiologist may interpret a chest radiograph for pneumonia in the morning, review a brain magnetic resonance scan for acute stroke shortly afterward, and later assess an abdominal computed tomography examination for malignancy staging. Each task demands different forms of anatomical knowledge, pathological recognition, and contextual reasoning. Traditional single purpose artificial intelligence systems are not inherently designed to accommodate this level of variability and complexity.

In response to these limitations, a new generation of artificial intelligence systems known as foundation models has begun to reshape the field of medical imaging. Foundation models differ fundamentally from earlier task specific systems because they are trained on extremely large and diverse datasets capable of supporting multiple downstream applications. Rather than being designed for a single diagnostic purpose, these models learn generalized representations of imaging data that can later be adapted to a wide range of clinical tasks with comparatively minimal additional training.

The concept underlying foundation models can be understood as a transition from isolated specialized tools toward a unified and flexible intelligence framework. Instead of maintaining numerous independent algorithms for separate clinical applications, researchers are developing adaptable systems capable of performing image classification, segmentation, disease detection, report generation, workflow prioritization, and multimodal data integration within a single architecture. This versatility more closely resembles the broad interpretive capabilities of human radiologists, who continuously apply transferable knowledge across different imaging contexts and disease processes.

Recent advances in deep learning architectures, self supervised learning, transformer models, and multimodal artificial intelligence have accelerated the development of these systems. Self supervised learning, in particular, allows models to learn meaningful imaging features from large volumes of unlabeled medical data, reducing dependence on labor intensive annotation processes. Transformer based architectures, originally developed for natural language processing, are increasingly being adapted for imaging analysis due to their ability to capture long range spatial relationships and contextual information within complex datasets.

Foundation models also offer major advantages in scalability and generalizability. Because they are trained on broad and heterogeneous datasets, they may perform more robustly across different patient populations, imaging devices, and healthcare institutions compared with narrowly trained algorithms. This has important implications for reducing performance variability and improving clinical reliability, especially in geographically diverse healthcare settings.

The potential clinical applications of foundation models extend far beyond automated image interpretation. These systems may assist with triaging urgent findings, reducing reporting turnaround times, enhancing image reconstruction quality, predicting disease progression, and integrating imaging findings with electronic health records, pathology data, and genomic information. In oncology, for example, foundation models could support precision medicine approaches by correlating radiographic features with molecular tumor characteristics. In emergency medicine, they may help prioritize critical findings such as intracranial hemorrhage or pulmonary embolism for immediate review.

Importantly, the emergence of foundation models may also help address global disparities in radiology access. Many regions worldwide continue to experience significant shortages of trained radiologists, leading to delays in diagnosis and limited imaging availability. Artificial intelligence systems capable of supporting multiple imaging tasks could improve healthcare delivery in resource constrained settings by augmenting local diagnostic capacity and facilitating tele-radiology workflows.

Despite their promise, foundation models introduce important scientific, clinical, and ethical challenges. Large scale training requires substantial computational infrastructure and access to diverse, high quality datasets, raising concerns regarding data privacy, security, and institutional ownership. Model interpretability remains another critical issue, as clinicians must understand how artificial intelligence systems generate recommendations in order to trust and safely integrate them into patient care. Bias within training datasets also presents a significant concern, as underrepresentation of certain populations may contribute to unequal diagnostic performance across demographic groups.

Regulatory and validation frameworks will therefore play a crucial role in determining how these technologies are implemented in clinical practice. Robust prospective studies, external validation across multiple healthcare systems, and continuous post deployment monitoring will be essential to ensure patient safety and maintain clinical effectiveness. Furthermore, integration into radiology workflows must be carefully designed to complement rather than replace physician expertise.

Ultimately, foundation models represent a remarkable evolution in the application of artificial intelligence to medical imaging. By moving beyond isolated task specific algorithms toward adaptable, multimodal systems, these technologies have the potential to transform radiology into a more efficient, scalable, and data integrated discipline. As research and clinical implementation continue to advance, foundation models may redefine how imaging is interpreted, how radiologists interact with technology, and how diagnostic care is delivered across the global healthcare landscape.

Understanding Foundation Models

Foundation models are AI systems trained on massive amounts of data to learn general patterns that can be applied to many different tasks. The concept started with language models like GPT, which learned to understand and generate text by studying billions of words from books, websites, and other sources. These models became so good at understanding language patterns that they could write essays, answer questions, translate between languages, and even write computer code.

The same idea is now being applied to medical images. Instead of training an AI system to look only at chest X-rays for pneumonia, researchers are training models on thousands of different types of medical images. These models learn general principles about how medical images work, what normal anatomy looks like, and how different diseases appear across various imaging methods.

The training process involves showing the AI system millions of images along with information about what those images show. The system learns to recognize patterns, shapes, textures, and relationships that appear across different types of scans. Once this training is complete, the model can be adapted to new tasks without starting from zero.

This approach offers several advantages over traditional AI systems. First, it reduces the need to collect large datasets for each new application. Second, it allows knowledge learned from one type of imaging to help with another type. Third, it makes it easier to deploy AI systems in hospitals because one model can handle multiple tasks instead of requiring many separate systems.

Current State of Foundation Models in Medical Imaging

Several research groups and companies are working on foundation models for medical imaging. Each approach has its own strengths and focuses on different aspects of the challenge.

One major effort involves adapting existing vision foundation models to work with medical images. These models were originally trained on regular photographs and then modified to understand medical scans. While this approach can work, medical images are quite different from everyday photographs, so the results are sometimes limited.

Another approach involves training foundation models specifically on medical images from the start. These models learn directly from radiological scans, pathology images, and other medical data. This method often produces better results for medical tasks, but it requires access to large amounts of medical data, which can be difficult to obtain.

Some foundation models focus on specific aspects of medical imaging. For example, some models specialize in understanding anatomy and can identify organs and structures across different types of scans. Others focus on detecting abnormalities and can spot potential problems in various imaging studies.

The performance of these models varies depending on the task and the type of images involved. In some cases, foundation models perform as well as or better than specialized AI systems. In other cases, they still lag behind more focused approaches. The results often depend on how much training data was available for specific tasks and how well the model was adapted for particular applications.

Recent studies have shown promising results in several areas. Foundation models have demonstrated good performance in image classification tasks, where they need to determine whether an image shows a particular condition. They have also shown ability in image segmentation, where they need to outline specific structures or abnormalities in scans.

Model Type	Primary Focus	Advantages	Limitations
Adapted Vision Models	General imaging tasks	Leverage existing knowledge	May miss medical-specific patterns
Medical-Specific Models	Healthcare applications	Better medical understanding	Require large medical datasets
Anatomy-Focused Models	Structure identification	Good at finding organs	Limited disease detection
Disease-Detection Models	Abnormality identification	Strong diagnostic capability	May miss subtle findings

Applications and Use Cases

Foundation models for medical imaging could be used in many different ways throughout healthcare systems. The most obvious application is in basic image interpretation, where these models could help radiologists identify normal and abnormal findings across different types of scans.

Emergency departments could benefit greatly from foundation models. When patients arrive with urgent symptoms, doctors often need quick answers from imaging studies. A foundation model could provide rapid preliminary reads of X-rays, CT scans, and other urgent studies, helping doctors make faster treatment decisions. The model could flag potential emergencies like pneumothorax, intracranial bleeding, or pulmonary embolism across different imaging modalities.

In routine screening programs, foundation models could help process large volumes of images more efficiently. Mammography screening, lung cancer screening with low-dose CT, and colonoscopy screening all involve reviewing many images to find relatively few abnormalities. A foundation model could help prioritize cases that need immediate attention and provide quality assurance for normal studies.

Teaching hospitals could use foundation models as educational tools. Medical students and radiology residents could practice with these systems, getting feedback on their interpretations and learning to recognize patterns across different imaging types. The models could also help standardize education by providing consistent examples of various conditions.

Smaller hospitals and clinics often struggle with access to specialized radiology expertise. Foundation models could help bridge this gap by providing preliminary interpretations for complex cases, helping local physicians decide when patients need transfer to larger medical centers, and ensuring that important findings are not missed in under-resourced settings.

Research applications represent another important use case. Foundation models could help researchers analyze large datasets more efficiently, identify patient populations for clinical studies, and discover new imaging patterns associated with different diseases. They could also help standardize image analysis across different research sites and institutions.

Quality assurance represents an often-overlooked but important application. Foundation models could help identify cases where human radiologists might have missed important findings, providing a safety net that catches potential errors before they impact patient care. They could also help ensure consistent interpretation standards across different radiologists and time periods.

Comparison with Traditional Radiology AI

Traditional radiology AI systems work like specialized tools. Each system is designed for a specific task, trained on data relevant to that task, and optimized to perform that one job very well. For example, a traditional AI system for detecting lung nodules would be trained only on chest CT scans with known nodules, and it would only be able to look for lung nodules in similar scans.

This specialized approach has several advantages. The AI system becomes very good at its specific task because all of its training focuses on that one problem. The development process is more straightforward because the goals are clear and limited. The validation process is also simpler because there is only one task to test and verify.

However, this approach also has important limitations. Each new application requires building a new system from scratch, which is time-consuming and expensive. Hospitals end up needing many different AI systems for different tasks, creating complexity in implementation and maintenance. Knowledge learned for one task cannot easily be transferred to related tasks.

Foundation models work more like general-purpose tools that can be adapted to different jobs. They learn broad patterns that apply across many different imaging tasks and can be fine-tuned for specific applications without starting from zero. This is similar to how radiologists work – they learn general principles about anatomy and pathology during training, then apply this knowledge to interpret many different types of studies throughout their careers.

The development timeline differs between these approaches. Traditional AI systems can often be developed and deployed more quickly for specific tasks because the scope is limited. Foundation models require more time and resources to train initially, but they can then be adapted to new tasks more rapidly.

Performance comparisons show mixed results. Traditional AI systems often perform better on the specific tasks they were designed for, especially when those tasks have been well-studied and large datasets are available. Foundation models may perform slightly worse on individual tasks but offer much greater flexibility and broader capability.

The practical implications for hospitals are quite different. Traditional AI systems require careful integration of multiple separate tools, each with its own interface, maintenance requirements, and potential failure modes. Foundation models could potentially simplify this by providing a single system that handles multiple imaging tasks, though this simplification comes with its own challenges.

There is an amusing parallel here to the evolution of medical practice itself. Early physicians were often specialists from the beginning – the bone-setter, the tooth-puller, the eye doctor. Over time, the concept of general practice emerged, with physicians who could handle a wide range of medical problems before referring to specialists when needed. We may be seeing a similar evolution in radiology AI, moving from highly specialized tools toward more general systems that can handle routine tasks across multiple areas.

Technical Challenges and Limitations

Building effective foundation models for medical imaging faces several major technical hurdles. The first challenge involves data quality and availability. Medical images come with complex metadata, varying protocols, and different standards across institutions. Unlike internet images used to train general vision models, medical images require careful curation and expert annotation.

Data privacy regulations add another layer of complexity. Medical images contain sensitive patient information that must be protected according to strict legal requirements. This limits how data can be collected, stored, and shared between institutions, making it harder to assemble the large datasets that foundation models need for training.

The diversity of medical imaging equipment creates additional challenges. The same type of scan can look quite different when performed on machines from different manufacturers, with different protocols, or at different institutions. Foundation models must learn to handle this variation while still recognizing the underlying medical patterns that matter for diagnosis.

Validation and testing present unique difficulties in medical applications. Unlike other AI applications where errors might be inconvenient, mistakes in medical imaging can directly harm patients. This means foundation models need much more rigorous testing before they can be used clinically. The testing must cover not just accuracy but also reliability, consistency, and behavior in edge cases.

The black box nature of many foundation models creates problems for medical applications. Radiologists need to understand why an AI system reached a particular conclusion, both for patient care and for legal liability reasons. Current foundation models often cannot provide clear explanations for their decisions, making them difficult to integrate into clinical workflows.

Computational requirements represent another practical barrier. Foundation models often require powerful hardware to run effectively, which may not be available in all healthcare settings. The models also need regular updates and maintenance, requiring technical expertise that many hospitals lack.

Bias in training data can lead to foundation models that work well for some patient populations but poorly for others. If training datasets are not representative of the full diversity of patients that the model will encounter, it may produce systematically different results for different groups of people.

Integration with existing hospital systems poses technical and practical challenges. Electronic health records, imaging systems, and clinical workflows were not designed with foundation models in mind. Retrofitting these systems to work with new AI approaches requires substantial effort and expense.

Regulatory and Ethical Considerations

The regulatory landscape for medical AI is still evolving, and foundation models present new challenges that existing frameworks may not adequately address. Traditional medical AI systems are typically approved for specific tasks with well-defined inputs and outputs. Foundation models, with their ability to be adapted to new tasks, do not fit neatly into this regulatory structure.

The Food and Drug Administration and similar agencies in other countries are working to develop new approaches for regulating more flexible AI systems. This includes concepts like predetermined change control plans, which would allow certain modifications to AI systems without requiring completely new approvals. However, the regulatory path for foundation models remains unclear.

Liability questions become more complex with foundation models. If a general-purpose AI system makes an error, who is responsible? The company that created the foundation model, the organization that adapted it for a specific task, the hospital that deployed it, or the radiologist who relied on its output? Current legal frameworks do not provide clear answers to these questions.

Patient consent represents another ethical challenge. Patients may understand consenting to the use of a specific AI tool for a defined purpose, but consenting to the use of a general AI system that could be applied to many different aspects of their care is more complex. How much detail about AI capabilities and limitations must be provided to patients?

Equity and fairness concerns arise when foundation models are trained primarily on data from certain populations or institutions. If these models do not work equally well for all patients, they could worsen existing healthcare disparities. Ensuring fair performance across diverse patient populations requires careful attention during both development and deployment.

The potential for bias extends beyond patient demographics to include institutional and geographic factors. Foundation models trained primarily on data from academic medical centers might not work as well in community hospitals. Models trained in one country might not perform optimally in healthcare systems with different patient populations or imaging protocols.

Professional autonomy represents an important consideration for radiologists. As foundation models become more capable, there may be pressure to rely on them more heavily, potentially reducing the role of human expertise in image interpretation. Maintaining appropriate human oversight while taking advantage of AI capabilities requires careful balance.

Future Directions and Research Opportunities

The field of foundation models for medical imaging is rapidly evolving, with several promising research directions emerging. One major area involves improving the training processes to create more robust and capable models. This includes developing better methods for combining data from different sources and institutions, creating more effective training objectives that capture the nuances of medical imaging, and finding ways to incorporate expert knowledge into the training process.

Multimodal foundation models represent an exciting frontier. These systems would combine medical images with other types of data such as patient history, laboratory results, and clinical notes. By integrating multiple sources of information, these models could potentially provide more accurate and contextually appropriate interpretations than systems that look only at images.

Federated learning approaches could help address some of the data privacy and sharing challenges that currently limit foundation model development. Instead of centralizing medical images from many institutions, federated learning allows models to be trained across multiple sites while keeping data local. This could enable the creation of more robust models while maintaining patient privacy.

Explainability research focuses on making foundation models more transparent and interpretable. This includes developing methods for the models to explain their reasoning, highlight the image features that influenced their decisions, and provide uncertainty estimates for their conclusions. These capabilities are essential for clinical adoption.

Continuous learning represents another important research direction. Instead of training foundation models once and then deploying them unchanged, researchers are exploring ways for these systems to continue learning and improving based on new data and feedback from clinical use. This could help models stay current with evolving medical knowledge and adapt to new imaging technologies.

Domain adaptation research aims to make foundation models work better across different clinical settings. This includes developing methods to quickly adapt models to new institutions, imaging protocols, or patient populations without requiring extensive retraining.

The integration of foundation models with existing clinical decision support systems offers opportunities to create more seamless workflows. This includes research on how to present AI insights in ways that support rather than interrupt clinical reasoning, and how to combine AI outputs with other clinical information.

Implementation Challenges in Clinical Practice

Moving foundation models from research laboratories to routine clinical use involves numerous practical challenges that extend beyond technical performance. Hospital information technology systems were typically designed to handle traditional radiology workflows and may need substantial modifications to support foundation models effectively.

Workflow integration requires careful consideration of how radiologists currently work and how AI systems can fit into these established patterns. Radiologists have developed efficient methods for reviewing studies, and any AI system that disrupts these workflows may face resistance regardless of its technical capabilities. Successful implementation requires understanding these workflows and designing AI interfaces that enhance rather than hinder productivity.

Training and education represent major implementation challenges. Radiologists and other medical staff need to understand how foundation models work, what their capabilities and limitations are, and how to interpret their outputs appropriately. This educational process must be ongoing as the models evolve and improve over time.

Quality assurance processes must be established to monitor foundation model performance in real-world clinical settings. This includes tracking accuracy rates, identifying when models may be struggling with particular types of cases, and ensuring that human oversight remains appropriate as AI capabilities change.

Cost considerations extend beyond the initial purchase or licensing of foundation models. Implementation requires hardware upgrades, staff training, workflow modifications, and ongoing maintenance and support. Many healthcare organizations struggle to make the business case for these investments, especially when the benefits may take time to materialize.

Change management becomes particularly important when implementing systems that could significantly alter how radiologists work. Some staff may be enthusiastic about AI capabilities, while others may be concerned about job security or skeptical about AI reliability. Successful implementation requires addressing these concerns and ensuring that all stakeholders understand the role that AI will play.

Technical support and maintenance requirements may exceed what many healthcare organizations can handle internally. Foundation models may require specialized expertise for ongoing optimization, troubleshooting, and updates. Organizations need to plan for these support needs as part of their implementation strategy.

Economic and Workforce Implications

The introduction of foundation models in radiology has important economic implications for healthcare organizations, radiologists, and the broader healthcare system. Understanding these implications is crucial for planning successful implementation and managing the transition to AI-augmented radiology practice.

For healthcare organizations, foundation models represent both an opportunity and a financial challenge. The potential benefits include increased efficiency in image interpretation, reduced need for subspecialist consultation in some cases, and improved consistency in radiology reports. However, the initial costs for implementation, training, and infrastructure upgrades can be substantial.

The return on investment for foundation models may differ from traditional radiology AI systems. While specialized AI tools typically show clear benefits for specific tasks, foundation models offer broader but potentially less dramatic improvements across many tasks. This makes the economic case more complex to evaluate and may require longer time horizons to demonstrate value.

For radiologists, foundation models could change daily work patterns in several ways. Routine cases might be processed more quickly with AI assistance, allowing more time for complex cases that require human expertise. However, this could also lead to pressure for increased productivity expectations as AI capabilities improve.

The workforce implications extend beyond radiologists to include radiology technologists, medical residents, and other staff involved in medical imaging. Training programs may need to incorporate AI literacy as a core competency, and continuing education will need to keep pace with rapidly evolving AI capabilities.

Subspecialty radiology could be particularly affected by foundation models. If general AI systems become capable of handling routine cases across multiple subspecialties, the demand for specialized radiology expertise might shift toward more complex cases and interventional procedures.

Rural and underserved areas could benefit substantially from foundation models if these systems can provide reliable preliminary interpretations for basic imaging studies. This could help address radiologist shortages in these areas and improve access to timely image interpretation.

Patient Safety and Quality Assurance

Patient safety must be the primary consideration when implementing foundation models in clinical radiology. The potential for AI systems to miss important findings or generate false positive results creates risks that must be carefully managed through appropriate oversight and quality assurance measures.

Human oversight remains essential even as AI capabilities improve. Radiologists must maintain the skills and knowledge necessary to independently verify AI findings and recognize when AI systems may be making errors. This requires ongoing training and calibration to ensure that radiologists neither over-rely on nor inappropriately dismiss AI outputs.

Error detection and reporting systems need to be established to monitor foundation model performance and identify patterns that might indicate problems with the AI system. This includes tracking cases where AI and human interpretations differ, analyzing cases where AI recommendations proved incorrect, and monitoring for systematic biases or blind spots.

Fallback procedures must be in place for situations where foundation models fail or are unavailable. Hospitals need to maintain the capability to operate effectively without AI assistance and ensure that staff remain competent to handle cases manually when necessary.

Patient communication about AI use in their care represents an important safety consideration. Patients have a right to know when AI systems are involved in their diagnosis, and they may have concerns or questions about AI reliability that need to be addressed appropriately.

Quality metrics for foundation models may need to be different from those used for traditional AI systems. The broad capabilities of foundation models make it challenging to define simple accuracy measures, and quality assessment may need to consider multiple factors including diagnostic accuracy, report consistency, and appropriate confidence calibration.

Key Takeaways

Foundation models for medical imaging represent a major shift from specialized AI tools toward more general-purpose systems that can handle multiple radiology tasks. This transition offers potential benefits in terms of efficiency, accessibility, and system simplification, but it also presents new challenges in validation, regulation, and implementation.

The technology is advancing rapidly, but significant work remains before foundation models are ready for routine clinical use. Current systems show promise but need improvement in reliability, interpretability, and integration with existing clinical workflows.

Successful implementation of foundation models will require collaboration between AI developers, radiologists, hospital administrators, and regulatory agencies. Technical capabilities alone are not sufficient; these systems must also meet clinical, economic, and regulatory requirements.

The future of radiology will likely involve close collaboration between human radiologists and AI systems rather than replacement of human expertise. Foundation models may handle routine tasks while radiologists focus on complex cases and patient interaction.

Ongoing research, education, and quality assurance will be essential as foundation models evolve and become more widely adopted. The radiology community must remain actively engaged in shaping how these systems are developed and deployed to ensure they truly serve patient needs and support high-quality medical care.

Frequently Asked Questions

What exactly is a foundation model in medical imaging?

A foundation model is an AI system trained on large amounts of diverse medical imaging data that can be adapted to perform many different radiology tasks. Unlike traditional AI systems that do only one specific job, foundation models learn general patterns that work across multiple types of scans and diagnostic tasks.

How do foundation models differ from the AI systems currently used in radiology?

Current radiology AI systems are typically designed for specific tasks, like detecting lung nodules or measuring heart function. Each system requires separate training and maintenance. Foundation models can handle multiple tasks with a single system, making them more flexible and potentially easier to manage.

Are foundation models accurate enough for clinical use?

Foundation models show promising accuracy in research studies, but most are not yet ready for routine clinical use. They need more testing, validation, and regulatory approval before they can be safely used for patient care. Current evidence suggests they perform well on many tasks but may not match specialized systems for specific applications.

Will foundation models replace radiologists?

No, foundation models are designed to assist radiologists rather than replace them. These systems can help with routine tasks and provide decision support, but human expertise remains essential for complex cases, quality oversight, and patient communication. The technology is more likely to change how radiologists work rather than eliminate their role.

What are the main challenges preventing widespread adoption of foundation models?

The primary challenges include regulatory approval processes, technical integration with hospital systems, training requirements for medical staff, cost considerations, and ensuring patient safety. Additionally, the models need further development to improve reliability and interpretability for clinical use.

How might foundation models affect smaller hospitals or rural healthcare facilities?

Foundation models could potentially improve access to radiology expertise in underserved areas by providing reliable preliminary interpretations of common imaging studies. However, these facilities would still need technical infrastructure and support to implement such systems effectively.

What should radiologists do to prepare for foundation models?

Radiologists should stay informed about AI developments through continuing education, learn about AI capabilities and limitations, and participate in discussions about how these systems should be integrated into clinical practice. Understanding how to work effectively with AI systems will become an important professional skill.

How will patients know if AI is being used in their care?

Healthcare organizations will need to develop policies for informing patients about AI use in their care. This may include consent processes, explanatory materials, and clear communication about the role of AI in diagnosis and treatment decisions.

What regulatory approvals do foundation models need?

Foundation models will need approval from agencies like the FDA, but the regulatory pathway is still being developed. These systems present new challenges for regulators because they can be adapted to multiple tasks, unlike traditional medical devices with single purposes.

When might foundation models become commonly available in hospitals?

The timeline for widespread adoption is uncertain and will depend on technical advancement, regulatory approval, and practical implementation challenges. Some limited applications may appear in the next few years, but broad adoption will likely take longer as the technology matures and integration issues are resolved.

References

Azizi, S., Mustafa, B., Ryan, F., Beaver, Z., Freyberg, J., Deaton, J., … & Natarajan, V. (2021). Big self-supervised models advance medical image classification. Proceedings of the IEEE/CVF International Conference on Computer Vision, 3478-3488.

Chen, R. J., Lu, M. Y., Chen, T. Y., Williamson, D. F., & Mahmood, F. (2021). Synthetic data in machine learning for medicine and healthcare. Nature Biomedical Engineering, 5(6), 493-497.

Dosovitskiy, A., Beyer, L., Kolesnikov, A., Weissenborn, D., Zhai, X., Unterthiner, T., … & Houlsby, N. (2020). An image is worth 16×16 words: Transformers for image recognition at scale. arXiv preprint arXiv:2010.11929.

Huang, S. C., Pareek, A., Seyyedi, S., Banerjee, I., & Lungren, M. P. (2020). Fusion of medical imaging and electronic health records using deep learning: a systematic review and implementation guidelines. NPJ Digital Medicine, 3(1), 136.

Irvin, J., Rajpurkar, P., Ko, M., Yu, Y., Ciurea-Ilcus, S., Chute, C., … & Ng, A. Y. (2019). CheXpert: A large chest radiograph dataset with uncertainty labels and expert comparison. Proceedings of the AAAI Conference on Artificial Intelligence, 33(01), 590-597.

Johnson, A. E., Pollard, T. J., Berkowitz, S. J., Greenbaum, N. R., Lungren, M. P., Deng, C. Y., … & Mark, R. G. (2019). MIMIC-CXR, a de-identified publicly available database of chest radiographs with free-text reports. Scientific Data, 6(1), 317.

Kirillov, A., Mintun, E., Ravi, N., Mao, H., Rolland, C., Gustafson, L., … & Girshick, R. (2023). Segment anything. arXiv preprint arXiv:2304.02643.

Liu, F., Zhou, Z., Jang, H., Samsonov, A., Zhao, G., & Kijowski, R. (2018). Deep convolutional neural network and 3D deformable approach for tissue segmentation in musculoskeletal magnetic resonance imaging. Magnetic Resonance in Medicine, 79(4), 2379-2391.

Moor, M., Banerjee, O., Abad, Z. S. H., Krumholz, H. M., Leskovec, J., Topol, E. J., & Rajpurkar, P. (2023). Foundation models for generalist medical artificial intelligence. Nature, 616(7956), 259-265.

Peng, C., Zhang, X., Yu, G., Luo, G., & Sun, J. (2017). Large kernel matters–improve semantic segmentation by global convolutional network. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 4353-4361.

Rajpurkar, P., Irvin, J., Zhu, K., Yang, B., Mehta, H., Duan, T., … & Ng, A. Y. (2017). CheXNet: Radiologist-level pneumonia detection on chest X-rays with deep learning. arXiv preprint arXiv:1711.05225.

Shamout, F. E., Shen, Y., Wu, N., Kaku, A., Park, J., Maki, T., … & Geras, K. J. (2021). An artificial intelligence system for predicting the deterioration of COVID-19 patients in the emergency department. NPJ Digital Medicine, 4(1), 80.

Tang, Y., Tang, Y., Xiao, J., & Summers, R. M. (2021). XLSor: A robust and accurate lung segmentor on chest X-rays using criss-cross attention and customized radiorealistic abnormalities generation. Medical Image Computing and Computer Assisted Intervention, 457-467.

Van der Velden, B. H., Kuijf, H. J., Gilhuijs, K. G., & Viergever, M. A. (2022). Explainable artificial intelligence (XAI) in deep learning-based medical image analysis. Medical Image Analysis, 79, 102470.

Wang, X., Peng, Y., Lu, L., Lu, Z., Bagheri, M., & Summers, R. M. (2017). ChestX-Ray8: Hospital-scale chest X-ray database and benchmarks for weakly-supervised classification and localization of common thorax diseases. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, 2097-2106.

Zhang, Y., Jiang, H., Miura, Y., Manning, C. D., & Langlotz, C. P. (2020). Contrastive learning of medical visual representations from paired images and text. Machine Learning for Healthcare Conference, 2020, 2-25.

Zhou, H. Y., Zhang, C., Li, H., Chen, D., Lu, C., & Yu, Y. (2023). A foundation model for generalizable disease detection from retinal photographs. Nature, 622(7981), 156-163.