Verily-NIH multimodal AI model boosts
Verily Health, with NVIDIA and the NIH All of Us program, developed a foundation model combining EHR and genomic data. The model's performance for identifying high-risk Type 2 diabetes cases improved significantly, achieving a threefold pre-training efficiency gain.
Verily Health, collaborating with NVIDIA and the National Institutes of Health (NIH) All of Us Research Program, has developed a multimodal foundation model integrating electronic health record and genomic data. The model demonstrated a substantial improvement in identifying individuals at high risk for Type 2 diabetes, according to a company announcement.
Integrating Diverse Health Data
Jonathan Amar, senior manager of Verily Data Science, explained that electronic health records are just one part of a patient's story. He stated that to truly understand disease risk, artificial intelligence must learn from multiple dimensions of human health, including both biology at birth and lived clinical experience. Combining EHR data with genomic information is seen as particularly promising for revealing inherited disease risk long before symptoms appear.
The research team set out to combine these two fundamentally different data types in a single foundation model. Amar noted the complexity of integrating static genetic data with dynamic clinical histories, alongside the computational demands that most CPU-based workflows cannot handle. Nation-level initiatives like the All of Us Research Program, which collects multimodal data including genomics, were crucial for providing the necessary datasets.
Model Performance and Efficiency
The Verily-led team integrated genetic risk scores with EHR data in a foundation model powered by NVIDIA GPUs. When genetic risk information was added, the model's ability to identify people at high risk for Type 2 diabetes improved substantially. It showed it could project risk over clinically meaningful timeframes of five to ten years.
On the computational side, the model achieved a significant efficiency gain. Using the NVIDIA NeMo AutoModel provided a threefold increase in pre-training efficiency compared to Hugging Face Accelerate on identical H100 GPUs. This translated to faster time to insight in a complex research environment.
| Model Component | Data Type Integrated | Key Performance Outcome |
|---|---|---|
| Forecast™ 1.0 | EHR and Genomic | Improved Type 2 diabetes risk identification |
| Training Framework | NVIDIA NeMo AutoModel | 3x pre-training efficiency vs. Hugging Face Accelerate |
Availability and Future Implications
This pioneering model, called Forecast™ 1.0, is now available to all researchers via GitHub. It gives global research teams hands-on access to accelerate their own disease risk discovery pipelines within a secure Trusted Research Environment.
The work provides an early blueprint for how multimodal foundation models could integrate many other dimensions of human health. These include wearables, clinical notes, medical imaging, and patient-reported outcomes to advance precision medicine. Verily continues to research improvements in predictive and generative models, driven by the expansion of multidimensional datasets.
Amar emphasized that by bringing together two types of typically mismatched data, they demonstrated how multimodal AI can improve disease detection. He stated this provides a preview of what's possible in pharmaceutical biomarker discovery, diagnostic risk stratification, and health system precision medicine programs. The model is available for researchers via GitHub.
