A New Kid On the Block: AI World Models In Biotech
LLMs are all the rage now, but they (likely) won't get us to real drug discovery revolution. So.. what's next?
Hi! This is our weekly newsletter, ‘Where Tech Meets Bio,’ where we talk about technologies, breakthroughs, and great companies moving the biopharma and medtech industries forward.
If you've received it, then you either subscribed or someone forwarded it to you. If that is the case, subscribe by clicking this button:
Let’s get to this week’s topics!
There is a new kid on the block and a new trend in AI: world models (also known as world simulators).
These models are gaining attention for their potential to bring AI closer to human-level reasoning by creating rich, realistic simulations of the world around us.
So, What Are World Models?
Unlike traditional AI that simply analyzes patterns, world models aim to "understand" how things interact in a real or virtual space, predicting behaviors based on physical and causal relationships.
Inspired by the way humans form mental models of the world, these systems can, for example, predict the movement of objects or simulate a realistic environment with an understanding of why things behave as they do.
Here Are Some Early Movers and Examples:
World Labs has raised $230 million to develop large world models.
Google DeepMind brought in OpenAI’s Sora video generator team to work on "world simulators."
Runway's Gen-3 video model can simulate realistic movements in generative video, avoiding the common "uncanny valley" effect.
How Do They Work?
First of all, world models incorporate data from photos, audio, videos, and text to form a "multisensory" representation.
These models learn basic physical principles, like why a basketball bounces, helping them make realistic predictions.
Some models can devise action sequences to reach an objective—e.g., visualizing steps to clean a room.
World models go step ahead of "traditional" Large Language Models (LLMs):
While LLMs can predict most likely words and build sentences, world models "understand" the interactions between objects and environments, mimicking basic cause-and-effect.
They go a step further than LLMs by attempting to build internal maps of a world, useful in both digital and physical realms.
For instance, world models could enable robots to interact more naturally with their surroundings by anticipating outcomes, an ability LLMs lack.
I first encoutered world models in the context of biotech in an article by a techbio startup NOETIK. The scientists there built OCTO, which is a multimodal, transformer-based world model designed to simulate cancer biology at a patient-specific level by integrating high-dimensional data, including protein staining, gene expression, DNA sequencing, and tissue structure.
It leverages a unique structured multimodal masking strategy to learn biologically relevant relationships across data types, allowing it to predict outcomes of therapeutic interventions through counterfactual simulations.
OCTO’s design enables it to act as a biological simulator for drug discovery, offering in silico screening of drug targets based on individualized patient data.
While we're likely years away from full potential of world models, I think they are shaping up to be a game-changer.
Btw, Yann LeCun is among big believers in world model strategy in AI.
Exciting times ahead for the AI field!
Read also:
11 Biopharma Trends to Watch in 2024
Thank you for explaining this AI news (which I didn't know about!) so directly, clearly and effectively. I would be really curious to know the many immediate implications it could have in different fields, starting from the medical, physical and chemical ones.