Hi! I am Andrii Buvailo, and this is my weekly newsletter, ‘Where Tech Meets Bio,’ where I talk about technologies, breakthroughs, and great companies moving the industry forward.
If you've received it, then you either subscribed or someone forwarded it to you. If the latter is the case, subscribe by pressing this button:
Now, let’s get to this week’s topics!
Below, I've compiled three recent studies rating various pharmaceutical companies in terms of either their adoption of advanced AI technologies or their productivity—i.e., how well AI is used to achieve results (for instance, generating molecules that reach clinical studies).
The first study is by CB Insights: The Pharma AI Readiness Index.
It evaluates the AI preparedness of leading pharmaceutical companies. This index ranks companies based on metrics like AI talent acquisition, project execution, and innovative prowess, utilizing datasets such as patents, partnerships, and product launches. Notably, Roche and Bayer have emerged as frontrunners due to their significant AI-related investments and initiatives. Firms like Eli Lilly and Company, AstraZeneca, and Sanofi are actively integrating AI.
Another study is by Christian Rutzer, Deputy Head of CIEB Universität Basel.
The analysis looks at how many publications by various pharmaceutical companies are related to AI-based research. The study methodology employed keyword searches in paper titles and abstracts from Scopus data.
According to this study, Swiss company Novartis emerges as a pioneer, with over 3% of its scientific publications between 2020 and 2022 highlighting AI applications. Following closely are Germany's Boehringer Ingelheim at 2.5% and the UK's AstraZeneca at 2%. As Dr. Christian Rutzer, Deputy Head CIEB University of Basel, comments on the results of this study, it's been observed that large US pharmaceutical companies have a slower AI incorporation rate in research compared to their European peers. Novartis's dominance, for example, might be influenced by its focus on big data, further amplified by its collaboration with Palantir Technologies.
This situation seems a bit counterintuitive to me, considering how many AI projects are being implemented in the US life science hubs, such as Boston, San Francisco, or San Diego, but these are the numbers. The study methodology is this:
”The analysis is based on Scopus data. We identified AI papers in pharma papers by keyword searches of paper titles and abstracts using the following terms: deep learning, natural language processing, machine learning, machine intelligence, artificial intelligence, ai technology, supervised learning,
learning algorithm, unsupervised learning, reinforcement learning”.
Finally, here is a 2022 study by BiopharmaTrend.com looking at how AI-driven emerging companies are using AI technology to accelerate drug candidate discovery and progression towards clinical trials.
In this study, disclosed information about the preclinical and clinical pipelines of more than 130 AI-driven companies was analyzed, and the 20 most productive companies were selected to build the BPT20 Artificial Intelligence in Drug Discovery Productivity Index. A formula used to calculate the index takes into account the number of drug candidates in various clinical trial stages, the number of preclinical candidates, and how long the companies have been on the market, as well as some other factors. It is calculated as a polinom, applying various weights to the corresponding variables and giving the resulting coefficient. The simple sorting of the coefficients for the 20 companies leads to the BPT20 index.
So, the company's position in the BPT20 index reflects its performance relative to the other 19 members. We are hoping to release the 2023 edition of the BPT20 index by the end of the year (some companies advanced heavily, while there were also setbacks in some others, like insufficiently optimistic results in this phase 2a trial, where the AI-inspired topical pan-Trk inhibitor failed to improve eczema symptoms or itch).
You can explore clinical-stage molecules by some of these companies in an interactive table here (only the leading drug candidates in each case were selected to be included in the table): The Roadmap of Drug Candidates Designed by AI.
The Paradigm of AI in Drug Discovery is Changing
Now, let’s discuss a quite profound change in the medical and life science AI fields, affecting how life science companies will be building and collaborating.
In 2016, when I started watching the market for machine learning based tools and companies in drug discovery, the area was dominated by highly specialized systems and platforms. The research models of then-founded AI-driven companies were based on task-specific training workflows, where a machine learning model would be trained on relatively small manually labeled datasets (thousands, hundreds of thousands, or perhaps millions of datapoints).
Most ML tools were primarily predictive in nature (i.e., with a really few examples of companies in the generative AI realm, e.g., Insilico Medicine, which pioneered the application of generative-adversarial networks (GANs) for de-novo molecular design). But even Insilico’s generative AI platform was designed to be task-specific.
Then, we entered the era of large models, with ChatGPT being the biggest popularizers of this AI concept.
GPT-4, Dall-E 2 and Bidirectional Encoder Representations from Transformers (BERT) are all foundation models. The term was coined in a 2021 paper, "On the Opportunities and Risks of Foundation Models," by authors from the Stanford Center for Research on Foundation Models and the Stanford Institute for Human-Centered Artificial Intelligence (HAI).
Among other things, the key event that essentially triggered this paradigmic shift in the AI field was the introduction of Transformer and Attention algorithms, was a landmark 2017 article “Attention is all you need” by Google scientists.
A foundation model (also called base model) is a large machine learning (ML) model trained on a vast quantity of data at scale (often by self-supervised learning or semi-supervised learning) such that it can be adapted to a wide range of downstream tasks.
The paradigm of foundation models is very different from the previous approach to building AI tools and platforms, including the following key considerations:
a) in contrast to earlier AI paradigms, foundation models are generalized to many tasks; they are domain-agnostic, essentially.
For instance, a foundation model-based ChatGPT can handle chemistry even though it was never specifically trained to answer chemical questions. It was shown that prompting ChatGPT can be efficient, for instance, for predicting synthesis conditions for metal organic frameworks (MOFs).
However, large language models (LLMs), being the most popular in the family of foundation models these days, have limitations when it comes to working with domain-specific tasks, like chemistry, biology, mathematics, and so on. They can handle some domain specific problems, but they often fail with other problems. Even worse, they provide wrong answers that look like correct answers, or make up facts altogether (the so-called ‘hallucination‘ of LLMs).
The other day, I was testing a tool, Glass AI 2.0, a creation of Glass Health, they call it "the first digital notebook designed for doctors". Glass AI 2.0 combines a base large language model (LLM) with a clinical knowledge database, created and maintained by clinicians, to create differential diagnosis (DDx) and Clinical Plan outputs.
The model gave very reasonable diagnostic plan and treatment plan for my own medical condition related to cardiovascular system. Previously I consulted with a number of doctors, and the model’s predictions matched their prescriptions.
Another very powerful strategy to improve the power of LLMs is to augment them with specialized software. OpenAI is already doing that with their Plugin Store. There, you can connect, say, to the Wolfram plugin, which provides access to powerful computation, accurate math, curated knowledge, real-time data, and visualization through Wolfram|Alpha and Wolfram Language.
In this scenario, a general purpose model is able to ‘talk‘ to third party software and make it run specific operations, like calculations or running code to perform certain tasks. The results are returned to a general purpose foundation model and interpreted by the model according to whatever the initial prompt to the foundation model was about. So, the foundation model takes on the role of an intelligence agent that uses the power of external tools, or plugins, to make things happen.
b) foundation models are trained mostly on unlabeled, unstructured, and structured data in very large quantities, which completely changes the strategy for data collection, storage, pre-processing, and collaboration with tech vendors.
This one is important in many ways. If the AI strategy of 5–7 years ago revolved around collecting as much data as possible in house and training custom task-specific models, the strategy with foundation models is different. You have to rely on one of the foundation models created by ‘big tech‘ companies, such as OpenAI or Google. And then fine-tune the model to your specific needs with your data, in local environment.
Moreover, research workflows involving foundation models require specialized data infrastructure, which is mostly impossible to build in-house. It involves, again, external services, like those from Google Cloud, MS Azure, Amazon Web Services, NVIDIA Clara™ for Drug Discovery, Palantir Foundry, and so on.
Those things come with a lot of costs that must be accounted for by the users. Large scale computational projects may run into millions or even tens of millions of dollars per year just for using the infrastructure.
In this way, life science and healthcare businesses become more dependent on external tech vendors than ever. This comes with new and unique kinds of challenges, and strategic risks.
c) there is a phenomenon of ‘emerging abilities‘.
It means, the model aquires unforeseen abilities with the size of training data.
Life Science companies are after genAI
Life science companies, big a small are after integrating foundation models into their workflows.
Ginkgo Bioworks, Inc. and Google just announced a 5-year partnership. Under the strategic partnership, Ginkgo will work to develop new, state-of-the-art large language models (LLMs) running on Google Cloud's Vertex AI platform across genomics, protein function, and synthetic biology, helping Ginkgo's customers accelerate innovation and discovery in fields as diverse as drug discovery, agriculture, industrial manufacturing, and biosecurity.
Bayer will also be utilizing Google Cloud’s Vertex AI and Med-PaLM 2 to refine the cumbersome clinical trials process. Namely, these tools will be able to better decipher large data sets, find correlations between disparate data points, and generate insights that can tangibly be used to advance the research and development lifecycle.
Large language models (LLMs) like OpenAI's ChatGPT are being used by biotech companies, such as Insilico Medicine, Ainnocence, and Exscientia, to aid drug discovery.
ChatGPT helps scientists interact with AI and machine learning tools, making data more accessible. For instance, Insilico Medicine employs ChatGPT to interact with their target discovery platform, PandaOmics, while Ainnocence's founder Lurong P. envisions specialized versions like Bio ChatGPT or Med ChatGPT. Exscientia utilizes LLMs to generate structured, mechanistic assertions for their knowledge graphs.
LLMs also serve as advanced search engines in biological science. Google and DeepMind's Med-PaLM chatbot provides answers to medical questions, while BenevolentAI seems excited about ChatGPT's retrieval plug-in feature, which could help customize the chat function based on their own data.
Despite the potential of LLMs in drug discovery, some companies like BioXcel Therapeutics, Inc. and Enterome remain cautious about adopting the technology, citing concerns over accuracy and privacy.