- Advertisement -
Tuesday, September 27, 2022
Home Pharma & Biotech Pharma & Life Sciences Transforming Clinical Development Using the Power of Big Data and AI–An Executive...

Transforming Clinical Development Using the Power of Big Data and AI–An Executive Overview

By Prakriteswar Santikary, Vice President of Technology, ConcertAI

With clinical trial costing $1B and taking about 10 years to complete before a new drug can be launched to the market, it’s no wonder life sciences and healthcare industries are looking to big data and AI to achieve efficiency, reduce cost and bring life-savings drugs and therapies to the market faster for better patient outcomes.


Randomized Clinical Trials (RCTs) are currently the method of choice – the gold standard – for pharma companies when it comes to assessing safety and efficacy of new medicines, but they have become more expensive and complex over time with very high failure rates. Reasons for failure are primarily attributed to delay in patient selection and recruitment as well as lack of continuous patient monitoring for compliance and data quality and stringent eligibility criteria.

No wonder why the clinical trial industry is ripe for disruption.

Enter Artificial Intelligence

AI applications – primarily machine learning (ML), deep learning (DL), natural language processing (NLP)  and optical character recognition (OCR), together with big data and modern data platform enabling its processing at scale–have a huge potential to transform every stage of clinical development – starting from identifying and recruiting patients to optimizing study design to patient monitoring and pharmacovigilance, and any complexities in between.

ML and DL, for example, can automatically find patterns of meaning in large datasets such as text, speech, or images. NLP can understand and correlate content in written or spoken language, and human–machine interfaces (HMIs) allow natural exchange of information between computers and humans.

OCR can help with the conversion of images of typed, handwritten, or printed text into machine-encoded text that NLP can use for further insight generation.

These AI technologies are, in fact, being used in clinical development to correlate large and diverse datasets such as electronic health records (EHRs), medical literature, medical claims, genomics, and trial databases for improved patient–trial matching and recruitment before a trial starts, as well as to monitor patients automatically and continuously during the trial, thereby allowing improved adherence control, data compliance against study protocol and yielding more reliable and efficient endpoint assessment.

Precision medicine, an approach to patient care that is based on targeted therapies, is another area where AI technologies can offer an alternative to the one-size-fits-all approach of traditional medicine. Armed with new insights into what makes patients healthy at the individual level, researchers can facilitate the development of new drugs, find new uses for existing drugs, suggest personalized combinations, and even predict disease risk.

In this article, we will focus on some specific use cases pertaining to clinical trials process and discuss how AI technologies are assisting.

Patient selection and recruitment

Poor patient recruitment is one of the most common reasons trials fail. Identifying and engaging a population of eligible patients, ensuring clinical trial diversity, and achieving an adequate sample size can all be significant stumbling blocks.

Using the power of cloud computing and modern data platform, sophisticated analytics methods are being employed to combine genomics data with electronic medical record (EMR) and other patient data, scattered among different locations, owners, and formats – from handwritten paper copies to digital medical imagery – to surface biomarkers that lead to endpoints that can be more efficiently measured, and thereby identify and characterize appropriate patient subpopulations. This presents a unique opportunity for NLP and computer vision algorithms such as OCR to automate the reading and compiling of this evidence.

AI is also helping to reduce population heterogeneity by parsing diverse EMR data to select patients with a higher probability of measurable clinical endpoints and using machine learning to look for key biomarkers. In this way, AI is helping select populations that stand a better chance of responding to treatment.

Patient eligibility

The complexity of trial eligibility criteria in terms of number and medical jargon generally makes it challenging for a patient to comprehend and assess their own eligibility. OCR and NLP are offering viable assistance with automatically finding needles in the EMR haystack.

NLP, for example, is being used to comprehend written and spoken language from a variety of structured and unstructured data types. AI technologies are analyzing EMR and clinical trial eligibility databases and finding matches between specific patients and recruiting trials and recommending these matches to doctors and patients, and helping to create synthetic control arm – “digital twins”-  by simulating how those patients may respond if assigned to the control group, enabling clinical trial sponsors to run trials that achieve the desired statistical power with smaller control groups.

Patient Monitoring

Recruiting the right patients into a clinical trial is a massive investment of both time and funding. The return on this investment can only be realized through successful completion of the trial. Hence, it is imperative that patients stay in the trial, adhere to trial procedures and rules throughout the trial, and that all data-points for monitoring the impact of the tested drug are collected efficiently and reliably for regulatory submission.

Improved patient monitoring and coaching methods during ongoing trials can be used to lower the adherence burden, make endpoint detection more efficient, and thus reduce dropout and non-adherence rates. AI techniques in combination with wearable technologies (smartphone, smart apps, biosensors, sleep trackers, etc) are offer new approaches to developing such power-efficient, mobile, real-time, and personalized patient monitoring systems.AI is also improving the safety of the trials through real-time data access and remote monitoring of participants, allowing researchers to keep more accurate tabs on biological changes, as well as identifying if a participant is responding to treatment in an adverse manner.

Looking Ahead

While the transformative power of big data and AI in clinical development is proven, as AI practitioners and data evangelists, we need to be continually cautious when applying AI to highly sensitive patient data with respect to maintaining safeguards around data privacy, security, and regulatory compliance.  Model explain ability, transparency and trust are keys, as is approaches towards dealing with data bias.

Data quality is key; AI models are as good as their inputs. In clinical industry, data interoperability is a challenge. Data is also often siloed and locked-in across various vendor systems. Extra attention must be paid towards data integration, aggregation, deidentification and enrichment before AI models can be trained.

Lastly, AI is a team sport so to be successful at it, working collaboratively as part of a cross-functional team consisting of clinicians, scientists, doctors, and SMEs is always a great start.


Dr. Santikary is an accomplished technology executive with over 25 years of industry experience inbuilding and commercializing software and data products, platforms and solutions utilizing modern techniques of distributed computing, cloud computing, and artificial intelligence.

In his current role as Global Vice President of Data Engineering, Architecture and Technology at ConcertAI, Dr. Santikary leads the vision, strategy and execution of the company’s global data and platform architecture, cloud architecture and AI-driven data products, services and solutions.

Before joining ConcertAI, Dr. Santikary held several technology executive roles across a variety of industries, including serving as Global Chief Data Officer at Clario, Head of Data Engineering and Director of Engineering at eBay, VP of Engineering at Zeta Global, Chief Architect and Technologist at Sunovion Pharmaceuticals, Director of Data Science and Engineering at EnerNOC, Chief Data Architect at PNC Financial and Research Scientist at The University of Michigan, Ann Arbor.

Dr. Santikary earned his PhD in Computer Simulation at Indian Institute of Science, Bangalore, India and post-doctoral research at The University of Michigan, Ann Arbor. He speaks regularly at global Big Data and AI conferences, IBM Chief Data Officer summit and MITCDOIQ.


Must Read

Data Sciences in the Medical Industry: How powerful is it?

When novel compounds were researched about, to meet the needs of patients who were in critical conditions, medical science had huge growth....

Related News

Finding a Little More Value across the Healthcare Expanse

The human arsenal has gone on to show some really valuable skills over the years, but if we are being honest, it hasn’t displayed...

Striking the Bio Balance

Human beings are known to be good at a ton of things, but if we are being honest, there is nothing we do better...

Fast Tracking Our Way to an Enhanced Healthcare Reality

Human beings tend to possess a lot of valuable traits, and yet the greatest trait they have ever possessed is the one that pushes...

Modernizing the Fight Against a Crisis

While human beings get to have a ton of valuable elements in their arsenal, we cannot really claim to possess anything more valuable than...

Taking Over a Different Landscape

Human arsenal, as we know, is loaded to the brim, and yet a deeper look would reveal how nothing there is more valuable than...