SCDM 2023: Our Take

We hosted a panel discussion on AI, “How to Overcome the ever-increasing Diversity of Data Sources in support of AI/ML.” This expert panel organized one of the most thorough discussions on all aspects of incorporating AI in clinical data management especially since during the keynote, Arshad Mohammed, Head of Global Data Management, GSK, noted that over 70% of clinical data now comes from sources other than EDC (eCRFs). External data has outstripped traditional data. This is driving a meaningful shift in the role of the data manager in clinical trials. Their job is now broader and no longer just EDC centric.

A poll taken during the session showed that nearly half of those in the room were already involved in AI projects at work with:

● 28 percent noting that their companies were beginning to be involved with AI and were working on at least one project

● 16 percent said their company had advanced Interest in working with AI

The Keynote:
AI at a Cross-Roads: Does Ethics Drive or Hinder the Way Forward?

Keynote speaker, Kay Firth-Butterfield, CEO, Good Tech Advisory LLC, is an attorney with a strong background in technology. In 2014, she was appointed as the first Corporate Executive Officer in AI or Chief AI Ethics Officer. She opened the conference with a cautionary note on the use and misuse of AI. Considering AI was the overwhelming focus of the conference, it seemed appropriate that she focus on the risks involved in the use of AI. For example, there was mention of the question of Trust and how with all the hype in the news about run-away AI, some half of respondents to a survey were concerned about “Super Intelligence.” Perhaps a better way to think of AI – as in any other tools – is that it can be a liability if misused.

As noted in our previous blogs, AI models and algorithms are only as good as the focus and quality of the training matter that was provided to them. “Model Bias,” resulting from uncontrolled training material is universally viewed as bad. That is because “Bias” is a negative word to begin with! I believe that the so-called Bias is something that can be viewed as a method to give the algorithm “focus.”  A better cautionary note about model bias is for the users and operators of the algorithm to maintain an awareness of the bias tendencies of the algorithm and to take advantage of it. For example, a model trained to have a focus on parsing insights for oncology is a good thing as long it is used for that purpose and not for other therapeutic areas. Replacing the phrase “Model Bias” with “Model Focus” then becomes an enhancement and not a detraction of AI usage.

Another prominent issue that Firth-Butterfield cautioned about was privacy. A true concern if operators are novices and do not understand public (CHATGPT) vs private model training. Unfortunately, in a real-life case a major corporation was using CHATGPT for the purposes of writing meeting notes from confidential internal meetings which were then discovered by their competitors. Fortunately, many major corporations now have clear guidelines about use of “public” AI services. Setup and maintenance of a private model that is trained specifically for a client using their own data enriched with public data in an isolated environment is necessary for any vendor in our industry who claims to offer AI deduced insights.

For now, the exuberance surrounding AI was omnipresent in about every presentation at the 2023 SCDM conference. 

In the first session, he emphasized that this is “a new AI era.” In the same session, Richard Young, Vice President, CDMS Strategy at Veeva, noted that the pace of change is making it more challenging to maintain scientific rigor.

In this same talk, Aruna Vattikola, eClinical Capability Director in eClinical Technologies, GDMS at Merck, made the point that for “EHR2EDC systems, the key is to have a middleware” technology layer to enable the exchange of data between the respective systems.

For the DCT and Digital Innovation session, speakers again focused on the speed of technology adoption and the challenges this brings. Clinical data scientists must be aware of these pitfalls as they lead adoption of modern technologies:

● Complexity is unavoidable
● No magic solution
● No one size fits all
● Be pragmatic
● It is complicated

Nadolny made the point that it is important to remember that the challenge is getting the right kind of patients. Protocol changes are often driven by changes in the patient population, and hence, technology solutions need to allow flexibility

Interestingly, a session attendee from the floor, Demetris Zambas, VP and Global Head of Data Monitoring and Management, Pfizer, pointed out that often, more automation equals less flexibility.

These points continue to encourage me as a data scientist and as a leader of a company that focuses on driving better data integration and interoperability. Companies are continually challenged to find flexible ways to manage new systems and sources and maintain rigorous standards in clinical research.