Changing Times Require Better Data Vigilance: A Perspective on Data Analytics and RWE

By Sina Adibi, CEO of Adaptive-Clinical Systems

There has been more and more buzz around using EHRs as the “single-source-of- truth” during clinical trials. The allure of getting real-time analytics as soon as a visit is complete, prompted many in clinical trials to race towards tighter integration with EHRs.

Our hope and expectation, as detailed in the Jan 2020 Transcelerate article1, was that existing clinical technologies combined with EHR can facilitate clinical innovation.

  • Improve, modernize, and streamline data collection , monitoring, and reporting
  • Improve access to electronic health data to advance/ enable machine learning for healthcare
  • Reduce data entry errors and minimize the effort required for source data verification
  • Promote real-time access for data review
  • Enable more rapid identification of safety and operational signals, as well as increase data integrity and quality.

Later, we expanded that to analyze past and present ongoing data and to look for patterns that could predict therapies and to be used as input to protocol design.

As we move towards this goal though, trial designers, clinical operations and data managers must be aware of dangers of relying on what could be “manipulated source data.” Recent news* highlights the dangers that exist where third-party commercial interests could influence the way that physicians behave. In their reliance on automation they can be unknowingly swayed to prescribe meds that then make their way into our eCRFs and begin to show in our analytics.

To connect single-sources-of-truth, here are some of the best practices we have adopted. 

  • Guarantee that the data is not in any way altered during the many transcription steps to which we have grown accustomed. (Not having to export data into a flat file, then load into excel or to run an R program against it to fix bad data, will make things faster and the data more accurate.)
  • Accelerate the historical 8 – 22 day lag between visit to eCRF. (In fact we can completely eliminate lag and have data appear in the EDC in real-time by eliminating double data entry or source data verification.)
  • Make many dashboards come alive and become more useful. (A challenge with existing analytics tools is that the lag time is too great and errors in data often translate to perceived outliers and distractions in our analysis work.)

In the work that we have done to date, we have seen that our data integration connectors and rules engine are able to address the operational challenges of bringing data together in real-time to help speed up studies and reduce error. This has been our industry’s primary goal so far.

So, what if the data that are coming in from the EMR/ EHR’s reflect biases that impact prescribing behavior or clinician’s treatment decisions? Modernized ICH E8(R1) has defined clinical trial quality (CTQ) factors as:

  • Protection of study subjects
  • Integrity of data
  • Reliability of results
  • Ability of studies to meet objectives

In other words what if our truth is false! Not only are your analytics wrong but you then fail to maintain the highest accepted standards for your trial data.

One answer is to not rely on a single source of truth. There are a couple of things that we can do to address that:

  • Incorporate sources of aggregated historical data, available from any number of vendors such as CiteLine and Cortellis™. This will enable us to double-check what we see from a given site’s EHR.
  • The second and more obvious method is to compare site data that are using different EHR tools. This would be a safety-check dashboard that would seek large deviations between groups of sites.
  • Don’t be too willing to dismiss outliers or inexplicable spikes in the view as data anomalies or otherwise bugs in the data. (In practice we have seen that our data integration rules engine has intercepted many of these data glitches long before they appear in the very dashboard that is shaping our only view of our clinical trial.)
  • Validate applications that collect and curate the source data for the purpose of patient safety. This demands that clinical decision support algorithms be based on unbiased and trustworthy evidence that promotes improvements in care.

If we are unaware of the pitfalls of source data, we are running the risk of unquestioningly trusting dashboards that are showing us biased data.

Here’s some advice on metrics and dashboards to help us avoid such source biases:

  • Pay careful attention to what we need to measure and then measure that. Having 50 focused dashboards is better than 300 that are not viewed and monitored. Then, take all measures to make sure that the sources feeding those key metrics are controlled in a validated environment.
  • Be sure that each metric combines data from different sources so that you are not falling victim to manipulated data streams. It’s best to choose among platforms that easily allow you to combine multiple disparate sources into a single cohesive feed.
  • Have comparison side-by-side benchmarks from commercial data vendors that are from independently curated sources. For this one, we must beware of not relying on any many of the “free” sources that allow sponsored data.
  • Finally, consider using a data broker that operates independent of the data sources and data analytics tools and use it to verify the authenticity of the data before it makes it into analytics framework. The data broker will need to support Rules or ML based capability to validate the data as it comes in.

It’s always a good time to be good stewards of data and ultimately patient safety. But in the current clinical climate, there’s never been a better time to focus on best practices and ensure data integrity with transparency.

About Adaptive Clinical

Adaptive Clinical Systems® created the only proven technology platform that lets you quickly go from integration to interoperability across all eClinical tools, leveraging a re-usable connector library and intelligent middleware. Our clinical rules engine not only moves, but can also transform data, all on a fully validated platform compliant with CFR 21 Part 11.  The proven cloud-based Adaptive eClinical Bus® solution unifies disparate data. It helps increase efficiency, enhance collaboration, and improve trial performance – and is modernizing data flow from any data source, now and in the future. To unify your data, go to adaptive-clinical.com, email [email protected] or call 856-452-0864.

* https://ehrintelligence.com/news/the-deadly-consequences-of-ehr-clinical-decision-support-tools

1 Parab, A.A., Mehta, P., Vattikola, A. et al. Accelerating the Adoption of eSource in Clinical Research: A Transcelerate Point of View. Ther Innov Regul Sci (2020). https://doi.org/10.1007/s43441-020-00138-y