What Warning Letters Reveal about Data Integrity

By Regina Fullin, VP of RA/QA Consulting at Compliance Team

After Looking at Over 6 Months of Data, Here’s What Warning Letters Reveal about Data Integrity

What Warning Letters Reveal about Data Integrity – In 2015, the FDA announced that it would no longer enforce 21 CFR Part 11, and was planning on enforcing “data integrity.”  In the industry, we didn’t really know what that meant, and hypothesized, based on the timing of the announcement, that this “data integrity” announcement was related to Executive Order 13636—Improving Critical Infrastructure Cybersecurity.  This executive order, which was signed in 2013, led to the believe that ”Data Integrity” was related to preventing the hackability of medical devices in hospitals and securing IT data at pharmaceutical and medical device manufacturers from destructive hackers, motivated by monetary gain or political subversion.

The 2015 hypothesis was only partially correct.  The overall picture of how the FDA enforces “Data Integrity” includes just a handful of violations for security of the IT infrastructure.  As many have observed, a great deal of the Data Integrity buzz revolved around laboratory instrumentation, and consistency of laboratory results. Many believe that Data Integrity issues are concentrated overseas, where data integrity concepts had not yet have become priority.

Data was collected on July 19, 2017 to test the hypothesis.  On that date, I searched all FDA Warning Letters to find the ones related to data integrity.  To narrow the search, only manufacturers of pharmaceutical and medical device product were included in the dataset, for Warning Letters posted from January 1, 2017 to July 19, 2017.  All observations were read, and a checklist was used to record findings. The Pareto chart below shows geographical findings from the search:

Surprisingly, China and the United States were tied for the number of warning letters related to Data Integrity, with India a close third.  Only one posted Warning Letter came from the European Union. The conclusion is that Data Integrity is not exclusively a foreign manufacturer problem.  Data Integrity is a worldwide problem.

The next exercise was to zero in on the specific Warning Letter findings, and identify what kinds of observations led to the Data Integrity finding.  The numbers will not match the numbers from the previous chart because the FDA would often cite several examples of Data Integrity violations in a Warning Letter. For each example, I would categorize it in a more general bucket.

The major issue with laboratory data surrounded a lax attitude toward the analyst’s ability to change data.  This broader category covers some of the other examples, such as Selectively Using Data, which a Testing into Compliance issue, Failure to Lock Data from Editing, which is when, upon inspection of audit trails on an instrument, they find that the data are overwritten.  One such finding determined that analysts were setting back the system clock so they could overwrite results with the same timestamp as the original, then setting the system clock back to the accurate time after altering the data.  The FDA does not tolerate such deception.

Nor does the FDA tolerate any attempts to hide data.  Two firms refused to provide product information that the FDA believed it had the rightful authority to inspect.  Two firms received citations for unexplained data loss.  The FDA’s stance for these situations is that a data loss event should trigger an investigation to identify the root cause of the data loss.  One firm, in this example, suggested that it might have been an electronic power issue, without an investigation to back up its claims.  In the hidden data department, the FDA inspected three separate firms in which discarded data were found in trash receptacles.  First, making one wonder, why a company would be so naïve as to think that the FDA wouldn’t find out, and second, why a company would waste an opportunity to use data that could yield improvement information that might help them become more profitable.

Some instances of data integrity violations appeared to be inadvertent, but, the FDA treated these instances with healthy skepticism.  For example, at one firm, the FDA found some test results that were invalidated due to technician error, but an investigation, stating specifically how the test error occurred, was not documented.  Another example showed manual data transcriptions that were inaccurately copied.  The manual transcriptions were sufficiently significant to change the results.

I did not include in the above chart the one-off situations described in the warning letters, but the list below names some favorites:

• Sharing usernames and passwords on a generic user accounts
• Batch records with blanks not filled in
• Omission of significant data necessary for Management to make sound decisions during Management review
• Inconsistent sample naming conventions, that created confusion about sample identity
• Data in batch records not directly recorded at the time it was generated
• Not enabling the audit trail function on an instrument that has this feature
• Device cybersecurity issues

Alas, one instance of a data integrity item related to device cyber security!  This proves that the FDA is concerned about cyber security, just not as much as the 2013 Executive order suggested.

As they say, “In God we trust; all others bring data.”  Because we use data to learn more about the process, it follows that we better have good data so our decisions are based on fact, not hope or conjecture.  The only hope is that these data are useful to your job, and may the data you generate in the future be of superior integrity!