Monitors pore over source documents to confirm that the raw data can be retrieved in its original form and has been recorded in a logical and consistent manner. They want to see that data can be linked to an identifiable study subject and that data can be attributed to the person at the study site who collected and/or recorded it. Monitors make sure that information in the source documents is accurately conveyed to the sponsor in the case report form. Until recently, most source data verification was performed by comparing paper source documents to paper case report forms (CRFs). But the paper era is rapidly ending.

Today, when monitors visit research sites at hospitals, clinics, research organizations, and private practices they are as likely to encounter electronic record-keeping and electronic data capture systems as they are to find paper. Principal investigators, study coordinators, and research subjects are all clicking, beaming, scanning, and emailing. Many hospitals now scan a study subject’s medical record into an imaging system and shred the original paper. Physicians, nurses, and other research personnel enter subject assessment data directly into bedside computers or handheld devices. Automated instrument recordings of physiologic measurements (temperature, blood pressure, volume measurements, flow rates, and others) become clinical trial source data.

Laboratory values are held in computerized systems at both research sites and central laboratories with which sponsors have contracts. Study subjects enter diary information into handheld computers, and the data is uploaded into a central system at scheduled study visits. Site personnel and research subjects enter data directly into Web-based data collection systems. Investigational pharmacists use computers to record treatment assignments, and in preparation and labeling of the study drug. Research personnel use a myriad of software products to record subject encounters and visit schedules. Clinical research coordinators make study tools and trackers from commercially available software products. The list is endless, and ever-increasing.

Often, a research subject’s source data is derived from multiple electronic systems. In a recent review of a subject’s medical record for an inpatient trial, I counted six different computerized systems that contributed source data for the trial, but institution personnel described their clinical milieu as “not computerized yet.”

In August of 1997, 21 CFR 11 (FDA’s Electronic Record, Electronic Signatures regulation) became effective. It describes the technical and procedural requirements that must be met when electronic records and/or signatures are used in lieu of paper records and signatures. The regulation works in conjunction with other FDA regulations that require record-keeping. Those laws and regulations (“predicate rules”) establish requirements for record content, signatures, and retention.² In short, 21 CFR 11 requires that computer systems and any electronic records that are created, modified, maintained, archived, retrieved, or transmitted under any records requirement set forth by FDA regulations:

• be validated.
• have the ability to generate accurate and complete copies in human-readable and electronic form, suitable for review by the agency.
• limit access to authorized individuals, trained in the use of the system.
• provide written policies to ensure user accountability for actions initiated under the user’s electronic signature.
• be protected and retrievable throughout the record-retention period.
• use computer-generated, time-stamped audit trails.
• control access to system operation and maintenance documentation.
• encrypt records, or otherwise protect record authenticity, integrity, and confidentiality if system access is “open.”
• meet specific requirements for electronic signature components and controls.³

Following an initial grace period, FDA began enforcing the regulation; the evidence is in recent 483s and Warning Letters that contain 21 CFR 11 citations. Although the vast majority of these citations occurred at pharmaceutical sponsors and manufacturing facilities, investigator site systems and the records they generate are subject to regulatory scrutiny under the regulation, and are likely to be inspected with increasing frequency in the future.

Thus, today’s monitors continue to perform source data verification as before, but in a whole new technological arena that currently seems to raise more questions than answers. As the answers emerge, monitors stand in a unique position; they are objective representatives of sponsors, and they are closest to the electronic source data throughout the duration of the trial. The monitor is likely to be the first to detect potential 21 CFR 11 compliance issues.

Compliance assessment during site selection
Initial 21 CFR 11 compliance assessments of a site’s systems should begin during the site selection process. Most research sites continue to be selected for participation in a given trial based on a two- to four-hour visit to the site by a representative of the sponsor, CRO, or SMO. (Sometimes the person that conducts the site evaluation and selection also functions as the site monitor or CRA. For purposes of this article, we assume that they are two separate persons.) It is important that initial questions about system security, SOPs, validation, training, and other factors be asked and answered at this meeting.

Some sponsors send a computer compliance survey or questionnaire to an investigator before the site evaluation visit so that the answers can be reviewed and clarified by their internal computer quality group. The sponsors then consider compliance issues before determining investigator participation. Sponsors may schedule an on-site computer compliance assessment by a qualified person, especially if particular concerns exist. And sponsors may insist on the use of a paper system if the compliance issues cannot be corrected in a reasonable time. These cautious evaluations seem wise and are helping to raise awareness of the 21 CFR 11 regulations on the part of investigators and site staff.4 Still, one wonders about the feasibility of conducting on-site system evaluations for research as general practice. For a large, multicenter trial, the cost of these visits alone would seem prohibitive.

So many, if not most, sites continue to be selected after the traditional two- to four-hour visit. During those visits, many agenda items—protocol and budget discussion, assessment of required patient population, tour of the facility, and visit to the pharmacy—compete for the time a busy researcher has allotted to the visit. At the time of site evaluation, it is quite likely that the study protocol may not be finalized by the sponsor, and more likely that the CRF will not yet be in its final form. Without a final protocol or CRF at this point, it can be difficult to identify (and evaluate) the system(s) that will ultimately provide the source data. Although it seems possible to detect flagrant 21 CFR 11 issues during a site selection visit, this meeting may not allow enough time and detail to find more subtle, potentially serious system and user issues.

Site initiation and electronic source documents
The monitor’s first contact with the site is often at the site initiation visit, several months (and maybe longer) after the site selection visit. In the interim, many changes are possible with the investigator’s systems and staff. During the site initiation visit, further dialogue should take place surrounding electronic source documents. Good questions to ask include, “What part(s) of the CRF have an electronic source? How will I access the system? How do you access the system? How were you trained to use this system (run the tests, take the measurements)? How are changes to the data made in the system? Have you ever lost data in the system, or had a system crash?”

Monitors, be prepared for the answers! Among others, I have heard:

• “You can’t get on the computer. We’ll just print out everything for you.”
• “Here, you can just use my password to get on.”
• “We’re not computerized yet” (when multiple systems and e-source documents are evident).
• “I don’t use this system, so I don’t know. I just copy the values off the printouts into the CRF.”

Even at prestigious academic and research centers, it is not unusual to find, at best, only a nodding acquaintance with the 21 CFR 11 regulations. Monitors can spend a good deal of time explaining concepts as basic as electronic documents and passwords.

E-documents. If a document is created (modified, maintained, archived, retrieved, or transmitted) in electronic form, the electronic document is the source, and should be what the monitor views. Paper printouts lack important “behind-the-scenes” data (metadata). Explaining that FDA expects to be on the same technological plane as the investigator site (hence, the monitor should see what FDA will see) can help to ease initial resistance to having a monitor access the system.

Passwords. The monitor should access the system(s) under a temporary (auditor) password, or at least looking “over the shoulder” of a registered investigator site user logged on to the system under his/her own ID and password. Study coordinators must often contact a site’s system administrator to obtain a temporary password. It is wise for the monitor to discuss this with the coordinator or investigator before they actually need to view records, because getting a password can take time and effort on the part of the research staff.

Once the monitor has access to electronic source data, he or she typically receives a “crash course” in system navigation, usually from the study coordinator. From that point forward, the monitor remains in close contact with site systems, electronic data, and site users throughout the duration of the trial.

Monitors spend many hours at investigative sites. They often work physically close to where data is being collected, and sometimes may even observe electronic data collection in progress. Monitors can become aware of site operational issues, including frustrated, overworked, and even angry coordinators, investigators, and ancillary staff. They may sense an impending resignation or political situation that could cloud judgment, or adversely motivate someone with access to electronic data. For the monitor, a sense of the site milieu and the ability to access and review the electronic data are a strong pair in detecting potential serious issues—no advanced computer skills required!

A need to understand the regulations
Malintent aside, at many investigator sites a large obstacle to 21 CFR 11 compliance seems to center upon a lack of awareness and understanding of the regulations. Broaching the subject of these laws at investigator sites generally results in blank stares. The existence and ramifications of the regulations don’t seem to be abundantly clear, even to educated and highly skilled personnel. I have, for example, witnessed one coordinator innocently “take over” for another coordinator during a lengthy electronic data collection session (without logging off, then back on) so that the first could go to lunch. Most medical personnel clearly understand the need for attribution in handwritten charting—they wouldn’t sign someone else’s progress note—but when a computer is involved, somehow all of that can be forgotten.

Not understanding can sometimes lead to resistance to questions or finding out the answers. When discussing electronic record-keeping issues, more than once I’ve heard, “We shouldn’t have to deal with this—that’s what IT is for.” Data is often collected using systems for which the investigator has no control of upkeep, training, back-up, or other services. Computerized systems, especially in institutions, often serve more than research purposes alone. Understandably, physicians who rely on their institution’s systems to conduct their research have difficulty being held accountable for systems not under their direct control.

A monitor’s lack of awareness of the Electronic Records, Electronic Signatures regulations and associated FDA guidance is also a significant concern. Taking the time to access and navigate a site’s computer system(s) certainly takes more time and initiative, and can be more cumbersome than looking at paper. Monitors who lack confidence in their own computer skills—or are pressed for time—may find that a pile of printouts provides a much greater comfort level.

Monitors generally have an excellent understanding of original paper source documents, but often have a poor grasp of what defines an electronic source. Additionally, many monitors fail to identify all the different electronic systems in use at a site, all of which can potentially contribute data to a given research subject’s case report form.

How should a monitor approach multiple systems, given that source data verification is but a portion of what needs to be checked in assessing proper trial conduct, and given that there are only so many hours within which to conduct a site visit?

Meeting the challenges to monitors
FDA’s Electronic Record, Electronic Signatures regulation presents significant challenges to traditional data monitoring. Armed with knowledge of the regulations, and supported by other agents of the sponsor, monitors can still be instrumental and effective in detecting potential compliance issues. Consider the following:

• In reviewing electronic data, a monitor determines her level of access to be greater than read-only.
• A monitor observes data being collected by means of a laptop computer that uses a program designed by the investigator. The laptop is left open and unattended in the presence of substance-addicted subjects.
• A monitor reviews computer printouts, on which all of the dates have reverted to the year 1904.
• The monitor ascertains that a computer upgrade performed on a hospital system renders previously restricted pharmacy screens (detailing blinded treatment assignments) unrestricted for many hospital personnel to view.

No advanced computer knowledge is required to understand the potential impact on data integrity revealed by these real-life examples. Additionally, even if site systems are deemed compliant at trial outset, human actions can contribute to compliance issues at any point in the trial.

Encountering situations such as these, monitors are likely to sound the first compliance alarm. Thereafter, resolution will certainly require the efforts of the team. The team may include sponsor IT experts for help with technical issues and sponsor QA experts to assist in properly identifying and escalating 21 CFR 11 issues, and for help in formulating corrective action. IT and QA personnel can help develop and implement comprehensive, systematic checklists and tools to ascertain system compliance during site selection and throughout trial duration, making sure that the right questions are asked—and that site evaluators and monitors know what to do with the answers.

Keenly aware of their regulated business, sponsors also seem likely to spearhead the 21 CFR 11 education efforts needed to enlighten their own research personnel and those at investigative sites. A good place to start could be training sessions, at start-up and other investigator meetings, focused on the Electronic Records, Electronic Signatures regulation.

In the midst of exponential increases in the use of electronic systems to collect and process data for clinical trials, 21 CFR 11 adds a whole new dimension to monitors’ responsibilities for “ensuring that the trial is conducted and documented properly.”1 Armed with knowledge of the regulations, proper tools, and back-up from knowledgeable sponsor experts, monitors can be key in furthering awareness of the regulatory requirements at investigator sites—thus helping to ensure compliance throughout the course of the trial.

References
1. Guideline for Good Clinical Practice, International Conference on Harmonisation, Federal Register 60 (90) 25691–25709 (9 May 1997).

2. Food and Drug Administration, Compliance Program Guidance Manual, Bioresearch Monitoring, Chapter 48 (FDA, Rockville, MD, 23 February 2001).

3. Code of Federal Regulations, Title 21, Part 11, Section 11.1–11.300 (U.S. Government Printing Office, Washington, DC).

4. Marta Haley Fields, “The Implications of 21 CFR Part 11 for Clinical Research,” American Pharmaceutical Review, Spring, 2002, 110–115.

Additional reading
1. Food and Drug Administration, Guidance for Industry, Computerized Systems Used in Clinical Trials, April, 1999.

This article is based on the author’s presentation at the 37th Annual Meeting of the Drug Information Association (held in July 2001 in Denver, Colorado).