QUALITY ASSURANCE / QUALITY CONTROL PROCEDURES FOR HEI STUDIES
PART 1. GENERAL QUALITY ASSURANCE / QUALITY CONTROL PROCEDURES
1.1. POLICY STATEMENT
The mission of the Health Effects Institute (HEI) is to provide high-quality, impartial, relevant scientific information on the health effects of pollutants from motor vehicles and other sources in the environment. All funded HEI studies are expected to have adequate QA/QC procedures in place to ensure that the data are collected according to a written protocol and Standard Operating Procedures (SOPs) and are traceable. The QA/QC guidelines provided in this appendix apply to all HEI-funded studies. For studies that involve human subjects and some animal studies of regulatory significance, HEI will implement Special Quality Assurance Procedures (described in Part II) that include an external audit by an HEI selected audit team. HEI will inform the investigator after approval of the study whether the Special QA procedures will apply to his/her study.
1.2. QUALITY ASSURANCE / QUALITY CONTROL COMPONENTS
QA procedures begin with the planning phase of the raw data collection and follow the subsequent transformations of the data. Generally, HEI requires that the investigators:
· Use a written protocol
· Use written standard operating procedures
· Involve qualified personnel
· Maintain written records
· Use appropriate data processing techniques
· Use quality control procedures for all data collected
A. A written research protocol defines the experimental objectives, research strategy, and methodologies to be used. The protocol will be sufficiently complete and detailed as to ensure that the data collected are of known and documented quality. It will include, as applicable:
1. Name of Principal Investigator and co-investigators
2. Study objectives
3. Scientific background and rationale
4. Anticipated significance of study results
5. Description of all experiments to be conducted with reference to a particular standard operating procedure when appropriate (see Section B)
6. Methods of data processing (see Section E)
7. Internal quality control procedures to be used (see Section F)
8. Safety precautions needed
9. Plans for archiving the completed project, including the anticipated address and physical location for storage of all raw data, records, electronic media, reports, SOPs, and any specimens that are expected to be retained
For studies involving human subjects, the protocol should also contain:
10. Subject selection procedures to be used, including inclusion and exclusion criteria (when applicable)
12. Procedures used to maintain subject confidentiality
13. Copy of the blank form used to obtain Informed Consent from subjects
14. IRB approval
The protocol may be amended as necessary to accommodate changes to the experimental design. Any changes to the original protocol considering items 1 through 14 shall be made in writing by preparing an amendment to the protocol that is signed and dated by the Principal Investigator. See also Section III, Roles of Institutions and Individuals in Achieving Quality Assurance, below. All amendments must be approved by HEI.
B. Written standard operating procedures will be used to document all routine, critical experimental procedures and measurement techniques for which variability must be minimized. Critical experimental procedures are those procedures that result in the acquisition of experimental samples or data used to draw scientific conclusions. Generally, SOPs cover procedures that are done routinely over time by the same person or by different individuals to minimize procedural variation.
Standard operating procedures will be developed by individuals knowledgeable of the specific procedures. They will describe what, when, where, how, and why in a stepwise manner. They will be sufficiently complete and detailed to ensure that the data collected are of known and documented quality and integrity and are generated to meet measurement objectives such that there is a minimum loss of data due to out-of-control conditions. Routine quality control procedures should be covered by an SOP. Other items covered by an SOP might include: use and calibration of laboratory instruments, chemical sampling and analyses, preventive maintenance, data handling, maintenance and storage, etc.
Standard operating procedures will be uniquely identified and dated, and updated as needed. Copies of all current SOPs should be readily available for reference by the study team or by a third party, as needed. All SOPs that have been superseded will be maintained in a historical file. Deviations from SOPs should be documented.
C. Qualified personnel will conduct the proposed research. The qualifications of all participating individuals, and any training they receive for the conduct of the study along with prior experience, should be documented in resumes that will be maintained as a part of the permanent record of the project.
D. Recordkeeping procedures. Written records will be maintained to document all aspects of the research effort. This shall include the use of bound notebooks, standard forms, and computer input and output. All entries shall be made in indelible ink. The entries should be dated and signed or initialed by the individual making the entry. Notebook entries shall be made in chronological order. If a blank space is left between entries, it shall be crossed-hatched to render it unusable. Entries shall not be erased or otherwise obscured. If any entry is to be changed because it is in error or for any other reason, a single line will be drawn through the entry and a correction made in the margin. The altered entry shall carry an explanation of the reason for the change, the date of the change, and the initials or the signature of the individual making the change.
The Principal Investigator for the project shall periodically review the records to verify their completeness and accuracy. This review shall be documented by the Principal Investigator signing and dating the reviewed record.
E. Data processing procedures should be documented. Data processing includes all manipulations performed on raw (i.e. “as collected”) information, validation, storage, transfer, reduction, and statistical analysis.
Data analysis frequently includes computation of summary statistics and their standard errors, confidence intervals, tests of hypotheses relative to the parameters, and model validation (goodness of fit tests). Specific statistical procedures, programs, and code to be used should be documented either in the protocol or in a separate document. HEI staff may require submissions of these procedures during the course of the study or the review of the final reports.
F. Quality control procedures should be documented for all data collected, i.e. procedures the investigator will use for ensuring the quality of the data during the data collection, sample analyses, and data processing.
1.3. ROLES OF INSTITUTIONS AND INDIVIDUALS IN ACHIEVING QUALITY ASSURANCE
The Principal Investigator and his/her institution have the primary responsibility for the preparation of the protocol, and all standard operating procedures and shall review and approve them by signing them. In addition, the Principal Investigator has the responsibility to prepare a Quality Assurance Plan, and submit it to HEI within the first months of the study (but no later than at the time of submission of the Year 1, 5-month progress report). HEI will work with the investigators to ensure that the QA plan is adequate and consistent with the agreed upon Statement of Work.
The QA plan shall include:
• The protocol, including the data analysis methods that will be used (see below)
• A list of SOPs
• A list of qualified personnel
• Record keeping procedures (how data will be collected, backed-up, collated, transferred, and stored)
• Documented data processing techniques
• Quality control procedures for all data collected
The protocol will be reviewed and approved by HEI. In many cases, the original Project Plan submitted with the HEI application can serve as the protocol, with added information as recommended by the HEI staff or the Research Committee. In some cases HEI may ask a group of investigators to work together to harmonize their study design and methods and develop a common or comparable protocol. Subsequent modifications to the protocol shall be submitted to HEI in the form of written amendments. All amendments are subject to HEI approval before they can be implemented.
The Principal Investigator has the responsibility for the actual conduct of the research, adhering to the protocol and SOPs. He or she has the primary responsibility of managing all aspects of data collection, validation, storage, transfer, reduction, and analysis. The Principal Investigator also has the responsibility for assuring that the research is conducted with qualified personnel and in accordance with this quality assurance plan. Technical and supporting personnel should have a detailed knowledge of the SOPs used in the conduct of their research activities.
HEI reserves the right to conduct a QA audit of an HEI-funded study if there are reasons to suspect that adequate procedures are not in place.
PART 2. SPECIAL QA/QC PROCEDURES
HEI uses third-party quality assurance (QA) procedures for most research projects involving human subjects and other projects with a high potential for use in regulatory decisions. The special procedures augment the QA/QC procedures applied to all HEI studies (described above in Part 1) and assure that data are collected under defined conditions and are reliable and traceable. Accurate scientific conclusions are dependent on the validity of the underlying data and the precision with which they are reported. If there is a QA program in place at the institute at which the research is being conducted, then HEI will assess its adequacy and modify its QA procedures as necessary.
2.1 THIRD-PARTY QA OVERSIGHT
HEI will generally engage one or more qualified individuals to serve as Quality Assurance consultants for the project. This individual will report to HEI’s Director of Science and be responsible for overseeing the implementation of this Quality Assurance plan. The QA consultant will review the (draft) protocol for adherence to the QA requirements and notify HEI staff if modifications are necessary. The QA consultant shall maintain signed copies of the protocol and all SOPs.
The QA consultant may conduct periodic audits of the research while in progress and when it is completed to ascertain compliance with the HEI’s special QA procedures. These audits shall include such matters as review of research procedures, notebooks, data forms, and data management activities. The audit shall be performed using the audit framework presented in the US Environmental Protection Agency’s Guidance on Technical Audits and Related Assessment for Environmental Data Operations (EPA QA/G-7 2000, available at www.epa.gov/quality/qs-docs/g7-final.pdf).
2.2. ELEMENTS OF A QA AUDIT
The key elements of a QA audit include:
1. Opening Meeting with the audit team, the Principal Investigator, and key project personnel.
2. Observation of the project activities being performed by the personnel who regularly perform such activities.
3. Review of written documents, such as QA Plans, calibration readouts, process data readouts, sample logs, custody papers, instrument logs, printouts from data spreadsheets, and maintenance notebooks (such records may be in electronic form).
4. Interviews with the project personnel to verify the results of observation and to clarify issues noted during document review.
5. Objective Evidence Compilation, such as copies of notebook pages, logs, instrument and model outputs, and QC charts.
6. Closing Meeting, during which the QA consultant provides a verbal summary to the Principal Investigator of significant findings that need to be addressed.
7. QA Audit Report. The QA consultant prepares a “Business Confidential” report of the audit. The report shall detail the nature of the audit, significant findings, and any requirements for corrective action(s). The audit report shall be provided to the HEI Director of Science, who will then transmit it to the HEI project manager for transmission to and discussion with the Principal Investigator. If corrective action is required, the Principal Investigator will ensure that such action is taken and return the summary to the HEI project manager with a copy to the QA consultant noting the action(s) taken. All copies of the audit report are to be marked as “Business Confidential” and are to be destroyed after use or maintained in a file separate from other records of the project. These audit reports are only to be released to people directly involved in management of the projects. To give these reports to people who are not directly involved violates the confidential nature of the audits and potentially reduce the degree of candor required in communications within the project on matters requiring corrective action. The QA consultant shall maintain a log of all audits indicating for each audit: the date conducted, participating personnel, and the nature of the audit.
2.3. TIMING OF QA AUDIT
While the exact timing of the audits varies across studies, the followed guidelines should be followed when defining the general plan and scope of the QA oversight for a study:
A. Audits during the course of the research period
1. Clinical studies
One QA audit should be conducted at the beginning of Year 1 to ensure that all SOPs are in place, the protocol is followed, and a data management plan is in place. This audit should occur fairly early in the study so that problems, if found, can be remedied before too many subjects have been studied.
One QA audit during Year 2 to audit a subset of the data collected to verify that the data management procedures are adequately implemented and the data collected are traceable, the informed consents are signed, and the protocol is followed consistently. This audit is optional and would depend on the outcome of the initial audit.
2. Epidemiologic, statistical, and other studies
One audit at the end of Year 1 or during Year 2 to ensure that data collection is done according to the protocol, the data collected are traceable, and a data management plan is in place. If problems are encountered and not addressed adequately, a follow-up visit may be needed.
B. Audit of the final report
Unless there are specific reasons to expedite the review of a final report, the timing of the final report QA audit will be decided during the first discussion of the draft final report by the Review Committee. The following guidelines will be followed:
1. If the Review Committee thinks that the draft final report does not require additional analyses, then a QA audit of the draft report should be scheduled immediately so the investigators can address all issues raised by the auditors in the revised report.
2. If the Review Committee thinks that the draft final report requires substantive changes and/or (partial) reanalysis of the data, the QA audit should be conducted on the revised final report, as soon as it is received.
3. Regardless of the timing of the final report audit, the auditors should always be provided with the final “accepted” version of the report and asked to review it before issuing the final QA Statement, which will be printed in the final, published report.
Quality Assurance and Quality Control (pdf of the text on this page)