Incidence and Types of Adverse Events and Negligent Care in Utah and Colorado Author(s): Eric J. Thomas, David M. Studdert, Helen R. Burstin, E. John Orav, Timothy Zeena, Elliott J. Williams, K. Mason Howard, Paul C. Weiler, Troyen A. Brennan Reviewed work(s): Source: Medical Care, Vol. 38, No. 3 (Mar., 2000), pp. 261-271 Published by: Lippincott Williams & Wilkins Stable URL: http://www.jstor.org/stable/3767190 . Accessed: 01/02/2012 14:30 Your use of the JSTOR archive indicates your acceptance of the Terms & Conditions of Use, available at . http://www.jstor.org/page/info/about/policies/terms.jsp JSTOR is a not-for-profit service that helps scholars, researchers, and students discover, use, and build upon a wide range of content in a trusted digital archive. We use information technology and tools to increase productivity and facilitate new forms of scholarship. For more information about JSTOR, please contact support@jstor.org. Lippincott Williams & Wilkins is collaborating with JSTOR to digitize, preserve and extend access to Medical Care. http://www.jstor.org MEDICAL CARE Volume 38, Number 3, pp 261-271 02000 Lippincott Williams & Wilkins, Inc. Incidenceand Types of Adverse Eventsand NegligentCare in Utahand Colorado ERIC J. THOMAS, MD, MPH,* DAVIDM. STUDDERT,LLB,ScD,*t HELENR. BURSTIN, MD, MPH,* E. JOHNORAV,PHD,* TIMOTHY J. ZEENA, BS,*t ELLIOTT WILLIAMS, JD,t K. MASON HOWARD,MD,? PAUL C. WEILER,LLM,II TROYENA. BRENNAN,MD, JD, MPH*t AND BACKGROUND. ongoing debate on the inThe cidence and types of iatrogenic injuries in American hospitals has been informed primarily by the Harvard Medical Practice Study, which analyzed hospitalizations in New York in 1984. The generalizability of these findings is unknown and has been questioned by other studies. We used methods similar to the OBJECTIVE. Harvard Medical Practice Study to estimate the incidence and types of adverse events and negligent adverse events in Utah and Colorado in 1992. DESIGNAND SUBJECTS. selected a repreWe sentative sample of hospitals from Utah and Colorado and then randomly sampled 15,000 nonpsychiatric 1992 discharges. Each record was screened by a trained nurse-reviewer for 1 of 18 criteria associated with adverse events. If >1 criteria were present, the record was reviewed by a trained physician to determine whether an adverse event or negligent adverse event occurred and to classify the type of adverse event. MEASURES. The measures were adverse events and negligent adverse events. RESULTS. Adverse events occurred in 2.9?0.2% (mean?SD) of hospitalizations in each state. In Utah, 32.6?4% of adverse events were due to negligence; in Colorado, 27.4?2.4%. Death occurred in 6.6?1.2% of adverse events and 8.8?2.5% of negligent adverse events. Operative adverse events comprised 44.9% of all adverse events; 16.9% were negligent, and 16.6% resulted in permanent disability. Adverse drug events were the leading cause of nonoperative adverse events (19.3% of all adverse events; 35.1% were negligent, and 9.7% caused permanent disability). Most adverse events were attributed to surgeons (46.1%, 22.3% negligent) and internists (23.2%, 44.9% negligent). CONCLUSIONS. incidence and types of adThe verse events in Utah and Colorado in 1992 were similar to those in New York State in 1984. Iatrogenic injury continues to be a significant public health problem. Improving systems of surgical care and drug delivery could substantially reduce the burden of iatrogenic injury. Key words: adverse events; medical malpractice; negligent care; error; patient safety. (Med Care 2000;38:261-271) From the Division of General Medicine, Department of Medicine, Brigham and Women's Hospital, Boston, Massachusetts. IlFromthe Harvard Law School, Cambridge, Massachusetts. tFrom the Department of Health Policy and Management, Harvard School of Public Health, Boston, Massachusetts. fFrom the Law Offices of Williams and Hunt, Salt Lake City, Utah. ?From the Colorado Physicians Insurance Company, Denver, Colorado. Funding was provided by the Robert Wood Johnson Foundation, Princeton, New Jersey. Address correspondence to: Eric J. Thomas, MD, MPH, University of Texas, Houston Medical School, 6431 Fannin MSB 1.122, Houston TX 77030. E-mail: ethomas@heart.med.uth.tmc.edu Received January 11, 1999; initial review completed March 22, 1999; accepted October 11, 1999. 261 THOMAS ET AL MEDICAL CARE In 1991, the Harvard Medical Practice Study investigators reported that adverse events occurred in 3.7% of hospitalizations in New Yorkin 1984 and that 28% of these were due to negligence.1 Commentators have extrapolated from this study to suggest that 180,000 people in the United States die each year from iatrogenic injury,2 and the study results encouraged substantial research on how to prevent adverse drug events.3-11 See p 247 and p 250 Such generalization of the Harvard Medical Practice Study may be inappropriate because the findings were limited to 1 state and 1 year. And although the New York study investigators relied heavily on a previous study undertaken by the Califoria Medical Association in 1976,12 their methodology for identifying adverse events has not been replicated in another large-scale study in the United States. Furthermore, a recent population-based study from Australia that used the HarvardMedical Practice Study methods13 and a study from a Chicago teaching hospital that used observational methods14 both found considerably higher rates of medical injury than the Harvard Medical Practice Study. The study reported here was designed in part to determine whether the New York findings are similar to those of other states in different time periods and to gather data to help focus research to prevent adverse events. (for profit, nonprofit, or goverment).16'17 Nonteaching hospitals had no interns or residents and were not affiliated with the Council of Teaching Hospitals; minor teaching hospitals had residency training programs but were not affiliated with the Council of Teaching Hospitals; and major teaching hospitals had residency programs and were members of the Council of Teaching Hospitals. Next, we created strata representing all possible combinations of these characteristics, and every hospital in each state was assigned to its appropriate stratum (11 strata in Utah and 15 in Colorado). At least 1 hospital from each stratum was then invited to participate on the basis of their potential importance to a no-fault medical malpractice demonstration project. Hospitals did not volunteer, none of the invited hospitals refused to participate, and the investigators had no knowledge of the adverse event rates of the hospitals. Psychiatric, rehabilitation, and drug/alcohol treatment diagnosis-related groups and hospitals that exclusively provide these services were excluded, as were Veterans Administration Hospitals. Each participating hospital approved the study, as did the human subjects committee at the Harvard School of Public Health. We then randomly sampled a total of 5,000 calendar year 1992 discharges in Utah and 10,000 in Colorado from comprehensive state data sets. The number of records sampled in each hospital was proportional to the number of discharges at each hospital relative to the total discharges of all hospitals in the study. Methods Our methods were based on those used in the HarvardMedical Practice Study except that (1) we used 1 physician-reviewer per case instead of 2, (2) our nurse- and physician-reviewer training process was more standardized, and (3) we conducted more extensive quality control of the chart review process because of marked differences in physician-reviewer adverse event detection rates found in the Harvard Medical Practice Study.15 Sampling Strategy First,we characterizedall hospitals in each state according to size (<8,000 or -8,000 discharges per year), location (urban or rural),teaching status (major, minor, or nonteaching), and ownership 262 Reviewer Training Nurses and physicians who had reviewed medical records for the peer review organizations in each state were asked to review records for this study. The nurses were all female, and the physicians were predominantly male and either board certified family practitioners or general internists. Each group underwent 1 day of training by 2 investigators. The data forms were reviewed and explained, terms were defined (including the definition of an adverse event), and examples of adverse events were discussed. The reviewers were compensated for their service at an hourly rate set by the peer review organizations in each state. Vol. 38, No. 3 INCIDENCE AND TYPESOF ADVERSEEVENTS Record Review Definition of Variables Nurses initially reviewed all sampled medical records for 1 of 18 criteria associated with an adverse event (see the Appendix). Records screened positive (and the completed nurse review form) were referred to a physician trained to use the adverse event analysis form.1 This form has a series of questions to facilitate reliable detection of adverse events and negligence. The physicianreviewers then described the event and graded their confidence that an adverse event was due to medical management and not the disease process. If the reviewer detected an adverse event, the adverse event analysis form directed that same reviewer to make a subsequent determination about whether the event was due to negligence. Each reviewer also classified the type of event and where it occurred. Because judgments about adverse events may be complex, we used a 6-point confidence scale: 1 = little or no evidence that medical management caused the event, 2 = slight evidence, 3 = not quite likely (<50:50 but close call), 4 = more likely than not (>50:50 but close call), 5 = strong evidence, and 6 = virtually certain evidence. We required a confidence score of -4 for determination of an adverse event. The same scale was used to make judgments about negligence. Specialist consultation was available for the reviewing physicians if requested. The reliabilityand validity of this process were extensively studied We during the HarvardMedical Practice Study.18,19 used data forms identical to those used in New York except we added 2 screening criteria (numbers 17 and 18; see the Appendix). A recent analysis of the Harvard Medical Practice Study data15found that much of the disagreement between reviewers occurred among those who had markedly low and high adverse event detection rates. In this study, we addressed reliability in 4 ways. First, we standardized physician and nurse training by conducting 2 training sessions for each group (1 in each state), and the sessions were conducted by the same 2 investigators. (The Harvard Medical Practice Study had several training sessions conducted by multiple trainers.) Second, we conducted a quality control study to identify physicians with low and high adverse event detection rates. Third,we conducted a reliability study. And fourth, investigators eliminated false-positive and, when possible, falsenegative events. An adverse event was defined as an injury caused by medical management (rather than the disease process) that resulted in either a prolonged hospital stay or disability at discharge. In accordance with standard tort criteria, negligence was defined as care that fell below the standard expected of physicians in their community.20 The other core components of a common-law negligence determination were addressed separately: Causation and the existence of an injurywere built into the decision-making criteria for the adverse event judgment, whereas a dutiful relationship between the doctor and patient was assumed. We required a confidence score of -4 for determination of an adverse event or negligent adverse event. All adverse events that contributed to the annual incidence rate were analyzed for this report. As in the Harvard Medical Practice Study, we counted adverse events caused by medical management that occurred before the sampled hospitalizations that either were also the cause of the sampled hospitalizations or were discovered during the sampled hospitalizations and adverse events caused by medical management during the sampled hospitalizations that were also discovered during the hospitalizations. Adverse events discovered after discharge were not counted in determination of the annual incidence of adverse events. The patient characteristics were gender, race (white and nonwhite), age (-15 years, 16 to 44 years, 45 to 64 years, and >65 years), and payer (uninsured, Medicare, Medicaid, private/other,and managed care). Hospital characteristics are defined above. Disability ratings were first made by the physician-chart reviewers using the National Association of Insurance Commissioners severity of injury scale.21 Next, 2 study investigators (E.J.T. and T.A.B.) reviewed all available information about each adverse event and corrected disability scores in cases in which reviewers clearly misapplied criteria. These scores were then reviewed by 4 medical malpractice insurance claims adjusters from Utah and 6 from Colorado, and consensus was reached on a final disability score for each patient. All adjusters had prior training and experience applying these scores to medical injury cases as part of their responsibilities for the medical malpractice insurers who employed them. 263 THOMAS ET AL MEDICAL CARE Quality Control Statistical Analysis To identify physician-reviewers with an unacceptably high false-negative rate, a study investigator (E.J.T.) re-reviewed 50 randomly selected records of physician-reviewers whose adverse event detection rate was 2 SD below the mean for the group of reviewers in their respective state. If -10% of the records were classified as adverse events by the investigator, their reviews were substituted with re-reviews by a different reviewer who was masked to the purpose of the re-review. We focused on outlier physician-reviewers because this was found to be a problem with the study on which our methods were based.15 After completion of all reviews, 2 investigators (E.J.T.and T.A.B.) removed false-positive events by independently reviewing each adverse event and negligent adverse event to ensure that all the events fulfilled the study definition. In addition, the investigators reviewed the adverse events to determine whether any events not originally judged as negligent were the result of substandard care (ie, false-negative negligent adverse event judgments). Adverse event rates are reported as the percentage of hospitalizations during which adverse events were detected. As in previous studies, negligent adverse event rates are reported as the percentage of adverse events judged negligent.1 If >1 adverse event was detected during a hospitalization, only the event that caused the most disability was analyzed. Population estimates of adverse event rates and adverse events caused by negligence were calculated by weighting of the observed data. For each of the sample hospitals, weights were calculated by dividing the number of discharges in the stratum during 1992 by the total number of records reviewed in the stratum. Standard errors for the weighted rates were calculated by use of weighted logistic regression models. Reliabilityof Physician Judgments After completion of the review, we re-reviewed a random sample of 500 records referredby nurses to physicians, maintaining the original 2:1 ratio of Colorado to Utah records (167 from Utah and 333 from Colorado). Of these records, 400 were originally judged to have no adverse event, and 100 were originally judged to have an adverse event. This mix of records provided a sample with a similar proportion of adverse events as the original review. Six physicians in Colorado and 4 in Utah reviewed these records; they all reviewed records that they had not previously reviewed. As in the original review, investigators reviewed all adverse events detected during the re-review to eliminate any false-positive findings. The reliability of the nurse screening process was not evaluated because the same process was used in the Harvard Medical Practice Study and was found to be an adequate screening tool, with a sensitivity of 84% and negative predictive value of 92%.18 264 Results There were 90 hospitals in Colorado (71 eligible) and 53 hospitals in Utah (41 eligible) in 1992. Thirteen hospitals in Utah and 15 in Colorado participated in the study. There were 2 major teaching (1 from each state), 8 minor teaching (2 in Utah, 6 in Colorado), 18 nonteaching (10 in Utah, 8 in Colorado), 7 government-owned (2 in Utah, 5 in Colorado), 8 for-profit (4 in Utah, 4 in Colorado), 13 nonprofit (7 in Utah, 6 in Colorado), 20 urban (9 in Utah, 11 in Colorado), 8 rural (4 in Utah, 4 in Colorado), 15 small (7 in Utah, 8 in Colorado), and 13 large (6 in Utah, 7 in Colorado) hospitals. We reviewed 4,943 (98.9%) of the 5,000 sampled records in Utah and 9,757 (97.9%) of the 10,000 records sampled in Colorado (2.6% of all discharges in these states). Among these, nursereviewers referred 854 Utah records (17.3%) and 2,014 Colorado records (20.6%) to physicians for further review. Physicians reviewed 842 (98.6%) of the referredUtah records and 1,978 (98.2%) of the referred Colorado records (the Figure). The rest were classified as missing. Twenty-two physicians reviewed records for the study (16 in Utah, 6 in Colorado). Among the 2,820 total records referredby nurses and reviewed by physicians in both states, 21.1% of the Colorado records (range of detection rates for individual reviewers was 2.6+4.9% to 27.9+2.8%) and 20.1% of the Utah records (range of detection rates Vol.38, No. 3 INCIDENCE AND TYPESOF ADVERSEEVENTS for individual reviewers was 3.2+6.2% to 53.8?27%) had adverseevents. On the basis of the the qualitycontrolcriteria, reviewsof 1 physician in Utah and 2 in Colorado (n = 739 records) were discarded and redone by different reviewers. After these records were re-reviewed, the entire sample of adverse events was reviewed by investigators, and 13 false-positive findings were excluded. No false-negative negligent adverse events were detected. The demographic characteristicsof the sampled discharges were similar to the characteristicsof all discharges in each state (Table1). The mean age of the patients was 38.9 years; that of all patients discharged from Colorado and Utah hospitals was 38.2 years. The physician-reviewers detected 169 adverse events in Utah and 418 in Colorado during the initial review (the Figure). One hundred thirtyeight of the Utah adverse events and 337 of the Colorado adverse events that were the result of medical management before or during the sampled hospitalization and were discovered during that hospitalization were used to calculate annual incidence rates. Adverse events caused by medical management during the sampled hospitalization but discovered after discharge (n = 112) were not counted in the determination of the annual incidence of adverse events. Counting such adverse events would have resulted in an overestimate of the rate (given our inclusion of events caused by medical management before the sampled hospitalization) because they would logically be linked to a sampled hospitalization at some later time. When weighted to the population of each state, 5,614 adverse events were estimated to have occurred in Utah and 11,578 in Colorado in 1992. The annual adverse event incidence rate was 2.9?0.2% (mean? SD) of hospitalizations in each state. Among the adverse events detected and verified, Utah had an estimated 1,828 (32.6?4.0%) and Colorado had 3,179 (27.5?2.4%) caused by negligence. Of the 500 records chosen for re-review as part of the reliability study, 488 were located and reviewed (97 of 100 adverse events and 391 of 400 nonadverse events). The percent agreement for adverse events was 79%, and the Kwas 0.4 (95% CI, 0.3 to 0.5). Death followed 6.6?1.2% of adverse events and 8.8+2.5% of negligent adverse events (Table 2). Approximately 439 patients were estimated to have died as a result of negligent care in these states in 1992. Another 160 patients suffered grave or major disability as a result of negligent care. Types of Adverse Events Operative adverse events comprised nearly half (44.9%) of all adverse events; 16.9% of them were negligent and 16.6% resulted in permanent disability (Table3). The leading cause of nonoperative adverse events was adverse drug events (19.3% of all adverse events; 35.1% negligent, and 9.7% caused permanent disability). Operative adverse events fell into several categories (Table4). Technical operative complications FIG. Overview record of reviewprocessin Colorado Utah. and 265 THOMAS ET AL MEDICAL CARE TABLE Characteristicsof the Study Population 1. Sample StatePopulation n % Discharges, n Discharges, Female White* Payer Medicare Medicaid Private/other Managedcare Uninsured WeightedPercentage 9,077 10,937 61.3 71.9 338,774 59.3 3,880 2,269 6,510 1,307 917 24.2 15.4 46.6 7.8 6.0 135,928 83,014 260,624 62,790 28,469 23.8 14.6 45.7 11.0 5.0 did *Hospitals not reportracein eitherstate in 1992. caused by poor surgical techniques accounted for 29.9% of operative adverse events, and 23.6% of these were negligent. Bleeding was second most common, followed by wound infections, nonwound infections, and deep venous thrombosis. The category of other wound problems included postoperative hernias and dehiscence. Medical complications of surgical cases, such as pulmonary embolus (3.7%) and myocardial infarction (3.8%), were also prevalent, although these were rarely due to negligence. Among adverse drug events, 24.9% were due to antibiotics, 17.4% to cardiovascular agents, 8.9% to analgesics, and 8.6% to anticoagulants (Table5). Among all the negligent adverse drug events, 20.9% were due to a provider prescribing the wrong drug, 7.9% to use of the wrong dose, 5.7% to a drug prescribed to a patient with a known allergy to that drug, 5.0% to a missed dose, 4.6% to an incorrectlyprescribed frequency, and 2.8% to a drug-drug interaction. Location and Provider Type Most adverse events (83.8%) occurred in the hospital, and most of these occurred in either the operating room (39.5%) or the patient's room (21.6%) (Table6). The highest percentage of negligent adverse events occurred in patients' rooms (45.8%) and emergency rooms (52.6%). Most adverse events outside the hospital occurred in physicians' offices (7.0% of all events) or patients' homes (3.4%). Finally,46.1% of the adverse events were attributed to surgeons (22.3% of these events were negligent) (Table 7). Internists were associated with 23.2% of the events, and 44.9% were negli- 2. TABLE Distribution of Adverse Events According to Disability, Projected to State Populations Disability Category Emotional Insignificant Minortemporary Majortemporary Minorpermanent permanent Significant Majorpermanent Grave Death Total* AdverseEvents,n % Mean+?SE, 56 1,848 7,243 5,429 899 274 114 158 1,139 17,160 0.3+0.4 10.8?1.4 *Total may differfromsum due to roundingerror. 266 42.2+2.3 31.6+2.1 5.2?1.0 1.6?0.6 0.7+0.5 0.9+0.5 6.6+1.2 Negligent AdverseEvents,n 0 358 2,069 1,701 130 145 32 134 439 5,007 Mean?SE, % 7.2+2.3 41.3?4.1 34.0?4.0 2.6?1.6 2.9?1.6 0.6+1.2 2.7+1.6 8.8+2.5 Vol. 38, No. 3 INCIDENCE AND TYPESOF ADVERSEEVENTS TABLE Types of Adverse Events 3. AdverseEvents 16.9 16.6 3.9 35.1 9.7 15.3 9.9 2.7 1.4 1,181(6.9) 736 (4.3) 620 (3.6) 93.8 56.8 25.5 20.1 30.9 0 20.1 9.0 0 25.3 29.8 19.3 32.7 65.8 0 24.7 24.7 0 24.7 256 (1.5) 17,192 Postpartum With Permanent Disability, % 532 (3.1) 226 (1.3) 220 (1.3) 66 (0.4) Incorrect delayeddiagnosis or Incorrect delayedtherapy or With Permanent Disability, % 2,315 (13.5) Drug Medical procedure With Negligence, % 7,715 (44.9) 3,325 (19.3) Operative Negligent AdverseEvents Adverse Events, n (%) Type AdverseEvents 59.9 30.4 0 0 Neonatal Anesthesia related Falls Fracture related Other Total gent. Although only 1.7% of adverse events were attributed to nursing staff and emergency physicians, a notable 74.2% and 94.8%, respectively, were negligent, mostly caused by failed diagnosis. Discussion These data from Colorado and Utah in 1992 are consistent with the findings of the Harvard MedTABLE 4. Type ical Practice Study conducted in New YorkState in 1984, with some important differences. Our estimates of the Utah and Colorado adverse event rates are lower than those of New York, but a direct comparison of adverse event rates is limited by temporal changes in health care and differences in patient and hospital characteristicsbetween the states. Notably, since 1984, the acuity of hospitalTABLE Drug-Related Adverse Events 5. According to Class of Drug Types of Surgical Adverse Events Estimated Adverse Adverse Events With Events, n (%) Negligence, % Technical 2,309 (29.9) 1,319 (17.1) Bleeding Wound infection 877 (11.4) Nonwound infection 775 (10.0) Deep venous thrombosis 391 (5.1) 295 (3.8) Myocardial infarction Other wound problem 288 (3.7) 272 (3.5) Pulmonary embolism 165 (2.1) Congestive heart failure Cerebrovascularaccident 163 (2.1) Other technical problem 101 (1.3) Pneumonia 80 (1.0) Other 501 (6.5) Unknown 180 (2.3) Total 7,716 23.6 9.8 20.8 7.5 19.1 0 32.5 0 19.1 0 0 29.0 28.9 0 Adverse Adverse EventsWith n (%) Negligence,% Events, Class Antibiotic Cardiovascularagent Analgesic Anticoagulant 828 (24.9) 579 (17.4) 6.8 38.9 297 (8.9) 286 (8.6) 33.3 25.1 Sedativeor hypnotic Antineoplastic agent Asthmatic agent 88 (2.6) 45 (1.4) 42 (1.3) Antidepressant 29 (0.9) 20 (0.6) Antipsychotic agent medication 14 (0.4) Antihypertension Antiseizure medication 13 (0.4) Potassium 13 (0.4) Other 603 (18.1) Unknown 468 (14.1) Total 3,325 100 0 68.3 0 100 0 100 100 31.0 267 THOMAS ET AL MEDICAL CARE TABLE Location of Adverse Events 6. Site Adverse Adverse EventsWith Events,n (%) Negligence,% In hospital 14,401(83.8) room 6,796(39.5) Operating Patient'sroom 3,711(21.6) Laborand delivery 1,110(6.5) area Cardiac catheterization 715 (4.2) laboratory Intensivecareunit 597 (3.5) Procedure room 587 (3.4) room 513 (3.0) Emergency Other 371 (2.2) Outsidehospital 2,744(16.0) office 1,200(7.0) Physician's 584 (3.4) Home 208 (1.2) Day surgery 103 (0.6) Nursinghome 649 (3.8) Other 46 (0.3) Unknown Total 17,191 6.8 18.1 45.8 27.8 12.0 31.8 5.4 52.6 18.5 0 33.2 22.1 35.8 35.3 74.7 0 ized patients has increased and nursing staff levels have decreased. Such factors would be expected to increase the adverse event rate. But differences between the patient populations, health care sys7. TABLE Adverse Events by Provider Type Provider Type Adverse AdverseEvents, EventsWith n (%) Negligence,% 7,935(46.1) 3,990(23.2) Obstetrics/midwifery 1,583(9.2) 1,158(6.7) Gynecology 754 (4.4) Familypractice 297 (1.7) Nursing 289 (1.7) medicine Emergency 165 (1.0) Radiology 149 (0.9) Pediatrics 120 (0.7) Anesthesia 528 (3.1) Other 226 (1.3) Missing/unknown Total 17,194 Surgery Internal medicine 268 22.3 44.9 25.8 24.2 51.7 74.2 94.8 34.9 55.0 42.3 84.1 tems, and other unknown factors may have contributed to lower rates in Utah and Colorado. In both studies, >50% of adverse events were minor, temporary injuries. However, in New York, a much higher percentage of adverse events led to death (New York, 14%, or 0.5 deaths per 100 discharges; Colorado and Utah, 6.6%, or 0.2 deaths per 100 discharges). Negligent adverse events leading to death were also more common in New York(New York,25%; Colorado and Utah, 8.8%). When extrapolated to all admissions in the United States, our data reveal that there were -976,404 adverse events (64,809 deaths) and 284,898 negligent adverse events (24,979 negligent deaths) in 1992. This burden of morbidity and mortality is striking although less than estimates derived from our New Yorkstudy. We found that surgical interventions (operative adverse events) caused approximately half of all adverse events, but a relatively small proportion (16.9%) were judged negligent. This proportion of operative adverse events is remarkably stable between studies done >8 years apart and in quite different states (Utah and Colorado versus New York). The heterogeneity of the postoperative medical complications suggests that research should focus on more than prevention of cardiac complications.22 As in previous population-based studies,23 drugs-especially antibiotics and cardiovascular agents-were the second leading cause of adverse events. But the distribution of adverse drug events in our study differed from a recent study of these events in 2 teaching hospitals.5 Although both studies found that 24% of adverse drug events were due to antibiotics (the most common cause in our study), the teaching hospital study found that analgesics were the most common cause (30%) of adverse drug events. Analgesics accounted for only 8.9% of adverse drug events in our study. Furthermore, cardiovascular drugs caused many more events in our study than in the teaching hospital study (16% versus 4%). These differences suggest that adverse events found in urban teaching hospitals may not be generalizable to other hospitals, and they highlight that different definitions of adverse events (the teaching hospital study did not require prolonged hospitalization or disability at discharge) may result in differing rates and types of events. Among locations, the emergency department had the highest percentage of negligent adverse events (52.6%), and a remarkable 94.8% of ad- INCIDENCE AND TYPESOF ADVERSEEVENTS Vol.38, No. 3 verse events attributedto emergencyphysicians werejudgednegligent.Thismaybe due to the task concurrent tasks,uncertainty, (multiple complexity and high workload)inherentin changingplans, emergency medical care, particularly trauma resuscitation.24 Emergency departments could cope with task complexity by improving teamwork and standardizing work procedures.24 Internists, family practitioners, and pediatricians also had relatively more adverse events caused by negligence than other provider types. This is related in part to adverse events resulting from incorrect or delayed diagnoses, which were usually attributed to these providers and were almost always (93.8%) negligent. These types of adverse events deserve more attention given their propensity to be negligent, to be disabling (20.1% resulted in permanent disability), and to result in litigation.25 The adverse event rates in Colorado and Utah were remarkably lower than the Quality in Australian Health Care Study,13 which found the incidence of events to be 13%. We are currently working with Australian investigators to understand whether subtle differences in study methodologies and clinical practices between the countries may have led to such disparities. Another study by Andrews et al14 at an urban teaching hospital observed surgical rounds and found an adverse event rate of 17.7%. This study cannot be readily compared with the New Yorkstudy or the present study because it used a definition of adverse events that would give a higher rate than our study and it studied only surgical patients, who suffer a greater proportion of adverse events than other patients. Adverse event rates of surgical patients are not comparable to our random sample of all hospitalized patients. The reliability of adverse event judgments (K = 0.4) is an important limitation of our study. However, this value is similar to those of the HarvardMedical Practice Study1 and the Australia study13; it is also similar to those measured for commonly used diagnostic tests, such as screening mammography (K = 0.47),26 and for the RAND appropriateness method for estimating overuse of hysterectomy (K = 0.52).27 Our investigator verifi- cation efforts provided a strong safeguard against false-positive adverse events, but re-review of medical records that reviewers judged not to involve adverse events could not feasibly be undertaken. Therefore, the moderate reliability score for adverse event judgments makes our estimate a lower bound of the true rate. The adverse event determination delineated the pool of cases eligible for subsequent judgments about negligence. Adverse event criteria are appropriate for delineating candidate negligent adverse events because "causation"and the presence of substantive injury,the crux of the adverse event judgment, are also pivotal criteria in the legal definition of negligence.20 The other key component of that definition-evidence that the injury was due to substandard care-was initially addressed by reviewers and then revisited by investigators, 1 of whom is a national expert on medical malpractice. This iterative process bears some resemblance to that used to decide the issue of negligence in court wherein multiple physician testimonies are weighed. Again, investigators were able to eliminate false-positive negligent adverse events quite conclusively using information from the record review. There was also some recourse to elimination of false-negative adverse events in the case of negligent adverse events; investigators reviewed the pool of adverse events to ensure that none had been overlooked as having been due to negligence. Other limitations include the facts that our method relies on documentation in medical records, that we counted only events that prolonged hospital stay or caused disability at discharge, and that we used general internists and family practitioners as reviewers. Although these generalists had access to specialists for consultation, they may have missed subtle adverse events resulting from care provided by specialists. Our hospital selection ensured a representative sample (for the characteristicssize, location, teaching status, and ownership), but the hospitals within each stratum were not randomly selected. Therefore, selection bias could have lowered our estimate of the adverse event rate if hospitals with poor quality of care did not participate. However, all of the hospitals approached agreed to participate, and investigators had no knowledge of any adverse event rate before the study. Finally, our results may not be generalizable to other locations. In summary, our study found that the morbidity and mortality associated with iatrogenic injury remains high, mostly because of operative complications and adverse drug events. How can these adverse events be prevented? On the basis of the experience of other industries28 and psychological theories about why errors occur,29 269 MEDICAL CARE THOMAS ET AL many errors in medicine are likely the result of system failures rather than just 1 individual practitioner committing an error.30Investigators have documented the system errors that lead to various adverse events31,32 and designed computer interventions to decrease these errors.33-35 However, new technologies should be introduced with care because they may bring new sources of error into the system.36 To prevent errors and adverse events in the operating room, researchers are looking to the use of flight simulation by the airline industry37 and its focus on teamwork and communication among cockpit crew members.38 These and other lines of inquiry offer exciting opportunities to improve patient safety. References 1. Brennan TA, Leape LL, Laird NM, Hebert L, Localio AR, LawthersAG, et al. Incidenceof adverse events and negligence care in hospitalizedpatients. N EnglJ Med 1991;324:370-376. 2. LeapeLL,LawthersAG, BrennanTA, Johnson WG. Preventing medical injury. Qual Rev Bull 1993;19:144-149. related 3. LesarTS, BricelandL, Stein BS. Factors in to errors medication 1997;277:312JAMA prescribing. 317. 4. Leape LL, Bates DW, Cullen DJ, Cooper J, Demonaco HJ, Gallivan T, et al. Systemsanalysisof adverse drug events: ADE PreventionStudy Group. 1995;274:35-43. JAMA 5. Bates DW, Cullen DJ, Laird M, Peterson LA, Small SD, Servi D, et al. Incidentsof drug adverse events and potentialadversedrug events:Implications for prevention:ADE PreventionStudy Group.JAMA 1995;274:29-34. 6. Zeroing in on medication errors. Lancet Editorial. 1997;349:369. and 7. RosemanC, BookerJM.Workload environmental factorsin hospitalmedicationerrors.Nurs Res 1995;44:226-230. 8. Cohen MR, Anderson RW,Attilio RM, Green L, Muller RJ, Pruemer JM. Preventingmedication Am in errors cancerchemotherapy. J HealthSystPharm 1996;53:737-746. of and 9. SendersJW.Theory analysis typicalerrors in a medicalsetting.HospitalPharm1993;28:505-508. 10. LesarTS, LomaestroBM, Pohl H. Medication errorsin a teachinghospital:A nine year prescribing ArchInternMed 1997;157:1569-1576. experience. 270 11. Classen DC, Pestonik SL, Evans SR, Lloyd JF, Burke JP. Adversedrug events in hospitalized patients.JAMA 1997;277:301-306. 12. Mills DH, Boyden JS, Rubamen DS. Report on the Medical Insurance Study. San Francisco, Calif: Sutter Publications; 1977. 13. WilsonRM,Runciman WB,GibberdRW,Harin rison BT,Newby L, HamiltonJD.Thequality Australian health-care study.Med J Aust 1995;163:458-471. 14. Andrews LB, Stocking C, Krizek T, Gottlieb for L, KricekC, VargishT, et al. An alternative strategy studying adverse events in medical care. Lancet 1997;349:309-313. 15. Localio AR, LawthersA, Brennan TA. Identifying adverse events caused by medical care: Degree of physician agreement in retrospective chart review. Ann Intern Med 1996;125:457-464. 16. The AHA Guide to the Health Care Field. Chicago, Ill: American Hospital Association, 1993. 17. The directory of U.S. hospitals. Baltimore, Md: Health Care Investment Analysts Inc; 1992. 18. BrennanTA, LocalioAR, LairdNM. Reliability and validity of judgments concerning adverse events and negligence. Med Care 1989;27:1148-1158. 19. Brennan TA, Localio AR, Leape LL, Laird of NM, Peterson L, Hiatt HH, et al. Identification adverse events occurring during hospitalization: A crosssectional study of litigation, quality assurance and medical records at two teaching hospitals. Ann Intern Med 1990;112:221-226. 20. Keeton WP. Prosserand Keetonon the Lawof Torts.5th ed. St Paul, Minn: West Publishing Company, 1984. 21. Sowka M, ed. NationalAssociationof Insurance Commissioners: Malpractice Claims: Final compilation. Brookfield,Wis: National Association of Insurance Commissioners; 1980. 22. Mangano DT, Goldman L. Preoperative assessment of patients with known or suspected coronary disease. N Engl J Med 1995;333:1750-1756. 23. Leape LL, Brennan TA, Laird N, Lawthers AG, Localio AR, Barnes BA, et al. The nature of adverse events in hospitalized patients: Results of the Harvard Medical Practice Study II. N Engl J Med 1991;324:377-384. 24. Xiao Y, Hunter WA, Mackenzie CF, Jeffries NJ, Horst RL. Task complexity in emergency medical care and its implications for team coordination, LOTAS group: Level one trauma anesthesia simulation. Hum Factors 1996;38:636-645. 25. Physicians and surgeons update. St Paul, Minn: Saint Paul Fire and Marine Insurance Company, Medical Services; 1997. INCIDENCE AND TYPESOF ADVERSEEVENTS Vol. 38, No. 3 26. Elmore JG, Wells CK, Lee CH, Howard DH, Feinstein AR. Variabilityin radiologists' interpretations of mammograms. N Engl J Med 1994;331:1493-1499. 27. Shekelle PG, Kahan JP, Bernstein SJ, Leape LL, Kamberg CJ, Park RE. The reproducibility of a method to identify the overuse and underuse of medical procedures. N Engl J Med 1998;338:1888-1895. 28. Perrow C. Normal accidents: Living with high risk technologies. New York,NY: Basic Books, 1984. 29. Reason JT. Human error.New York,NY: Cambridge University Press, 1990. 30. Leape LL. Error in medicine. JAMA 1994;282:1851-1857. 31. Leape LL, Bates DW, Cullen DJ. Systems analysis of adverse drug events: ADE prevention study group. JAMA 1995;274:35-43. 32. Petersen LA, Brennan TA, O'Neil AC, Cook EF, Lee TH. Does housestaff discontinuity of care increase the risk for preventable adverse events? Ann Intern Med 1994;121:866-872. 33. Bates DW, Leape LL, Cullen DJ, Laird N, Peterson LA, Teich JM, et al. Effect of computerized physician order entry and a team intervention on pre- vention of serious medication errors.JAMA 1998;80:13111316. 34. Evans RS, Pestotnik SL, Classen DC, Clemmer TP, Weaver LK, Orme JF Jr, et al. A computerassisted management program for antibiotics and other infective agents. N Engl J Med 1998;338: 232-238. 35. Petersen LA, Orav EJ, Teich JM, O'Neill AC, Brennan TA. Using a computerized sign-out programto improve continuity of inpatient care and prevent adverse events. Jt Comm J Qual Improv 1998;24:77-87. 36. Cook RI, Woods DD. Adapting to new technologies in the operating room. Hum Factors 1996; 38:593-613. 37. Howard SK, Gaba DM, Fish KJ, Yang G, Sarnquist FH. Anesthesia crisis resource management training: teaching anesthesiologists to handle critical incidents. Aviation Space Environ Med 1992;63:763-770. 38. Helmreich RH, Davies J. Human factors in the operating room: interpersonal determinants of safety, efficiency and morale. Baillieres Clin Anaesthesiol 1996;10:277-293. Appendix Screening Criteria Used by Nurse-Reviewers 1. Was there prior hospitalization within 1 year and was the patient <65 years of age, or was there prior hospitalization within 6 months and the patient 265 old? 2. subsequent admission to any hospital after this discharge? 3. hospital-incurred trauma? 4. an adverse drug reaction while in hospital? 5. transferred from a general care unit special unit? 6. another acute 7. did return operating room during admission? 8. treatment operation performed because damage organ systems invasive procedure? 9. have myocardial infarction, stroke, pulmonary embolism 10. new neurological deficit at time 11. die? 12. patient's temperature>101?F (38.3?C) on the day before discharge? 13. Did the patient have a cardiac/respiratory arrest? 14. Was there injury or complication related to abortion, amniocentesis, or labor and delivery? 15. Were there other undesirable outcomes? 16. Was there correspondence in the medical record suggesting litigation? 17. Was there an injury or complication related to ambulatory surgery or an invasive procedure that resulted in admission or emergency department evaluation? 18. Was there an unplanned open surgical procedure or admission after a planned ambulatory laparoscopic or closed procedure? 271