Research Article REVIEW OF THE LONG-TERM EFFECTIVENESS OF COGNITIVE BEHAVIORAL THERAPY COMPARED TO MEDICATIONS IN PANIC DISORDER Deepa N. Nadiga, M.D., Paula L. Hensley, M.D., and E. H. Uhlenhuth, M.D. Panic disorder is a recurrent and disabling illness. It is believed that Cognitive Behavioral Therapy (CBT) has a long-term protective effect for this disorder. This would offer CBT considerable advantage over medication management of panic disorder, as patients often relapse when they are tapered off their medications. This is a review of the literature about the long-term effectiveness of CBT. We searched for follow-up studies of panic disorder using CBT. Of the 78 citations produced in the initial search, most bad major methodological flaws, including ignoring losses to follow-up, not accounting for interval treatment, and unclear reporting. Three papers met strict methodological criteria, and two of these demonstrated a modest protective effect of CBT in panic disorder patients. We make recommendations for well-designed studies involving comparisons of medications and cognitive behavior therapy. Depression and Anxiety 17:58-64, 2003. 2003 Wiley-Liss, Inc. Key words: adult; antidepressants; cognitive behavioral therapy; panic disorder; follow-up studies INTRODUCTION Panic disorder is a common [Weissman et al., 1997] and disabling [Klerman et al., 1991; Pollack and Marzol, 2000] illness. The disorder involves recurrent unexpected panic attacks accompanied by one of the following: a persistent concern about having additional attacks, worry about the implications of the attacks, and a significant change of behavior (DSM-IV). It is often accompanied by agoraphobia [Klerman et al., 1991]. Medications (SSRIs, TCAS, MAOIs, and benzodia- zepines) treat panic disorder effectively and achieve remission rates upwards of 70% [Ballenger, 1993; Toni et al., 2000]. However, relapse rates of 50% are common on discontinuation of antidepressant medica- tion [Toni et al., 2000]. These rates are higher with benzodiazepines [Noyes et al., 1991]. Cognitive beha- vioral therapy (CBT) is also effective in treating panic disorder with 75% of patients achieving panic-free end states [American Psychiatric Association, 1998]. In contrast, cognitive behavioral therapy is reputed to maintain its treatment gains over time, and it is in this area that CBT claims a distinct advantage over medications. Here, we assemble the evidence for this claim. The companion to this article examines the 2003 WILEY-LISS, INC. long-term effectiveness of CBT in studies of depres- sion (Hensley et al., submitted). MAJOR ISSUES IN FOLLOW-UP STUDIES DROPOUTS Acute treatment studies. Most acute treatment studies nowadays use an intent-to-treat analysis (ITT). In such an analysis, all randomly assigned subjects are included, usually using the latest available set of clinical measures (last observation carried forward; LOCF). Department of Psychiatry, School of Medicine, University of New Mexico Health Sciences Center, Albuquerque, New Mexico *Correspondence to: Dr. Deepa N. Nadiga, UNM Department of Psychiatry, 2400 Tucker N.E., 4th floor, Albuquerque, NM 87131. E-mail: dnadiga@salud.unm.edu Received for publication 14 December 2001; Accepted 8 Novem- ber 2002 Published online in Wiley InterScience (www.interscience.wiley. com). DOI 10.1002/da.10084 Subjects who drop out are considered to be treatment failures, either because of non-response or adverse events [Rickels and Schweizer, 1998]. An ITT analysis of a study prevents the "sieve effect" (i.e., patients who are benefited by treatment tend to stay on), with narrowing of the contrast between active and control conditions. Follow-up studies. Now that ITT analysis is accepted in short-term therapeutic research, the idea that dropouts have to be taken into account should be extended to follow-up studies. Currently, follow-up studies usually include only the improved subjects completing the acute treatment as a baseline. The problem is that the original randomization is lost and now we have biased subsamples. Patients who selec- tively survive an ineffective acute treatment of months' duration are likely to be hardy and at low risk of relapse after treatment is withdrawn, whereas patients who respond to an active treatment are more likely to relapse when it is withdrawn. Furthermore, patients who fail acute treatment may improve spontaneously a year or two later. Apart from ignoring attrition in the acute phase of the study, most studies also ignore attrition in the follow-up phase. The percentage of patients who maintain wellness is calculated using the total number of survivors at follow-up as a base. An assumption is made that there will be some attrition over time and these losses do not affect the results. However, this may result in over estimation of treatment effects. A better solution would be to use the last available observation. Where such data are not available, it would be more conservative to assume that losses to follow-up actually represent non-response to treatment or adverse events, as in acute treatment studies [Rickels et al., 1993]. The argument against this line of reasoning is that not all losses to follow-up represent failure of treat- ment. Subjects simply may tire of the follow-up process once gains have been made. Other losses to follow-up may occur because of subjects moving out of the geographical area. This argument certainly holds for some attrition that occurs in longer follow-up studies, but it is hard to make this case for follow-ups of two years or less. In the absence of data, all of the aforementioned arguments are speculative. The issue can be argued both ways. For the purpose of this paper we have decided to take the approach of treating the dropouts as failures of treatment. We believe that an ITT analysis using the original number randomized as the base would result in a conservative but clear outcome. CRITERIA FOR LONG-TERM OUTCOMES Many studies use cross-sectional evaluations of continuous variables to record improvement in patients with a particular intervention. This procedure fails to detect relapses that occur between evaluations. For example, a patient may have a relapse, recover completely, and present for follow-up. A cross-sec- tional assessment at this point would fail to document the interval relapse. For this purpose, full interval histories and binary definition of clinical status is required. USE OF TREATMENT IN THE FOLLOW-UP PERIOD Even in studies that are carefully carried out in the acute treatment phase, patients are free to use any treatment that they need in the follow-up period. It would be ethically inappropriate to deny such treat- ment in long-term follow-up studies. However, many of these studies do not report the number of patients who take other treatment in each experimental group. If patients require additional treatment, they should be classified as having failed the criterion of long-term effectiveness of the experimental treatment. Follow-up studies that look only at cross-sectional measurements of clinical status at different points in time without accounting for additional treatment during the inter- vals are underepresenting failures. THE HYPOTHESIS It is well known that many patients with panic disorder relapse on discontinuation of medications. The belief that even a few weeks of cognitive behavioral therapy (CBT) results in favorable long- term outcomes is widely held. We present a critical review of the evidence relevant to this belief. We identified follow-up studies that involved comparisons of CBT with medications. We wanted to examine if any of the claims of significant difference made in the original paper would remain if we performed a recalculation using ITT and preferably LOCF analysis. Using this standard, we predicted that the evidence supporting the longitudinal effectiveness of CBT in panic disorder would be modest. METHODS The initial search for relevant reports included journal articles indexed in Medline, PsychInfo, Sci- Search, SocSciSearch, and the Cochrane Library. The terms "panic disorder", "Agoraphobia", and "follow-up studies" were used. New references from reviews and primary sources were traced and added. This search produced 78 citations. The following criteria were applied to the products of the initial search: The review focused on follow-ups of straightfor- ward head-to-head comparisons of CBT and control treatments (including medications) in general samples of adult panic disorder patients. Studies of patients aged less than 18 years were excluded. Follow-up studies lacking a credible control group were excluded. Credible control groups for this purpose were taken to be patients treated with medication, who were expected to relapse after treatment was withdrawn, and patients who received a clearly inferior treatment, such as a waiting list or placebo. This requirement alone eliminated 43 studies. Studies of special samples, such as patients who failed pharmacotherapy, were excluded. Studies of exposure only (behavior treatment) were excluded, since some would argue that this is less effective and enduring than CBT. This eliminated two studies. Studies of specific elements of CBT ("decomposi- tion studies") were excluded. Follow-up studies of less than six months were excluded. Although relapse often occurs early, effects lasting less than six months can hardly be considered "long-term." This eliminated two stu- dies. There had to be a binary measure clearly defining patients who maintained remission. Histories of relapse and use of treatments during the intervals between follow-up interviews were re- quired. Only primary sources were included in this review. Eight of the originally identified citations were excluded because they were reviews, and 12 were excluded because they were on some other topic entirely. RECALCULATION Eleven studies met the previously stated criteria. For each of these studies, we determined whether appro- priate data were presented in the paper to enable a recalculation using an ITT analysis based on the original number randomized. This reduced the number of studies to three. Survivors then were identified from the follow-up group. Survivors were defined as subjects who: Are present (i.e., have not dropped out), Are doing well by standardized criteria as defined by each study, or Did not require any additional treatment in the interim. The percentage of these survivors over the original number randomized to each arm was calculated. A Fishers exact test was performed to see if the differences were significant. Paired comparisons that were significant at P TABLE 1. Sharp DM, et al. [1996] Fluvox Plac Fluvox+CBT Plac+CBT CBT Overall exact P Evaluation At randomization Measure of remission n % n % n % n % n % 36 100.0 37 100.0 38 100.0 36 100.0 43 100.0 Patients maintaining Hamilton Anxiety Scale 11 30.6 8 21.6 18 47.4 18 50.0 15 34.9 0.065 remission Symptom Rating Test 4 11.1 4 10.8 10 28.3 11 30.6 11 25.6 0.100 FQ-AG 10 27.8 7 18.9 16 42.1 17 47.2 15 34.9 0.079 At follow-up 6 months after acute treatment Effect size compared with Hamilton Anxiety Scale 0.17 0.19 0.04 Symptom Rating Test 0.19 0.19 0.14 FQ-AG 0.14 0.15 0.07 Effect size compared with Hamilton Anxiety Scale Symptom Rating Test FQ-AG 0.00 0.09 0.15 0.16 Fluvox, fluvoxamine; Plac, placebo; CBT, cognitive behavior therapy; FQ-AG, Fear Questionnaire, agarophobia subscale. Effect sizes are calculated as the difference between two portions. "Number of patients with "clinically significant." improvement from baseline 0.09 0.26 0.28 0.13 0.16 0.20 0.23 0.28 TABLE 2. Loerch B et al. [1999] + CBT+Plac CM+Moc CM+Plac Overall Exact p Evaluation Measure of remission 22 % 22 % 22 % 22 % At randomization 14 100.0 14 100.0 16 100.0 11 100.0 Patients maintaining remission Fear questionnaire 10 or less reduced 50%+ 8 57.1 2 14.3 0 0.0 0 0.0 0.000 10 71.4 3 21.4 0 0.0 0 0.0 0.000 Effect size compared with CM+Moc Fear questionnaire 10 or less reduced 50% 0.57 0.14 0.71 0.21 Effect size compared with CM+Plac Fear questionnaire 10 or less reduced 50%+ 0.57 0.14 0.00 0.71 0.21 CBT, cognitive behavior therapy; Moc, moclobemide; Plac, placebo; CM, clinical management. Effect sizes are calculated as the difference between two proportions. Paired comparisons in bold font denote difference significant at P <0.05 obtained. Unfortunately, the end-of-treatment status of patients who did not participate in follow-up was not clear, so that LOCF analysis was not feasible. Our ITT analysis of follow-up status at six months took data from Loerch's Figure 3 and classified all missing patients as non-remitted (see Table 2 for data recalculation). This analysis indicated that more patients treated with moclobemide plus CBT main- tained remission than patients in the other treatment groups (P-00). The effect size analysis revealed small to moderate effect sizes from 0.14-0.71. It is notable that the moderate to moderately large effect sizes were all in the CBT plus moclebomide group, while the CBT plus placebo group had uniformly small effect sizes. This study produces a wide spread of results, with moclebomide producing no results on its own but performing most robustly in combination with CBT. In conclusion, this study suggests that CBT has some long-term protective effect. Barlow et al. [2000] performed an elegant study randomizing 312 panic disorder patients to one of five treatment arms: CBT, imipramine (Imip), pill placebo (Plac), CBT plus imipramine, and CBT plus pill placebo. This study had a three-month acute treatment phase and a six-month maintenance treatment phase, followed by a follow-up of 6 months. CBT consisted of 11 sessions. Imipramine plus desipramine plasma levels at six weeks averaged about 220 ng/ml. Patients were interviewed six months after the end of maintenance treatment and interval treatment data were collected. Response was determined by a 40% reduction from baseline on the clinician rated Panic Disorder Severity Scale (PDSS). The other binary variable was Clinical Global Impression Scale (CGI) with a Panic disorder anchor added to it. To be a responder, a patient needed to achieve a score of 2 (much improved) or better while being rated as 3 (mild) or less on CGI severity. Receipt of additional treatment was noted at each assessment point and patients who received nonstudy treatment for anxiety or panic were determined to be non-respon- ders. This study is remarkable for the maintenance of a placebo control even in the follow-up phase, the large number of patients, the comprehensiveness of the analyses, and the clarity of presentation. The authors did an ITT/LOCF analysis of status at follow-up. This study demonstrates that CBT continues to be effective over a six-month follow-up period, (P=.001) but all the effect sizes for this study were uniformly small (see Table 3). DISCUSSION LONG-TERM EFFECTIVENESS OF CBT Of the three studies included in our review, two indicate that CBT does have a long-term effect but with small effect sizes. More surprisingly, the data analysis reveals inconsistent results, with certain similar treatment arms performing erratically. Though the Barlow et al. [2000] study showed a modest effect, the results are impressive because of the excellent design of the study. However, a confident conclusion that CBT is effective for the long-term in panic disorder would require replication of this finding in additional trials that meet the admittedly stringent methodological criteria that we adopted. LACK OF A CREDIBLE CONTROL GROUP A review of citations on follow-up of patients receiving CBT for panic disorder reveals that there are a large number of studies. However, there is a lack of randomized controlled studies in this area. In our review, the requirement that studies have a credible control group reduced the total number of citations by more than half (from 78 to 35 citations). Randomized controlled trials are the gold standard in scientific research. The abundance of uncontrolled, naturalistic follow-up studies in the area of CBT and panic disorder can only suggest a protective effect but not prove it reliably. OTHER PROBLEMS IN METHODOLOGY This review finally identified 11 studies that looked specifically at the durability of CBT in panic disorder. However, only three of these studies met our criteria for adequate design and reporting. Some of the commonly encountered drawbacks in these studies are the following. They did not clearly provide the numbers rando- mized to each treatment. They mentioned dropouts but did not state from which cells the drop outs occurred. Some of them used percentages of subjects im- proved without stating what number was being used as the base for the percentage. Some studies used different criteria for improve- ment in the acute and follow-up phases. In some studies, subjects were allowed to use alternative treatment during the follow-up, but this was not factored into results. For some studies, relapses between follow-up points were not taken into account. In retrospect, it seems clear that most studies were not designed with a follow-up phase at the very outset. When a decision is made, at a later stage, to add on a follow-up phase to an acute phase study, it leads to discrepancies in the design of the two phases. LIMITATIONS OF THE REVIEW One of the limitations of this review is the small number of studies identified. Any conclusions that we reach are limited in their scope. These three studies also report inconsistent results. It seems that the field TABLE 3. Barlow DH et al. [2000] CBT Imip Plac CBT+limp CBT+Plac Overall exact p Measure of remission Evaluation At randomization At follow-up 6 months after and Patients maintaining remission of maintenance treatment Effect size compared imipramine 17 13 100.0 77 100.0 24 100.0 31.) 19.5 3 12.5 $2.9 15 19.5 2 8.3 0.12 0.15 0.19 0.07 Clinical global impression 23 PDSS reduced 40%+ PDSS reduced 40%+ Clinical g al global Impression Effect size compared with placebo Clinical global impression 0.11 855 100.0 62 00.0 25.4 25 40.3 0.028 25.4 25 40.3 0.009 0.06 0.21 0.06 0.21 0.13 0.17 0.28 0.32 CBT, Cognitive behavior therapy; Imip, imipramine; Plac, placebo; PDSS, Panic Disorder Severity Scale. Effect sizes are calculated as the difference between two proportions. "Excludes 17 patients assigned to another study prior to follow-up in this study. still requires more data before any strong conclusions about the durability of CBT can be made. It is certainly true that to take the approach that all dropouts are failures is a matter of perspective. On the other hand to assume that dropouts can be ignored is also a matter of perspective. We do not know the reasons why these patients dropped out of treatment. In the absence of this data, the only alternative is to present the analysis in a variety of ways. This paper chooses a particular way to present the data. RECOMMENDATIONS We recommend a change in perspective for future studies. The longer-term studies reported here have been conceptualized as composed of two independent phases: acute treatment and no-treatment follow-up, largely limited to patients who improve and remain readily accessible. Among other problems, this leads to much discussion about response, remission, relapse, and recurrence that ultimately ends in essentially arbitrary definitions that differ among investigators, with increased difficulty in comparing the results of different studies. In addition, this framework leads to inconsistencies in measures, definitions of remission, efforts to maintain contact with patients and their clinical condition, data analyses, and so forth even in the two pxhases of the same study. Many of the problems we have noted in these follow-up studies stem from this context within which they were designed. One might even suspect that the follow-up phase in some studies was not even envisioned until after the original acute study was completed. In contrast, we propose conceptualizing longer-term studies as trials designed from the start in a unified, consistent manner to span a longer period, even though treatment may not extend for the entire period. (In the case of maintenance studies, however, treatment might continue throughout.) Many of the accepted principles of acute trials might apply to longer-term studies designed from this perspective. The perspective that we suggest carries many implications about design. Every subject who enters the study needs to be followed up. An ITT, LOCF analysis needs to be performed on all randomized subjects, including dropouts. All patients who receive interval treatment need to be considered as non-remitters. Certainly, procedures within a study should be consistent throughout. Remission would be defined and measured in a manner that is informative over longer as well as shorter intervals between assessments. Of the studies that we evaluated, the one done by Barlow et al. [2000] fulfills many of these criteria; it is conceived as a study that evaluates the comparative efficacy of Imipramine and CBT both in the acute and follow-up phase. The placebo group is also followed up for 6 months. All the patients are accounted for and are analyzed in the groups to which they were randomly assigned. The reporting is clear, precise, and complete. 64 Nadiga et al. It should be a model for other studies that look at similar issues. CONCLUSION We set out to evaluate evidence regarding the long- term effectiveness of CBT compared to medications for panic disorder. While we found that the evidence for this claim was modest, we also found that there were few studies that evaluated this question using strong research methodology. The field suffers from a lack of data, and the existing data reveals erratic results. Thus the conclusion of this review is not that CBT does or does not have enduring results, but rather that more reliable data are needed. REFERENCES American Psychiatric Association. 1998. Practice guideline for the treatment of patients with panic disorder. Am J Psychiatry 155:S1- S34. Ballenger JC. 1993. Panic disorder: Efficacy of current treatments. Psychopharmacol Bull 29:477-486. Barlow DH., Gorman JM, Shear MK, Woods SW. 2000. Cognitive- behavioral therapy, imipramine, or their combination for panic disorder: A randomized controlled trial. JAMA 283: 2529-2536. Klerman GL, Weissman MM, Ouelette R, Johnson J, Greenwald S. 1991. Panic attacks in the community. JAMA 265:742-746. Loerch B, Graf Morgenstern M, Hautzinger M, Schlegel S, Hain C, Sandmann J, Benkert O. 1999. Randomised placebo-controlled trial of moclobemide, cognitive-behavioral therapy and their combination in panic disorder with agoraphobia. Br J Psychiatry 174:205-212. Noyes R, Garvey MJ, Cook B. 1991. Controlled discontinuation of benzodiazepene treatment for patients with panic disorder. Am J Psychiatry 148:517-523, 1991 Pollack MH, Marzol PC. 2000. Panic: Course, complications and treatment of panic disorder. J Psychopharmacol 14:$25-$30. Rickels K, Schweizer E. 1998. Panic disorder: Long-term pharma- cotherapy and discontinuation. J Clin Psychopharmacol 18:12S-18S. Rickels K, Schweizer E, et al. 1993. Maintenance drug treatment for panic disorder. II. Short- and long-term outcome after drug taper. Arch Gen Psychiatry 50:61-68. Sharp DM, Power KG, Simpson RJ, Swanson V, Moodie E, Anstee JA, Ashford JJ. 1996. Fluvoxamine, placebo, and cognitive behavior therapy used alone and in combination in the treatment of panic disorder and agoraphobia. J Anxiety Disorders 10:219-242. Toni C, Perugi G, Frare F, Mata B, Vitale B, Mengali F, Recchia M, Serra G, Akiskal HS. 2000. A prospective naturalistic study of 326 panic-agoraphobic patients treated with antidepressants. Pharma- copsychiatry 33:121-131. Weissman MM, Bland RC, Canino GJ, Faravelli C, Greenwald S, Hwu HG, Joyce PR, Karam EG, Lee CK, Lellouch J, Lepine JP. 1997. The cross-national epidemiology of panic disorder. Arch Gen Psychiatry 54:305-309.