Module 6 Statistical Tests Solutions Geremy Gully PSY-845/Dr. Bielek Researchers wanted to explore self-esteem in adolescent boys and adolescent girls. Each respondent completed a 10-item self-esteem scale (they chose one rating for each item from a Likert-type scale, 1 = strongly disagree and 5 = strongly agree). The sum of the 10 ratings was each respondent's self-esteem score. Their results were: t = 2.01, d = .90 (40 girls, 40 boys). In an essay of 250-500 words, use the scenario presented in part 1a, above, to thoroughly answer the following questions: 1) What statistical test did the researchers use to determine if there was a statistically significant difference in levels of self-esteem between the boys and the girls? The researcher used independent sample t test for testing the average levels of selfesteem between the boys and the girls. The two samples (adolescent boys and adolescent girls) are independent so t test is the most suitable to test this hypothesis. 2) What was the purpose of calculating a Cohen's d? When is a Cohen's d calculated? Interpret d=.90. What does it mean in this example? Cohen's d is an effect size. The purpose of calculating the effect size is to quantifying the difference between two groups. Cohen's d calculated when we want to measure to size of difference. This can be used when comparing two means. The formula for Cohen's d is simply the difference in the two groups' means divided by the average of their standard deviations. The value of d =0.90, tells us that the two groups' means differ by 0.90 standard deviation. This value of Cohen's d above 0.80 depicts that it is a large effect size. 3) What if the researcher compared the adolescent boys before treatment and again after treating them for depression? What type of t-test would be most appropriate in this case, and why? In this case we have a dependent sample of same adolescent boys before and after the treatment for depression. Whenever the sample is related or dependent, we always use Paired Sample t test. It takes care of the dependencies between the subjects, that is why we are using this test whenever the samples are dependent. What statistical test was used in Nelson et al., (2010)? Authors have used a descriptive technique to evaluate the correlation between the Praise notes and ODRs. They have effectively used the bivariate correlation to test an existence of correlation between the two events. Later they used the test for difference between independent correlations of students with and without ODRs. Did the authors use the correct statistical test? In other words, what was their rationale for using this test (i.e., were the variables discrete or continuous and was the test appropriate for this type of data?) The authors have correctly implemented the tests. With continuous variables, the correlation can be easily measured. One can also observe a linear relationship between praise notes and ODR, because of their similar judgmental outcome. The test was perfect the data as in case of discrete values the variations are large, but In continuous data much variation is due to inherent changes in variable and not just numerical approximation. What was the research question? How did the statistical test address and answer the research question? Research Question was \"whether teachers' use of praise notes to students demonstrating competency with social skills would influence number of ODRs?\" The statistical test established a negative correlation which is quite true to the inherent definition of ODR and praise notes. But, as mentioned in the discussion part of the study, there are several other factors which contribute to ODRs. Considering only praise notes as factor to be correlated will lead to a result but whether it is a considerably significant amongst other factors included or not, is still an unanswered question. How did the authors interpret the results of this study? Authors have very well crafted the results of the study. However, they have neglected an aspect that teachers were positively reinforced for praise notes. This led to their better approach in handling foul actions. The authors have also neglected the fact that Students have also responded to positive appraise from both parents and teachers as the praise notes when shared with parents also create a double positive impact . Using Teacher-Written Praise Notes to Promote a Positive Environment in a Middle School Julie A. Peterson Nelson, Benjamin J. Young, Ellie L. Young, and Gregory Cox ABSTRACT: Teachers in 1 middle school learned about the positive school adults intentionally seek opportunities to build and strengthen adult-youth relationships, they may actually be decreasing the likelihood that students will act out in the future (Young, Black, Marchant, Mitchem, & West, 2000). To meet educational goals, educators must do more than merely prevent antisocial behaviors; they must increase prosocial behaviors (Winette & Winkler, 1972). Unfortunately, schools in the United States continue to encourage punishment for problem behaviors (Noguera, 2003), which, in the absence of a positive schoolwide system of support, has been associated with increased aggression, vandalism, tardiness, truancy, and dropouts (Mayer & Sulzer-Azeroff, 1991). Other researchers have suggested that reinforcing positive behaviors rather than punishing inappropriate behaviors is an effective behavior-management system (Walker, Colvin, & Ramsey, 1995). Hence, creating a positive school environment requires emphasizing preventive and positive measures rather than punitive procedures (Mitchem, Young, & West, 2000). Research has empirically shown that one effective positive behavior-management strategy is teacher praise (Lewis & Bullock, 2004). Other research has identified a functional relation between teacher praise and student behavior. Specifically, when teachers praise appropriate behavior, disruptive behavior decreases (Madsen, Becker, & Thomas, 1968). Further research has shown that praise delivered appropriately increases students' on-task behavior (Ferguson & Houghton, 1992), classroom motivation (Thomas, 1991), and academic success (Sutherland & effects of writing praise notes to students, which is 1 component of a positive behavior support. The authors intended for this procedure to promote a positive school environment and reinforce the appropriate use of social skills. Also, the authors instructed the teachers to use a direct instruction model to teach social skills lessons during 1st-period classes and praise students when they effectively demonstrated these skills. The authors analyzed the data to determine whether students receiving praise notes were less likely to receive an office discipline referral (ODR). The data revealed a significant negative correlation between the number of praise notes and number of ODRs that students received, indicating that as praise notes increased, the rate of ODRs decreased. The authors provide several hypotheses for this relation. KEYWORDS: positive behavior support, praise, schoolwide intervention, teacher-written praise EFFECTIVE SCHOOLWIDE MANAGEMENT of disruptive behaviors is an ongoing national concern (Lewis & Sugai, 1999; Scott, 2001; Turnbull et al., 2002). School violence, discipline, and safety have been among the top concerns for U.S. educators (American Federation of Teachers, 1995-1996; Elam, Rose, & Gallup, 1998; U.S. Department of Education, 1995, 2005). When addressing students with problem behaviors, many schools continue to rely on punitive strategies (e.g., office or administrative disciplinary interventions, suspensions, expulsions) that do little to create a safe and positive educational environment (Lewis & Garrison-Harrell, 1999). These types of interventions tend to be reactionary rather than preventive and proactive. In addition, these types of responses do little to teach new behaviors or to increase the likelihood that positive replacement behaviors would be used in the future (Knoff, 2003). Punitive disciplinary measures can certainly be one approach to behavior management, but if punishment is the only approach used, student behaviors are unlikely to change over the long term. When administrators and other This study was funded in part by U.S. Office of Special Education Programs Grant No. H324C030124. Benjamin J. Young and Ellie L. Young contributed equally to this article. Address correspondence to Julie A. Peterson Nelson, Psychology Department, 1030 SWKT Brigham Young University, Provo, UT 84602, julie_nelson@byu.edu. Copyright 2010 Heldref Publications 119 120 Preventing School Failure Wehby, 2001). In addition, praise has been widely recommended as effective in providing encouragement to the student, building self-esteem, and promoting closer relationships between teachers and students (Brophy, 1981). Teacher praise appears to have several meaningful outcomes that contribute to increased positive behaviors. However, few researchers have explored the use of written praise, especially as it relates to students' use of social skills. In addition, the connection between written praise and a decrease in negative behaviors that lead to office disciplinary action has not been explored, especially among early adolescent youth. Hence, this article addresses how a specific positive behavioral intervention (i.e., written teacher praise) can be a useful strategy for junior high or middle school teachers and administrators to improve the school climate, possibly influencing problem behaviors and lowering rates of office disciplinary referrals (ODRs). Positive Behavior Support (PBS) PBS has been widely adopted to create positive behavioral expectations, teach appropriate behaviors, and manage problem behaviors (Scott, 2001), and its use for affecting students' behaviors and school climate is well documented (Sugai, 1998; Sugai & Horner, 1994; Walker et al., 1996). PBS is a team-based system designed to facilitate student success by using evidence-based interventions and preventive strategies at a schoolwide, classroom, or individual level. The PBS model creates positive behavioral expectations for students, makes these positive expectations explicit, and communicates them widely. For example, students recognize and can articulate the expectation that, while at school, they must behave in a respectful, safe, and caring way. This positive expectation is emphasized in schoolwide efforts rather than on the behaviors that students should not do. The behaviors are explicitly taught through direct instruction of social skills (Lewis & Sugai, 1999). Direct instruction of social skills teaches students the behavioral expectations of the school community. Just as a student could be expected to successfully complete a long division math problem only after specific and directed instruction in long division, students who have had direct instruction in social skills are more likely to enact those skills, thereby meeting the expectations of the adults in the school. When the student displays the newly learned social skill, peers and adults should respond positively, thereby reinforcing the desirable behavior (Gresham, 1998; Lewis, Chard, & Scott, 1994; Lewis & Sugai, 1996; Lipsey, 1991; Mayer, 1995; Peacock Hill Working Group, 1991; Sugai & Horner, 1994; Sugai & Lewis, 1996; Walker et al., 1996). Teacher praise should be coupled with social skills training to reinforce such skills by specifically praising the students for their positive behavior, thus increasing the possibility Vol. 54, No. 2 of the students' using the skill in the future. Understanding how written praise notes may influence disruptive behaviors, as measured by ODR, would add to the understanding of effective components of the PBS model. ODR ODRs have historically been used as an index of student behavior for guiding and developing schoolwide programs and interventions (Skiba, Peterson, & Williams, 1997; Sugai, Sprague, Homer, & Walker, 2000; Tobin & Sugai, 1999; Tobin, Sugai, & Colvin, 1996, 2000). The following three purposes for using ODR data have been identified: (a) as a guide in the development or selection of specific environmentally appropriate interventions (e.g., if a significant number of ODRs are being written for disruptive lunchroom behavior, interventions need to focus on teaching positive behaviors in the lunchroom), (b) as an outcome measure to evaluate the effectiveness of programs, and (c) as a screening procedure to identify students who may benefit from targeted interventions (Nelson, Benner, Reid, Epstein, & Currin, 2002). ODR data continue to have practical and empirical uses. On a practical level, ODRs are used to manage and monitor students with problematic behaviors. These data are easily obtained and monitored in most schools. ODRs can also be used to consider progress toward solving situational concerns (e.g., disruptive lunchroom behavior). On an empirical level, ODRs have been related to poor student outcomes such as school failure and juvenile delinquency (Shinn, Ramsey, Walker, Stieber, & O'Neill, 1987; Sprague et al., 2001; Tobin & Sugai, 1999; Walker, Shinn, O'Neill, & Ramsey, 1987; Walker & Stieber, 1998; Walker, Stieber, Ramsey, & O'Neill, 1990). However, other research has indicated that ODR data for individual students have been much less predictive of poor outcomes than teacher ratings or direct observation (Walker et al., 1990) and that a teacher's use of such data may be influenced by a variety of variables such as classroom management, discipline policies, and teacher tolerance (Sugai et al., 2000; Wright & Dusek, 1998). Additional research has compared ODR data with teachers' ratings on the Teacher Report Form (Auchenbach, 1991) and found that the use of ODR records failed to identify many students whom teachers rated as meeting borderline or clinical cutoff scores for students at risk for emotional and behavioral disorders (Nelson et al., 2002). Although the use of ODR records is one way of measuring outcomes of schoolwide PBS interventions, these records measure negative and ineffective behaviors rather than positive replacement behaviors that are being taught and reinforced. Creating a way to measure the socially appropriate behaviors of youth as they demonstrate their mastery of social skills and other positive behaviors could Vol. 54, No. 2 be a meaningful and important component of evaluating intervention outcomes. In addition, carefully reviewing praise note data (e.g., frequency, content, distribution) may be an alternative means of documenting progress toward PBS goals. Summary This descriptive research considered the use of praise notes to evaluate PBS-focused objectives. Specifically, we considered how instructing teachers about praise and then reinforcing teachers' use of praise notes to students demonstrating competency with social skills would influence ODRs. The use of written praise notes has not been adequately explored in the research literature, especially in the middle or junior high school setting. Method Participants and Setting Participants were 70 teachers (48 women, 22 men) and 1,809 sixth- and seventh-grade students (927 boys [51%], 882 girls [49%]; 86% Caucasian, 11% Hispanic, 1% Native American, and 1% Pacific Islander, African American, or Asian) at secondary schools in the western part of the United States. Approximately 39% of these students qualified for free or reduced-price lunch. This school was in the 3rd year of implementing a schoolwide PBS model. A school planning committeecomprising school administration, selected teachers, and representatives from a local universitydiscussed concerns and developed schoolwide goals. School faculty and staff members addressed these goals by providing social skills lessons, instructing students on expectations for their behavior, and agreeing to increase positive feedback to students. Procedure We instructed the teachers that during this study, which was conducted across 2 consecutive school years, they were to write praise notes to students whose behavior exemplified schoolwide PBS goals. At the beginning of the school year, as a part of a 2-day PBS training sequence, teachers were taught how to effectively praise students. Teachers were given blank praise notes with instructions on how to fill them out. Measures Praise notes were printed in triplicate on no-carbonrequired paper. Students were given the original copy. Teachers turned in a copy for drawings and prizes; we used this copy for data analysis. Last, the third copy was given to parents during parent-teacher conferences. Praise note data (e.g., name of student, name of teacher, date, behavior for which the student was praised) were entered into Peterson Nelson, Young, Young, & Cox 121 a database. Fewer than 1% of notes were incomplete and therefore eliminated from the analyses. The names of students who had received praise notes were entered into a weekly drawing for a candy bar, and winners were recognized during morning announcements. Teachers were given neither incentives to write praise notes nor feedback regarding the notes they had written. During the first 7 months of the intervention, rates of notes written were somewhat low (0-2 praise notes per 100 students per day). To increase praise note rates, teachers were given reinforcements during the final 2 months of the school year and throughout the 9 months of the following school year. Teachers received gift certificates to local restaurants when they reached benchmark numbers of notes written (e.g., 25, 60, 100, 150). The requirements to earn gift certificates increased slightly during the 2nd year of the study because teachers exceeded goals from the previous year. The number of praise notes written was reported to teachers, and praise notes were placed in a box for drawings for prizes during faculty meetings. Teachers were also given a list of students who had not received a praise note that year. Data Analysis We tracked students' ODRs using a district-maintained database and collected teacher-written praise notes for the 2005-2006 and 2006-2007 school years. Praise note and ODR data were analyzed quantitatively using SPSS statistical analysis software (Version 15.0). The unit of analysis was number of praise notes written per day per 100 students. This measure allowed for all months to be compared equally despite differences in number of days per month. It was also consistent in the event of changes in student body size. The unit of analysis for ODRs was also number of ODRs written per 100 students per day. We used bivariate correlations to examine the relation between total praise notes written and number of ODRs for each month. In addition, data were analyzed separately for a subgroup of students who had received one or more ODRs to determine whether students with ODRs received praise notes at the same rate as students without ODRs and to determine whether receiving praise notes influenced ODRs to students who had previously received one or more ODR. For these analyses, we divided the student body into two groups: (a) students who received at least one ODR during the study, and (b) students who did not receive any ODRs. Students who received praise were categorized similarly. Again, we used bivariate correlations to examine the relation between praise notes received and ODRs received among students in this subgroup. To test whether students who did not receive ODRs were praised more frequently than were students who received at least one ODR, we used a test for difference between independent correlations. This procedure examines whether two correlations significantly differ. For 122 Preventing School Failure this analysis, we converted each correlation coefficient into a Fischer's z and ran a z test. Figure 1 shows the number of praise notes written per 100 students per day, and Figure 2 shows the number of ODRs written per day. Results Over the course of this 2-year study, 14,527 praise notes were written, and 2,143 ODRs were recorded (see Figures 1 and 2). There was a significant negative correlation between the total number of praise notes written to the student body and the number of ODRs for the student body (r .551, p .05), indicating that, as praise notes increased, ODR rates decreased. In addition, for the subgroup of students who received at least one ODR, there was a significant negative correlation between praise notes received and number of ODRs: As praise notes increased among students with at least one ODR, their rates of ODR decreased (r .553, p .05). Teachers wrote an average of 0.88 notes per day per 100 students during the first 7 months of the study. Praise notes written increased 672% to an average of 5.91 notes per day per 100 students for the remaining 2 months of the 1st year and the entire 9 months of the 2nd school year when incentives were given to teachers for writing praise notes. Further examination of the data revealed that 28.4% of all students received one or more ODRs during the study. Students with ODRs received 5.2 praise notes per day per 100 students, whereas students with no ODRs received 7.5 praise notes per day per 100 students. Although students with ODRs were praised slightly less than the rest of the student body, praise trends for students with ODRs were significantly correlated with praise trends for the rest of the student body, r .94, p .001. A test for difference between independent correlations indicated no statistical differences Vol. 54, No. 2 between praise trends for students with ODRs and praise trends for the rest of the student body, indicating that all students were praised at similar rates: z .02, p .05. Discussion The general aim of this study was to explore how teachers' use of praise notes to students demonstrating competency with social skills would influence ODRs. The results showed that praise notes and ODRs had a significant negative correlation: As praise notes increased, rates of ODR decreased. Hence, the data provide some evidence that increasing teacher praise notes may have been influencing the decrease in ODRs. However, more closely controlled research is needed. As with any descriptive research, the results of this study should be considered as correlationalnot causalrelations. There are several variables that could have contributed to a decrease in ODRs: Social skill instruction may have been a sufficient intervention to decrease ODRs. Also, ODRs may have decreased as administrators and teachers became more skilled in responding to behaviors that led to ODRs. It is also possible that in noticing and praising positive student behavior, teachers may have overlooked or become less focused on inappropriate behaviors. Although the cause of lower ODR rates cannot be determined by this descriptive study, it appears that teacher praise contingent upon the use of social skills had positive outcomes for students and for the overall school climatereinforcing positive behaviors and decreasing rates of ODR. Recommendations for Principals and Administrators We implemented this intervention to reach PBS goals and encourage the use of social skills. When several FIGURE 1. Average number of praise notes per day per 100 students, by month. Vol. 54, No. 2 Peterson Nelson, Young, Young, & Cox 123 FIGURE 2. Average number of office disciplinary referrals per day for 100 students, by month. important strategies were in place, teachers' use of praise notes increased. Teachers received specific and targeted instruction about praise at the beginning of the school year, including reminders that praise should be delivered sincerely and that praise notes should reinforce specific and directly communicated behavioral expectations. The effectiveness of praise notes would probably have more meaning and focus when integrated into a comprehensive PBS model. Toward the end of the school year, we reviewed the data with teachers and administrators, who were then encouraged and reinforced for writing praise notes. Periodically, teachers who had written praise notes had their names placed in a lottery for prizes to be given during faculty meetings. In addition, when teachers had written 25 praise notes, they were given a gift certificate to a local restaurant. Certificates to higher quality restaurants were given when the teacher had written 60, 100, and 150 praise notes. This monitoring and public reporting seems to have been vital to motivating and reinforcing teachers. Teachers were made aware of those students who had not yet received a praise note and were encouraged to watch for positive behaviors of students who had not been recognized. Some teachers expressed concerns that they were being encouraged to write notes to students with behavior problems, whereas a few students with appropriate behaviors may have been overlooked. Other teachers opined that only exemplary students deserved the recognition. Additional training, combined with data, addressed these concerns and reminded teachers that the purpose of praise notes was to reinforce the use of social skills that were being taught weekly in the classrooms. Even students with behavior problems showed appropriate social skills at times, and it was appropriate to recognize their efforts. It is possible that the praise notes written to students with a history of behavior problems contributed to the decrease in the number of ODRs. Hence, encouraging teachers to recognize specific, positive behaviors of all students was an important component of this project. It was not until data on praise notes were summarized and feedback was given to teachers that they began to recognize and understand the effect of praise notes on student behavior and ODRs. Summarizing the data on praise notes took a fair amount of time and resources for counting notes, as did determining those students who had or had not received them and monitoring which teachers had or had not been writing notes. However, it appears that these resources were appropriately spent because fewer ODRs were given, meaning that more students were in the classroom and receiving instruction. Processes for recording data were funded as part of a major research project; however, in schools without such funding, teachers could assign students or parent volunteers to assist or make data gathering and analysis a project for mathematics or statistics courses or service-learning classes if the information was not considered confidential. In addition, teachers need to be informed that the number of praise notes they write would become public information in the school. Depending on factors such as administrators' style or school culture, identifying information about teachers' rates of praise notes written may need to remain confidential, although we found that providing intermittent feedback, which included specific data regarding the effectiveness of praise notes, increased teachers' motivation to praise students. Lessons Learned Teachers and administrators implementing praise notes as a schoolwide intervention can benefit from the lessons learned by following these recommendations: 124 Preventing School Failure Vol. 54, No. 2 1. Provide teachers with ongoing, specific instruction on effective praise.1 Consistently encourage teachers to write praise notes; our intervention was discussed in faculty meetings, and teachers were reminded to deliver praise to students using the social skills that were being taught schoolwide. Teachers' personal stories of how praise notes made a positive difference were shared in faculty meetings or in e-mails. 2. Give teachers specific feedback regarding the number of notes written and the names of students who had and had not received praise notes. This feedback increased teachers' awareness of their progress toward a schoolwide goal and facilitated data-based decision making. 3. Review praise notes and ODR data with teachers to demonstrate the possible effect of written praise on student behavior and ODRs. 4. Most important, reinforce teachers for writing praise notes. When teachers had an opportunity to earn gift certificates for praising students, the total number of praise notes written dramatically increased. 5. This intervention seemed to be most effective and easily implemented as a component of an integrated PBS model designed to increase students' social skill use. The combination of these strategies appeared to significantly influence rates of praise notes. ODRs decreased. This approachwhich emphasizes positive and preventive measures rather than punitive and reactionary measuresappears to be beneficial in creating a more positive school environment. Limitations and Future Research As with any descriptive research, the findings of this study are correlational, and causal relations should not be assumed. Several variables may have influenced the decrease in ODRs, such as teachers' skills in responding to students' inappropriate behavior, administrators' skills at teaching more positive behaviors to students sent to the office for discipline purposes, and effects of reporting data to faculty. Moreover, teachers and administrators were making concentrated efforts to decrease rates of ODR by responding more positively to students as part of the PBS model. In addition, this post hoc analysis lacked a priori awareness of other confounding factors not controlled for or identified before or during the research that may have influenced the outcomes. The results of this study may have limited generalizability to other schools. Administrators and teachers at this school were responsive to innovative methods implemented to achieve PBS goals. In addition, this school had limited ethnic diversity. These contextual factors should be considered in future applications. Moreover, other PBS goals and objectives were being implemented at this school that could have influenced the results. Regardless of limitations, the findings of this study are encouraging. When teachers and administrators attended to positive behaviors, praising students for demonstrating social skills, students' problem behaviors and, consequently, REFERENCES NOTE 1. One Web-based article that could be used to educate teachers and administrators about the steps for delivering praise is titled High Rates of Positive Responses, available from http://www.usu .edu/teachall/text/behavior/LRBIpdfs/Hrates.pdf. AUTHOR NOTES Julie A. Peterson Nelson is a visiting assistant professor in the psychology department at Brigham Young University. Her research interests include gender issues in social development, praise, and sexual harassment. Ellie L. Young is an associate professor and coordinator of the school psychology program at Brigham Young University. Her research interests include gender and educational issues, as well as working with youth with behavioral and emotional concerns. Benjamin J. Young is in mental health counseling and is a data and technology specialist at Brigham Young University's Positive Behavior Support Initiative. His research interests include positive psychology, early intervention and screening, and technology in schools. Gregory Cox is a public school superintendent currently working in a school district on the Wind River Indian Reservation in Wyoming. His research interests include the practical application of educational and behavioral research to ongoing student and school improvement. American Federation of Teachers. (1995-1996). Elements of an effective discipline strategy. American Educator, 19(4), 24-27. Auchenbach, T. M. (1991). Manual for the Teacher Report Form. Burlington: University of Vermont Press. Brophy, J. (1981). Teacher praise: A functional analysis. Review of Education Research, 51, 5-32. Elam, S. J., Rose, L. C., & Gallup, A. M. (1998). 28th annual Phi Delta Kappa/Gallup poll of the public's attitudes toward the public schools. Kappan, 80, 41-59. Ferguson, E., & Houghton, S. (1992). The effects of contingent teacher praise, as specified by Canter's assertive discipline. Educational Studies, 18(1), 11-18. Gresham, F. M. (1998). Social skills training: Should we raze, remodel, or rebuild? Behavioral Disorders, 24, 19-25. Knoff, H. M. (2003). Character education vs. social skills training: Comparing constructs vs. behavior. Communiqu, 32(3), 39-40. Lewis, T. J., & Bullock, L. M. (2004). Scientifically supported practices in emotional and behavioral disorders: A proposed approach and brief review of current practices. Behavioral Disorders, 29, 247-259. Lewis, T. J., Chard, D., & Scott, T. (1994). Full inclusion and the education of children and youth with emotional and behavioral disorders. Behavioral Disorders, 19, 277-293. Lewis, T. J., & Garrison-Harrell, L. (1999). Effective behavior support: Designing setting specific interventions. Effective School Practices, 17(4), 38-46. Lewis, T. J., & Sugai, G. (1996). Descriptive and experimental analysis of teacher and peer attention and the use of assessment based intervention to improve the pro-social behavior of a student in a general education setting. Journal of Behavioral Education, 6, 7-24. Vol. 54, No. 2 Lewis T. J., & Sugai, G. (1999). Effective behavior support: A systems approach to proactive schoolwide management. Focus on Exceptional Children, 31(6), 24-47. Lipsey, M. W. (1991). The effect of treatment on juvenile delinquents: Results from meta-analysis. In F. Losel, D. Bender, & T. Bliesener (Eds.), Psychology and law (pp. 131-143). New York: Walter de Gruyter. Madsen, C. H., Becker, W. C., & Thomas, D. R. (1968). Rules, praise, ignoring: Elements of elementary classroom control. Journal of Applied Behavior Analysis, 1, 139-150. Mayer, G. R. (1995). Preventing antisocial behavior in the schools. Journal of Applied Behavior Analysis, 28, 467-478. Mayer, G. R., & Sulzer-Azeroff, B. (1991). Interventions for vandalism. In G. Stoner, M. K. Shinn, & H. M. Walker (Eds.), Interventions for achievement and behavior problems (pp. 559-580). Washington, DC: National Association of School Psychologists. Mitchem, K. J., Young, K. R., & West, R. P. (2000). Changing student, parent, and faculty perceptions: School is a positive place. Equalizing Learning Opportunities, 35, 248-262. Nelson, J. R., Benner, G. J., Reid, R. C., Epstein, M. H., & Currin, D. (2002). The convergent validity of office discipline referrals with the CBCL-TRF. Journal of Emotional and Behavioral Disorders, 10, 181-197. Noguera, P. A. (2003). Schools, prisons, and social implications of punishment: Rethinking disciplinary practices. Theory Into Practice, 42, 341-350. Peacock Hill Working Group. (1991). Problems and promises in special education and related services for children and youth with emotional and behavioral disorders. Behavioral Disorders, 16, 299-313. Scott, T. M. (2001). A schoolwide example of positive behavioral support. Journal of Positive Behavioral Interventions, 3(2), 88-94. Shinn, M. R., Ramsey, E., Walker, H. M., Stieber, S., & O'Neill, R. E. (1987). Antisocial behavior in school settings: Initial differences in an at-risk and normal population. Journal of Special Education, 21, 69-84. Skiba, R. J., Peterson, R. L., & Williams, T. (1997). Office referrals and suspensions: Disciplinary intervention in middle schools. Education and Treatment of Children, 20, 295-315. Sprague, J. R., Walker, H. M., Stieber, S., Simonsen, B., Nishioka, V., & Wagner, L. (2001). Exploring the relationship between school discipline referrals and delinquency. Psychology in the Schools, 38, 197-206. Sugai, G. (1998). Postmodernism and emotional and behavioral disorders: Distraction or advancement. Behavior Disorders, 23, 171-177. Sugai, G., & Horner, R. (1994). Including students with severe behavior problems in general education settings: Assumptions, challenges, and solutions. In J. Marr, G. Sugai, & G. Tindal (Eds.), The Oregon conference monograph (Vol. 6, pp. 102-120). Eugene: University of Oregon. Sugai, G., & Lewis, T. (1996). Preferred and promising practices for social skill instruction. Focus on Exceptional Children, 24(4), 1-16. Sugai, G., Sprague, J. R., Homer, R. H., & Walker, H. M. (2000). Preventing school violence: The use of office discipline refer- Peterson Nelson, Young, Young, & Cox 125 rals to assess and monitor school-wide discipline intervention. Journal of Emotional and Behavioral Disorders, 8, 94-112. Sutherland, K. S., & Wehby, J. H. (2001). The effect of self-evaluation on teaching behavior in classrooms for students with emotional and behavioral disorders. Journal of Special Education, 35, 161-171. Thomas, J. (1991). You're the greatest! A few well-chosen words can work wonders in positive behavior reinforcement. Principal, 71, 32-33. Tobin, T., & Sugai, G. (1999). Using sixth-grade school records to predict school violence, chronic discipline problems, and high school outcomes. Journal of Emotional and Behavioral Disorders, 7, 40-53. Tobin, T., Sugai, G., & Colvin, G. (1996). Patterns in middle school discipline records. Journal of Emotional and Behavioral Disorders, 4, 82-94. Tobin, T., Sugai, G., & Colvin, G. (2000). Using discipline referrals to make decisions. NASSP Bulletin, 84, 106-117. Turnbull, A., Edmondson, H., Griggs, P., Wickham, D., Sailor, W, Freeman, R. et al. (2002). A blueprint for schoolwide positive behavior support. Exceptional Children, 68, 377-402. U.S. Department of Education. (1995). The condition of education, 1995. Washington, DC: National Center for Education Statistics. U.S. Department of Education. (2005). The condition of education, 2005. Washington, DC: National Center for Education Statistics. Walker, H. M, Colvin, G., & Ramsey, E. (1995). Antisocial behavior in school: Strategies and best practices. Pacific Grove, CA: Brooks/Cole. Walker, H. M., Homer, R. H., Sugai, G., Bullis, M., Sprague, J. R., Bricker, D. et al. (1996). Integrated approaches to preventing antisocial behavior patterns among school-age children and youth. Journal of Emotional and Behavioral Disorders, 4, 193-256. Walker, H. M., Shinn, M. R., O'Neill, R. E., & Ramsey, E. (1987). A longitudinal assessment of the development of antisocial behavior in boys: Rationale, methodology, and first-year results. Remedial and Special Education, 8(4), 7-16, 27. Walker, H. M., & Stieber, S. (1998). Teacher ratings of social skills as longitudinal predictors of long-term arrest status in a sample of at-risk males. Behavioral Disorders, 23, 222-230. Walker, H. M., Stieber, S., Ramsey, E., & O'Neill, R. E. (1990). School behavioral profiles of arrested versus nonarrested adolescents. Exceptionality, 1, 249-265. Ward, J. (1976). Behavior modification in education: An overview and model for programme implementation. Bulletin of the British Psychological Society, 29, 257-268. Winette, R. A., & Winkler, R. C. (1972). Current behavior modification in the classroom: Be still, be quiet, be docile. Journal of Applied Behavior Analysis, 5, 499-504. Wright, J. A., & Dusek, J. B. (1998). Research into practice: Compiling school base rates for disruptive behaviors from student disciplinary referral data. School Psychology Review, 27, 138-147. Young, K. R., Black, S., Marchant, M., Mitchem, K. J., & West, R. P. (2000, August). A teaching approach to discipline: An alternative to punishment. Marriage and Families, 9-15. Reproduced with permission of the copyright owner. Further reproduction prohibited without permission. What statistical test was used in Nelson et al., (2010)? Authors have used a descriptive technique to evaluate the correlation between the Praise notes and ODRs. They have effectively used the bivariate correlation to test an existence of correlation between the two events. Later they used the test for difference between independent correlations of students with and without ODRs. Did the authors use the correct statistical test? In other words, what was their rationale for using this test (i.e., were the variables discrete or continuous and was the test appropriate for this type of data?) The authors have correctly implemented the tests. With continuous variables, the correlation can be easily measured. One can also observe a linear relationship between praise notes and ODR, because of their similar judgmental outcome. The test was perfect the data as in case of discrete values the variations are large, but In continuous data much variation is due to inherent changes in variable and not just numerical approximation. What was the research question? How did the statistical test address and answer the research question? Research Question was \"whether teachers' use of praise notes to students demonstrating competency with social skills would influence number of ODRs?\" The statistical test established a negative correlation which is quite true to the inherent definition of ODR and praise notes. But, as mentioned in the discussion part of the study, there are several other factors which contribute to ODRs. Considering only praise notes as factor to be correlated will lead to a result but whether it is a considerably significant amongst other factors included or not, is still an unanswered question. How did the authors interpret the results of this study? Authors have very well crafted the results of the study. However, they have neglected an aspect that teachers were positively reinforced for praise notes. This led to their better approach in handling foul actions. The authors have also neglected the fact that Students have also responded to positive appraise from both parents and teachers as the praise notes when shared with parents also create a double positive impact . Using Teacher-Written Praise Notes to Promote a Positive Environment in a Middle School Julie A. Peterson Nelson, Benjamin J. Young, Ellie L. Young, and Gregory Cox ABSTRACT: Teachers in 1 middle school learned about the positive school adults intentionally seek opportunities to build and strengthen adult-youth relationships, they may actually be decreasing the likelihood that students will act out in the future (Young, Black, Marchant, Mitchem, & West, 2000). To meet educational goals, educators must do more than merely prevent antisocial behaviors; they must increase prosocial behaviors (Winette & Winkler, 1972). Unfortunately, schools in the United States continue to encourage punishment for problem behaviors (Noguera, 2003), which, in the absence of a positive schoolwide system of support, has been associated with increased aggression, vandalism, tardiness, truancy, and dropouts (Mayer & Sulzer-Azeroff, 1991). Other researchers have suggested that reinforcing positive behaviors rather than punishing inappropriate behaviors is an effective behavior-management system (Walker, Colvin, & Ramsey, 1995). Hence, creating a positive school environment requires emphasizing preventive and positive measures rather than punitive procedures (Mitchem, Young, & West, 2000). Research has empirically shown that one effective positive behavior-management strategy is teacher praise (Lewis & Bullock, 2004). Other research has identified a functional relation between teacher praise and student behavior. Specifically, when teachers praise appropriate behavior, disruptive behavior decreases (Madsen, Becker, & Thomas, 1968). Further research has shown that praise delivered appropriately increases students' on-task behavior (Ferguson & Houghton, 1992), classroom motivation (Thomas, 1991), and academic success (Sutherland & effects of writing praise notes to students, which is 1 component of a positive behavior support. The authors intended for this procedure to promote a positive school environment and reinforce the appropriate use of social skills. Also, the authors instructed the teachers to use a direct instruction model to teach social skills lessons during 1st-period classes and praise students when they effectively demonstrated these skills. The authors analyzed the data to determine whether students receiving praise notes were less likely to receive an office discipline referral (ODR). The data revealed a significant negative correlation between the number of praise notes and number of ODRs that students received, indicating that as praise notes increased, the rate of ODRs decreased. The authors provide several hypotheses for this relation. KEYWORDS: positive behavior support, praise, schoolwide intervention, teacher-written praise EFFECTIVE SCHOOLWIDE MANAGEMENT of disruptive behaviors is an ongoing national concern (Lewis & Sugai, 1999; Scott, 2001; Turnbull et al., 2002). School violence, discipline, and safety have been among the top concerns for U.S. educators (American Federation of Teachers, 1995-1996; Elam, Rose, & Gallup, 1998; U.S. Department of Education, 1995, 2005). When addressing students with problem behaviors, many schools continue to rely on punitive strategies (e.g., office or administrative disciplinary interventions, suspensions, expulsions) that do little to create a safe and positive educational environment (Lewis & Garrison-Harrell, 1999). These types of interventions tend to be reactionary rather than preventive and proactive. In addition, these types of responses do little to teach new behaviors or to increase the likelihood that positive replacement behaviors would be used in the future (Knoff, 2003). Punitive disciplinary measures can certainly be one approach to behavior management, but if punishment is the only approach used, student behaviors are unlikely to change over the long term. When administrators and other This study was funded in part by U.S. Office of Special Education Programs Grant No. H324C030124. Benjamin J. Young and Ellie L. Young contributed equally to this article. Address correspondence to Julie A. Peterson Nelson, Psychology Department, 1030 SWKT Brigham Young University, Provo, UT 84602, julie_nelson@byu.edu. Copyright 2010 Heldref Publications 119 120 Preventing School Failure Wehby, 2001). In addition, praise has been widely recommended as effective in providing encouragement to the student, building self-esteem, and promoting closer relationships between teachers and students (Brophy, 1981). Teacher praise appears to have several meaningful outcomes that contribute to increased positive behaviors. However, few researchers have explored the use of written praise, especially as it relates to students' use of social skills. In addition, the connection between written praise and a decrease in negative behaviors that lead to office disciplinary action has not been explored, especially among early adolescent youth. Hence, this article addresses how a specific positive behavioral intervention (i.e., written teacher praise) can be a useful strategy for junior high or middle school teachers and administrators to improve the school climate, possibly influencing problem behaviors and lowering rates of office disciplinary referrals (ODRs). Positive Behavior Support (PBS) PBS has been widely adopted to create positive behavioral expectations, teach appropriate behaviors, and manage problem behaviors (Scott, 2001), and its use for affecting students' behaviors and school climate is well documented (Sugai, 1998; Sugai & Horner, 1994; Walker et al., 1996). PBS is a team-based system designed to facilitate student success by using evidence-based interventions and preventive strategies at a schoolwide, classroom, or individual level. The PBS model creates positive behavioral expectations for students, makes these positive expectations explicit, and communicates them widely. For example, students recognize and can articulate the expectation that, while at school, they must behave in a respectful, safe, and caring way. This positive expectation is emphasized in schoolwide efforts rather than on the behaviors that students should not do. The behaviors are explicitly taught through direct instruction of social skills (Lewis & Sugai, 1999). Direct instruction of social skills teaches students the behavioral expectations of the school community. Just as a student could be expected to successfully complete a long division math problem only after specific and directed instruction in long division, students who have had direct instruction in social skills are more likely to enact those skills, thereby meeting the expectations of the adults in the school. When the student displays the newly learned social skill, peers and adults should respond positively, thereby reinforcing the desirable behavior (Gresham, 1998; Lewis, Chard, & Scott, 1994; Lewis & Sugai, 1996; Lipsey, 1991; Mayer, 1995; Peacock Hill Working Group, 1991; Sugai & Horner, 1994; Sugai & Lewis, 1996; Walker et al., 1996). Teacher praise should be coupled with social skills training to reinforce such skills by specifically praising the students for their positive behavior, thus increasing the possibility Vol. 54, No. 2 of the students' using the skill in the future. Understanding how written praise notes may influence disruptive behaviors, as measured by ODR, would add to the understanding of effective components of the PBS model. ODR ODRs have historically been used as an index of student behavior for guiding and developing schoolwide programs and interventions (Skiba, Peterson, & Williams, 1997; Sugai, Sprague, Homer, & Walker, 2000; Tobin & Sugai, 1999; Tobin, Sugai, & Colvin, 1996, 2000). The following three purposes for using ODR data have been identified: (a) as a guide in the development or selection of specific environmentally appropriate interventions (e.g., if a significant number of ODRs are being written for disruptive lunchroom behavior, interventions need to focus on teaching positive behaviors in the lunchroom), (b) as an outcome measure to evaluate the effectiveness of programs, and (c) as a screening procedure to identify students who may benefit from targeted interventions (Nelson, Benner, Reid, Epstein, & Currin, 2002). ODR data continue to have practical and empirical uses. On a practical level, ODRs are used to manage and monitor students with problematic behaviors. These data are easily obtained and monitored in most schools. ODRs can also be used to consider progress toward solving situational concerns (e.g., disruptive lunchroom behavior). On an empirical level, ODRs have been related to poor student outcomes such as school failure and juvenile delinquency (Shinn, Ramsey, Walker, Stieber, & O'Neill, 1987; Sprague et al., 2001; Tobin & Sugai, 1999; Walker, Shinn, O'Neill, & Ramsey, 1987; Walker & Stieber, 1998; Walker, Stieber, Ramsey, & O'Neill, 1990). However, other research has indicated that ODR data for individual students have been much less predictive of poor outcomes than teacher ratings or direct observation (Walker et al., 1990) and that a teacher's use of such data may be influenced by a variety of variables such as classroom management, discipline policies, and teacher tolerance (Sugai et al., 2000; Wright & Dusek, 1998). Additional research has compared ODR data with teachers' ratings on the Teacher Report Form (Auchenbach, 1991) and found that the use of ODR records failed to identify many students whom teachers rated as meeting borderline or clinical cutoff scores for students at risk for emotional and behavioral disorders (Nelson et al., 2002). Although the use of ODR records is one way of measuring outcomes of schoolwide PBS interventions, these records measure negative and ineffective behaviors rather than positive replacement behaviors that are being taught and reinforced. Creating a way to measure the socially appropriate behaviors of youth as they demonstrate their mastery of social skills and other positive behaviors could Vol. 54, No. 2 be a meaningful and important component of evaluating intervention outcomes. In addition, carefully reviewing praise note data (e.g., frequency, content, distribution) may be an alternative means of documenting progress toward PBS goals. Summary This descriptive research considered the use of praise notes to evaluate PBS-focused objectives. Specifically, we considered how instructing teachers about praise and then reinforcing teachers' use of praise notes to students demonstrating competency with social skills would influence ODRs. The use of written praise notes has not been adequately explored in the research literature, especially in the middle or junior high school setting. Method Participants and Setting Participants were 70 teachers (48 women, 22 men) and 1,809 sixth- and seventh-grade students (927 boys [51%], 882 girls [49%]; 86% Caucasian, 11% Hispanic, 1% Native American, and 1% Pacific Islander, African American, or Asian) at secondary schools in the western part of the United States. Approximately 39% of these students qualified for free or reduced-price lunch. This school was in the 3rd year of implementing a schoolwide PBS model. A school planning committeecomprising school administration, selected teachers, and representatives from a local universitydiscussed concerns and developed schoolwide goals. School faculty and staff members addressed these goals by providing social skills lessons, instructing students on expectations for their behavior, and agreeing to increase positive feedback to students. Procedure We instructed the teachers that during this study, which was conducted across 2 consecutive school years, they were to write praise notes to students whose behavior exemplified schoolwide PBS goals. At the beginning of the school year, as a part of a 2-day PBS training sequence, teachers were taught how to effectively praise students. Teachers were given blank praise notes with instructions on how to fill them out. Measures Praise notes were printed in triplicate on no-carbonrequired paper. Students were given the original copy. Teachers turned in a copy for drawings and prizes; we used this copy for data analysis. Last, the third copy was given to parents during parent-teacher conferences. Praise note data (e.g., name of student, name of teacher, date, behavior for which the student was praised) were entered into Peterson Nelson, Young, Young, & Cox 121 a database. Fewer than 1% of notes were incomplete and therefore eliminated from the analyses. The names of students who had received praise notes were entered into a weekly drawing for a candy bar, and winners were recognized during morning announcements. Teachers were given neither incentives to write praise notes nor feedback regarding the notes they had written. During the first 7 months of the intervention, rates of notes written were somewhat low (0-2 praise notes per 100 students per day). To increase praise note rates, teachers were given reinforcements during the final 2 months of the school year and throughout the 9 months of the following school year. Teachers received gift certificates to local restaurants when they reached benchmark numbers of notes written (e.g., 25, 60, 100, 150). The requirements to earn gift certificates increased slightly during the 2nd year of the study because teachers exceeded goals from the previous year. The number of praise notes written was reported to teachers, and praise notes were placed in a box for drawings for prizes during faculty meetings. Teachers were also given a list of students who had not received a praise note that year. Data Analysis We tracked students' ODRs using a district-maintained database and collected teacher-written praise notes for the 2005-2006 and 2006-2007 school years. Praise note and ODR data were analyzed quantitatively using SPSS statistical analysis software (Version 15.0). The unit of analysis was number of praise notes written per day per 100 students. This measure allowed for all months to be compared equally despite differences in number of days per month. It was also consistent in the event of changes in student body size. The unit of analysis for ODRs was also number of ODRs written per 100 students per day. We used bivariate correlations to examine the relation between total praise notes written and number of ODRs for each month. In addition, data were analyzed separately for a subgroup of students who had received one or more ODRs to determine whether students with ODRs received praise notes at the same rate as students without ODRs and to determine whether receiving praise notes influenced ODRs to students who had previously received one or more ODR. For these analyses, we divided the student body into two groups: (a) students who received at least one ODR during the study, and (b) students who did not receive any ODRs. Students who received praise were categorized similarly. Again, we used bivariate correlations to examine the relation between praise notes received and ODRs received among students in this subgroup. To test whether students who did not receive ODRs were praised more frequently than were students who received at least one ODR, we used a test for difference between independent correlations. This procedure examines whether two correlations significantly differ. For 122 Preventing School Failure this analysis, we converted each correlation coefficient into a Fischer's z and ran a z test. Figure 1 shows the number of praise notes written per 100 students per day, and Figure 2 shows the number of ODRs written per day. Results Over the course of this 2-year study, 14,527 praise notes were written, and 2,143 ODRs were recorded (see Figures 1 and 2). There was a significant negative correlation between the total number of praise notes written to the student body and the number of ODRs for the student body (r .551, p .05), indicating that, as praise notes increased, ODR rates decreased. In addition, for the subgroup of students who received at least one ODR, there was a significant negative correlation between praise notes received and number of ODRs: As praise notes increased among students with at least one ODR, their rates of ODR decreased (r .553, p .05). Teachers wrote an average of 0.88 notes per day per 100 students during the first 7 months of the study. Praise notes written increased 672% to an average of 5.91 notes per day per 100 students for the remaining 2 months of the 1st year and the entire 9 months of the 2nd school year when incentives were given to teachers for writing praise notes. Further examination of the data revealed that 28.4% of all students received one or more ODRs during the study. Students with ODRs received 5.2 praise notes per day per 100 students, whereas students with no ODRs received 7.5 praise notes per day per 100 students. Although students with ODRs were praised slightly less than the rest of the student body, praise trends for students with ODRs were significantly correlated with praise trends for the rest of the student body, r .94, p .001. A test for difference between independent correlations indicated no statistical differences Vol. 54, No. 2 between praise trends for students with ODRs and praise trends for the rest of the student body, indicating that all students were praised at similar rates: z .02, p .05. Discussion The general aim of this study was to explore how teachers' use of praise notes to students demonstrating competency with social skills would influence ODRs. The results showed that praise notes and ODRs had a significant negative correlation: As praise notes increased, rates of ODR decreased. Hence, the data provide some evidence that increasing teacher praise notes may have been influencing the decrease in ODRs. However, more closely controlled research is needed. As with any descriptive research, the results of this study should be considered as correlationalnot causalrelations. There are several variables that could have contributed to a decrease in ODRs: Social skill instruction may have been a sufficient intervention to decrease ODRs. Also, ODRs may have decreased as administrators and teachers became more skilled in responding to behaviors that led to ODRs. It is also possible that in noticing and praising positive student behavior, teachers may have overlooked or become less focused on inappropriate behaviors. Although the cause of lower ODR rates cannot be determined by this descriptive study, it appears that teacher praise contingent upon the use of social skills had positive outcomes for students and for the overall school climatereinforcing positive behaviors and decreasing rates of ODR. Recommendations for Principals and Administrators We implemented this intervention to reach PBS goals and encourage the use of social skills. When several FIGURE 1. Average number of praise notes per day per 100 students, by month. Vol. 54, No. 2 Peterson Nelson, Young, Young, & Cox 123 FIGURE 2. Average number of office disciplinary referrals per day for 100 students, by month. important strategies were in place, teachers' use of praise notes increased. Teachers received specific and targeted instruction about praise at the beginning of the school year, including reminders that praise should be delivered sincerely and that praise notes should reinforce specific and directly communicated behavioral expectations. The effectiveness of praise notes would probably have more meaning and focus when integrated into a comprehensive PBS model. Toward the end of the school year, we reviewed the data with teachers and administrators, who were then encouraged and reinforced for writing praise notes. Periodically, teachers who had written praise notes had their names placed in a lottery for prizes to be given during faculty meetings. In addition, when teachers had written 25 praise notes, they were given a gift certificate to a local restaurant. Certificates to higher quality restaurants were given when the teacher had written 60, 100, and 150 praise notes. This monitoring and public reporting seems to have been vital to motivating and reinforcing teachers. Teachers were made aware of those students who had not yet received a praise note and were encouraged to watch for positive behaviors of students who had not been recognized. Some teachers expressed concerns that they were being encouraged to write notes to students with behavior problems, whereas a few students with appropriate behaviors may have been overlooked. Other teachers opined that only exemplary students deserved the recognition. Additional training, combined with data, addressed these concerns and reminded teachers that the purpose of praise notes was to reinforce the use of social skills that were being taught weekly in the classrooms. Even students with behavior problems showed appropriate social skills at times, and it was appropriate to recognize their efforts. It is possible that the praise notes written to students with a history of behavior problems contributed to the decrease in the number of ODRs. Hence, encouraging teachers to recognize specific, positive behaviors of all students was an important component of this project. It was not until data on praise notes were summarized and feedback was given to teachers that they began to recognize and understand the effect of praise notes on student behavior and ODRs. Summarizing the data on praise notes took a fair amount of time and resources for counting notes, as did determining those students who had or had not received them and monitoring which teachers had or had not been writing notes. However, it appears that these resources were appropriately spent because fewer ODRs were given, meaning that more students were in the classroom and receiving instruction. Processes for recording data were funded as part of a major research project; however, in schools without such funding, teachers could assign students or parent volunteers to assist or make data gathering and analysis a project for mathematics or statistics courses or service-learning classes if the information was not considered confidential. In addition, teachers need to be informed that the number of praise notes they write would become public information in the school. Depending on factors such as administrators' style or school culture, identifying information about teachers' rates of praise notes written may need to remain confidential, although we found that providing intermittent feedback, which included specific data regarding the effectiveness of praise notes, increased teachers' motivation to praise students. Lessons Learned Teachers and administrators implementing praise notes as a schoolwide intervention can benefit from the lessons learned by following these recommendations: 124 Preventing School Failure Vol. 54, No. 2 1. Provide teachers with ongoing, specific instruction on effective praise.1 Consistently encourage teachers to write praise notes; our intervention was discussed in faculty meetings, and teachers were reminded to deliver praise to students using the social skills that were being taught schoolwide. Teachers' personal stories of how praise notes made a positive difference were shared in faculty meetings or in e-mails. 2. Give teachers specific feedback regarding the number of notes written and the names of students who had and had not received praise notes. This feedback increased teachers' awareness of their progress toward a schoolwide goal and facilitated data-based decision making. 3. Review praise notes and ODR data with teachers to demonstrate the possible effect of written praise on student behavior and O