[b] effects of classroom c... + EFFECTS OF CLASSROOM CELL PHONE USE ON EXPECTED AND ACTUAL LEARNING ARNOLD D. FROESE CHRISTINA N. CARPENTER DENYSE A. INMAN JESSICA R. SCHOOLEY REBECCA B. BARNES PAUL W. BRECHT JASMIN D. CHACON Sterling College Studies of driving indicate that the conversational aspects of using cell phones generate high risks from divided attention. Prior surveys document high rates at which students carry phones to and use them during class. Some experiments have demonstrated that cell phones distract students from learning. The present studies combined survey and experimental methods to determine student expectations about and actual performance under cell phone use conditions. On the survey, students esti- mated the number of questions they could answer out of 10 when texting and when not texting. For the experiment, we used a repeated measures design with simulated classroom presenta- tions and measured performance on a 10-item quiz. Students expected to lose close to 30% on a quiz and actually did lose close to 30% when texting. We discuss implications of our methodology and our findings for improving student learning. Studies of drivers using cell phones reveal that the cognitive distraction of con- versations significantly increases accident risk. The National Safety Council (2010) published a literature review explaining why cognitive load from cell phones pro- duces inattention blindness for drivers. Strayer and Johnston (2001) showed that listening to music or even to a recorded book did not produce high accident risks, as did conversing on cell phones. These findings are important for con- sidering the potential effects of classroom texting on students' ability to learn pre- sented material. Texting is conversational, though it involves visual instead of audi- tory "listening" as students read incoming 323 messages, and manual instead of verbal "talking" as they reply. If conversational cognitive load increases accident risk for drivers, the same cognitive load should increase errors on tests of lesson material presented while students are texting. Survey Research Researchers have explored the dis- tracting effects of cell phones in classrooms using surveys. Many students admit to using cell phones for social networking purposes in the classroom (Bayer, Klein, & Rubinstein, 2009; Besser, 2007; Kennedy & Smith, 2010; Rubinkam, 2010). Some studies documented percep- tions of distraction from phone ringing 324/College Student Journal (Campbell, 2006) and from texting or send- ing instant messages during a class or study session (Besser, 2007; Kennedy & Smith, 2010; Levine, Waite, & Bowman, 2007). These studies employed survey responses to evaluate effects. The typical measurement scales for such reports are quantitatively weak. For example, Besser (2007) and Kennedy and Smith (2010) measured student percep- tions of the effects of cell phone use on class performance using statements with which respondents either agreed or dis- agreed, Besser's statement was about cell phone use as a serious safety risk. Nev- ertheless, 46% of those claiming that such use was an "extremely serious risk still reported using their phones while drivi within 30 days prior to the intervi Kennedy and Smith (2010) reported si ilar discrepancies in student behavi Although students generally "agreed" tha cell phones disrupted classroom learning, they persisted in using their cell phones in the classroom. Levels of agreement do not clearly indicate the size of the expected effect. If respondents agree that risk is increased, but perceive that the risk is low, 6:17 (2010) published a literature review explaining why cognitive load from cell phones pro- duces inattention blindness for drivers. Strayer and Johnston (2001) showed that listening to music or even to a recorded book did not produce high accident risks, as did conversing on cell phones. These findings are important for con- sidering the potential effects of classroom texting on students' ability to learn pre- sented material. Texting is conversational, though it involves visual instead of audi- tory "listening" as students read incoming 323 drivers, the same cognitive load should increase errors on tests of lesson material presented while students are texting. Survey Research Researchers have explored the dis- tracting effects of cell phones in classrooms using surveys. Many students admit to using cell phones for social networking purposes in the classroom (Bayer, Klein, & Rubinstein, 2009; Besser, 2007; Kennedy & Smith, 2010; Rubinkam, 2010). Some studies documented percep- tions of distraction from phone ringing 2/11 324/College Student Journal (Campbell, 2006) and from texting or send ing instant messages during a class or study session (Besser, 2007; Kennedy & Smith, 2010; Levine, Waite, & Bowman, 2007). These studies employed survey responses to evaluate effects. The typical measurement scales for such reports are quantitatively weak. For example, Besser (2007) and Kennedy and Smith (2010) measured student percep- tions of the effects of cell phone use on class performance using statements with which respondents either agreed or dis- agreed. Besser's statement was about texting drawing attention away from class, and Kennedy and Smith's statement was about these activities helping class per- formance. These nominal measurements do not provide information about the quan- tity of expected information loss. Other researchers (Campbell, 2006; Levin, Waite, & Bowman, 2007) have expanded the num- ber of response options. For example, Campbell (2006) used a 5-point Likert scale ranging from strongly agree to strongly disagree to evaluate student atti- tudes about the disruptive effects of ringing phones. Although these scales increase response variability, there is no clear rela- tionship between level of agreement with a statement such as "when a mobile phone rings during class, it is a serious distrac- tion" and any quantity of information loss. The absence of clarity about the expect- ed size of the effect presents additional interpretive problems. Some researchers have found a difference between expressed attitudes about phone risks and actual behavior. An American Automobile Asso- ciation Foundation for Traffic Safety (2008) survey showed that drivers viewed cell phone use as a serious safety risk. Nev- ertheless, 46% of those claiming that such use was an "extremely serious risk" still reported using their phones while driving within 30 days prior to the interview. Kennedy and Smith (2010) reported sim- ilar discrepancies in student behavior. Although students generally "agreed" that cell phones disrupted classroom learning, they persisted in using their cell phones in the classroom. Levels of agreement do not clearly indicate the size of the expected effect. If respondents agree that risk is increased, but perceive that the risk is low, they may feel justified in ignoring the risk. Experimental Research Some researchers have employed exper- imental techniques to assess actual effects of cell phone activity on classroom-relat- ed performance. Bowman, Levine, Waite, and Gendron (2009) and Fox, Rosen, and Crawford (2009) compared comprehen- sion scores for students who were or were not sending instant messages during a non- class reading task. Neither study revealed differences in comprehension, but com- pleting the reading took significantly longer for those engaged in instant messaging. These results do not generalize to a lec- ture or discussion-based classroom environment where students do not con- trol the timing of information. Other researchers have experimentally explored distraction from a cell phone ring- ing in a classroom. In two studies, researchers compared classroom scores for material when no phone was ringing to scores when a phone was ringing (End, Worthman, Mathews, & Wetterau, 2010; Shelton, Elliott, Eaves, & Exner, 2009). In Effects of Cell Phone Use on Learning... / 325 both studies, performance deteriorated sig- nificantly for material presented during the ringing condition. Performance decrements ranged from 25-40% during ringing of expected quiz score changes with exper- imental performance scores. Finally, we designed an experiment that approximat- ed both the classroom environment and phones. Although response variability, there is no clear rela- tionship between level of agreement with a statement such as "when a mobile phone rings during class, it is a serious distrac- tion" and any quantity of information loss. The absence of clarity about the expect- ed size of the effect presents additional interpretive problems. Some researchers have found a difference between expressed attitudes about phone risks and actual behavior. An American Automobile Asso- ciation Foundation for Traffic Safety (2008) survey showed that drivers viewed for those engaged in instant messaging. These results do not generalize to a lec- ture or discussion-based classroom environment where students do not con- trol the timing of information. Other researchers have experimentally explored distraction from a cell phone ring- ing in a classroom. In two studies, researchers compared classroom scores for material when no phone was ringing to scores when a phone was ringing (End, Worthman, Mathews, & Wetterau, 2010; Shelton, Elliott, Eaves, & Exner, 2009). In 3/11 Effects of Cell Phone Use on Learning... / 325 both studies, performance deteriorated sig- nificantly for material presented during the ringing condition. Performance decrements ranged from 25-40% during ringing periods. These two studies addressed dis- traction effects for bystanders and left open the question of distraction for texting per- formers. Ellis, Daniels, and Jauregui (2010) most directly assessed the effects of texting on performers in a real classroom context. Students in the experimental condition sent three text messages to the instructor dur- ing the lecture. The control group presumably had turned their phones off. Experimental students scored significant- ly lower than control students did on a pop quiz at the end of class. Although this experiment comes directly from a class- room setting, sending a text message to a teacher who does not respond is likely not as distracting as a conversational texting dialogue. Purpose The above studies begin to explore how texting changes classroom learning. How ever, their limitations suggested the following research strategies. First, we designed both a survey to assess how much information students thought they would lose if they were texting, and a corre- sponding experiment to explore the actual loss of information. Second, we generat- ed a survey response scale that had stronger numerical properties than dichotomous or Likert-scale response options. Third, our survey response scale had numerical prop- erties that matched those of our experimental outcome variable. This match allowed us to compare quantity estimates of expected quiz score changes with exper- imental performance scores. Finally, we designed an experiment that approximat- ed both the classroom environment and students' texting experiences. Hearing a cell phone ringing in a class distracts learn- ers from lesson content. However, if increased cognitive load explains learning deficits from texting distraction, the most invasive distraction should occur for students actively engaged in texting con- versations during a class. Implementing these developments permitted us to com- pare expected and actual effects of non-class-related texting on classroom learning. We expected that students would be aware of learning decrements produced by texting, and that their actual perfor- mance would confirm that expectation. Study 1 This study employed a self-report sur- vey to assess students' cell phone activity in classes and their expectations of the effects of such activity on learning out- comes. Unlike previous studies using self-report measures, we created a mea- sure of anticipated learning deficits from texting based on measurements common to classroom settings. Method Participants. We collected surveys from 693 students at seven colleges and universities across the United States dur- ing October through December, 2009. Seven teachers at these schools adminis- tered the surveys in their classes during class time. Participants' average age was 20.5 years. Ninety-nine percent owned cell phones. They had owned cell phones an 326/College Student Journal Table 1 Verbal and Quantitative Comparison of Self-Described Texting How Often Do You Text in a Day? ed a survey response scale that had stronger numerical properties than dichotomous or Likert-scale response options. Third, our survey response scale had numerical prop- erties that matched those of our experimental outcome variable. This match allowed us to compare quantity estimates universities across the United States dur- ing October through December, 2009. Seven teachers at these schools adminis- tered the surveys in their classes during class time. Participants' average age was 20.5 years. Ninety-nine percent owned cell phones. They had owned cell phones an 326/College Student Journal Table 1 Verbal and Quantitative Comparison of Self-Described Texting How Often Do You Text in a Day? 0-25 26-50 51-75 76-100 100+ times times times times times Total How Would Emergency-only 5 0 0 00 5 You Describe Minimal 53 7 1 1 1 63 Yourself as a Moderate 84 87 46 23 14 25 Text User? Avid 21 54 70 76 139 34/11 Total 163 148 117 100 154 6 ::: average of 5.4 years and used texting func- Results tions an average of 4.1 years. Instrument. Our survey requested demographic information from students (summarized above), and information about frequency of carrying their phones and texting frequency in various daily activity contexts. Participants also esti- mated their expected learning performance if they texted during class. Our metric for performance was the question, "If you were listening to some information, and some- one asked you 10 factual questions about that information, estimate the number of questions you might be able to correctly answer?" Participants answered that ques- tion for two conditions-if they were and were not texting while they listened to the information. Procedure. Instructors read an intro- ductory script to their classes that provided instructions and the informed consent option of not completing the survey. Sur- veys were confidential, and students completed them during a 6-minute time limit. More than half (52.8%) of our respon- dents described themselves as "avid users" and 90% described themselves as moder- ate or avid users. These verbal categories corresponded with reported number of texts sent per day, r, (682) = .612, p Effects of Cell Phone Use on Learning.../327 328/ College Student Journal ticipants sat on different sides of the room to reduce distraction. Co-experimenters sat in the room across the hall. We told all participants that they would watch an informational presentation; they could take notes if they desired; and they should try to retain the presented infor- mation for a quiz following a 5-minute break. During the break, participants had access to refreshments. They were told not to discuss the content of the presentation. We identified the texting condition for each participant before each presentation. The texting participants set their phones on vibrate, and were free to respond imme- diately to any texts that arrived. The non-texting participants turned off the vibrate function, placed their phones out of sight and did not use their phones dur- ing the presentation. Following the first quiz, the groups switched conditions for the second presentation. The co-experimenters confirmed phone functionality with participants before the experiment began. Following confirma- tion, the experimenter signaled co-experimenters to begin texting the par- ticipants. When all texting participants received their first message, the experi- menter started the PowerPoint presentation. Co-experimenters exchanged messages as quickly as possible with assigned partici- pants throughout the presentation. We prepared a list of texting topics involving general introductory information, but allowed texting content to develop spon- taneously throughout the interactions. Results Quiz scores were significantly lower when students texted (M = 6.02, SD = 2.224) than when they did not text (M = 8.25, SD = 1.597), (39) = 5.34, p < .01, effect size (t/N)= .84. The difference in scores represented a 27% decline during texting from the non-texting performance. Neither the story during which they texted, nor the order of texting and non-texting, produced different results. For a convenience sample of 15 stu- dents, we recorded the time participants actually spent reading or texting on their phone during the texting phase. Partici- pants spent an average of 2.69 minutes engaged in texting during the presentation. The range of texting times was from 1.5 to 4.25 minutes. Time engaged in texting was negatively, though not significantly, cor- related with quiz score in the texting phase, r(13) 472, p = .076. Discussion Our data support a prior report (Ellis, Daniels, & Jauregui, 2010) of deleterious effects of texting on classroom learning.. Score reductions for texting conditions were greater in our experiment than in the prior experiment. Our methodological addition of conversational texting may account for our greater score reductions. Although the correlation between tex- ting time and texting score was not significant, the direction and size of the correlation leave open possibilities that level of engagement in texting is a factor in losing classroom information. Our method presents a stron evaluating the effects of texting 6/11 Effects of Cell Phone Use on Learning... / 329 ing. The counterbalanced, repeated-mea- sures design controlled subject and order variables. The pre-recorded presentations equated lesson materials for all partici- pants across testing sessions. Nevertheless, due to phone connectivity differences, par- ticipants spent widely differing amounts of time actually engaged in texting. We expect that methodological refinements could demonstrate even greater loss of information than we found. General Discussion Our research successfully implement ed a survey measure of students' expectations about the effects of texting on learning that was comparable to typi- cal classroom measures-predicted quiz scores. The measure is quantitatively strong-a ratio measurement scale-and easy for respondents to understand. The data confirmed that self-report measures can provide information that is verified in engaged participants in conversation, a pro- cedure that the driving studies (National Safety Council, 2010; Strayer & Johnston, 2001) suggested as a source of distraction and one that was missing from the Ellis, Daniels, and Jaurgui (2010) study. This engagement likely accounted for more information loss in our study than Ellis, Daniels, and Jaurgui (2010) found. Fur- thermore, the conversations occurred simultaneously with the lesson presenta- tion, unlike the studies reported by Bowman, Levine, Waite, and Gendron (2009) and Fox, Rosen, and Crawford (2009). The differences in information loss that we obtained, in contrast to Bowman, Levine, Waite, and Gendron (2009) and Fox, Rosen, and Crawford (2009) support the idea that cognitive load increases when information presentation conflicts with tex- ting communications. One remaining difference between our experimental set- ting and a real classroom is that some Effects of Cell Phone Use on Learning.../329 ing. The counterbalanced, repeated-mea- sures design controlled subject and order variables. The pre-recorded presentations equated lesson materials for all partici- pants across testing sessions. Nevertheless, due to phone connectivity differences, par- ticipants spent widely differing amounts of time actually engaged in texting. We expect that methodological refinements could demonstrate even greater loss of information than we found. General Discussion Our research successfully implement ed a survey measure of students' expectations about the effects of texting on learning that was comparable to typi- cal classroom measures-predicted quiz scores. The measure is quantitatively strong-a ratio measurement scale-and easy for respondents to understand. The data confirmed that self-report measures can provide information that is verified in experimental outcome studies. One remaining limitation is that students may fail to account for chance performance lev- els associated with multiple-choice questions. With four response alternatives, that chance level-25%-represents no significant learning. It is likely that those students who predicted scores lower than chance did not understand this baseline minimum. The texting manipulation in the simu- lated classroom environment more closely approximated texting during real class ses- sions than previous experiments. Students in the texting condition responded to mes- sages from their own friends as well as from co-experimenters. The messages engaged participants in conversation, a pro- cedure that the driving studies (National Safety Council, 2010; Strayer & Johnston, 2001) suggested as a source of distraction and one that was missing from the Ellis, Daniels, and Jaurgui (2010) study. This engagement likely accounted for more information loss in our study than Ellis, Daniels, and Jaurgui (2010) found. Fur- thermore, the conversations occurred simultaneously with the lesson presenta- tion, unlike the studies reported by Bowman, Levine, Waite, and Gendron (2009) and Fox, Rosen, and Crawford (2009). The differences in information loss that we obtained, in contrast to Bowman, Levine, Waite, and Gendron (2009) and Fox, Rosen, and Crawford (2009) support the idea that cognitive load increases when information presentation conflicts with tex- ting communications. One remaining difference between our experimental set- ting and a real classroom is that some students commented about how different it was to freely text during a classroom presentation. Our data confirm that students expect texting to disrupt their classroom learning, and that texting does disrupt learning. The real score declines (27%) approximated the expected declines (33%). The some- what higher expected declines could have occurred as students failed to account for the 25% chance baseline and from texting requirements that did not occupy all of the lesson time. The corresponding declines for self-report and experimental measure- ments suggest that students are aware that using cell phones for personal communi- cation in class compromises classroom 330/College Student Journal learning. Thus, our data support the value of self-reports of the effects of using cell phones on learning, at least as presented with the measurement tools we used. Survey participants varied considerably in their score predictions under texting con- ditions. Some participants expected no detrimental effects of texting. Similarly, experimental participants varied consid- erably in their quiz scores under texting conditions. Some texting participants answered all questions correctly. We do not know if each participant's expected and actual performance measures were cor- related because different participants completed the survey and the experiment. These data could reflect the same kind of discrepancies reported by the American Automobile Association Foundation for Traffic Safety (2008) between participants' expectations of safety risks for others but false immunity from risk for self. Further research could solicit information loss expectations from experimental partici- pants to determine whether students can accurately predict their own distractibili- ty. Stanovich (2009) summarized two aspects of rationality-epistemic and instrumental. Epistemic rationality exists 8/11 educated, yet choose to engage in coun- terproductive behaviors. Given that students generally expect texting to disrupt their learning, researchers can reasonably ask why students risk potential failure to maintain social con- tact? Wei and Wang (2010) recently explored two models of student motiva- tion for classroom texting. They predicted that instructor immediacy-making eye contact, calling students by name, talking with students outside of class, among other behaviors could enhance students' moti- vation to learn and thus reduce texting. Alternatively, students' habits and gratifi- cations they receive from the activity could maintain texting. Their data confirmed that immediacy enhanced motivation to learn, but that motivation did not correlate with texting rates. They concluded that the habits and gratifications model better fits their data. These results raise questions about how phone carrying habits and phone checking impulses relate to instructional variables. Students may benefit from know- ing whether carrying their phones to class increases their impulses to check for mes- sages. Likewise, teachers may want to know if interruptions to lesson flow increase students' urges to check their 330/College Student Journal learning. Thus, our data support the value of self-reports of the effects of using cell phones on learning, at least as presented with the measurement tools we used. Survey participants varied considerably in their score predictions under texting con- ditions. Some participants expected no detrimental effects of texting. Similarly, experimental participants varied consid- erably in their quiz scores under texting conditions. Some texting participants answered all questions correctly. We do not know if each participant's expected and actual performance measures were cor- related because different participants completed the survey and the experiment. These data could reflect the same kind of discrepancies reported by the American Automobile Association Foundation for Traffic Safety (2008) between participants' expectations of safety risks for others but false immunity from risk for self. Further research could solicit information loss expectations from experimental partici- pants to determine whether students can accurately predict their own distractibili- ty. Stanovich (2009) summarized two aspects of rationality-epistemic and instrumental. Epistemic rationality exists when a person's view of the way the world works matches the way it actually works. The correspondence of average expected and actual losses in our studies suggests a degree of epistemic rationality. Participants really do know what happens when stu- dents text. Instrumental rationality is evident when a person sets a goal and fol- lows appropriate steps to achieve that goal. Our data suggest deficits in instrumental rationality for students who pay to become educated, yet choose to engage in coun- terproductive behaviors. Given that students generally expect texting to disrupt their learning, researchers can reasonably ask why students risk potential failure to maintain social con- tact? Wei and Wang (2010) recently explored two models of student motiva- tion for classroom texting. They predicted that instructor immediacy-making eye contact, calling students by name, talking with students outside of class, among other behaviors could enhance students' moti- vation to learn and thus reduce texting. Alternatively, students' habits and gratifi- cations they receive from the activity could maintain texting. Their data confirmed that immediacy enhanced motivation to learn, but that motivation did not correlate with texting rates. They concluded that the habits and gratifications model better fits their data. These results raise questions about how phone carrying habits and phone checking impulses relate to instructional variables. Students may benefit from know- ing whether carrying their phones to class increases their impulses to check for mes- sages. Likewise, teachers may want to know if interruptions to lesson flow increase students' urges to check their phones. These possibilities present fertile ground for future research. Finally, faculty variations in handling texting events in classrooms may affect student behaviors in ways that alter learn- ing. Further research could explore differences between faculty and students in perceptions of the effects of texting as well as of techniques for handling unwant- ed texting in class. Knowing such perceptions and the effectiveness of inter- Effects of Cell Phone Use on Learning.../331 vention techniques in the context of the demonstrated effects of texting could improve classroom environments and enhance student learning. Author Note Denyse A. Inman is now at the School of Behavioral Sciences, California Baptist University. Christina N. Carpenter is now at Bailey, Colorado. Jasmin D. Chacon is now at the Department of Psychology, Gal- laudet University. We thank Brian Allen, Andrea Mehringer, and Jeffrey Ropp for assistance in conducting the research, coding data, and discussing our ideas. Address correspondence concerning this article to Arnold D. Froese, Psychol- ogy Department, Sterling College, Sterling, KS 67579. E-mail: afroese46@gmail.com References American Automobile Association Foundation for Traffic Safety. (2008). 2008 Traffic Safety Cul- ture Index. Washington, DC: AAA Foundation for Traffic Safety. Downloaded from 8/11 Bowman, L. L., Levine, L. E., Wa Gendron, M. (2009). Can studen titask? An experimental study messaging while reading. Computers & Edu- cation, 54,927-931.doi: 10.1016/j.compedu.2 009.09.024. Campbell, S. W. (2006) Perceptions of mobile phones in college classrooms: Ringing, cheat- ing, and classroom policies. Communication Education, 55, 280-294. doi:10.1080/0363 4520600748573. Ellis, Y., Daniels, B., & Jauregui, A. (2010). The effect of multitasking on the grade perfor- mance of business students. Research in Higher Education Journal, 8, 1-10. End, C. M., Worthman, S., Mathews, M. B., & Wetterau, K. (2010). Costly cell phones: The impact of cell phone rings on academic per- formance. Teaching of Psychology, 37, 55- 57. doi:10.1080/00986280903425912. Fox, A. B., Rosen, J., & Crawford, M. (2009). Dis- tractions, distractions: Does instant messaging affect college students' performance on a con- current reading comprehension task? CyberPsychology & Behavior, 12, 51 - 53. doi:10.1089/cpb.2008.0107. Kennedy, J., & Smith, C. A. (2010, August). Rela- tionship between classroom attitudes and distractions. Poster presented at the 118th meeting of the American Psychological Asso-