Scott, a second-year graduate student in educational measurement, had just gotten off the phone with his sister Gail, who had recently finished her nursing degree. Gail and her friend, Tammy, had taken the CAT version of the National Council Licensure Examination for Registered Nurses (NCLEX-RN) a few weeks earlier. Gail and Tammy had just received their results. Gail had passed and Tammy had failed. Passing this licensing exam is required of all nursing students who hope to practice as registered nurses in a given state within the United States. While Gail was glad that she had passed, she was disappointed for her friend Tammy. Gail didn’t really understand how the CAT worked, and, reflecting back on their conversation, Scott felt he had had a difficult time trying to explain it to her. “It doesn’t seem fair that Tammy and I actually took different tests. Don’t the tests have to contain the same items to be able to compare them?” asked Gail. “Well, it doesn’t have to,” Scott began as he tried to explain. “The test is called ‘adaptive’ because it adapts to your ability level.” “How does a computer know what my ability level is if I haven’t taken the test yet?” asked Gail, somewhat perplexed. “Well, the computer starts with a moderately difficult item and then, depending on whether you answer that one correctly or not, it gives you an easier item if you get it wrong, or a harder item if you get it correct,” Scott explained. “Then the computer uses that information to compute an estimate of your ability in nursing,” Scott added. 

“But how could it do that with just a few questions?” Gail wondered out loud. “Well, the estimate of your ability isn’t very good at first. That’s why the computer has to give you more than just a couple of questions,” Scott tried to explain. “In fact, I just looked up information on the NCLEX-RN on the Internet and it says that they have to administer a minimum of 75 questions.” “I kind of understand, but I still don’t know why Tammy had to answer so many more questions than I did and she still failed. In fact, she was there for five hours and it only took me about half that time to complete the test,” said Gail, somewhat frustrated. “Well, you must have been more consistent in your responding than Tammy. In addition, for a licensing exam, the key is to score above the cutoff score, so as soon as the computer is relatively confident that you are above the cutoff score it will stop administering questions. So, my guess is that the computer was able to say with confidence that you were above the cutoff, but it took much longer for Tammy. In fact, the information I found on the NCLEXRN says the maximum time limit is five hours, so Tammy simply ran out of time and never reached the maximum number of 265 questions,” Scott explained. “Ah, I think I’m starting to understand,” said Gail with a wry smile on her face. “But, I still don’t understand why Tammy and I couldn’t just answer the same questions.” Somewhat discouraged, Scott said, “Okay, let me try to explain it to you another way ….” 

Questions 

1. If you were Scott, how would you go about explaining what a CAT was to Gail? 

2. What are some of the major differences between a CAT and a paper-and-pencil test that might highlight the advantages of CAT over paper-and-pencil testing for Gail? 

3. Are there other reasons that Tammy might have had to answer more questions than Gail? Is there a better way to explain this than what Scott said? 

4. What other stopping procedures might a CAT use to decide when to end the testing session besides a maximum number of items or a time limit? Will it be different for licensing exams versus other more traditional testing situations? 

5. Are there other examples of the use of CAT that you can think of that might help Gail better understand what a CAT is?