Image transcriptions

Show all

ETHICS GUIDE MIS-DIAGNOSIS Fred Bolton stared at his computer screen un his eyes glazed over. He had been working 15-hour days for could do to improve the system's "perception" of the product. During his testing, Fred found that minor modifications to the past week trying to solve a serious problem that could have a devastating impact on the future of his employer. Fred the drug's profile made a big difference. But some of the numbers he used to modify the profile were not accurate. had worked at A+Meds for almost a decade, and he was Even if they were, the changes he made would warrant a proud to be affiliated with a world-leading pharmaceutical regulatory review, which could take an extensive amount of company. He started out at the bottom of the IT department time. The financial damage to the company would be done but had moved up quickly. He was a fast learner with a never- long before the review was complete. Fred was not looking give-up attitude. But today he was on the verge of giving up. forward to reporting his findings. Fred was astounded at how much the pharmaceutical industry had changed over the previous 10 years. When he Information Manipulation first started at A+Meds, the company could drive up sales Fred kept looking at the clock on his computer monitor. It using direct marketing techniques. Doctors met with com- was time for his meeting, but he was trying to find an excuse pany representatives who convinced them that A+Meds were to linger at his desk. He came up empty handed and headed the best on the market. Now, technology had started to per- meate every aspect of the healthcare industry. Doctors were over to the boardroom. He took a seat at the end of a long conference table and joined Patricia Tanner, a high-level relying more and more on artificial intelligence (AI)-driven A+Meds sales executive. "So, Fred, what did you find out?" expert systems to select the most appropriate medications and Patricia asked. "Good news, I hope!" Fred explained that, in treatments. These systems made recommendations based on spite of his extensive analysis of the recommendation sys- drug profiles submitted to the system by pharmaceutical com- tem, he was unable to identify a solution that would cause panies. The companies could update the drug profiles if any the system to select their product over competing products- aspect of the drug changed, but this didn't happen very often unless they tweaked the profile. if the changes were minor. 'But our drug is superior and safer!" she exclaimed. "I Recently, the sales of a new drug had been underper- was a sales executive at our competitor when they were put- forming. A+Meds had invested tens of millions of dollars in ting a similar drug through trials, and I know for a fact that developing it. Company executives and new product develop- our drug is the better choice." ers were convinced that the product was superior to com- peting drugs. The problem, they believed, was that the expert systems used by doctors were not recommending the product. Sales were suffering, profitabil ity was down, and employee compensation was in jeopardy. Fred had been tasked with doing a rigorous investigation of the AI recommendation sys- tem. He was supposed to iden- tify the problem and see if there was something the company Source: Sergey/Fot 50ETHICS GUIDE MIS-DIAGNOSIS Fred Bolton stared at his computer screen un his eyes glazed over. He had been working 15-hour days for could do to improve the system's "perception" of the product. During his testing, Fred found that minor modifications to the past week trying to solve a serious problem that could have a devastating impact on the future of his employer. Fred the drug's profile made a big difference. But some of the numbers he used to modify the profile were not accurate. had worked at A+Meds for almost a decade, and he was Even if they were, the changes he made would warrant a proud to be affiliated with a world-leading pharmaceutical regulatory review, which could take an extensive amount of company. He started out at the bottom of the IT department time. The financial damage to the company would be done but had moved up quickly. He was a fast learner with a never- long before the review was complete. Fred was not looking give-up attitude. But today he was on the verge of giving up. forward to reporting his findings. Fred was astounded at how much the pharmaceutical industry had changed over the previous 10 years. When he Information Manipulation first started at A+Meds, the company could drive up sales Fred kept looking at the clock on his computer monitor. It using direct marketing techniques. Doctors met with com- was time for his meeting, but he was trying to find an excuse pany representatives who convinced them that A+Meds were to linger at his desk. He came up empty handed and headed the best on the market. Now, technology had started to per- meate every aspect of the healthcare industry. Doctors were over to the boardroom. He took a seat at the end of a long conference table and joined Patricia Tanner, a high-level relying more and more on artificial intelligence (AI)-driven A+Meds sales executive. "So, Fred, what did you find out?" expert systems to select the most appropriate medications and Patricia asked. "Good news, I hope!" Fred explained that, in treatments. These systems made recommendations based on spite of his extensive analysis of the recommendation sys- drug profiles submitted to the system by pharmaceutical com- tem, he was unable to identify a solution that would cause panies. The companies could update the drug profiles if any the system to select their product over competing products- aspect of the drug changed, but this didn't happen very often unless they tweaked the profile. if the changes were minor. 'But our drug is superior and safer!" she exclaimed. "I Recently, the sales of a new drug had been underper- was a sales executive at our competitor when they were put- forming. A+Meds had invested tens of millions of dollars in ting a similar drug through trials, and I know for a fact that developing it. Company executives and new product develop- our drug is the better choice." ers were convinced that the product was superior to com- peting drugs. The problem, they believed, was that the expert systems used by doctors were not recommending the product. Sales were suffering, profitabil ity was down, and employee compensation was in jeopardy. Fred had been tasked with doing a rigorous investigation of the AI recommendation sys- tem. He was supposed to iden- tify the problem and see if there was something the company Source: Sergey/Fot 50

ETHICS GUIDE MIS-diagnosis "That may be," Fred replied cautiously, "but our profile 35 is based on our current approval guidelines. The drug's cur- were numerous patient-related incidents of a serious rent profile is causing us to lose out to competing drugs." nature. They both sat for a minute before Patricia slowly Patricia looked over at him with a funny look on her face replied, "What if we submit a new profile that the system and said, "Do you think it is right to have people using what perceives as more favorable, even though some of the data we know to be an inferior drug simply based on how this sys- was a bit of a stretch?" tem interprets drug profiles? What if people get sick or if something more serious happens to them because they Fred couldn't believe she'd just asked that question. He should have taken our drug but didn't because of the system? wasn't sure how to respond without putting his job in jeop- Wouldn't you feel bad about that?" Fred hadn't thought ardy. "Wouldn't the addition of inaccurate information to about it like that. Maybe Patricia was right. Fred did believe the system be considered a violation? Wouldn't we be liable their drug was the better choice. But he wasn't a doctor. if something happened to a patient who took our drug Adhering to federal regulations seemed like the right choice, based on altered information?" Fred asked. Patricia replied but not at the risk of keeping people from the medication that drug companies did stuff like this all of the time. Inves- they should be getting. He let out a sigh and leaned back in his tigations were extremely rare and only occurred if there chair. He wasn't sure what to say.ETHICS GUIDE MIS-diagnosis "That may be," Fred replied cautiously, "but our profile 35 is based on our current approval guidelines. The drug's cur- were numerous patient-related incidents of a serious rent profile is causing us to lose out to competing drugs." nature. They both sat for a minute before Patricia slowly Patricia looked over at him with a funny look on her face replied, "What if we submit a new profile that the system and said, "Do you think it is right to have people using what perceives as more favorable, even though some of the data we know to be an inferior drug simply based on how this sys- was a bit of a stretch?" tem interprets drug profiles? What if people get sick or if something more serious happens to them because they Fred couldn't believe she'd just asked that question. He should have taken our drug but didn't because of the system? wasn't sure how to respond without putting his job in jeop- Wouldn't you feel bad about that?" Fred hadn't thought ardy. "Wouldn't the addition of inaccurate information to about it like that. Maybe Patricia was right. Fred did believe the system be considered a violation? Wouldn't we be liable their drug was the better choice. But he wasn't a doctor. if something happened to a patient who took our drug Adhering to federal regulations seemed like the right choice, based on altered information?" Fred asked. Patricia replied but not at the risk of keeping people from the medication that drug companies did stuff like this all of the time. Inves- they should be getting. He let out a sigh and leaned back in his tigations were extremely rare and only occurred if there chair. He wasn't sure what to say.

? DISCUSSION QUESTIONS According to the definitions of the ethics principles defined previously in this book: inaccurate information because the drug may be better a. Do you think that manipulating the recommenda- and safer than the competition? tion of an AI system even though the new rec- 3. How should Fred handle the fact that Patricia made the ommendation may be for the better drug is ethical suggestion to manipulate the drug's profile? Is her will- according to the categorical imperative (pages ingness to use this type of tactic cause for concern in its own right? 23-24)? 4. How do you feel about the growing use of AI and other b. Do you think that manipulating the recommenda- technological solutions in helping people make deci- tion of an AI system even though the new rec- sions? Would you want a doctor treating you based on ommendation may be for the better drug is ethical recommendations from an automated system? Con- according to the utilitarian perspective (pages sider other arenas as well. For example, would you 60-61)? trust the recommendation of an automated financial 2. How would you respond if you were placed in Fred's investment system over the advice of a human financial shoes? Do you think it is appropriate to submit advisor?? DISCUSSION QUESTIONS According to the definitions of the ethics principles defined previously in this book: inaccurate information because the drug may be better a. Do you think that manipulating the recommenda- and safer than the competition? tion of an AI system even though the new rec- 3. How should Fred handle the fact that Patricia made the ommendation may be for the better drug is ethical suggestion to manipulate the drug's profile? Is her will- according to the categorical imperative (pages ingness to use this type of tactic cause for concern in its own right? 23-24)? 4. How do you feel about the growing use of AI and other b. Do you think that manipulating the recommenda- technological solutions in helping people make deci- tion of an AI system even though the new rec- sions? Would you want a doctor treating you based on ommendation may be for the better drug is ethical recommendations from an automated system? Con- according to the utilitarian perspective (pages sider other arenas as well. For example, would you 60-61)? trust the recommendation of an automated financial 2. How would you respond if you were placed in Fred's investment system over the advice of a human financial shoes? Do you think it is appropriate to submit advisor?