QUESTION A:

Background for Question A: Jordan works for a company that has developed a college admissions platform. A unique URL is set up for each college so the platform appears to be operated by the college. When applying to a college, prospective students input their applications and an AI that has been trained on that college's admissions data identifies who should be given early admission, who should be kept on a waiting list and who should be rejected.

Jordan notices that STATE COLLEGE is accepting what appears to be a five-times more individuals who identify as male than all of the other types of individuals combined. Jordon also notices that IVY COLLEGE is rejecting every applicant that indicates the need for financial assistance. Jordan verifies that the AI making decisions for each college is making decisions as it was trained to do (i.e. there is no apparent malfunction of the technology but there is a problem with the training data). Jordan wants to tell the affected colleges about the problem immediately. Jordan's boss does not want Jordan to do it. STATE COLLEGE and IVY COLLEGE are two of the company's biggest clients. They will be embarrassed if it comes out that their admission processes are biased. They may try to blame the problems on the platform or the AI. They may stop paying their fees.

Question A.1 - With reference to the principles from Responsible AI (as covered in class) are applicable when analyzing what to do in this scenario?

Question A.2 - Using a virtue-based framework and only the spheres of action discussed in class, help Jordan decide what to do?

Question A.3 - Set out below is what ChaptGPT advised Jordan to do in response to Question A.2. What did ChatGPT get wrong?

CHATGPT'S ANSWER:

In this scenario, Jordan faces a moral dilemma where loyalty to the company conflicts with the virtue of honesty and integrity. Let's analyze Jordan's situation using a virtue-based framework and considering the spheres of action.

1. Individual Sphere: From an individual perspective, Jordan's integrity and honesty are at stake. Remaining silent about the biased admissions processes would compromise these virtues. Jordan should prioritize personal integrity and the pursuit of truth.
2. Community Sphere: Jordan is part of a community consisting of both the company and the colleges it serves. From a communal perspective, Jordan has a responsibility to ensure fairness and equity in college admissions processes. By reporting the bias, Jordan contributes to the greater good of the community by fostering transparency and accountability.
3. Professional Sphere: Within the professional sphere, Jordan's loyalty to the company may conflict with ethical obligations. While Jordan's loyalty to the company is important, it should not come at the expense of compromising ethical principles. As a professional, Jordan has a duty to ensure the integrity of the services provided by the company.

Given these considerations, here's what Jordan could do:

1. Express Concerns Internally: Jordan should voice concerns to superiors about biased admissions processes, proposing solutions like revisiting AI training data.
2. Document Concerns: If internal efforts fail, Jordan should document concerns for future reference.
3. Consider External Reporting: As a last resort, Jordan may consider reporting the issue externally to relevant authorities or stakeholders.

Balancing loyalty with ethical obligations is crucial for maintaining personal integrity and promoting fairness.

QUESTION B:

Background for Question B: ABC Inc. is a professional services company located in Toronto, Ontario. Among other services, ABC offers an online service that allows its customers to track their energy consumption: monitoring devices are installed at the client site(s); they feed usage information back to a central database; and customers may access their usage information through a secure web site. Last year, this service accounted for approximately 20% of ABC's gross revenue, and the percentage is increasing year-to-year. ABC hosts the secure web site internally, but wants to change this by starting to use Host Corp. hosting services. Host Corp. is based in San Francisco, California and Dublin, Ireland.

Four sections of Host Corp.'s proposed agreement are set out below:

Section 8 - This Agreement is governed by the laws of the State of California. All claims, legal proceedings or litigation arising in connection with the Services will be brought solely in the federal or state courts located in San Francisco County, California, United States, and you consent to the jurisdiction of and venue in such courts and waive any objection as to inconvenient forum.

Section 14 - HOST CORP. SHALL NOT BE LIABLE TO ABC FOR ANY INDIRECT, SPECIAL OR CONSEQUENTIAL DAMAGES OR LOST PROFITS, INCLUDING BUT NOT LIMITED TO LOSS OF DATA, ARISING OUT OF OR RELATED TO THIS AGREEMENT OR THE PERFORMANCE OR BREACH OF THIS AGREEMENT, EVEN IF SUCH PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES OR LOSSES. NOTWITHSTANDING ANY OTHER TERM OF THIS AGREEMENT, IN NO EVENT SHALL XYZ'S LIABILITY HEREUNDER EXCEED THE AMOUNT PAID BY ABC DURING THE TWO (2) MONTH PERIOD PRECEDING THE DATE ON WHICH ABC MAKES ITS FIRST CLAIM FOR DAMAGES.

Section 26 - ABC grants to Host Corp. a non-exclusive, perpetual, irrevocable, worldwide license to use, sample, collect, and compile ABC's hosted data in aggregated, de-identified form (i.e., it is stripped of all persistent identifiers (e.g. device identifiers, IP addresses, and cookie ID's)), for the purposes of providing or maintenance of, improvement to, and operation of the Services and for any new or different products or services.

Question B.1 - You represent ABC Inc. What changes do you try to negotiate to those provisions? Explain why each change should be made.

Question B.2 - Set out below is ChatGPT's response for what to do with section 26. What did ChatGPT get right? What did ChatGPT get wrong?

Section 26 - Data Usage License: While it is reasonable for Host Corp. to require a license to use ABC's hosted data for service maintenance and improvement, the perpetual and irrevocable nature of the license is concerning. I would negotiate to limit the duration of the license to the term of the agreement and require Host Corp. to delete or return ABC's data upon termination of the agreement.

Question B.3 - Using a utilitarian framework to guide your decision making, decide whether or not it makes sense to negotiate changes to section 14.