Lab assignment - Bias Business analytics tools are increasingly used in problems that may have a significant impact on people's lives in policy areas like criminal justice, education, public health, and social services. Aequitas, an open source bias audit toolkit developed by the Center for Data Science and Public Policy at the University of Chicago, can be used to evaluate whether predictions are biased towards certain groups of people. Let's say that you're interested in opening a new boba tea store in one of three possible communities. You know the level of education of all adults living and working in each location, but not their income. Academic success is often associated with greater opportunities, income, and discretionary income. You have a model that predicts whether a person's income exceeds $50,000/ vear using 1994 US Census Data of adults, but you don't know if the model created is biased towards any particular group of individuals. You have the ability to evaluate whether bias exists in education, gender, and/or race in this model, but let's just focus on whether there's any bias in the level of education attained. A biased model towards education may result in selecting a location that either too risky (i.e., the community has less discretionary income than predicted) or undervaluing another location (i.e., the community has more discretionary income than predicted). - Step 1: Select "US Adult Income Doto" sample data set to example from Auquitas. - Step 2: Select only "Educotion" as the protected attribute and set the reference group to "Bochelors" - Step 3: Select fairness metrics "False Positive Rote Parity" and "Folse Negotive Rate Parity"; set the disparity intolerance at " 80 percent; select "Generote Foimess Report" Enter the followine scores usina the "Audit Results: Bias Metrics Values" table - Note: Bachelors scores are 1.0 since it's the reference group all other groups are compared with Onily Bacheiars (17\%), HS-grad (33\%), and Some-college (23\%) results are examined as they comprise the majority of the somple data set (73\%) - see "Audit Results; Group Metrics Values" toble Interpret the results - if bias does evist in this model, how does this bias impoct your lacation decision? - A value that is Green suggests that the model is unbiased compared with the reference group (Bachelors) - A value that is Red suggests that the model is biased relative to the reference group (Bachelors) - Scores greater than 1.0 is \% more likely (i.e., 1.38=38% more Whely FP or FN relative to the reference group) - Scores less than 1.0 is the % less likely i.e, 0.74=26% less fikely FP or FN relative to the reference group) - Folse Positive Rate Porily: is the rate of False Positives (predict >$50K income, but $50K income) for the protected group (HS-grad or Some college) similar to the reference group (Bachelors)? - Folse Negative Rate Pority: is the rate of False Negatives (predict $50% income, but >$50s income) for the protected group (HS-grad or Some college) similar to the reference group (Bachelors\}? This model which predicts income based on education is biased/unblased/indeterminate (select one) compared with the individuals with a Bachelor's degree as evident by s - A screenshot of the Fairness Report from Aequitas. You don't need to display all the details, just enough to confirm completion. (1 pts) - Responses to the questions in the instructions: 1. "Interpret the results - if bias does exist in this model, how does this bias impact your location decision?" (2 pts) 2. "This model which predicts income based on education is biased/unbiased/indeterminate (select one) compared with the individuals with a Bachelor's degree as evident by >" (2 pts) Lab assignment - Bias Business analytics tools are increasingly used in problems that may have a significant impact on people's lives in policy areas like criminal justice, education, public health, and social services. Aequitas, an open source bias audit toolkit developed by the Center for Data Science and Public Policy at the University of Chicago, can be used to evaluate whether predictions are biased towards certain groups of people. Let's say that you're interested in opening a new boba tea store in one of three possible communities. You know the level of education of all adults living and working in each location, but not their income. Academic success is often associated with greater opportunities, income, and discretionary income. You have a model that predicts whether a person's income exceeds $50,000/ vear using 1994 US Census Data of adults, but you don't know if the model created is biased towards any particular group of individuals. You have the ability to evaluate whether bias exists in education, gender, and/or race in this model, but let's just focus on whether there's any bias in the level of education attained. A biased model towards education may result in selecting a location that either too risky (i.e., the community has less discretionary income than predicted) or undervaluing another location (i.e., the community has more discretionary income than predicted). - Step 1: Select "US Adult Income Doto" sample data set to example from Auquitas. - Step 2: Select only "Educotion" as the protected attribute and set the reference group to "Bochelors" - Step 3: Select fairness metrics "False Positive Rote Parity" and "Folse Negotive Rate Parity"; set the disparity intolerance at " 80 percent; select "Generote Foimess Report" Enter the followine scores usina the "Audit Results: Bias Metrics Values" table - Note: Bachelors scores are 1.0 since it's the reference group all other groups are compared with Onily Bacheiars (17\%), HS-grad (33\%), and Some-college (23\%) results are examined as they comprise the majority of the somple data set (73\%) - see "Audit Results; Group Metrics Values" toble Interpret the results - if bias does evist in this model, how does this bias impoct your lacation decision? - A value that is Green suggests that the model is unbiased compared with the reference group (Bachelors) - A value that is Red suggests that the model is biased relative to the reference group (Bachelors) - Scores greater than 1.0 is \% more likely (i.e., 1.38=38% more Whely FP or FN relative to the reference group) - Scores less than 1.0 is the % less likely i.e, 0.74=26% less fikely FP or FN relative to the reference group) - Folse Positive Rate Porily: is the rate of False Positives (predict >$50K income, but $50K income) for the protected group (HS-grad or Some college) similar to the reference group (Bachelors)? - Folse Negative Rate Pority: is the rate of False Negatives (predict $50% income, but >$50s income) for the protected group (HS-grad or Some college) similar to the reference group (Bachelors\}? This model which predicts income based on education is biased/unblased/indeterminate (select one) compared with the individuals with a Bachelor's degree as evident by s - A screenshot of the Fairness Report from Aequitas. You don't need to display all the details, just enough to confirm completion. (1 pts) - Responses to the questions in the instructions: 1. "Interpret the results - if bias does exist in this model, how does this bias impact your location decision?" (2 pts) 2. "This model which predicts income based on education is biased/unbiased/indeterminate (select one) compared with the individuals with a Bachelor's degree as evident by >" (2 pts)