THIS QUESTION IS BASED ON UNDERSTANDING TASK $1$ TO TASK $5$ AND ANSWER TASK $6$ REGARDING MODEL OPTIMIZATIONS IN PYTHON

Technical Assessment

Introduction:

Hello! Welcome to the Open Cities Lab $($OCL$)$ data technical assessment. This assignment is intended to help you demonstrate your skills and for us to get a sense of how you think about and approach data$-$driven problem solving and decision$-$making.

At OCL, there is as much a focus on the empathetic, real$-$world usage of data as the leveraging of cutting$-$edge, hyper$-$accurate methods and models. Accordingly, we are as interested in how you frame and contextualize the problem as we are in your ability to use advanced data science and machine learning algorithms. So$,$ please don$$t hold back in letting us know why you did what you did at each step, any challenges you identify, concerns that you have in the problem statements, potential biases in the source data, and how you would augment the data provided with other sources if you had the chance. All the best!

Source Data:

You have been provided $($link$)$ with several datasets containing information indexed with StatsSA$$s Small Area Layer $($SAL$)$ codes, as well as complementary geospatial data.

Task $1$: Preprocessing, Data Exploration and Visualization

$$ Clean up any data files that contain extraneous information.

$$ Handle missing values if any.

$$ Encode categorical variables if necessary.

$$ Normalize or standardize numerical features if required.

$$ Perform an exploratory data analysis to understand the structure and properties of the datasets.

$$ Visualize important features and relationships in the data using appropriate plots and charts.

Task $2$: Feature Selection$/$Engineering

$$ Select which features you consider to be most relevant to the prediction tasks in $3$a and $3$b$.$ Please make a note of these before running.

$$ Engineer new features if you think they can improve model performance and briefly describe your reasoning.

Task $3$a: Supervised Model Building

$$ Create a target variable that represents a high probability of well earning females being present in a SAL.

$$ Split the dataset into a training set and a test set.

$$ Build a machine learning model to predict the target without using the features used to construct it$.$ You can choose any model you think is appropriate $($e$.$g$.,$ logistic regression, decision tree, random forest, gradient boosting, etc.$).$

Task $3$b: Unsupervised Model Building

$$ Create a $2-$dimensional $(2$D$)$ topology that represents individual SALs in terms of two indicators, representing relative Urbanization and Socio$-$Economic Standing respectively. Briefly describe your logic in creating these indicators and how they are defined.

$$ Identify and visualize interesting clusters $/$ segments within this $2$D space and map these geospatially by joining this data to the polygons provided and colouring by cluster $/$ segment ID$.$ Briefly discuss why these clusters are interesting $($or not$).$

Task $4$: Model Evaluations

$$ Evaluate both your models$$ performances using appropriate metrics $($e$.$g$.,$ accuracy, precision, recall, F$1-$score, AUC$-$ROC, Silhouette Coefficient, etc.$).$

$$ Discuss why you chose these metrics and what they tell you about your models$$ performance.

Task $5$: Model Optimizations

$$ Try to improve your models$$ performance. This could involve tuning hyperparameters, trying different models, utilizing additional indicators to your original set, or through further feature engineering.

$$ Discuss what steps you took to optimize your models and why. What would you do if you had more time?

$********************************************************************************************************************************$

QUESTION TO ANSWER :

Task $6$: Insights & Conclusions

$$ Based on the models for $3$a and $3$b$,$ statistically assess which features could potentially contribute most to the predictions of your target function $(3$a$)$

and influence the relative locations on the $2$D topology $(3$b$)$ most.

$$ What possible insights could we potetially derive from the models that could be useful for cities to improve service delivery and reverse spatial apartheid?

$$ How could organizations use these insights or principles to consider grass roots programs to promote opportunities for under$-$represented groups?

This answer does not have to be limited to the group represented by the target function developed for this exercise.

$********************************************************************************************************************************$