SIADS 503: Data Science Ethics · University of Michigan · March 2026
This project was completed as part of SIADS 503: Data Science Ethics at the University of Michigan. The assignment tasked students with selecting an AI algorithm deployed in a real organizational context, identifying its ethics risks, applying course frameworks to analyze those risks, and delivering findings as a video presentation to a dual audience: the organization's stakeholders and University of Michigan instructors.
The scenario centers on a resume scoring AI at a hypothetical talent solutions firm — Apex Talent Solutions — that automatically filters job applicants before any human review occurs. The analysis examines two ethics risks embedded in the system: the provenance of the data used to train the model, and the power imbalance created when applicants are evaluated by a system they are unaware of and cannot contest.
Skills demonstrated in this project
Apex Talent Solutions operates a resume scoring AI that assigns each applicant a score from 0 to 100. Top-scoring candidates are passed to a human recruiter for final review; low-scoring candidates are filtered out automatically and never seen by a human. The model was trained on ten years of Apex's internal placement and hiring records, supplemented by applicant profiles purchased from third-party data brokers — profiles that were not originally compiled for the purpose of assessing job suitability.
Three groups are affected by the system: job applicants, who are scored and potentially rejected without their knowledge; human recruiters, who receive a pre-filtered candidate pool and may not recognize what has been excluded; and client companies, who only ever see candidates that have already cleared the AI's threshold. The algorithm's decisions are consequential and largely invisible to the people most directly affected by them.
The training data reflects ten years of Apex's internal hiring decisions — decisions made by humans who may have held biases, operated under different hiring norms, or served a client base that was not demographically representative. As Mimi Onuoha argues, datasets are not neutral records of reality; they are the products of human processes, decisions, and contexts. A model trained on this history learns to replicate those patterns, not to identify the best candidates. The third-party broker profiles compound the problem: compiled from thousands of attributes across many sources, they introduce data of unknown origin, accuracy, and relevance — and broker datasets have been shown to enable multiple forms of discrimination.
Applicants bear all of the risk in this system while having none of the visibility. They are scored using data they cannot see, by a model they are unaware of, with no mechanism to investigate or contest the result. Pratyusha Kalluri argues that the right question about AI systems is not whether they are "good" or "fair" in the abstract — it is who they shift power toward and away from. At Apex, the system concentrates decision-making power with the firm and its clients while removing applicants' ability to participate in, understand, or appeal decisions made about them. The applicant — the person whose employment prospects are directly shaped by the algorithm's output — has no standing in the process at all.
The recommendation centers on establishing informed consent and data transparency — not as a public relations measure, but as an ethical obligation rooted in the frameworks above.
The reasoning is grounded in both frameworks. Onuoha argues that data collection is a transaction involving real people on both sides — not a passive extraction from an indifferent world. Applicants are participants in that transaction and should be treated as such. Kalluri argues that people evaluated by AI systems should have the ability to investigate and contest them. Transparency and contestability are not enhancements to the system — they are the minimum conditions under which using such a system is ethically defensible.
The findings were delivered as a five-to-seven minute video presentation structured for a dual audience: organizational stakeholders who needed to understand the ethics risks in plain language, and University of Michigan instructors evaluating the application of course frameworks. The presentation walks through the scenario, system diagram, course concepts, and recommendation in sequence.