Rater Sensitivity and Bias in Adverse Impact Decision-Making: A Signal Detection Theory Approach
| dc.contributor.advisor | Oswald, Frederick L. | en_US |
| dc.creator | Alexander, Leo | en_US |
| dc.date.accessioned | 2021-12-06T20:00:45Z | en_US |
| dc.date.created | 2021-12 | en_US |
| dc.date.issued | 2021-12-03 | en_US |
| dc.date.submitted | December 2021 | en_US |
| dc.date.updated | 2021-12-06T20:00:45Z | en_US |
| dc.description | EMBARGO NOTE: This item is embargoed until 2027-12-01 | en_US |
| dc.description.abstract | The concept of adverse impact (AI) was created to help identify employment discrimination and is defined as a meaningful difference in the selection rates (e.g., hiring, admissions, promotion) between two groups of employees or applicants (e.g., men and women). Researchers studying fairness in organizations have treated AI almost entirely as if it is only a quantitative problem to be addressed by quantitative tests (e.g., statistical testing) and have neglected to study the decision-making processes of individuals charged with the task of evaluating the fairness of employee selection outcomes (detecting AI). The purpose of this study is to understand how well people can detect AI and what factors influences people’s judgements of AI. Signal detection theory (SDT) is a framework that is well-suited to modelling individuals’ perceptions of the fairness of employee selection outcomes (AI) because it was developed to model human decision-making in the presence of uncertainty. In addition to global evaluations of accuracy (whether or not one can accurately determine whether a hiring decision is fair or unfair), SDT allows researchers to calculate people’s sensitivity to AI stimuli (d′; the ability to detect differences in the selection rates of two groups), as well as people’s response bias in doing so (c ;the difference in selection rates between two groups at which a rater tends to decide a selection decision is unfair). In this study, 234 participants (58% women) were asked to decide if 36 selection scenarios where men and women were hired at differing rates were fair (or unfair), based on the Equal Employment Opportunity Commission’s definition of AI. Participants also completed various individual difference measures hypothesized to influence sensitivity: i.e., cognitive ability, numeracy, and graph literacy. Similarly, participants completed various individual difference measures that were hypothesized to influence response bias: i.e., Procedural Justice Beliefs for Others, Distributive Justice Beliefs for Others, and the Ambivalent Sexism Inventory. Other person and environment characteristics were also recorded: i.e., gender and stimulus type (whether hiring decisions were presented as icon array graphs or numeric text). Participants’ sensitivity and response bias in AI decision-making were analyzed using both traditional and mixed-effects-regression-based SDT approaches. Cognitive ability, numeracy, and graph literacy all emerged as significant predictors of participant sensitivity in the analyses. Procedural Justice Beliefs for Others and gender emerged as significant predictors of response bias in the analyses. The implications of these results as well as the suitability of traditional and mixed-effects-regression-based SDT models as a framework for analyzing AI decision-making are discussed. | en_US |
| dc.embargo.lift | 2027-12-01 | en_US |
| dc.embargo.terms | 2027-12-01 | en_US |
| dc.format.mimetype | application/pdf | en_US |
| dc.identifier.citation | Alexander, Leo. "Rater Sensitivity and Bias in Adverse Impact Decision-Making: A Signal Detection Theory Approach." (2021) Master’s Thesis, Rice University. <a href="https://hdl.handle.net/1911/111759">https://hdl.handle.net/1911/111759</a>. | en_US |
| dc.identifier.uri | https://hdl.handle.net/1911/111759 | en_US |
| dc.language.iso | eng | en_US |
| dc.rights | Copyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder. | en_US |
| dc.subject | adverse impact | en_US |
| dc.subject | employee selection | en_US |
| dc.subject | decision-making | en_US |
| dc.subject | signal detection theory | en_US |
| dc.subject | individual differences | en_US |
| dc.title | Rater Sensitivity and Bias in Adverse Impact Decision-Making: A Signal Detection Theory Approach | en_US |
| dc.type | Thesis | en_US |
| dc.type.material | Text | en_US |
| thesis.degree.department | Psychology | en_US |
| thesis.degree.discipline | Social Sciences | en_US |
| thesis.degree.grantor | Rice University | en_US |
| thesis.degree.level | Masters | en_US |
| thesis.degree.name | Master of Arts | en_US |