Hiring as Exploration
Titel: Hiring as Exploration (with Peter Bergman and Lindsey Raymond)
Abstract: In looking for the best workers over time, firms must balance exploitation (selecting from groups with proven track records) with exploration (selecting from under-represented groups to learn about quality). Yet modern hiring algorithms, based on “supervised learning” approaches, are designed solely for exploitation. In this paper, we view hiring as a contextual bandit problem and build a resume screening algorithm that values exploration by evaluating candidates according to their upside potential. Using data from professional services recruiting within a Fortune 500 firm, we show that this approach improves both the quality (as measured by eventual offer and acceptance rates) and the diversity of candidates selected for an interview relative to the firm’s existing practices. The same is not true for traditional supervised learning based algorithms, which improve quality but select far fewer minority applicants. In an extension, we show that exploration-based algorithms are also able to learn more effectively about simulated changes in applicant quality over time. Together, our results highlight the importance of incorporating exploration in developing hiring algorithms that are potentially both more efficient and equitable.