Fairlearn is an open-source Python package that gives data scientists and developers practical tools for evaluating and mitigating fairness issues in machine learning systems. The toolkit focuses on two categories of harm: allocation harms where AI systems unfairly extend or withhold opportunities, and quality-of-service harms where systems perform worse for certain groups. Rather than claiming to fully debias models, Fairlearn enables humans to understand trade-offs and make informed decisions about how to balance fairness and performance.
The assessment component provides a comprehensive set of fairness metrics including demographic parity, equalized odds, and worst-case accuracy rates for classification, plus worst-case mean squared error and log loss for regression. These metrics quantify how differently a model treats groups defined by sensitive features like age, gender, or ethnicity. The interactive visualization dashboard lets teams compare multiple models side by side, exploring how different fairness constraints affect both accuracy and group-level performance across various metrics simultaneously.
The mitigation component offers three categories of algorithms that follow scikit-learn conventions for easy adoption. Pre-processing methods like CorrelationRemover transform input features before training. In-processing methods like ExponentiatedGradient constrain the training process itself to satisfy fairness requirements. Post-processing methods like ThresholdOptimizer adjust prediction thresholds per group to meet parity constraints. This flexibility lets teams choose the intervention point that best fits their workflow and constraints. Fairlearn is MIT licensed with over 2,200 GitHub stars and is deeply integrated with Azure Machine Learning's Responsible AI capabilities.
