Lecture 5 Additional Software Tools

• Python:
  • mlxtend
  • SALib
  • themis-ml
• R:
  • DALEX
  • drifter
• Other:
  • Manifold
  • What-If Tool

Lecture 5 Additional Software Examples

• Advanced residual analysis example
• Advanced sensitivity analysis examples
• Basic sensitivity and residual analysis example
• Debugging a Machine Learning Model
• Machine Learning for High-Risk Applications: Chapter 8 code examples

Lecture 5 Additional Reading

• Introduction and Background:

  • AI Incident Database
  • Debugging Machine Learning Models
  • Identifying and Eliminating Bugs in Learned Predictive Models
  • Real-World Strategies for Model Debugging
  • Safe and Reliable Machine Learning
  • Why you should care about debugging machine learning models

• Debugging Approaches and Information:

  • A Comprehensive Study on Deep Learning Bug Characteristics
  • DQI: Measuring Data Quality in NLP
  • Identifying and Overcoming Common Data Mining Mistakes
  • Predicting Good Probabilities With Supervised Learning
  • Underspecification Presents Challenges for Credibility in Modern Machine Learning