Sommelier API - model governance beyond the notebook

Problem

The assessment notebook moved from a Decision Tree to a class-weighted Random Forest, but production still served the old classifier. The API was not broken; the evidence contract between coursework and deployment was.

Approach

• Compared 22 model/treatment runs plus a majority baseline across untreated, SMOTE, and class-weighted variants.
• Selected the classifier through predeclared gates for ROC-AUC, sensitivity, specificity, balanced accuracy, and interpretability tie-breaks.
• Recorded model artifact contracts: source commits, hashes, feature order, target semantics, split parameters, and metrics.
• Updated the FastAPI and Streamlit surfaces so clients could consume the changed model without silent contract drift.

Stack

Outcome

• Replaced the production classifier with a documented class-weighted Random Forest rather than a notebook-only claim.
• Improved held-out ROC-AUC from 0.7923 to 0.8337 and specificity from 0.7252 to 0.8063, while documenting the sensitivity trade-off.
• Turned reproducibility, artifact size, and model-selection evidence into first-class engineering concerns.