Skip to content

Add ordinal + calibration metrics, CV reporting, and CI-built model artifact - #2

Merged
AmirhosseinHonardoust merged 3 commits into
mainfrom
enhance/ordinal-calibration-metrics-and-model-artifact
Jul 4, 2026
Merged

Add ordinal + calibration metrics, CV reporting, and CI-built model artifact#2
AmirhosseinHonardoust merged 3 commits into
mainfrom
enhance/ordinal-calibration-metrics-and-model-artifact

Conversation

@AmirhosseinHonardoust

Copy link
Copy Markdown
Owner

Closes the remaining polish gaps from the previous overhaul.

Modeling / metrics

  • Report cross-validated accuracy (0.429 ± 0.036) alongside the single-split
    0.483 so the headline isn't a lucky fold.
  • Add ordinal-aware metrics (MAE, quadratic-weighted kappa) + log loss to
    metrics.json.
  • Add src/ordinal.py (Frank & Hall) and a reproducible flat-vs-ordinal
    comparison (reports/metrics/ordinal_comparison.json). Flat RF wins on every
    metric, so it stays default; ordinal kept as a tested alternative.
  • Add reliability diagram + Expected Calibration Error (src/model_metrics.py).
    RF ECE=0.09; sigmoid recalibration documented as a reported trade-off.

Ops

  • models/*.joblib is now a git-ignored build artifact (removed from the repo).
  • CI adds a pipeline job that runs generate->prep->train->evaluate->explain and
    uploads the model + reports as artifacts, exercising the training path in CI.

Tests: 12 -> 20 (ordinal classifier, metric helpers). Gate green locally:
ruff/black/mypy clean, 20 passed. Full pipeline reproducible & deterministic.

@AmirhosseinHonardoust AmirhosseinHonardoust self-assigned this Jul 4, 2026
@AmirhosseinHonardoust AmirhosseinHonardoust added the enhancement New feature or request label Jul 4, 2026
@AmirhosseinHonardoust
AmirhosseinHonardoust merged commit 7a03ded into main Jul 4, 2026
2 checks passed
@AmirhosseinHonardoust
AmirhosseinHonardoust deleted the enhance/ordinal-calibration-metrics-and-model-artifact branch July 4, 2026 12:17
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant