85 lines
4.9 KiB
Markdown
85 lines
4.9 KiB
Markdown
# WashData ML subsystem (experimental, gated)
|
|
|
|
Compact, **NumPy-only** models plus the runtime that trains and consumes them.
|
|
No new dependencies (NumPy is already in `manifest.json`). Everything here is
|
|
gated by flags in `const.py` and is inert until enabled, so the proven
|
|
detection/matching/ETA code paths are unchanged by default.
|
|
|
|
## Feature flags (const.py)
|
|
|
|
- `SHOW_ML_LAB` - show the ML Lab panel tab (shadow-mode comparison + review).
|
|
- `ENABLE_ML_SUGGESTIONS` - surface ML-calibrated setting suggestions alongside
|
|
the classic ones (`MLSuggestionEngine`).
|
|
- `ENABLE_ML_TRAINING` - allow the scheduled/manual on-device training loop.
|
|
- `CONF_ENABLE_ML_MODELS` (per-device option) - opt-in gate (via
|
|
`ml_models_enabled(options)`) for feeding ML signals into runtime decisions;
|
|
default off so callers keep existing behavior. (No runtime consumer wires this
|
|
yet - the live ML paths below run under their own flags.)
|
|
|
|
## What ships here
|
|
|
|
- `promoted_manifest.json` + `<name>_model.py` - the embedded **baseline** models
|
|
(the broad-corpus models trained offline in `/root/ml_washdata`). Each module
|
|
is self-contained and exposes `score()`, `predict()`, `FEATURE_COLUMNS`,
|
|
`THRESHOLD`, `MODEL_METRICS`. `<name>_feature_contract.json` documents the live
|
|
data each feature comes from; `<name>_parity.json` are golden feature→score
|
|
cases the tests assert against.
|
|
- `feature_extraction.py` - NumPy-only runtime feature extractors
|
|
(`latest_end_event_features`, `live_match_features`, `quality_features`,
|
|
`profile_expectation`, energy integration) matching the models' `FEATURE_COLUMNS`.
|
|
- `engine.py` - `resolve_scorer(capability, store)`, the single bridge that
|
|
returns a **classifier** scoring callable preferring an on-device trained spec
|
|
over the embedded baseline (`"on_device"` vs `"baseline"`); `resolve_regressor(
|
|
capability, store)` is its **regression** twin for `standardized_linear` heads
|
|
that have no shipped baseline (returns `(None, None)` until one is promoted),
|
|
plus `ml_models_enabled` (opt-in gate) and `available_models` (manifest provenance).
|
|
- `trainer.py` - NumPy-only training for two spec kinds: logistic classifiers
|
|
(`fit_logistic`, `select_threshold`, `binary_metrics`, `auc`, `build_spec`/
|
|
`score_spec` - byte-compatible with the embedded `score()` math) and ridge
|
|
**regressors** (`fit_ridge`, `regression_metrics`, `build_regression_spec`/
|
|
`predict_value_spec` - standardized features + standardized target).
|
|
- `training_task.py` - on-device orchestration: derives labels from the device's
|
|
own cycles (end events from trace geometry; quality from status + ML-Lab review
|
|
labels; live_match from match-ranking-history snapshots), synthesises
|
|
completion-fraction examples for the regression capabilities, trains, and
|
|
promotes a classifier only when its held-out AUC is within margin of the
|
|
baseline (a regressor only when its held-out MAE beats the naive elapsed/
|
|
expected projection).
|
|
- `matching_tuner.py` - `tune_matching_config(cycles)`: NumPy-only, executor-safe
|
|
leave-one-out tuning of the matcher's bounded scoring weights (`corr_weight`,
|
|
`duration_weight`, `energy_weight`, `dtw_ensemble_w`) over the device's own
|
|
labelled cycles. Same promotion discipline as the models (gate on a held-out
|
|
split by a margin); it only ever changes the emphasis between shape/level/energy,
|
|
never structural matching behaviour.
|
|
|
|
Models (all standardized-logistic; only models that beat their baseline are shipped):
|
|
- `hybrid_curve_quality_model` - P(finished cycle is a problem).
|
|
- `live_match_commit_model` - P(top-1 live program match is correct).
|
|
- `cycle_end_detector_model` - P(a low-power event is the true end vs a pause).
|
|
|
|
(No regression baseline is shipped: the `remaining_time` and `total_energy`
|
|
completion-fraction regressors did not beat the `expected_duration - elapsed`
|
|
heuristic on the broad corpus, so they stay inert until on-device training
|
|
promotes a per-device spec that beats that naive projection.)
|
|
|
|
## How trained models reach inference
|
|
|
|
`resolve_scorer(capability, store)` is used by the ML Lab shadow comparison
|
|
(`ws_api._compute_ml_comparison`) and by `MLSuggestionEngine`. If the profile
|
|
store holds an on-device spec for that capability (trained by `training_task` and
|
|
persisted under `ml_model_versions`), it is used; otherwise the embedded baseline
|
|
module is used. The shipped baseline is a broad-corpus model - per-user accuracy
|
|
gains come from on-device training, not from replacing the baseline.
|
|
|
|
## Regenerating the embedded baseline (offline lab only)
|
|
|
|
```bash
|
|
cd /root/ml_washdata
|
|
./ml.sh experiment # retrain + verify the determinism gate
|
|
python promote_to_integration.py --target <this directory> # reads output/promoted/
|
|
```
|
|
|
|
`promote_to_integration.py` refuses to copy any model whose encode/decode round
|
|
trip is not deterministic. On-device training never touches these baseline files;
|
|
it writes trained specs into the profile store instead.
|