ML and experiments

Models are trained by ML recipes in the flow; a training run registers a saved model; experiments keep a ledger of runs with their parameters, metrics and artifacts. See Machine learning for the platform side.

Train a model

rec = project.train_model(
    "churn_clf",                               # recipe name (also the default model name)
    input_table="marts.dim_customer",
    target_column="churned",
    task_type="classification",
    algorithm="random_forest",
    features=["tenure_months", "monthly_spend", "plan"],
    test_size=0.2,
)
print(rec.last_result["status"], rec.last_result["metrics"])
model_id = rec.last_result["model_id"]

train_model(name, *, input_table, target_column, task_type, algorithm, features=None, hyperparams=None, test_size=None, max_rows=None, model_name=None, run=True, wait=True) creates a train_model recipe and, with run=True (the default), runs it at once. The result (status, metrics, model_id, or error) is on rec.last_result. task_type is classification or regression, and algorithm must suit it (for example logistic_regression or random_forest for classification). features=None lets the runner choose the columns.

The other helpers follow the same pattern:

project.score_model("churn_score", input_table="marts.dim_customer_new",
                    model_id=model_id, output_table="churn_scores")
project.cluster_model("customer_segments", input_table="marts.dim_customer",
                      algorithm="kmeans", hyperparams={"n_clusters": 5},
                      output_table="customer_segments")
project.evaluate_model("churn_holdout", input_table="marts.dim_customer_holdout",
                       target_column="churned", model_id=model_id)

score_model writes predictions to <output_schema>.<output_table>, by default ml_outputs.scored_<model>.

The MlRecipe handle

The helpers return an MlRecipe. For full control, create one directly:

rec = project.create_ml_recipe(
    "churn_clf_v2", "train_model",
    inputs=["marts.dim_customer"],
    config={"input_table": "marts.dim_customer", "target_column": "churned",
            "task_type": "classification", "algorithm": "logistic_regression"},
)
rec.config                                    # the stored run configuration
rec.update_config({"test_size": 0.25})        # merges; merge=False replaces
result = rec.run(timeout=1800)                # synchronous; timeout bounds the HTTP call
rec.runs(limit=10)
rec.delete()                                  # trained models are kept

recipe_type is train_model, score_model, cluster_model or evaluate_model. run() returns once training finishes, so its timeout (seconds) needs to cover the whole run (0.1.8+; 0.1.7 ignored it and used the client's 60-second default). The wait argument is accepted for symmetry and has no effect.

Saved models

for m in project.list_models():                 # SavedModel handles
    print(m.model_id, m.name, m.version, m.task_type, m.metrics)

model = project.get_model(model_id)
model.metrics                 # {"accuracy": ..., ...}
model.feature_importance      # {feature: importance}
model.feature_columns
model.list_versions()         # every version of this model name, newest first
model.activate()              # make this version the active one for its name
model.delete()                # refused (409) for the active version while others exist

list_models(name=...) narrows to one model's versions. The properties load the model record on first use; get_info() re-reads it.

Experiments (0.1.8+)

An experiment groups runs; each run carries parameters, metrics, tags and artifacts. Runs logged by the platform's analysis tools and runs you log from code sit side by side.

exp = project.create_experiment("churn", description="Q3 sweep")

with exp.start_run("rf depth=6", params={"max_depth": 6}) as run:
    # train however you like…
    run.log_metrics({"f1": 0.83, "auc": 0.91})
    run.log_artifact("roc.png")
# the run is finished on exit, or closed as "failed" if the block raised

runs = exp.list_runs()
matrix = exp.compare([r.run_id for r in runs])
print(matrix["metrics"]["f1"])

Call	Does
project.list_experiments() / get_experiment(id)	List or open experiments.
exp.log_run(run_name, *, params=None, metrics=None, tags=None, status="running")	Open a run; pass status="finished" to record a completed run in one call.
exp.start_run(...)	log_run as a context manager that always closes the run.
run.log_metrics / log_params / log_tags	Merge values into the run; re-logging a key overwrites it.
run.log_artifact(path, *, name=None)	Upload a local file.
run.download_artifact(name, *, dest=None)	Read an artifact back (bytes); dest also writes it to disk.
run.finish(status="finished", *, metrics=None)	Close the run; the first terminal write wins.
run.promote(model_name)	Register the run's model as a saved model.
exp.compare(run_ids)	A metrics and parameters matrix; ids not in the experiment are listed under missing_run_ids.

promote works only for a finished run that the platform's analysis tools produced, because those are the runs it can rebuild a training recipe from; any other run raises HoneyframeValidationError saying why. experiments.iter_runs(exp, limit=200) yields the newest limit runs (up to 1,000), oldest first. Artifacts are held in memory during upload and download; use a managed folder for large files.

---

Verified against honeyframeapi 0.1.8.