Autonomous model-training agent

Most teams that need a custom vision model are gated by the same bottleneck: there isn't enough ML-engineering time to go around. Choosing an architecture, tuning hyperparameters, running the experiments, reading the results, deciding what to try next — it's slow, expert work, and it's the reason a labelled dataset can sit untouched for months.

I designed and built a system that removes the human from that loop entirely. An autonomous agent does the ML research engineer's job: it forms a hypothesis, rewrites the training code, runs the experiment on a commodity cloud GPU, scores the result against a fixed bar, keeps what works, discards what doesn't, and goes again — for hundreds of cycles, unattended. No engineer picks the architecture. No engineer tunes the run.

The hard part of a system like this isn't getting an agent to write training code. It's stopping it from fooling itself over a long campaign. The engineering is all in the guardrails.

the guardrails

• An evaluation harness the agent cannot edit.

  The dataset split, the metric, and the scoring code are locked; the agent owns the training script and nothing else. Because the test never moves, every experiment stays scientifically comparable — and the agent can't “win” by quietly making the problem easier.

• Git as the experiment ledger.

  Every run is a commit. Every result maps back to the exact code that produced it and a versioned copy of the model. A campaign of hundreds of experiments stays fully reproducible, not a pile of unlabelled checkpoints.

• A deployability rule, not just an accuracy score.

  A model that posts a great headline number by collapsing on one class is useless in production. The agent discards any model that fails on a single class, even when the aggregate looks excellent. Product sense, encoded into the objective.

• A loop built to survive its own length.

  Long agent runs degrade as context fills up. This one keeps its working memory in files, not in the conversation, and reconstructs state every cycle — so the thousandth experiment is run with the same discipline as the first.

Pointed at real, messy image-classification problems, the system reached production-grade accuracy with no ML engineer involved, on the kind of compute bill you'd lose in the rounding. The result I care about most: aimed at different datasets, it arrived at genuinely different — and individually correct — architectural choices. That's evidence of real search, not a default that happened to fit.

What it is: a proven core — an autonomous agent that does an expert's technical job reliably and cheaply. What it isn't, and I'll say so on a call: a turnkey product. The work to wrap a serving API, a feedback loop and data hygiene around it is well-understood engineering, not unsolved research. The risk is sequencing, not science — which is exactly the risk profile you want to be buying.

The judgment this represents is the one I sell: how to make an autonomous agent do real work without gaming, drifting, or quietly breaking — anti-gaming harnesses, reproducibility, a deployability bar, context discipline. It's the same discipline I bring to agents, evals and retrieval in someone else's stack.