Why a live Agent can get worse, not better

Start with a counter-intuitive fact we keep hitting: the most dangerous thing in production is not that the Agent is wrong on day one, but that it quietly degrades. We call this drift, and it has at least five sources — none of which require you to touch a single line of code:

• Model version upgrades. The underlying model swaps a version; few-shot behavior, refusal boundaries, and long-context trade-offs all shift. A prompt that was stable last week starts misbehaving.
• Stale policies and knowledge. Amazon category rules, refund logic, and ad policies change every quarter. The “correct” in the Agent’s head may be three months out of date.
• Tool schema changes. A downstream API adds a required field or changes its response shape. The Agent doesn’t error — it just starts deciding on the wrong field. That is the sneakiest class of bug.
• Example contamination. You stuff historical conversations into few-shot as exemplars, and a few are six months old. The Agent learns from them and gets older by the day.
• Upstream data drift. The real-time data source it depends on changes units — say price moves from USD to local currency — and the Agent innocently reasons on wrong facts while producing a “correct” inference.

This is why we firmly oppose the popular phrase “let the Agent rewrite its own SOP.” Letting an Agent edit its own rules is like letting a patient rewrite their own prescription — it lacks the full context of “why this was written that way” and cannot judge whether one change will detonate in another market. The first law of Agent self-improvement: any update that changes its behavior must be human-confirmed, sourced, and reversible — never decided by the Agent mid-run.

The foundation of Agent self-improvement: what a single execution should log

To talk about improvement, you first need observability. No trace, no iteration — only guessing. When we deploy for clients, we require at least eight fields per execution. Drop one, and the “why did it fail” puzzle becomes unsolvable:

Of these eight, human edit and business outcome are the most underrated. The former is free, intent-labeled training signal. The latter is the only anchor that aligns “the Agent thinks it’s right” with “the business is actually better.” We have seen teams log only “which tool was called,” then when things break, shrug and guess “is the model acting up again.”

The fuel of Agent self-improvement: turning human edits into signal, not noise

Here is the key realization: human correction is the most precious fuel for Agent self-improvement, but the raw edit is not data — only the curated edit is. Dumping every “place a human touched” into your eval set or knowledge base earns you a pile of noise. We recommend a four-step curation:

1. Label intent: did the human fix a fact (wrong retrieval), a flow (wrong order), an expression (wrong wording), or a strategy (shouldn’t have touched this action)? The same red line means completely different things.
2. Dedupe and sample: one bug edited ten times by ten people counts once. We require the eval set to cluster by failure mode, not pile up by ticket count.
3. Attribute to root cause: was it prompt-induced, wrong tool return, or stale knowledge? Fix the wrong layer and you award full marks to a wrong answer.
4. Version everything: every sample carries “model version + data date.” Evaluating today’s model against three-month-old samples yields only noise.

A concrete target: an eval set of at least 200 cases covering no fewer than 12 failure modes. 200 sounds like a lot, but it is the minimum scale where “running it weekly and seeing a trend” is meaningful. Below that, your “92% pass rate” is meaningless — too few samples, overfit to a handful of cases. Note: the eval set is a stratified, human-labeled set, not “auto-ingest all online edits.” This discipline is what separates your Agent self-improvement from alchemy.

Offline eval, shadow, A/B, online monitoring — how the four layers stack

Many teams reduce “evaluation” to a one-off offline benchmark and call it done. Not nearly enough. A stable Agent self-improvement is four layers of insurance stacked, each catching a different landmine:

• Offline evaluation (weekly regression): take that fixed 200-case set, run it fully after every prompt/skill change, and watch pass rate and per-failure-mode movement. This is the gate before release.
• Shadow mode (≥2 weeks): new and old versions run side by side; the new one only logs, never commits. Compare their behavior on real traffic. It answers “will swapping it in blow up?” without touching real users.
• A/B test (10% traffic, ≥1 week): once shadow is clean, give 10% real traffic to the new version and watch business metrics (dispute rate, reopen rate, conversion) — not just “model score.”
• Online monitoring (continuous): after full rollout, watch human-edit rate, anomaly rate, outcome distribution. Set alert thresholds — e.g., human-edit rate jumping >15% in a single week triggers an immediate rollback to the last stable version.

The value of this combo: it turns “Agent gets better” from a slogan into a pipeline that is observable, interceptable, and reversible. The worst incident we have seen was skipping shadow and going straight to full rollout — a tiny prompt tweak tripled the refund-misjudgment rate, and the team found out two days later from customer complaints. Two weeks of shadow is the cheapest insurance you will ever buy.

When to change prompt, skill, tool, or knowledge base?

The easiest mistake in Agent self-improvement is “fix where it broke” — a fact error sends you frantically tuning the prompt until it becomes soup. The right move is to judge failure type first, then pick the lever. Our internal decision matrix:

One sentence captures the table: aim the lever at the root cause, not the symptom. We strongly recommend writing “what changed” into the version record — every release notes “edited prompt section X to fix failure mode Y, eval pass rate 88% → 91%.” Without that record, three months later you won’t know which change saved the day and which buried a landmine.

Why you must not feed raw error cases to the model

This is the most counter-intuitive, most commonly botched rule in Agent self-improvement. The moment teams hear “train on data,” they dump every online error case into the training set or knowledge base — that is poisoning the Agent.

Three reasons. First, error cases carry noise: that mistake might be a bad upstream value, a human mislabel, or a user trolling — treat it as gold and the model learns “that one lucky wrong hit,” not the correct behavior. Second, error cases are severely unbalanced — you only collect failures, so normal samples are rare, and the model overfits to “how to handle anomalies” while forgetting “how to work normally.” Third, error cases written straight into the knowledge base fight the correct knowledge, and which one gets retrieved is a coin flip.

The discipline: error cases pass through human review + stratified sampling + root-cause tagging first; only samples that are clearly sourced, human-confirmed, and deduped enter the eval set or knowledge/skill update. And — consistent with the permission article — every update keeps a rollback point. Fine-tuning is heavy artillery; for most business scenarios, start with reversible, explainable, gradable levers like RAG / knowledge-base updates before reaching for model retraining. We have also read the arXiv work on LLM production monitoring and self-improvement (e.g., 2510.06674, 2606.08867); the conclusions match our front-line scars exactly: without observability and controlled evaluation, “self-improvement” is basically uncontrolled drift.

A concrete example: a refund Agent’s six-week flywheel

Frameworks are not enough; here is a real cadence we have seen. A cross-border support refund Agent wanted to ship a “more policy-aware” new prompt. Weeks 1–2 in shadow: comparison showed the new version’s false-rejection rate on “cross-border gift cards” was 4 points higher than the old, but the old leaked far more on “duplicate refunds” — looking at either side alone misleads; only side-by-side reveals both. Week 3 A/B at 10% traffic: confirmed the new version cut overall dispute rate by 1.8 points and reopen rate by 0.6 — business metrics genuinely improved, not just “model score.” Week 4 full rollout.

The real lesson is week 5: the model vendor silently changed a refusal boundary in a routine upgrade, telling no one. The monitoring dashboard showed human-edit rate jumping from 6% to 19%; the alert fired that week, and the team rolled back to the last stable version within two hours — zero customer awareness. Week 6, we curated those 13% new human edits line by line, built 30 fresh eval samples, and filed them under “model-upgrade regression.” Next week’s regression ran green. Across those six weeks, not one word of “let the Agent learn on its own” — just a controlled flywheel turning. And what it produced was real, explainable, reversible improvement.

Original observation: an enterprise’s “learning rate” is not set by model parameters

After a full round of this, we are more convinced than ever of one thing: the industry talks about Agent self-improvement as if it were about “will the model learn on its own” or “how to write the prompt so it’s smarter.” But for enterprises, the real bottleneck is never on the model side.

An enterprise’s learning rate is not set by the model’s learning rate; it is set by whether the business can stably record and evaluate its own decisions. A team that fully logs every execution trace, runs weekly regressions on a human-labeled eval set, and rolls back within two hours of drift — even on a year-old model — will see its Agent improve month over month. Conversely, a team where every error is a “who’s acting up again?” shout in the group chat, with no eval set and no version record, will see its Agent spin in place or regress no matter how new the model. Without observability, self-improvement is just unexplained drift wearing a nicer name.

What this means for Amazon e-commerce Agents

For Amazon ops teams, this Agent self-improvement framework is a mirror. An Amazon ops Agent can touch far too much: checking BSR, watching ad placements, adjusting bids, reading review sentiment, even emailing buyers. Whether it “improves” depends on having an evaluation loop — and the loop’s most common missing piece is a reliable ground truth.

Example: the Agent suggests raising a keyword’s bid by 10%; two weeks later ACOS actually dropped — is that “right”? Not necessarily; it could be seasonality. To judge whether the bid advice is genuinely good, you need continuous, trustworthy, cross-13-market real-time data as a control, not the Agent scraping web pages and guessing. This is exactly why we separate the “real-time fact layer” from the “decision Agent”: the former is provided by a real-time data layer like Amazon Data API — price, rank, ad placement, reviews, all traceable — while the latter, any action that mutates the Amazon backend, must run through the flywheel of trace logging + human-edit curation + shadow/A/B + monitoring alerts. Amazon policies shift violently by market, the prime breeding ground for systemic drift; without an eval set and weekly regression, your ops Agent will quietly burn the budget while you are unaware.

Also, whether the Agent’s review-sentiment calls are accurate, or its “does this negative review touch compliance” judgment is right, both need continuously calibrated human-labeled eval sets. If you want to feed Amazon real-time data directly into Agent workflows, see the Amazon Data MCP documentation — it standardizes the interface between the data layer and the Agent layer, at least making “which data version did the Agent decide on” traceable.

Conclusion: Agent self-improvement governs observability and evaluability, not “making the model try harder”

Back to the opening — stop believing “it gets smarter after launch.” An Agent does not graduate by running longer; it gets genuinely, stably, explainably better only when you have a controlled flywheel that logs traces, curates human edits, regresses on an eval set, intercepts risk via shadow/A/B, and catches drift via monitoring. Put away the romance of “self-rewriting SOP,” and focus on action-layer observability: leave a trace on every execution, label intent on every edit, aim every release at the root cause, and keep a rollback point. When your business can stably evaluate its own decisions, the Agent’s learning rate finally sits in your hands.

Further reading: Amazon Data MCP Documentation. This is a sub-article in Pangolinfo’s enterprise AI transformation series; the pillar is Amazon Enterprise AI Transformation, with prior pieces AI Transformation SOP or Decision Flow, Enterprise AI Knowledge Governance, and Enterprise Agent Permission Management.