Autonomy is a knowledge problem, not a model problem

The short version

The goal is right: clients should move to full autonomy as soon as they can trust it, and they should be able to see that trust grow. The engineering underneath is careful. It refuses to send when it is unsure, which is why nothing embarrassing goes out. It is also why so little goes out on its own.

The cause is structural. Every reply asks a general language model to rediscover the client in the same few seconds it has to answer the customer: what this brand sells, what this post is about, which item in the photo is the product, what the brand is trying to achieve this week, and how this person would phrase it. A model can be good at any one of those. Asking it to do all five, live, on every comment, is where invented discount codes, wrong products and slightly-off voice come from.

Move the knowledge out of the moment of reply. Maintain a living model of each client (an ontology), mine each part of it ahead of time with a tool specialized for that part, let autonomy act only on facts those tools have already established, and restrict it to a declared graph of actions for each job. Learn voice from what the people at the brand actually wrote, separately for each kind of conversation. That is a product a competitor cannot copy with a better prompt.

Where the product stands

The surface is complete and coherent: one inbox across social channels, a store connection so a commenter can also be a buyer, people records with advocacy and loyalty signals, analytics on conversations, a publishing surface, brand settings, a knowledge base filled from the brand's own site, and agents that reply in public and in direct messages.

The agent path is defensive in the right places. Deterministic code rejects ineligible events before any model is called. Code, not the model, decides which agents may answer. A hallucinated choice becomes a pass instead of a send. Impossible action combinations are refused. Every send is rechecked at the last moment. Review is the default and autonomy is opt-in per agent. Nightly offline evaluations run a large bank of scenarios and are tracked over time.

What is missing is not safety. It is knowledge held ahead of time. Three patterns show it:

• The product in the picture. A new drop is a plain tee, jeans and a cap, and the cap is the launch. A general vision pass sees an outfit. It can only name a product when a label is legible, and the match to the catalog is a text comparison. An unlabeled cap is invisible, so the agent either guesses or passes.
• Facts at the edge of the catalog. Codes, variants, sizes and specs are exactly what a customer asks about and exactly where an unanchored model fills gaps. Rules forbid inventing them, but rules are not information.
• Voice nuance. "Thank you, that's awesome" and "thanks" are different brands of warmth. A single written brief cannot hold the difference across a complaint, a first-time buyer, a repeat fan and a creator collaboration.

The nightly evaluations measure behavior on synthetic cases. They are valuable for regressions, and they cannot tell a client that its own agent is ready, because they never look at that client's posts, catalog or people.

Why a model-only approach is not a position

A product whose intelligence is "the best available model plus a well-written prompt" has three problems, and none of them are fixed by the next model release.

The model should be the last step that phrases a decision, not the first step that discovers the business.

The concept: a client ontology

Treat each client as a structured, continuously maintained model of the business. Not a prompt, not a brief: a graph of entities with sources, freshness and confidence on every fact. Agents read from it. Nothing sends that the graph cannot support.

Two properties matter more than the shape. First, the ontology is sector-specific. An apparel client has garments, fits, sizes, colorways and drops. A food client has products, flavors, pack sizes, ingredients and allergens. The branches differ, so the tools that fill them differ. Second, it is maintained, not configured. It updates when the catalog syncs, when a post goes up, when an agent is edited, and when a person rewrites a draft.

Specialized miners for each part of the ontology

Each branch gets an owner: a tool built for that kind of information, run at the moment that information appears, writing facts with confidence back to the graph. Some are small vision or classification models, some are deterministic parsers, some are language models with a narrow job. None of them is asked to reply to anyone.

Why specialization beats a bigger general model here

• A garment segmenter trained or tuned on apparel imagery finds an unlabeled cap that a general model describes as "a person in casual clothing." Linking that crop to a catalog image is a retrieval problem with a measurable accuracy, not a language problem.
• Each miner can be evaluated on its own job, with a number that means something to the client ("we identified the featured product correctly on 46 of your last 50 posts"), instead of one opaque pass rate.
• Mining at publish time is done once per post, not once per comment. The expensive perception step is amortized across every comment that post ever gets.
• A miner that is unsure writes low confidence, and the gap is visible to the client before a customer ever asks. The fix becomes "confirm this cap is the new drop" in the app, not a bad reply in public.

Action graphs as guardrails for autonomy

Today the model plans which actions to take, and code rejects plans that are impossible. That is a blacklist. For autonomy, invert it: each agent job is a declared graph of the actions it may take, in what order, and the ontology facts each step requires. Full autonomy may walk the graph. It may not leave it.

Graphs become templates per sector and per job: drop questions, order status, restock alerts, creator outreach, complaint triage. A new client starts from the sector's template and inherits its tested edges. That is how onboarding gets fast without getting reckless.

Voice: learn it from the people, by context

A written brand voice describes an average. Customers never meet the average. They meet a social lead answering a complaint at night, or a brand owner thanking a repeat buyer. The system already sees those replies, in the inbox and in every edit a person makes to a draft. Use them.

1. Read what each person actually sent. Every human reply and every edited draft, attributed to the person who wrote it, on the brand's own accounts.
2. Split by context. Public or private. Praise, question, complaint, purchase intent. New person, fan, repeat buyer, creator. Launch week or ordinary day. Each combination gets its own profile.
3. Store a voice card per context. Length, warmth, emoji, sign-offs, what they never say, and three to five real replies as examples. Real examples carry nuance a description loses.
4. Write with the matching card. The agent replying to a repeat buyer's praise in public uses that card and its examples, not a single brand paragraph.
5. Measure distance on every edit. When a person changes a draft, record how far it moved and in which context. Shrinking distance per context is the clearest evidence that voice is learned. A context that keeps getting rewritten stays on review.

This also answers a question clients ask and rarely say: "will it sound like us, or like a bot?" The honest answer becomes a chart of their own edits trending toward zero.

Trust you can show

Autonomy should be granted on evidence from the client's own traffic, per job and per edge, and taken back automatically when the evidence weakens. Offline evaluations stay as the internal regression net. The client sees something else.

With those, the product can make a specific, defensible offer: "Your drop questions have run at high coverage and near-zero edits across your last 80 conversations. Turn on autonomy for this job?" That is a one-click decision a client can make in week one, and one a skeptical client can reverse with a clear reason. It is very different from asking a client to trust a toggle.

Autonomy becomes a ladder per job: observe, draft, autonomous on covered edges, autonomous on the whole graph. Clients climb it on their own numbers.

Where the competitive advantage comes from

• Compounding per client. Every post mined, every product link confirmed and every edited draft makes that client's ontology better. A competitor starting fresh starts with none of it. That is switching cost earned, not imposed.
• Compounding per sector. Apparel miners and graphs improve across every apparel client. The tenth clothing brand onboards faster and safer than the first. Model vendors and platform auto-replies will not build garment segmentation linked to a merchant's catalog.
• Measurable claims. "Identifies your featured product on most posts" and "your team's edits fell by most of the way in a month" are sales lines a generic tool cannot say.
• Model independence. When the writer is the last step, the product can swap models freely for cost or quality. The value lives in the graph, not in the vendor.
• Better unit economics. Perception once per post, small specialized models for narrow jobs, and a general model only to phrase the final reply keep the cost per conversation low as clients scale, which protects the markup on the most-used surfaces.

Sequence

Concentrate where the clients are. Most are apparel, so apparel goes first.

Hold back on new channels and on rewriting the general prompt. Both add surface without adding knowledge.

Risks to manage

• Lifestyle imagery is hard. Crops, layering and lighting will defeat segmentation on some posts. That is acceptable when low confidence routes to a person and asks the client to confirm. The failure to avoid is a confident wrong link.
• Catalogs are messy. Product types are inconsistent and images vary. The normalizer is unglamorous and decisive. Budget for it.
• Graphs can become bureaucracy. Keep each job's graph small, readable and owned. If a client cannot understand the graph in a minute, it is too big.
• Voice must stay the brand's. Mine only replies sent from the brand's accounts, attribute them, let a person remove examples, and never mix one client's voice into another's.
• Specialized models are an operating commitment. Hosting, evaluation and drift monitoring per miner. Start with one sector, measure, and add the next only when the first pays for itself.