To autonomize, or not to autonomize?

AI agents are really cool, don't you think? but we don't always want the coolest system. we want the simplest one which still performs well, meets our product requirements and fulfils our business goals.

The discretion to choose between a deterministic system, an LLM application and an agentic system is extremely valuable and must be developed. Each kind of system has its own level of autonomy and therefore, its own trade-offs.

my North Star:

let's take a real problem: say, you want to automate customer support in your e-commerce company.

first step: understand what kind of customer issues exist in the present day. this takes us back to looking at data: analysing existing customer care cases to

• identify common issue types
• understand which resolutions satisfy customers
• investigate which behaviour of the support agent works well with customers
• <more insights>

it's possible that a majority of the customer care issues belong to only a few types of problems, so we can think of solving for those problems first and then handle the rest gradually.

for example, say the major classes are refunds, order cancellations, and undelivered orders and if we build a deterministic system to handle only these issues, 75% of issues get automated.

this might be good enough for your company if getting a human agent to handle 25% of the problems is cheaper than managing a complex AI system targeted at handling 100% of the problems.

second step: choose the level of autonomy in the system to solve for the use cases.

I start with thinking of deterministic solutions.

Q: Can I make the chat interface a menu of categories that the user can select from?

A: If a majority of the problems fall under those categories, yes that's the way to go. for other categories, involve a human agent.

what if we just allow users to send free-form chat messages from the start?

that's much more impressive. but all the more complex. because now we need to:

• generate clean training data to fine-tune/train our model OR make custom knowledge bases(/graphs) of our business customer care policies (domain knowledge) that third-party models would use as a rulebook
• generate validation data covering hundreds of use cases to test our system
• ensure that our data is representative of the customer demographics
• develop evals to evaluate objective metrics - precision, recall, f1
• develop LLM-as-judge evals to cover subjective metrics - conciseness, completeness, correctness, faithfulness
• add AI and human safety checks to prevent harmful conversations
• continuously update our evals and training data to keep up with the changing user behaviours, business policies and use cases
• continuously evaluate and re-train (if needed) our model using the updated data

but all this, not to say to never use autonomous solutions. but to use them wisely, with a serious consideration of the benefits versus the cost.

third step: choose which components of the system you want to give agentic autonomy to.

here's an example:

• Agentic flow for problem understanding: decides whether it has enough context to understand the problem fully. until then, it keeps asking questions to the user to gather information. then it identified all subproblems and classifies each subproblem into one of our predefined categories (undelivered order, missing item, poor quality items)
• Deterministic flow for undelivered order: Check driver status, order status in database, associated notes if any and report back to the user using pre-defined messages (if order is delivered but user didn't get it, if order is delayed etc)
• Deterministic flow for missing item: Ask for a picture of delivered order, if the item price is low, issue a refund. Else, involve a human agent to figure out where the missing item went and whether or not to hold the customer liable.
• Deterministic flow for poor quality items: Asks for a picture and sends to its image recognition model, or issues the refund directly if it's a low price item or involves a human agent if the item does not seem bad quality.

fourth step: make your system significantly less cool. add restrictions to limit your agent's capability.

here are a few examples:

• prevent infinite loop: limit the number of turns that the agent can take
• cap the context limit: this controls token usage as well as prevents lost-in-the-middle issues
• enforce intermediate result format: for each intermediate result and reasoning step, use a specified format and enforce it strictly (USE CAPS IN PROMPTS or use code checkers) - this controls the reasoning quality of the agentic system and prevents ad-hoc agentic behaviour
• <there are others but we will talk about them in a separate blog>

lastly, add deterministic checks to your agent's answer before it returns one to the user. these can be used as yes/no validity checks of your agent's reasoning, groundedness and correctness.

For example, is the item mentioned by the agent the same as the item mentioned by the user? Does the item price match the order details?

add an evaluator agent only if you need to: a fresh set of "eyes" that can help add objectivity and improve the quality of your agent's answers. this increases latency of your response, so only add one at critical checkpoints.

For example, if your agent is the one who decides to issue a refund, then you can have an evaluator agent judge its reasoning behind that to verify the claim.

that's how conservative we must be if we want an AI system that actually works in production.

a simple system is easy to understand, evaluate, debug and fix.

hence, it's easy to maintain.

If you enjoyed this, I'm glad. I'm here to talk about real AI-powered projects from experience. from building, failing and learning so you can start from a stronger footing. always happy to hear from readers and fellow Applied AI builders! let me know what resonated with you!