When an organisation approves an AI agent, the business case usually covers the work everyone can see: scoping, integrations, building the knowledge base, testing and launch. Far less attention goes to what happens in the months after go-live, when the agent is answering real customers, the business keeps changing around it, and the AI models underneath it keep changing too.


Gartner June 2025 predicted that more than 40% of agentic AI projects will be cancelled by the end of 2027 for similar reasons, warning that hype can hide the real cost of running agents at scale.


None of this is new to people who have run machine learning systems in production. The same principle applies to the AI agents businesses are deploying today, and the practical answer is a maintenance plan that names the people, the processes and the recurring costs before the project is approved.
‍

Why an AI Sales & Support agent needs work post-launch

The most obvious source of change is the business itself.

Products are added and discontinued, prices and policies are revised, opening hours change over holidays, and new services launch with their own set of customer questions.

Every one of those changes has to reach the agent's knowledge base, and someone has to check that the agent is giving the new answer rather than the old one, because customers will treat whatever the agent tells them as the organisation's position.

The second source of change sits underneath the agent, in the AI models it runs on.

Model providers retire older models on a regular schedule, and when a model is retired, anything still calling it stops working. Anthropic, for example, commits to giving at least 60 days' notice before retiring a publicly released model, and it notified developers on 30 September 2026 that Claude Sonnet 4.5 will be retired on 30 November 2026, roughly fourteen months after that model was released.

Moving to a replacement model is rarely a matter of changing one setting, because a new model can respond differently to the same instructions, so the agent's behaviour needs to be tested again across the questions it handles before the switch is made.
‍
‍
Models can also behave differently over time even when nobody on your side has changed anything. Researchers from Stanford and UC Berkeley compared versions of GPT-4 released in March and June 2023 and found that its accuracy on one task, identifying prime numbers, fell from 84% to 51% between the two versions, while other tasks improved. 
‍
The study is a few years old now, but the lesson has held up: an agent that passed testing at launch is not guaranteed to keep passing, and the only way to know is to keep checking.

Customer behaviour adds a further layer of change, since the questions people ask in the first month after launch are rarely the same mix they ask a year later.
‍

The Maintenance plan needs People

Every AI agent needs an owner inside the business, someone accountable for how it performs and with the authority to decide what changes.

This is the first of the six practices in Australia's Guidance for AI Adoption, published by the National AI Centre, which asks organisations to decide who is accountable for each AI system.

In a smaller organisation the owner may be a customer experience or operations manager who spends part of each week on the agent, while larger organisations often assign a product owner whose role includes it.

‍

Around the owner, a few kinds of work need to be covered, whether by one person or several. Someone needs to read a sample of real conversations regularly and turn what they find into knowledge updates, which is work for a person who understands the products and policies rather than a developer.

Someone with technical skills needs to look after the integrations with systems such as the CRM, ecommerce platform or booking system, test the agent when a model changes, and investigate when something breaks.

The team that receives escalated conversations is part of the plan too, because the quality of the handover from agent to person depends on how well both sides are set up, and their feedback is one of the best sources of information about where the agent is falling short.
‍

‍

‍

Maintenance is easier to budget when it is broken into routines with a set frequency. For a customer-facing agent, the right frequency for each task will depend on conversation volume, how often the business changes and how much risk sits in the agent's answers.

Test and monitor is also one of the six practices in the Guidance for AI Adoption, so a documented review routine is useful evidence for governance and audit purposes as well as for performance.
‍

The recurring costs to put in the budget

The first line in the budget is the cost of running the agent itself: platform fees and the cost of the model usage behind each conversation.

These usually scale with volume, so the budget should allow for busy periods rather than an average month, and for any per-conversation charges that apply above an agreed quota.


The second and usually larger line is people's time. 

‍

Model migrations deserve their own allowance, because they arrive on the provider's timetable rather than yours and involve retesting the agent's behaviour across a wide range of questions.

Integration upkeep is another, since the systems the agent connects to will have their own updates, and a change to an API or a permission can stop the agent from looking up an order or booking an appointment.
‍

Finally, there is governance work. Australian organisations covered by the Privacy Act must explain in their privacy policies how they use personal information in automated decisions that could significantly affect individuals, and businesses with AI agents involved in those kinds of decisions will need to keep that disclosure accurate as the agent's role changes.

‍

Making the maintenance plan part of the business case

The most useful time to plan for maintenance is before the project is approved, while the ongoing costs can still be weighed against the expected return.
‍

Some organisations have the people and skills to manage every part of this in-house. Others prefer to work with a partner that takes care of the ongoing work.

This is the model Clevertar uses. Our team manages the day-to-day operation of your agent, from reviewing conversations and updating its knowledge to testing model changes, monitoring performance and identifying opportunities to improve. We schedule the reviews, address emerging issues and keep the agent aligned with your business as it changes.

Your team's role is to tell us what the business needs, set the priorities and give us the direction to improve the agent. Our role is to handle the ongoing work that keeps it delivering value.


You get the benefits of a customer-facing AI agent without having to build an internal team around maintaining it.

If you're considering an AI agent, or already have one live, an AI Advantage session with our team can help you work out what needs to happen behind the scenes, who should be responsible and how to make ongoing maintenance part of the plan from day one.

‍

‍

‍

Sources:

• Gartner, "Gartner Predicts Over 40% of Agentic AI Projects Will Be Canceled by End of 2027", 25 June 2025: https://www.gartner.com/en/newsroom/press-releases/2025-06-25-gartner-predicts-over-40-percent-of-agentic-ai-projects-will-be-canceled-by-end-of-2027
• Technology Decisions, "Gartner predicts 30% of GenAI projects to be abandoned before 2026": https://www.technologydecisions.com.au/content/it-management/news/gartner-predicts-30-of-genai-projects-to-be-abandoned-before-2026-1522022543
• Sculley et al., "Hidden Technical Debt in Machine Learning Systems", NeurIPS 2015: https://proceedings.neurips.cc/paper/2015/file/86df7dcfd896fcaf2674f757a2463eba-Paper.pdf
• Anthropic, Model deprecations: https://platform.claude.com/docs/en/about-claude/model-deprecations
• Chen, Zaharia and Zou, "How Is ChatGPT's Behavior Changing over Time?": https://arxiv.org/abs/2307.09009
• Department of Industry, Science and Resources, Guidance for AI Adoption: https://www.industry.gov.au/publications/guidance-ai-adoption
• Allens, "Mandatory automated decision-making disclosure requirements are coming", October 2026: https://www.allens.com.au/insights-news/insights/2026/10/mandatory-automated-decision-making-disclosure-requirements-are-coming-key-takeaways-from-the-oaic-guidance/

‍