DataHill

Support & Maintenance

We watch over the AI systems you have in production, so quality, cost, and security stay where you set them long after the project team has moved on.

Overview

AI systems do not fail the way ordinary software does. They keep running while the answers quietly get worse, because the data changed, a provider updated a model, or a small prompt edit had a side effect. Costs creep up with usage. Without someone watching, you hear about it from a customer. Support and maintenance gives the system an owner who checks it every day and fixes it before anyone outside notices.

Quality is watched, not assumed
We run your test set on a schedule and sample live answers, so a drop in quality shows up as an alert.
Costs stay inside a budget
Usage and spend are tracked per feature, with alerts well before a limit is reached.
Changes are tested first
Updates to models, prompts, and dependencies are checked against your test set before they reach users.

What we look after

The routine work that keeps an AI system trustworthy once it is live.

Monitoring and alerting
Dashboards and alerts for availability, speed, errors, and usage, routed to the people who can act.
Quality and drift checks
Scheduled evaluations against your own cases, plus review of a sample of live answers each week.
Cost and usage control
Spend broken down by feature and model, with budgets, alerts, and suggestions for bringing it down.
Incident response
A written process, named contacts, and response times agreed by severity, with a short write-up after every incident.
Model and dependency updates
Provider changes, library patches, and security fixes applied through a tested release, with a way back.
Continuous improvement
A monthly review of what users struggled with, turned into a short list of fixes ranked by value.

When something goes wrong

The situations that catch teams out, what we do in each, and what you see as a result.

  • Answers start to get worse

    What we do

    We compare current results with the baseline, find the cause in the data, prompt, or model, and fix it or roll back.

    What you see

    Quality back above the agreed threshold, with a note on what changed.

  • The monthly bill jumps

    What we do

    We trace the increase to a feature, a user group, or a prompt, and put limits or a cheaper model in place.

    What you see

    The cause explained, spend back in range, and a new alert so it cannot repeat unnoticed.

  • A provider retires or changes a model

    What we do

    We test the replacement on your cases before the deadline and plan the switch with you.

    What you see

    A scheduled change with scores, not an emergency.

  • A user reports a wrong or harmful answer

    What we do

    We reproduce it, check the sources, decide whether it is a one-off or a pattern, and add it to the test set.

    What you see

    A reply to the user, a fix, and a test that guards against a repeat.

  • A connected system changes

    What we do

    We watch for failing integrations, update the connection, and rerun the tests that depend on it.

    What you see

    The integration repaired, usually before your users notice.

Approach

We take over in a fortnight, then settle into a steady monthly rhythm.

  1. Phase 1

    2 weeks

    Take over

    • Review the architecture, access, runbooks, and past incidents
    • Record baselines for quality, speed, and cost
    • Agree severity levels, contacts, and what needs your approval

    You receive

    Service plan, baselines, and gaps in the runbooks closed.

  2. Phase 2

    2 to 3 weeks

    Set up watching

    • Add monitoring, dashboards, and alert rules
    • Schedule the evaluation runs
    • Rehearse an incident with your team

    You receive

    Live dashboards, alert rules, and a tested incident process.

  3. Phase 3

    Ongoing

    Run

    • Watch the dashboards and respond to alerts
    • Apply updates and fixes through a tested release
    • Keep a log of every incident and change

    You receive

    A monthly service report and an incident log.

  4. Phase 4

    Quarterly

    Improve

    • Review quality, cost, and user feedback with your leadership
    • Tune prompts, models, and retrieval where there is a clear gain
    • Retire what is unused and plan what comes next

    You receive

    Quarterly review and an updated improvement backlog.

Engagement models

Pick the level of cover that matches how much the system matters to your business.

Watch

Lower-risk systems where you want early warning and a monthly check-up.

Monthly, business hours

  • Monitoring and alerts
  • Weekly quality sample
  • Monthly service report
  • Fixes quoted as they come up

Run

Systems that people depend on every day and need a fast response when they fail.

Monthly, extended or 24/7 cover

  • Everything in Watch
  • Incident response within agreed times
  • Updates and patches included
  • Quarterly review with your leadership

Run and improve

Teams that want the system to keep getting better, not only stay up.

Monthly, with dedicated engineer days

  • Everything in Run
  • Engineer days for improvements each month
  • New cases added to the test set
  • Roadmap input for the next release

Access and tools

What we are allowed to touch, how we are held to account, and the tools we use.

How we protect your systems

Least-privilege access
We get the access the job needs, through named accounts, and every session is logged.
Change control
Changes are tested, recorded, and approved by the person you name before they go live.
Written service levels
Response times by severity are in the agreement, and the monthly report shows how we did against them.
Your data stays in your environment
We work inside your cloud account and monitoring tools. We do not copy production data elsewhere.
Easy to leave
Dashboards, runbooks, and test sets belong to you, so another team can pick them up at any time.

Tools we use

Observability
OpenTelemetry, Grafana, Datadog, Langfuse
Evaluation
Your own test sets, run on a schedule with custom harnesses
Alerting and on-call
PagerDuty, Opsgenie, Slack, and Microsoft Teams
Cost tracking
Provider usage data and cloud cost dashboards
Ticketing
Jira, ServiceNow, or the system your team already uses
Cloud
AWS, Microsoft Azure, Google Cloud

We use the monitoring you already have wherever we can.

Frequently asked questions

Can you support a system you did not build?

Yes. The two-week takeover is designed for that. We review the system, write any missing runbooks, and set baselines before we take responsibility for it.

What counts as an incident?

Anything that breaks the service levels we agree in phase 1: the system is down, answers fall below the quality threshold, or spend goes past a limit. We grade each by severity and respond within the agreed time.

Do updates and fixes cost extra?

Patches and provider changes are included in Run and Run and improve. In Watch, we quote larger fixes before starting. Improvements beyond that use the engineer days in Run and improve or a separate quote.

How will we know what you are doing?

You get a monthly report with quality, cost, incidents, and changes, and read access to the same dashboards we use.

Do you offer round-the-clock cover?

Yes, in the Run and Run and improve levels. We agree which systems need it, because not every system does.

What if we want to take it back in-house?

We hand over the runbooks, dashboards, and test sets, and work alongside your engineers for a notice period so nothing is lost.

Have an AI system live and no one watching it? Tell us what runs where and we will say what we would put in place.

Book a discovery call

Start a conversation

Tell us your challenge. We'll show you what's possible with AI.

hello@datahill.com