A managed support service cut mean incident duration by 85% for a global SaaS provider, from around 197 hours to under 30 hours within three months of the service starting. Codurance supplied Level 2 support through provision of a focused senior team of Software Craftspeople in the UK, Portugal and Brazil, covering structured triage, runbooks, proactive monitoring, root cause analysis and 24/7 coverage. Product teams were pulled out of routine incident response, and the service was handed over to a locally recruited team with a documented process.
This case study is written for CTOs and Engineering Leaders who are weighing whether a managed support service suits a platform that is outgrowing its original support model.
|
Metric |
Result |
|---|---|
|
Mean Time to Recover (MTTR) |
Down 85%, from around 197 hours to under 30 hours within three months |
|
Support coverage |
24/7, including US business hours covered from Brazil |
|
Incident-free periods |
Periods with no incidents at all, a first according to the client's CTO |
|
Delivery team |
Senior Level 2 team across the UK, Portugal and Brazil |
|
Exit |
Handed over to a locally recruited team at a lower run rate |
A managed support service relieves product teams of incident response, shortens resolution time and extends coverage when support demand outgrows the model that got the platform this far. As a platform and its complexity grows, incidents can often multiply, product teams are pulled away from strategic work and customers lose confidence, even when the underlying product is strong.
The client in this case study was a global SaaS provider whose platform is used every day by businesses across North America, Europe and Asia-Pacific. The core product had been in the market for more than five years and was complemented by a newer mobile application. It ran on a multi-tenant architecture, with each server hosting a large number of customers. That model is efficient and scalable, but when one server is under pressure, every customer on it feels the effect.
Several of the clients engineering teams were regularly drawn into incident response, which limited the time they could spend on planned product work. The client needed to:
Four factors made the problem harder than it first looked:
Downtime is expensive for most mid-size and large enterprises, which is why reliability work tends to pay back quickly. ITIC's 2024 Hourly Cost of Downtime Survey polled over 1,000 firms worldwide and found that a single hour of downtime costs more than $300,000 for over 90% of mid-size and large enterprises.
Google's Site Reliability Engineering book offers three principles that shaped this engagement:
A managed support service reduces resolution time through five practices: structured triage, runbooks for tactical fixes, proactive monitoring, data-led root cause analysis and platform improvements made in partnership with the client. Codurance embedded a small, senior team of Software Craftspeople based in the UK, Portugal and Brazil. The team acted as Level 2 support, worked with the client's Level 1 support staff, and brought in the client's engineering teams as Level 3 only when a fix genuinely needed them.
Structured triage speeds up resolution because every incident has one owner, a consistent severity and a regular update cadence. Codurance took ownership of triage in the client's incident management tool. Duplicate reports were merged, severities were applied consistently and communications improved, with Level 1 and other stakeholders kept in the loop at all times. Older incidents were worked down at the same time as new ones arrived.
Using runbooks ensures that recurring problems are resolved quickly and safely at Level 2, which keeps large numbers of incidents away from product teams altogether. Where a permanent fix was not yet a priority, the Codurance team wrote runbooks for the tactical fix so that the same problem did not need engineering time each time it appeared.
Proactive monitoring turns early warning signs in the logs into incidents automatically, so the team can act before customers notice a problem. A blended team of Codurance and the client's platform engineers converted monitoring alerts into incidents. The change was introduced gradually to avoid flooding on-call engineers with noise.
Root cause analysis reduces future incidents by replacing anecdote with data-led evidence about where reliability problems sit. Codurance set aside time to investigate yesterday's incidents as well as today's. Analysing incidents by component, server and severity showed leadership where the hotspots were, and the findings were shared with the wider product function to inform reliability priorities. One long-standing fault had generated a large number of incidents by causing errors for customers returning to an inactive session. It was traced and resolved at source.
Incident analysis can point directly to platform improvements that remove whole categories of incident. In this engagement the analysis identified configuration drift across the server estate, and Codurance worked with the client's platform team as they brought servers back into line. The team also supported the migration of Asia-Pacific customers onto dedicated infrastructure, so that scheduled overnight jobs for one region no longer affected users in another. A major operating system upgrade across a 200+ server estate followed, and the client's platform team went on to complete the majority of it.
Out-of-hours support works best when it is staffed from time zones that match your customers' working hours. In this engagement, Codurance first provided cover during the working day. As the relationship matured, the cover extended into an on-call rota and became the 24/7 support desk.
The team's location mattered as much as its schedule. Codurance Craftspeople based in Brazil covered US business hours, so the client's US customers had support online when their UK colleagues were finishing for the day. Overnight incident volumes were modest, but the coverage brought real reassurance to customers and to the client's own leadership. Some planned infrastructure changes, such as customer migrations, were also scheduled out of hours to minimise the impact on users.
Software craftsmanship shifts the measure of success from how quickly tickets are closed to how few tickets are needed. It does this by treating every recurring incident as a prompt to remove its cause, rather than to close it faster.
|
Principle |
What it meant in this engagement |
|---|---|
|
Move from reactive to proactive |
APAC customers were co-located on specific servers, where scheduled jobs were tailored to those time zones to minimise operational impact. |
|
Fix causes, not symptoms |
A long-standing fault that had generated a large number of incidents was traced and resolved at source. |
|
Automate the repeatable |
Runbooks and automated alerting turned tribal knowledge into shared, repeatable practice. |
|
Let the evidence lead |
Incident data gave the client an objective view of where reliability work would pay back most. |
|
Build relationships, not just queues |
The Codurance UK lead craftsperson worked on site with the client's support staff, and clear, regular communication built trust across support, engineering and leadership. |
|
Work towards a graceful exit |
Codurance trained and mentored the client's own support staff, then handed over to a locally recruited team with a well-documented process, giving the client a sustainable model at a lower run rate. |
In this engagement, mean incident duration fell by 85% within three months, incidents dropped sharply after the platform upgrade, and product teams regained their focus. Mean Time to Recover (MTTR, as defined by DORA) fell from around 197 hours to under 30 hours within three months of the service starting. See the note on how MTTR is measured at the end of this section.
|
Result |
Detail |
|---|---|
|
85% reduction in mean incident duration |
MTTR fell from around 197 hours to under 30 hours within three months. |
|
81% reduction in average incident time |
Average incident time fell by 81% after Codurance began working with the client. |
|
Approximately 75% reduction in total incidents raised |
Within 12 months, total incidents raised fell by approximately 75% between the peak and the low month. |
|
Periods with no incidents at all |
The client's CTO noted that it was the first time they could remember a period with none. |
|
A sharp fall in incidents after the platform upgrade |
The combined effect of proactive alerting, automation and the operating system upgrade was that incidents "fell off a cliff". |
|
Lower compensation costs |
As stability improved, the client needed to offer fewer service credits and compensation payments to customers. |
|
Outcome |
Detail |
|---|---|
|
Product teams got their focus back |
Engineering teams were involved in incident response only when their expertise was truly required. |
|
Stronger confidence in support |
Level 1 support staff reported greater trust that incidents were under control, with clearer communication and fewer surprises. |
|
A smoother peak season |
During the client's busiest trading period, the VP of Customers described the support experience as the smoothest she had seen in seven years. |
|
Customers who could rely on the platform |
Fewer disruptions meant fewer worried calls, and customers gradually stopped assuming every hiccup was a platform problem. |
|
A change in conversation |
Discussions at the client moved from "how do we stop this going down?" to "what features should we build next?" |
DORA, the DevOps Research and Assessment programme, tracks how quickly teams recover from failures, and has defined the metric more narrowly over time as failed deployment recovery time. In this case study we report mean incident duration across all incidents, measured from the time reported to the time resolved.
A software consultancy gives you experienced engineers who take on triage and Level 2 support, cover the hours your customers work, and fix the causes of recurring problems so that support demand falls over time. Many fast-growing software businesses reach a point where support demand outpaces the model that served them well.
A managed support service from Codurance suits engineering leaders who need to:
Codurance partners with your people to deliver pragmatic, well-crafted results and to hand over a stronger position than we found. To discuss managed support for your platform, get in touch with Codurance.
A managed support service places a dedicated team of experienced engineers alongside your own, handling incident triage, Level 2 support and on-call cover. The team also works to fix root causes so that incident volumes fall over time.
Codurance introduced a consistent triage process, built runbooks for common problems, used monitoring alerts to spot issues early, and analysed incident data to find and fix recurring causes. Together these cut average resolution time from around 197 hours to under 30 hours.
Level 2 support sits between a client's front-line support staff (Level 1) and its engineering teams (Level 3). In this engagement, Codurance acted as Level 2, worked with the client's Level 1 staff, and brought in engineering teams as Level 3 only when a fix genuinely needed them.
In this engagement, mean incident duration fell from around 197 hours to under 30 hours within three months of the service starting.
Codurance operates an on-call rota covering evenings, overnight and weekends, and places team members in time zones that match your customers' working hours. In this engagement, colleagues in Brazil provided cover for US business hours.
A managed service suits teams that need experienced engineers for triage, Level 2 support and out-of-hours cover while they build their own capability. In this engagement, Codurance trained and mentored the client's support staff throughout and handed over to a locally recruited team with a well-documented process, at a lower run rate.
No. Codurance builds your capability as it goes, through runbooks, documentation, data and mentoring, so that your team can take the service on if they want to. In this engagement, the service was handed over to a locally recruited team.
It shifts the focus from closing tickets to removing the reasons they exist: automating repeatable work, investigating root causes and communicating openly with everyone affected.
Natalie Gray is Director of Marketing & Growth at Codurance, a global AI-first software engineering consultancy that supports businesses and their investors to drive value through building and modernising sustainable software and platforms at all stages of the investment lifecycle. With more than 20 years in the tech industry, Natalie believes the power to innovate is accelerated when people are able to try new ideas, fail fast and collaborate. This is why, to her, community is at the heart of her approach to aligning business and technology outcomes. Outside of her commercial role, Natalie runs a number of tech meetups, is an advocate for women in tech and enjoys spending time with her family or protecting her 1000 day streak on Duolingo.