Metrics for the CareOps Practice

CareOps is a nascent field. Until it has seen the industry dialog it requires we need to look elsewhere to inform us about relevant metrics to drive the CareOps practice.

There are obvious parallels with the field of DevOps, which in turn has taken a lot from scientifically grounded movements like Total Quality Management and the Theory of Constraints. Therefore, this first throw is mainly a transposition of what success means in those fields applied to healthcare and the CareOps practice.

To recap, the CareOps practice aligns key teams at the provider organization - clinical, product, technology and compliance - along the CareOps lifecycle and around three key principles: focus on value, cross-functional collaboration and accelerated iteration. The ultimate achievement is a care provider organization that continuously delivers better outcomes at lower cost than its peers.

Before we can define metrics, let's take another look at the stages in the CareOps Lifecycle. The illustration below is valid for different kinds of care provider organizations. For example a provider organization A could have care teams using paper / PDFs during care delivery (top right corner). Provider organization B on the other hand has built all care flows into the relevant applications so care teams and patients interact with software during the care journey. In reality, often a mix exists where processes documented on paper / PDF are complemented with partially implemented workflows in software.

The CareOps Lifecycle
The CareOps Lifecycle and its components applicable to most care provider organizations

What do provider organizations running a successful CareOps practice look like?

Where we discussed performance in each of the stages of the lifecycle in the CareOps Lifecycle, the following attributes describe care provider organizations that are successful in their entire CareOps practice.

Ultimately it is not about mastering each stage but about tying them together in a way that accelerates the organization's ability to excel in the 4 following areas:

They build more integrated processes, faster

In the design phase, successful provider organizations design care delivery processes that go beyond the single care setting and clinical intervention. They start from a multidisciplinary view including aspects related to social and psychological wellbeing of the patient. They also build outcomes collection and automation with other systems into their care flows from the outset. When Tia Care built an on-line check-in for patients, they rallied technical, clinical and administrative teams:

Before even setting out to design or build tech, we said to them, “what are the ways in which you use this information? What are your biggest pain points around it? What do you wish you could do?”. These answers factored directly into the systems that we designed — both technical and operational.”

Felicity Yost, Co-founder and CPO/COO at Tia Care

Besides designing more integrated processes, best performers' lead times are also shorter than their peers for projects of similar complexity. This is because they have the habit of cross-functional collaboration and the toolset to support a fast handover between design, validate, build and operate.

CareOps - Comparing two versions of Design, Validate, Build
The ability to go from Validate straight to Build accelerates the entire cycle

What we see in this illustration is that the ability to skip flowcharts in PDFs, documents describing protocols, excel sheets, Miro boards and other accelerates a care provider's ability to go from Design & Validate straight to the Build stage.

They show less unintended variation

Unintended variation is when the correct practice is clear but not applied consistently. It has many causes but some include deficits in evidence, care provider knowledge, lack of documentation and dysfunctional communication.

High performing provider organizations support their care teams and patients with software that shows a clear historical path and possible or suggested next steps at any given time. This removes the need for the individual care provider to keep all of the complexity in the head and rely on experience or memory alone. It leads to better workflow across multidisciplinary team members, a better experience because everyone in the care journey understands what has been done and what should happen next. It also creates the ability to shift routine clinical tasks to other care team members than the highly paid doctors, enabling them to focus on the more difficult decision making.

At the time of writing, future will tell whether the prescribing policies that got Cerebral in trouble were enforced top-down, a result of individual provider freedom or just SOPs that were based on the wrong guidelines. In case of the last one, Cerebral could very well be the canary in the coal mine as many (virtual) care providers might be running care off of outdated playbooks and the only reason why it became apparent at Cerebral is because tall trees tend to catch a lot of wind.

In any case, having clear guardrails in the software application used by the care team immediately leads to lower unintended variation and ultimately to better outcomes and cost management.

CareOps - Comparing 2 implementations of Build & Operate
Processes built into software reduce the amount of unintended variation on care team and patients side

They extract more insights from monitoring and outcomes collection

The healthcare world is flush with data. But data in itself is useless, it’s the insights extracted from data that make the data actionable. Including the data team (as part of the technology team) in the CareOps lifecycle from the outset facilitates extraction of insights downstream.

In the previous illustration, working off PDFs, flowcharts in Lucidchart, Miro boards or written protocols in documents means there is no monitoring on many of the core activities in the care delivery process as they are not captured - or not in sufficient granular detail.

Following the adage "you can't improve what you don't measure" it's also important to identify which aspects of care flow performance should be monitored to get relevant insights and come to the right decisions on what to improve. If you only measure financial performance, this might have an impact on clinical quality. Therefore we advocate monitoring metrics in 4 buckets: care flow performance metrics (such as drop-off rate and completion rate), financial metrics (such as revenue per patient, cost per provider per patient), clinical outcomes (vital signs, lab results) and patient reported outcomes or PROMs (PHQ-9, PROMIS-10, GAD7, etc.).

They iterate more frequently on their processes

Insights gained from monitoring and outcomes collection are all too often destined for publication in medical journals and subsequently in contractual negotiations with customers. What they should be used for more often, and this is what high performing care providers do, is to use insights to further drive improvements to their care flows.

The ability to iterate more frequently creates a powerful competitive advantage for any provider organization. A quarterly iteration means four opportunities to learn and improve vs. only one for a competitor that iterates once a year. The worst performers have even no ability at all to iterate:

Our patient onboarding is this monolithic piece of code that evolved over time, adding bits and pieces to it, kind of like Frankenstein. Our reality today is that it's impossible to evolve it further and the project to take it apart is too big and too risky given our current patient volume so it gets postponed indefinitely.

Director of Engineering (anonymous) at virtual care provider that raised > $ 150mio

Besides having a culture of continuous improvement, the prerequisite to iterate more frequently is mastering the full CareOps Lifecycle. If you've built a strong product and engineering organization but the design and validate stage takes too long, your iteration frequency will be low. If you fail to extract insights, you're depending on outside insights such as published medical research, meaning you're dependent on an outside source for your iteration cadence.

Mastering the full CareOps Lifecycle
Mastering the full CareOps Lifecycle

Mastering the full CareOps Lifecycle means:

  1. Having clinical, product, engineering and compliance work efficiently together in identifying the inputs for care processes
  2. The ability to continuously deploy improvements to care processes, instead of having an in-between step with requirements
  3. Rapid building by using modular, flexible infrastructure and applications
  4. Automation of activities in the care delivery process
  5. Real time monitoring of care flow performance and outcomes
  6. The right tools and experts to analyse the data and make sense of it
  7. Being able to package insights as input for the next cycle of iteration

Metrics for the CareOps practice

Given this is a nascent field, these are just initial proposals of metrics that can be tracked to measure a provider’s ability to practice CareOps. See them as a v1, they will evolve over time and feel free to contribute with your thoughts. These metrics can only evolve in the right direction through constructive dialog.

Lead time of a change in a care delivery process

High lead times are a signal of low agility and iteration capability. Shorter lead times are an indicator of more integrated, collaborative teams that can iterate on their processes more often.

Iteration frequency of a care delivery process

New evidence and insights from collected outcomes should lead to updates to care processes. In practice, they are hardly updated at all. Organizations iterating more often in a given timeframe will improve faster than their peers. In a field where your competitors are running 4 iterations in a year and you can do only one iteration, this means you are potentially learning and improving at a 4x lower speed.

% of the process built into clinical operations software

If the care team is reading the protocol off a Word document or following a flowchart designed in Miro or Lucidchart at any point during the care journey, then that process is not built into the clinical operations software (also not if it's uploaded in that software and available to view).

Care flows built into software show prompts for activities to the right stakeholder at the right time, provide an overview of all activities in a given care plan / care program / pathway and orchestrate data synchronization between point solutions to automate the workflow.

The higher the % of the care process built into the clinical operations software, the more automation drives down cost, the lower the amount of errors in clinical practice for the care team, and the better the experience for the patient.

We are aware that this is a very difficult metric to track. We are currently looking into how Value Stream mapping is enabling organizations to measure similar activities but suggestions are welcome.

Real time monitoring

Lots of care providers capture data, only to look at it somewhere down the line to get some insights out of them or publish them in medical research. When a high % of the process is embedded in clinical operations software, real time monitoring of performance metrics, financial metrics, clinical outcomes and patient reported outcomes allows to intervene early, identify bottlenecks as they arise and course correct when needed.

CareOps Metrics
The four key metrics of CareOps success


We propose CareOps as a foundational field for care providers to continuously drive better outcomes and lower cost. Our view is heavily informed by scientific programs to drive improvements in production systems as early as the 50s and 60s as well as the more recent field of Devops driving improvements in the software development lifecycle.

We formulate CareOps as a combination of culture, tools and practices organized along the CareOps Lifecycle as well as metrics to drive and improve the practice.

We cannot state enough that this is only a first throw at defining this field. We are looking at the community of next generation care provider organizations to further evolve the concepts in this field over the months and years to come.

Join the conversation

These articles are only a first throw at what we think is a major revolution in healthcare in the making. Discover other posts on this website ranging from our State of CareOps Surveys to guest posts by authors explaining their best practice.

If any of the above resonates with you, or you'd like to shine and tell the world how you've solved some of the challenges described in the CareOps practice, we'd love to give you the stage. Please reach out to to see how we can collaborate with you on this.


Stay up to date on what tools and tactics you can use to drive clinical & operational efficiency

Get ahead of the curve and learn from leading care providers and companies such as Cityblock Health, Ro, Bicycle Health, Recora, Boulder Care, Ophelia and more
We send max. 1-2 messages per month on best practices and the topic of CareOps. No commercial stuff.