Interpreting data for learning: what major investigations teach us about the signals we miss
Insights from Alex Kafetz, Non-Executive Director at the Care Quality Commission, recorded at the RLDatix Wales: Listening and Learning for Improvement and Safe Care event, 23 June 2026, St. David’s Cardiff
The signals were in the data. In every major healthcare investigation of the past two decades, the information needed to identify harm existed before the inquiry was commissioned. The question was never whether the data was there. It was whether anyone looked at it, connected it and was prepared to act. Alex Kafetz, Non-Executive Director at the Care Quality Commission and former leader of the Dr Foster healthcare analytics programme, walked the room through three cases he was personally involved in.
Key takeaways from this session
- The signals are always in the data. In every major investigation, the information needed to identify harm existed before the inquiry began. The failure was not data. It was leadership, culture and curiosity.
- Whole-system failure looks different from rogue individuals, but both are detectable. Mid Staffordshire showed what happens when an entire organisation fails. Shipman and Patterson showed what happens when individuals operate within otherwise functioning systems. Both left statistical signals.
- Financial data can be as revealing as clinical data. In the Patterson case, the pattern of unnecessary treatment after mastectomy created a financial signal that should have prompted investigation, even if the clinical data was disputed.
- Inquiry recommendations are frequently not implemented. Key recommendations from the Francis Inquiry and the Patterson Inquiry remain unactioned years later. The pattern Helen Hughes described as the implementation gap is clearly visible here.
- Three indicators can tell you a great deal about a hospital. Risk-adjusted mortality, the Friends and Family Test and the staff survey question on whether you would want your loved ones treated where you work provide a powerful baseline.
From Florence Nightingale to the mortality wheel
Using data to identify preventable deaths is not new. Acting on it still is.
Alex opened with a historical anchor. Florence Nightingale’s work during the Crimean War is widely known for advancing nursing practice. Less widely known is that she produced the first recorded example of using data to identify preventable deaths in hospitals, colour-coding mortality by cause to show where harm was most concentrated.
Alex and colleagues built on that foundation over a decade ago, creating a “mortality wheel” that mapped every hospital in England against four measures: in-hospital mortality, deaths within 30 days of discharge, failure to rescue (deaths after surgery where survival was expected) and deaths in low-risk conditions.
The variation was striking. Almost every hospital had at least one area where outcomes were strong enough to celebrate, and at least one where further investigation was warranted. The data was not a verdict. It was an invitation to ask questions.
That work, published through the Dr Foster Good Hospital Guide, also surfaced the finding that patients who had surgery on a Friday had approximately 1.5% higher mortality than those operated on Monday to Thursday, a finding that drove significant political and workforce debate.
"The days are gone when the NHS can act as a secret society.”
Alan Milburn, as quoted by Alex Kafetz
Mid Staffordshire: when a whole system fails
The data flagged concerns for years. The response was to commission a paper disputing the data.
As the mortality analytics programme grew, one hospital stood out: Stafford Hospital, part of Mid Staffordshire NHS Trust. Its risk-adjusted mortality rate was statistically high almost every year over a sustained period. Alex ‘s team tried to alert the Department of Health. They published the data. At the same time, a group of patients who had lost loved ones at the hospital began meeting in a cafe in Stafford, sharing stories and recognising patterns.
The hospital’s response was not to investigate. It was to commission statisticians to publish a paper in the BMJ arguing that the data was flawed.
"At the very least, you might want to assure yourselves that what we were saying and what this group of patients were saying wasn’t right. Unfortunately, that’s not what they did.”
The subsequent public inquiry, chaired by Sir Robert Francis, produced a series of recommendations including duty of candour, restrictions on non-disclosure agreements and clear lines of leadership accountability. Alex ‘s assessment was that many of those recommendations have not been implemented as fully as they should have been.
The human reality behind the data was captured in the story of Julie Bailey, whose mother was so dehydrated that Bailey had to take flowers out of vases and feed her mother the water because she could not find any staff or any water on the ward.
Harold Shipman: detecting a rogue operator statistically
A simple chart of time of death told the story that years of practice had hidden
Shipman, a general practitioner in Manchester and Britain’s most prolific serial killer, presented a different kind of challenge. The system around him was not failing. He was a rogue individual operating within a functioning practice.
Alex worked on the Shipman Inquiry and used a methodology called CUSUM, borrowed from Toyota’s lean manufacturing approach. Each GP was represented as a line on a chart. Every time a patient died under their care where survival was statistically expected, the line went up. Every time a patient survived where death was expected, it went down.
Shipman’s line tripped the threshold, though it was not the most extreme outlier. What was far more revealing was a simple analysis of time of death. Compared to other GPs, whose patients died at broadly random times, Shipman had a massive spike of recorded deaths between 2pm and 4pm, the window when he was making home visits to patients he knew would be alone.
"Even if the mortality data is quite complicated, this is a very simple indicator. If you found another GP and all their patients were dying around a particular time, that’s something people might want to investigate.”
The Shipman Inquiry recommendations, including medical revalidation and public access to GMC fitness-to-practice records, have been more widely implemented than most. Alex noted that this is worth examining: why do some inquiry recommendations get actioned and others do not?
Ian Patterson: clinical data, financial data and the culture that protected him
Over a thousand women harmed. The signals were there in both the clinical and the financial data.
Ian Patterson, a breast cancer surgeon in Birmingham, was sentenced to prison for harming 15 women. By the time the inquiry concluded, the number was over a thousand. He operated across one NHS hospital and two private hospitals, telling patients in the private sector they had cancer when they did not, and in the NHS performing mastectomies but leaving cancer tissue behind.
Alex was asked whether there was enough data that Patterson should have been spotted. The answer was yes. Clinical data showed signals. But what was equally revealing was financial data. After Patterson’s mastectomies, breast care nurses were sending patients for chemotherapy and radiotherapy, treatment that NICE guidance would not have indicated after a full mastectomy. The insurers and finance teams should have questioned why they were paying for treatment that did not follow the clinical pathway.
"Even if you don’t believe the clinical data, why didn’t the finance director of the hospital say, why are we paying for this treatment which doesn’t follow the NICE guidance?”
When a new CEO, Mark Newbold, arrived in 2011, he suspended Patterson the next day. There was no new data. The difference was that someone looked at the same information and decided to act.
The inquiry classified witnesses into three categories: should have known, could have known and must have known. Those in the “must have known” category were referred to the police, the GMC or the Nursing and Midwifery Council.
Alex drew a parallel from Ronan Farrow’s reporting on Harvey Weinstein:
"With painful frequency, stories of abuse by powerful people are also stories of a failure of poor culture. That’s absolutely right in the NHS as well.”
The inquiry’s lead recommendation, that there should be a single database of consultants setting out their practice, privileges and critical performance data including the number of times they have performed a procedure, has never been implemented. The Department of Health in England has not accepted it.
Three indicators that tell you if a hospital is any good
Alex closed with a practical framework. A colleague he worked with argued that three pieces of information give you a strong indication of what is going on in any hospital:
- Risk-adjusted mortality data, which shows whether more patients are dying than statistically expected
- The Friends and Family Test, where 95% of patients in England would recommend their hospital care
- The NHS Staff Survey question: would you want your loved ones to be treated in the hospital where you work? Approximately 15% of staff in England would not
Those three data points, combining clinical outcomes, patient experience and workforce sentiment, provide a baseline that no single measure can offer alone.
The common thread
Every case Alex walked through carried the same lesson. The data existed. The signals were visible. What was missing was the curiosity, the culture and the leadership to investigate and act.
Mid Staffordshire was a whole-system failure where data was dismissed and patients were ignored. Shipman was a rogue operator whose patterns were statistically detectable. Patterson was protected by a culture of deference that refused to confront what the clinical and financial data was showing.
"All these people are in the data. We just need to surface them and have leadership that’s prepared to investigate.”
The message for NHS Wales was clear: the data you already collect through Datix Cymru and other systems contains signals. The question is whether your organisations have the culture, the curiosity and the leadership to look at them, connect them and act.
FAQs
Risk-adjusted mortality uses statistical methods to compare the number of deaths in a hospital against the number that would be expected, given the age, conditions and complexity of the patients treated. Measures such as HSMR (Hospital Standardised Mortality Ratio) and SHMI (Summary Hospital-level Mortality Indicator) allow hospitals to be compared fairly. A statistically high rate does not prove poor care, but it is a signal that warrants investigation.
The Mid Staffordshire NHS Foundation Trust Public Inquiry, chaired by Sir Robert Francis QC, examined widespread failures in care at Stafford Hospital between 2005 and 2009. The inquiry found that patients were subjected to serious neglect and that the organisation had prioritised institutional reputation over patient safety. Its recommendations included duty of candour, restrictions on non-disclosure agreements and clearer leadership accountability. Many of these recommendations have not been fully implemented.
The Independent Inquiry into the Issues raised by Paterson examined how breast cancer surgeon Ian Patterson was able to harm over a thousand women across NHS and private hospitals in Birmingham over many years. The inquiry found that both clinical and financial data contained signals that should have prompted earlier intervention, and that a culture of deference and a lack of board-level curiosity allowed him to continue practising. Its lead recommendation, for a single national database of consultant performance data, has not been accepted by the Department of Health.
CUSUM (Cumulative Sum) is a statistical quality control method originally developed in manufacturing and adapted for healthcare. It tracks the cumulative performance of an individual over time, flagging when performance deviates significantly from what would be expected. In the Shipman Inquiry, CUSUM was used to compare GP-level mortality against expected rates, demonstrating that statistical monitoring could identify outlier practitioners who warrant investigation.
The three indicators are: risk-adjusted mortality data (are more patients dying than expected?), the NHS Friends and Family Test (would patients recommend their care?) and the NHS Staff Survey question on whether staff would want their loved ones treated at the hospital where they work. Together, these combine clinical outcomes, patient experience and workforce sentiment into a practical baseline assessment.
