A surgeon who I liked working with, was struck off the hospital’s specialist registry following one of his patients dying on the operating table. It seemed that his relationships to his Director of General Surgery at this particular hospital was uphill even prior to this event.
The hospital got behind the Director of Surgery and accused my friend of unacceptably high complication rates. He was reported to the authorities for investigation. It took him many months to sift through his medical charts and prove that his surgical complication rates were well within the expected range. It took him 18 months to resume his surgical practice, during which time he earned no income.
In the name of safety my colleague was named and shamed, unfairly harmed and “managed” by administrators until he was able to prove his medical fitness to practice again. We documented many more cases in our recent survey among Australian O&G specialists.
A few years ago, a group of US investigative journalists went public on a project scoring the surgical performance of 16,827 surgeons performing 8 common, elective surgical procedures. The journalists focussed on surgeons, who they named and shamed publicly for being responsible for patients’ deaths and causing avoidable surgical complications.
ProPublica analysed U.S. health administrative data of 2.3 million common, surgical procedures, which typically are associated with reasonably good survival outcomes, such as cholecystectomy and hip replacement treated from 2009 to 2013. In an attempt to create a homogenous sample they also excluded emergency cases (non-elective) as well as revision surgery.
Complications were defined as death during hospital admission or readmission to any hospital within 30 days. An expert panel manually checked the causes for readmission and selected those who likely constitute a complication. Fewer than 50% of the readmissions were deemed to be a complication by their expert panel.
The journalists calculated a Health Score reflecting patients’ medical co-morbidites using the Elixhauser index. This index was developed from ICD codes that were supposed to predict inpatient mortality. The project used administrative (government) data that is routinely used for medical billing.
And this is their verdict:
Hundreds of surgeons across the US have complication rates double and triple the national average.
A total of 63,173 patients had to be readmitted to hospital within 30 days from surgery and 3,405 patients died subsequent to surgery.
The report identifies surgeons with the highest and the lowest surgical complication rates. The names of these surgeons were listed on their website.
Apart from the surgeon-hostile attitude and apart from the primary goal to achieve sensationalistic headlines (rather than to contribute to good health outcomes), there are several scientific concerns of this study that need to be very clearly stated:
- Health administrative data used for medical billing are an unreliable source of data to calculate surgical complication rates.
- From the above example it appears that 756 surgeons did not have a single surgical complication and another 1,423 surgeons only had one surgical complication during the five-year period. Unfortunately, a zero complication rate for 13% of surgeons is unrealistic and questionable.
- A data set used for medical billing is inadequate to determine the influence of confounders, such as medical co-morbidities. While the journalists claim that they adjusted complication rates for medical co-morbidities, that claim is not substantiated. A comparison of a small sample that determined medical co-morbidities prospectively with the administrative data would have helped to determine how reliable the administrative data really is.
- Readmissions have been used to indicate the presence of complications. However, every surgeon knows that a readmission does not necessarily reflect the presence of a surgical complication. While that may be the case, elderly patients and patients with poor social support are far more likely to be readmitted due to not-coping with the daily chores of life after surgery. That has nothing to do with an operation gone wrong. On the other hand some complications (e.g., DVT, PE) will constitute true complications but would have gone unnoticed here because they often will not require a readmission.
- The interpretation of the data is flawed: By definition, 50% of every sample will have a complication rate higher than average. That should not make 50% of surgeons outliers.
- The journalists selected the “acceptable” boundaries so narrowly that the majority of surgeons are finding themselves outside these boundaries. For knee replacement surgery the 95%-confidence interval of complication rates comes to lie outside the acceptable complication rates for approximately 80% of surgeons.
- What the journalists’ defined as a “high risk surgeon” was arbitrary. It is highly unlikely that a surgeon with a complication rate of 3.1% is an outlier if the national average is 2.3%.
- In every statistical sample –even in standardised environments – variation of outcomes are expected and normal. In some months our complication rates would be zero and in other months it would be scaringly high. While all surgeons try to standardize the conditions in an operating theatre (same patient positioning, same scrub nurse, same surgical assistant, etc.) an operating room cannot be standardized completely. We operate on patients at different ages and body sizes, with different co-morbidities, different diseases, and different scrub nurses and on the ward our patients are looked after by different nurses every few hours on every day. Variation of surgical outcomes within the hospital and over time is perfectly normal and expected.
These days we hear about rogue endometriosis and other surgeons, with the names and headshots of the surgeons in highly regarded news outlets.
The truth is that there are outliers amongst us; surgeons whose complication rates for straight forward, standard cases are 3 or 4 times higher than expected.
However, they are very few but account for a disproportionate large number of patient harm. Typically these surgeons are well known to their peers and hospital administrators; unfortunately hospitals and governing bodies typically delegate the responsibility to manage these outliers back to each other and lack the courage to effectively manage them.
Accusing individual surgeons of poor surgical performance openly through mass media has become increasingly common. It is becoming increasingly easier to name and shame a surgeon as we can see from the above examples. Due to public outcry, the impression will be created that those surgeons are guilty (until they prove that they are OK).
When confronted with such serious accusations, most surgeons would feel vulnerable. However, an increasing number of surgeons will be aware of their own clinical outcomes and can comment and answer questions.
Individual surgeons can defend themselves by collecting data on their own surgical outcomes on an ongoing basis.
Collecting data via SurgicalPerformance on all your cases will address that vulnerability and enable you to address accusations. It is the right thing to do and to demonstrate accountability. It will enable a surgeon to respond to accusations of surgical performance within a few hours (not months) quoting actual data and facts; it will allow you to continue with your practice because you have patient data at hand. It will enable you to monitor your surgical performance continuously and be the first to become aware if things start going wrong.