Oxford-NHS Study validates GE Healthcare AI medical software for assisting in the diagnosis of collapsed lung
A study by Oxford clinicians has highlighted the advantages of using a software tool to support decision-making and provide patient benefit.
A collapsed lung is a serious condition and can be life threatening. Occurring when air escapes from the lung and builds up in the chest outside the lung, the pressure means that the lung cannot expand. Accurate and efficient diagnosis is critical to identify and manage this condition. However, the turnover of imaging staff in Emergency Departments means that the experience base required for accuracy is hard to grow and maintain.
GE Healthcare has developed a Critical Care Suite Pneumothorax Detection Artificial Intelligence algorithm, CCS PTX AI, as an assistance tool for the detection and triage of cases of collapsed lung, or PTX, on X-rays. It has been favourably demonstrated against the diagnoses of expert Consultant Radiologists but had yet to be compared with other clinicians who are more likely to routinely assess chest X-Rays.
A study, facilitated by the Consulting Services team at Oxford University Innovation, was undertaken to compare the reporting accuracy of different clinician groups when acting alone versus when assisted by GE’s Critical Care Suite. Eighteen clinicians with three experience levels – senior (five or more years), mid-level (about three years) and junior (up to 18 months) – were recruited from the Thames Valley Emergency Medicine Research Network by the Principal Investigator Professor Alex Novak based at Oxford’s John Radcliffe Hospital. The clinicians were all routinely involved in the diagnosis and management of collapsed lung within the NHS.
Each clinician reviewed 400 anonymised X-ray images containing both collapsed lung and other conditions routinely encountered in clinical practice. The images were collected by NHS Trusts who were members of the NCIMI Consortium, a partnership coordinated by the University of Oxford. Using Oxford University spinout company RAIQC’s Report And Image Quality Control image viewer and reporting tool, the clinicians scored the images for the presence or absence of collapsed lung, as well as their confidence in making the decision. The clinicians rescored the images four weeks later, but now with the assistance of GE’s Critical Care Suite that highlighted the area of interest.
Comparison of the clinicians’ performance with and without the AI tool showed an overall improvement in accuracy of 12% (from 67% to 79%). The differences between the experience levels was marked: the less experienced junior clinicians improved by 35%, essentially increasing their performance to that of the more experienced clinicians.
The results of this study highlighted the advantages of using the GE AI software tool to support decision-making by junior clinicians and ultimately provide patient benefit. The findings have major potential benefits for Emergency Medicine Departments within the NHS, showing how AI tools can provide a buffer against the loss of accuracy that often accompanies high staff turnover, and a large and varied number of patients. This will help provide an evidence base for AI vendors and stakeholders to demonstrate efficacy and impact.
