Implementation of a national AI technology program on cardiovascular outcomes and the health system

Patient characteristics

Between April 2017 and December 2020, 102,616 CCTAs were performed at 27 sites across a widespread geographic distribution, at secondary and tertiary hospitals in England that are representative of NHS clinical practice (Extended Data Figs. 1 and 2). There were 289 (0.28%) patients without an NHS number, 5,674 (5.5%) patients withdrew their consent, 6,100 (5.9%) CCTA were repeat studies on the same patient during the study period and 20 (0.0001%) patients had a post-mortem CCTA. The final study population of 90,553 patients consisted of 35,688 who had undergone CCTA before the introduction of FFR-CT, and 54,865 CCTA after FFR-CT was available at their hospital (Fig. 1). The mean age was 58 ± 13 years, 48.1% female, with varied ethnicity (78.7% white British or Irish, 2.2% black, 1.4% mixed race, 8.2% Asian, 2.4% other, 7.1% unstated).

Data provided by the 27 NHS England hospitals with subsequent identification of patient numbers, repeat tests and patients excluded due to withdrawal of consent (national ‘Opt out of research’ database).

The median follow-up for the total population was 1,211 (interquartile range (IQR) 535) days, with 98.1% (n = 88,842) completing 2 years follow-up. The cohorts were well matched for demographic and cardiovascular disease risk factors (Table 1) with clinically small but statistically significant differences in patients’ age, hypertension, heart failure, valve disease, chronic obstructive pulmonary disease (COPD) and chronic kidney disease. Among 54,865 CCTA patients who had FFR-CT testing available to their hospital, 7,863 (14.1%) went on to undergo FFR-CT analysis. This cohort were older (63 (IQR 55–71) years), with greater cardiovascular risk factors compared with the total population as they were selected for further testing on the basis of the presence of CAD (Supplementary Table 1).

All-cause mortality and cardiovascular outcomes

There were 2,746 deaths, of which 1,082 were cardiovascular, 1,129 myocardial infarction (MI) and 13,903 ICA performed at 2 years. The 90-day, 1-year and 2-year numbers (%) for each group are reported in Table 2.

There was lower all-cause (n = 1,134 (3.2%) versus 1,612 (2.9%), hazard ratio (HR) 0.92 (0.856–0.996), P = 0.04) and cardiovascular mortality (n = 465 (1.3%) versus 617 (1.1%), HR 0.86 (0.765–0.973), P = 0.02) rates observed in the FFR-CT available group at 2 years compared with the unavailable group (Fig. 2). There was no significant difference in MI events (n = 425 (1.2%) versus 704 (1.3%), HR 1.08 (0.96–1.22), P = 0.24). Rates of all ICA inclusive of those progressing to revascularization (n = 5,720 (16.0%) versus 8,183 (14.9%), HR 0.92 (0.89–0.96), P < 0.001) and ICA with no subsequent revascularization (n = 3,117 (8.7%) versus 4,002 (7.3%), HR 0.83 (0.79–0.87), P < 0.001) were significantly lower in the FFR-CT available group.

a–c, The incidence of all-cause death (a), cardiovascular death (b) and MI (c) rates. d, The incidence of ICA without subsequent revascularization. The shaded areas indicate the 95% CIs.

Nonbalanced prognostic factors at baseline were entered into a multivariable Cox-regression model (age, hypertension, heart failure, COPD, valve disease and chronic kidney disease). Once adjusted for baseline differences in co-morbidities, there was no significant difference in all-cause (adjusted HR (aHR) 1.00 (0.93–1.08), P = 0.97) or cardiovascular mortality (aHR 0.96 (0.85–1.08), P = 0.48) risk between the FFR-CT available and FFR-CT unavailable groups at 2 years. Adjusted risk of MI was higher in the FFR-CT available group (aHR 1.18 (1.05–1.34), P = 0.006) (Table 2 and Extended Data Fig. 3). The risk reduction in all ICA (aHR 0.93 (0.90–0.97), P < 0.001) and ICA without revascularization (aHR 0.84 (0.80–0.88), P < 0.001) remained after covariate adjustment (Table 2) and sensitivity analysis (Supplementary Table 2). Assessment of the proportional hazards assumption by testing for a zero slope in the scaled Schoenfeld residuals for each Cox model showed proportionality for all outcomes. Propensity score matching (PSM) resulted in two cohorts of 30,665 patients (FFR-CT unavailable and FFR-CT available) (Extended Data Fig. 4 and Supplementary Table 3). The PSM analysis showed that all-cause, cardiovascular death and MI at 2 years did not significantly differ (P = 0.95, P = 0.85 and P = 0.17) between groups. Lower all ICA and ICA without revascularization was still observed in the FFR-CT available cohort compared with the unavailable cohort (P < 0.005 and P < 0.001) (Extended Data Fig. 5).

Downstream secondary cardiovascular tests (excluding ICA) were selected from 92 different diagnostic codes (Supplementary Table 5). These cardiac imaging modalities were subcategorized into cardiovascular magnetic resonance imaging (MRI), cardiac CT, nuclear medicine, stress echocardiography and invasive intracoronary imaging (optical coherence tomography, intravascular ultrasound and invasive FFR). A total of 15,942 subsequent non-ICA cardiovascular tests were performed within 2 years of the index CCTA (178.5/1,000 patients). Noninvasive cardiovascular tests performed were lower at 2 years after FFR-CT (n = 6,777 (189/1,000 patients) versus 9,169 (167/1,000), P < 0.001) with a 12% relative risk reduction in the FFR-CT available cohort (HR 0.88 (0.85–0.92), P < 0.001) of having a downstream test. There was reduced likelihood of having a repeat cardiac CT (HR 0.87 (0.80–0.93), P < 0.001), second-line stress echocardiogram (HR 0.52 (0.44–0.62), P < 0.001) or nuclear stress testing (HR 0.61 (0.56–0.67), P < 0.001). The number of cardiovascular magnetic resonance scans performed was higher in the FFR-CT available cohort (HR 1.06 (1.00–1.13), P = 0.042). Invasive intracoronary imaging represented a small number of second-line tests (4.6/1,000 patients), but these increased significantly (HR 1.70 (1.36–2.12), P < 0.001) after FFR-CT availability (Fig. 3).

Subcategorized into noninvasive tests (cardiovascular MRI, cardiac CT, nuclear medicine (positron emission tomography and radionuclide imaging) and echocardiography (excluding transthoracic echocardiography)) and invasive tests (total ICA and intracoronary imaging (optical coherence tomography (OCT), intravascular ultrasound (IVUS) and invasive FFR (FFR))). The error bars indicate the 95% CIs.

Coronary revascularization

There was an early increase in PCI that was sustained at 2 years in the FFR-CT available group (n = 1,912 (5.4%) versus 3,161 (5.8%), aHR 1.09 (1.03–1.15, P = 0.002) (Table 2 and Supplementary Fig. 3). The likelihood of receiving coronary artery bypass grafting (CABG) did not change (n = 691 (1.9%) versus 1,020 (1.9%), aHR 1.01 (0.91–1.11), P = 0.89). Total revascularization rates (PCI and CABG) were higher in the FFR-CT available group (n = 2,603 (7.3%) versus 4,181 (7.6%), aHR 1.06 (1.01–1.11), P = 0.02). The proportion of patients going to ICA who received revascularization (revascularization ratio) was higher in the FFR-CT available cohort (48.3% versus 52.9%, P < 0.001) (Table 2). There was no significant difference in the treatment time from CCTA to revascularization between the FFR-CT unavailable and FFR-CT available cohorts nor the group who had adjunct FFR-CT analysis (Extended Data Fig. 6).

FFR-CT subgroup analysis

Among the 7,863 patients who received FFR-CT analysis, 7,091 (90.1%) had a stenosis-specific result and 7,844 (99.7%) had a distal vessel result, leaving 19 (0.2%) with no FFR-CT value. A stenosis-specific positive FFR-CT (≤0.80) was observed in 4,390 (55.8%) patients. A positive FFR-CT predicted cardiovascular mortality (HR 3.00 (1.33–6.76), P = 0.008), MI (HR 4.76 (2.91–7.77), P < 0.001) and revascularization (HR 13.47 (10.74–16.89), P < 0.001) at 2 years. ICA rates after a positive FFR-CT were higher than for those with a negative FFR-CT, or no stenosis-specific FFR-CT value (FFR-CT ≤0.80, n = 2,406 (54.9%), FFR-CT >0.80, n = 299 (11.1%), no FFR-CT value, n = 36 (4.8%), P < 0.001) (Table 3).

AI implementation and learning

From a baseline in which no NHS site was commissioned in March 2018, within 12 months, 27 different hospitals implemented the AI technology. At the end of the program, 54 sites were commissioned and utilizing the AI technology in routine healthcare settings. National implementation was geographically equitable with balanced representation from across England, urban and rural, academic and nonacademic centers (Extended Data Fig. 1). The median time from funding to starting an FFR-CT program was 4.7 months (IQR 2.4–7.6) with some variability between centers and regions (North 4.6 (2.3–8.3), Midlands 0.8 (0.0–8.0), Southeast 5.4 (3.5–11.0), Southwest 5.1 (2.0–8.6)) (Extended Data Fig. 2b). The indices of multiple deprivation (IMD) score, which ranks populations from the least deprived to the most deprived areas, varied between hospitals according to geographic region and population served. There was no difference in the population IMD score before or after FFR-CT introduction (mean 20.58 (±15.9), pre-FFR-CT 20.61, post-FFR-CT 20.52), nor in the FFR-CT tested cohort (20.1), indicating that the availability and use of FFR-CT was nondiscriminatory according to the level of deprivation or social class (Supplementary Table 6).

The learning curve was determined on an institutional basis by assessing the patient outcomes and resource utilization from the FFR-CT tested cohort (n = 7,863) by the center experience (first 75 FFR-CT cases versus >75 cases). This demonstrated a change in practice and learning over time. The frequency of positive FFR-CT results increased (55% versus 60%), with no change in ICA (24.2% versus 23.8%) or revascularization (19.4% versus 19.4%) rates, resulting in an increased revascularization ratio (53.4% versus 56.2%). There was an associated reduction in the number of second-line downstream tests performed over time (220.5 versus 183.2/1,000 patients) (Supplementary Table 7).