Evaluation of the transferability of survival calculators for stage II/III colon cancer across healthcare systems

Adjuvant! Online Inc (A!O), the Memorial Sloan Kettering Cancer Center (MSKCC), MD Anderson (MDA) and Mayo Clinic (MC) provide calculators to predict survival probabilities for patients with resected early-stage colon cancer, trained on data from United States (US) patient cohorts or patients enrolled in international clinical trials. Limited data exist on the transferability of calculators across healthcare systems. Calculator transferability to Australian community practice was evaluated for 1,401 stage II/III patients. Calibration and discrimination were assessed for overall (OS), cancer-specific (CSS) or recurrence-free survival (RFS). The US patient cohort-based calculators, A!O, MSKCC and MDA, significantly overestimated risks of recurrence and death in Australian patients, with 5-year OS, CSS and RFS prediction differences of −6.5% to −9.9%, −9.1% to −14.4% and − 3.8% to −6.8%, respectively (p < 0.001). Significant heterogeneity in calibration was observed for subgroups by tumor stage and treatment, age, gender, tumor location, ECOG and ASA score. Calibration appeared acceptable for the clinical trial patient-based MC calculator, but restricted tool applicability (stage III patients, ≥12 examined lymph nodes, receiving adjuvant treatment) limited the sample size. Compared to AJCC 7th edition tumor staging, calculators showed improved discrimination for OS, but no improvement for CSS and RFS. In conclusion, deficiencies in calibration limited transferability of US patient cohort-based survival calculators for early-stage colon cancer to the setting of Australian community practice. Our results demonstrate the utility for multi-feature survival calculators to improve OS predictions but highlight the importance for performance assessment of tools prior to implementation in an external health care setting.