Original Article|Articles in Press

Derivation and validation of a machine learning-based risk prediction model in patients with acute heart failure

Published:March 15, 2023DOI:


      • A risk score to predict mortality in acute heart failure was developed.
      • The newly developed 4 V-RS comprises four readily available variables.
      • The 4 V-RS performs equally to the more complex pre-existing risk models.



      Risk stratification is important in patients with acute heart failure (AHF), and a simple risk score that accurately predicts mortality is needed. The aim of this study is to develop a user-friendly risk-prediction model using a machine-learning method.


      A machine-learning-based risk model using least absolute shrinkage and selection operator (LASSO) regression was developed by identifying predictors of in-hospital mortality in the derivation cohort (REALITY-AHF), and its performance was externally validated in the validation cohort (NARA-HF) and compared with two pre-existing risk models: the Get With The Guidelines risk score incorporating brain natriuretic peptide and hypochloremia (GWTG-BNP-Cl-RS) and the acute decompensated heart failure national registry risk (ADHERE).


      In-hospital deaths in the derivation and validation cohorts were 76 (5.1 %) and 61 (4.9 %), respectively. The risk score comprised four variables (systolic blood pressure, blood urea nitrogen, serum chloride, and C-reactive protein) and was developed according to the results of the LASSO regression weighting the coefficient for selected variables using a logistic regression model (4 V-RS). Even though 4 V-RS comprised fewer variables, in the validation cohort, it showed a higher area under the receiver operating characteristic curve (AUC) than the ADHERE risk model (AUC, 0.783 vs. 0.740; p = 0.059) and a significant improvement in net reclassification (0.359; 95 % CI, 0.10–0.67; p = 0.006). 4 V-RS performed similarly to GWTG-BNP-Cl-RS in terms of discrimination (AUC, 0.783 vs. 0.759; p = 0.426) and net reclassification (0.176; 95 % CI, −0.08–0.43; p = 0.178).


      The 4 V-RS model comprising only four readily available data points at the time of admission performed similarly to the more complex pre-existing risk model in patients with AHF.

      Graphical abstract


      To read this article in full you will need to make a payment

      Purchase one-time access:

      Academic & Personal: 24 hour online accessCorporate R&D Professionals: 24 hour online access
      One-time access price info
      • For academic or personal research use, select 'Academic and Personal'
      • For corporate R&D use, select 'Corporate R&D Professionals'


      Subscribe to Journal of Cardiology
      Already a print subscriber? Claim online access
      Already an online subscriber? Sign in
      Institutional Access: Sign in to ScienceDirect


        • Okura Y.
        • Ramadan M.M.
        • Ohno Y.
        • Mitsuma W.
        • Tanaka K.
        • Ito M.
        • et al.
        Impending epidemic - future projection of heart failure in Japan to the year 2055.
        Circ J. 2008; 72: 489-491
        • Benjamin E.J.
        • Muntner P.
        • Alonso A.
        • Bittencourt M.S.
        • Callaway C.W.
        • Carson A.P.
        • et al.
        Heart disease and stroke statistics-2019 update: a report from the American Heart Association.
        Circulation. 2019; 139: e56-e528
        • Conrad N.
        • Judge A.
        • Tran J.
        • Mohseni H.
        • Hedgecott D.
        • Crespillo A.P.
        • et al.
        Temporal trends and patterns in heart failure incidence: a population-based study of 4 million individuals.
        Lancet. 2018; 391: 572-580
        • Saku K.
        • Yokota S.
        • Nishikawa T.
        • Kinugawa K.
        Interventional heart failure therapy: a new concept fighting against heart failure.
        J Cardiol. 2022; 80: 101-109
        • Cook C.
        • Cole G.
        • Asaria P.
        • Jabbour R.
        • Francis D.P.
        The annual global economic burden of heart failure.
        Int J Cardiol. 2014; 171: 368-376
        • Heidenreich P.A.
        • Bozkurt B.
        • Aguilar D.
        • Allen L.A.
        • Byun J.J.
        • Colvin M.M.
        • et al.
        2022 AHA/ACC/HFSA guideline for the management of heart failure: a report of the american College of Cardiology/American Heart Association joint committee on clinical practice guidelines.
        Circulation. 2022; 145: e895-e1032
        • Peterson P.N.
        • Rumsfeld J.S.
        • Liang L.
        • Albert N.M.
        • Hernandez A.F.
        • Peterson E.D.
        • et al.
        A validated risk score for in-hospital mortality in patients with heart failure from the American Heart Association get with the guidelines program.
        Circ Cardiovasc Qual Outcomes. 2010; 3: 25-32
        • Fonarow G.C.
        • Adams K.F.
        • Abraham W.T.
        • Yancy C.W.
        • Boscardin W.J.
        ADHERE scientific advisory committee, study group, and investigators. Risk stratification for in-hospital mortality in acutely decompensated heart failure: classification and regression tree analysis.
        JAMA. 2005; 293: 572-580
        • Abraham W.T.
        • Fonarow G.C.
        • Albert N.M.
        • Stough W.G.
        • Gheorghiade M.
        • Greenberg B.H.
        • et al.
        Predictors of in-hospital mortality in patients hospitalized for heart failure. Insights from the Organized Program to Initiate Lifesaving Treatment in Hospitalized Patients With Heart Failure (OPTIMIZE-HF).
        J Am Coll Cardiol. 2008; 52: 347-356
        • Mortazavi B.J.
        • Downing N.S.
        • Bucholz E.M.
        • Dharmarajan K.
        • Manhapra A.
        • Li S.X.
        • et al.
        Analysis of machine learning techniques for heart failure readmissions.
        Circ Cardiovasc Qual Outcomes. 2016; 9: 629-640
        • Obermeyer Z.
        • Emanuel E.J.
        Predicting the future - big data, machine learning, and clinical medicine.
        N Engl J Med. 2016; 375: 1216-1219
        • Goldstein B.A.
        • Navar A.M.
        • Carter R.E.
        Moving beyond regression techniques in cardiovascular risk prediction: applying machine learning to address analytic challenges.
        Eur Heart J. 2017; 38: 1805-1814
        • Pavlou M.
        • Ambler G.
        • Seaman S.R.
        • Guttmann O.
        • Elliott P.
        • King M.
        • et al.
        How to develop a more accurate risk prediction model when there are few events.
        BMJ. 2015; 351: 7-11
        • Matsue Y.
        • Damman K.
        • Voors A.A.
        • Kagiyama N.
        • Yamaguchi T.
        • Kuroda S.
        • et al.
        Time-to-furosemide treatment and mortality in patients hospitalized with acute heart failure.
        J Am Coll Cardiol. 2017; 69: 3042-3051
        • Ho K.K.
        • Anderson K.M.
        • Kannel W.B.
        • Grossman W.
        • Levy D.
        Survival after the onset of congestive heart failure in Framingham heart study subjects.
        Circulation. 1993; 88: 107-115
        • Ueda T.
        • Kawakami R.
        • Horii M.
        • Sugawara Y.
        • Matsumoto T.
        • Okada S.
        • et al.
        High mean corpuscular volume is a new indicator of prognosis in acute decompensated heart failure.
        Circ J. 2013; 77: 2766-2771
        • Nakada Y.
        • Kawakami R.
        • Matsui M.
        • Ueda T.
        • Nakano T.
        • Takitsume A.
        • et al.
        Prognostic value of urinary neutrophil gelatinase-associated lipocalin on the first day of admission for adverse events in patients with acute decompensated heart failure.
        J Am Heart Assoc. 2017; 6e004582
        • Nakada Y.
        • Kawakami R.
        • Matsushima S.
        • Ide T.
        • Kanaoka K.
        • Ueda T.
        • et al.
        Simple risk score to predict survival in acute decompensated heart failure: A2B score.
        Circ J. 2019; 83: 1019-1024
        • Austin P.C.
        • Lee D.S.
        • Ko D.T.
        • White I.R.
        Effect of variable selection strategy on the performance of prognostic models when using multiple imputation.
        Circ Cardiovasc Qual Outcomes. 2019; 12: 1-14
        • Misumi K.
        • Matsue Y.
        • Nogi K.
        • Sunayama T.
        • Dotare T.
        • Maeda D.
        • et al.
        Usefulness of incorporating hypochloremia into the get with the guidelines-heart failure risk model in patients with acute heart failure.
        Am J Cardiol. 2022; 162: 122-128
        • Pencina M.J.
        • D’Agostino R.B.
        • Steyerberg E.W.
        Extensions of net reclassification improvement calculations to measure usefulness of new biomarkers.
        Stat Med. 2011; 30: 11-21
        • Miyashita N.
        • Matsushima T.
        • Oka M.
        • Japanese Respiratory Society
        The JRS guidelines for the management of community-acquired pneumonia in adults: an update and new recommendations.
        Intern Med. 2006; 45: 419-428
        • Johnston S.C.
        • Rothwell P.M.
        • Nguyen-Huynh M.N.
        • Giles M.F.
        • Elkins J.S.
        • Bernstein A.L.
        • et al.
        Validation and refinement of scores to predict very early stroke risk after transient ischaemic attack.
        Lancet. 2007; 369: 283-292
        • Harrell Jr., F.E.
        Regression modeling strategies.
        2nd ed. Springer, New York2015
        • Judd C.M.M.G.
        • Ryan C.S.
        • Data analysis.
        Model comparison approach.
        2nd. ed. Routledge, New York City2015
        • Ambler G.
        • Seaman S.
        • Omar R.Z.
        An evaluation of penalised survival methods for developing prognostic models with rare events.
        Stat Med. 2012; 31: 1150-1161
        • Lee D.S.
        • Austin P.C.
        • Rouleau J.L.
        • Liu P.P.
        • Naimark D.
        • Tu J.V.
        Predicting mortality among patients hospitalized for heart failure: derivation and validation of a clinical model.
        JAMA. 2003; 290: 2581-2587
        • Aronson D.
        • Mittleman M.A.
        • Burger A.J.
        Elevated blood urea nitrogen level as a predictor of mortality in patients admitted for decompensated heart failure.
        Am J Med. 2004; 116: 466-473
        • Klein L.
        • Massie B.M.
        • Leimberger J.D.
        • O’Connor C.M.
        • Piña I.L.
        • Adams K.F.
        • et al.
        Admission or changes in renal function during hospitalization for worsening heart failure predict postdischarge survival: results from the outcomes of a prospective trial of intravenous milrinone for exacerbations of chronic heart failure (OPTIME-CHF).
        Circ Heart Fail. 2008; 1: 25-33
        • Sands J.M.
        Mammalian urea transporters.
        Annu Rev Physiol. 2003; 65: 543-566
        • Kiuchi S.
        • Ikeda T.
        Management of hypertension associated with cardiovascular failure.
        J Cardiol. 2022; 79: 698-702
        • Gheorghiade M.
        • Abraham W.T.
        • Albert N.M.
        • Greenberg B.H.
        • O’Connor C.M.
        • She L.
        • et al.
        Systolic blood pressure at admission, clinical characteristics, and outcomes in patients hospitalized with acute heart failure.
        JAMA. 2006; 296: 2217-2226
        • Lourenço P.
        • Pereira J.
        • Ribeiro A.
        • Ferreira-Coimbra J.
        • Barroso I.
        • Guimarães J.-T.
        • et al.
        C-reactive protein decrease associates with mortality reduction only in heart failure with preserved ejection fraction.
        J Cardiovasc Med (Hagerstown). 2019; 20: 23-29
        • Geenen L.W.
        • Baggen V.J.M.
        • van den Bosch A.E.
        • Eindhoven J.A.
        • Kauling R.M.
        • Cuypers J.A.A.E.
        • et al.
        Prognostic value of C-reactive protein in adults with congenital heart disease.
        Heart. 2020; 107: 474-481
        • Yamada T.
        • Haruki S.
        • Minami Y.
        • Numata M.
        • Hagiwara N.
        The C-reactive protein to prealbumin ratio on admission and its relationship with outcome in patients hospitalized for acute heart failure.
        J Cardiol. 2021; 78: 308-313
        • Mendall M.A.
        • Patel P.
        • Asante M.
        • Ballam L.
        • Morris J.
        • Strachan D.P.
        • et al.
        Relation of serum cytokine concentrations to cardiovascular risk factors and coronary heart disease.
        Heart. 1997; 78: 273-277
        • Valentova M.
        • von Haehling S.
        • Bauditz J.
        • Doehner W.
        • Ebner N.
        • Bekfani T.
        • et al.
        Intestinal congestion and right ventricular dysfunction: a link with appetite loss, inflammation, and cachexia in chronic heart failure.
        Eur Heart J. 2016; 37: 1684-1691
        • Klein L.
        • O’Connor C.M.
        • Leimberger J.D.
        • Gattis-Stough W.
        • Piña I.L.
        • Felker G.M.
        • et al.
        Lower serum sodium is associated with increased short-term mortality in hospitalized patients with worsening heart failure: results from the outcomes of a prospective trial of intravenous milrinone for exacerbations of chronic heart failure (OPTIME-CHF) study.
        Circulation. 2005; 111: 2454-2460
        • Grodin J.L.
        • Simon J.
        • Hachamovitch R.
        • Wu Y.
        • Jackson G.
        • Halkar M.
        • et al.
        Prognostic role of serum chloride levels in acute decompensated heart failure.
        J Am Coll Cardiol. 2015; 66: 659-666
        • Grodin J.L.
        • Sun J.L.
        • Anstrom K.J.
        • Chen H.H.
        • Starling R.C.
        • Testani J.M.
        • et al.
        Implications of serum chloride homeostasis in acute heart failure (from ROSE-AHF).
        Am J Cardiol. 2017; 119: 78-83
        • Hanberg J.S.
        • Rao V.
        • Ter Maaten J.M.
        • Laur O.
        • Brisco M.A.
        • Perry Wilson F.
        • et al.
        Hypochloremia and diuretic resistance in heart failure: mechanistic insights.
        Circ Heart Fail. 2016; : 9
        • Kotchen T.A.
        • Luke R.G.
        • Ott C.E.
        • Galla J.H.
        • Whitescarver S.
        Effect of chloride on renin and blood pressure responses to sodium chloride.
        Ann Intern Med. 1983; 98: 817-822