It sounds clichéd to say that addressing the needs of the most medically and socially complex patients is challenging. An article published by researchers affiliated with the Abdul Latif Jameel Poverty Action Lab (J-PAL) based at MIT in last week’s New England Journal of Medicine shows that improving care for such patients is challenging indeed. The new study was a randomized controlled trial, or RCT, of the “hot-spotting” model for care of complex patients developed by the Camden Coalition of Healthcare Providers. The article reports on one main outcome, hospital readmissions — patients coming back to the hospital after an initial stay.
The study’s authors found that, on average, patients who were randomly assigned to the Camden Coalition’s care management intervention were readmitted to the hospital just as often as patients who received “usual care.”
The result, while disappointing, is not likely to entirely surprise close observers of the complex care field. Leadership at the Camden Coalition has long vigorously called attention to the failure of our health system to meet the needs of complex patients, but they never promoted their model as a magic bullet and consistently called for careful study of its impacts.
It is now incumbent on all of us who seek to improve care for the most marginalized patients to fully understand the lessons from this new trial, and, importantly, to take account of what lessons it does not offer.
RCTs are the gold standard of proof in biomedical research. When well executed, such trials rule out myriad possible biases that are common in studies without random assignment. But RCTs also have limitations. For one, they are typically conducted under ideal circumstances which do not necessarily reflect the real world and often focus on narrow classes of patients or a narrow set of outcomes, limiting their generalizability. Further, according to Princeton University’s Angus Deaton, randomized trials are especially inadequate for helping us understand why interventions work or, by extension, why they fail to work.
Health services researchers like me have long understood the importance of the readmission rate as a marker of the care adequacy. Among patients with complex needs, readmissions may result from shortcomings in the safety net. Given the diversity of needs of complex care populations, we also understand that whatever the result of this RCT, it would be premature to judge from one trial examining one outcome whether the Camden Coalition model should continue or how it can be strengthened. To achieve those goals, much more work remains to be done.
Tackling failures in delivery of health care
The work of the Camden Coalition is well-known to health policy wonks, safety net service providers, and patient advocates. Dr. Jeffrey Brenner founded the organization that would become the Camden Coalition in 2002 to tackle a failing health care delivery system and dismal health outcomes for residents of Camden, one of the nation’s poorest cities. The coalition’s work was considered a breakthrough by many. Brenner and his colleagues used innovative methods to analyze data to identify hot spots of excessive hospital emergency department and inpatient care and to deploy intensive intervention teams of social workers, nurses, and community health workers.
The approach was considered a paradigm shift, and was featured in the New Yorker magazine and other high-profile media and ultimately won Brenner a MacArthur fellowship “genius” award. The approach was considered so promising that it received a large federal Innovation Award grant to expand the work, and to demonstrate and test its potential on a broader scale. Although Brenner was optimistic about the possibility of this new approach, the coalition team, with which I have worked closely, viewed it as experimental and likely to require multiple refinements over time given the complexity of the patient population, their life circumstances, and the care delivery systems involved.
Like so many others, my group at Rutgers Center for State Health Policy saw the potential of the model. Working with Brenner and the coalition, we were also awarded a federal Innovation Award to facilitate the adaptation of the hot-spotting model in four diverse healthcare settings serving populations experiencing poverty across the country.
Our work with the Camden Coalition and four clinical sites could not have been more amazing and gratifying. We got to go along for the ride with some of the most committed health and social services professionals one could imagine, and observe as they worked tirelessly to turn around lives of people who were suffering from multiple chronic illnesses, disenfranchisement, and extreme social disadvantage. Sometimes they did practical things that we don’t think of as “health care.” They advocated for a man to get his broken wheelchair replaced, enabling him to go to church and build a social support network. They gave air conditioners to people who frequented emergency departments with out-of-control asthma. They also systematically tracked peoples’ engagement in health care in their communities. They accompanied hundreds of patients to doctor visits following lengthy hospital stays, and they sorted out piles of free medication samples for people who lost track of how to manage their countless health problems. We had no doubt that people were being made better off, and independent evaluation showed that our projects reduced hospital readmission rates among Medicare patients and suggested other promising findings.
Despite these apparent successes, as researchers we knew more study was needed to demonstrate program impact. To their credit, Brenner and his successors at the Camden Coalition felt the same way. That is why they reached out to work with the independent researchers at J-PAL.
Results of the randomized trial
The J-PAL investigators, working closely with the Camden Coalition, used the same high RCT design standards that the FDA would require of a trial of a new drug. Their work was rigorous, but in many respects the Camden Coalition intervention did not easily fit the RCT rubric. It is not a pill or standardized medical procedure.
Notably, most trials focus on narrowly defined populations, such as those with a specific diagnosis but with limited or no co-occurring complications. Homogeneity of a study population maximizes the prospect for finding the hypothesized outcome. But by design, the Camden Coalition intervention was applied to an extraordinarily heterogeneous population, not just in terms of medical conditions but also social circumstances. Some were homeless, many had multiple chronic illnesses, many lacked social support. The hope was that a brief, mostly standardized intervention would be enough to reverse the revolving door of hospitalizations that many of these diverse patients experienced.
As I noted above, another key limitation was that the RCT focused on a single outcome — readmission to the hospital within 180 days of an initial stay. The study was originally also designed to examine emergency department utilization, but did not because of data limitations. (We may be able to remedy this gap: more below). While the readmission rate is an important marker of the care adequacy, it is just one of many possible outcomes of interest. It would be premature to declare the intervention failed for not achieving the readmission measure alone.
In addition, typically RCTs do not modify their intervention design during the test period. A hallmark of the coalition’s work is that it is adaptive, with care management approaches changing to address diverse patient needs in the dynamic health system. The intervention was tested during a time of extraordinary health system change. Medicare, for instance, was experimenting with “value based payment” and Medicaid was implementing a broad population health improvement incentive program. Unfortunately, the RCT was not designed to test whether intervention effectiveness improved over the course of the trial as the model was refined and adjusted.
Kudos both to the Camden Coalition and J-PAL for their pathbreaking work. But we need to view the RCT as a single data point, and should not take it as the final verdict. It is typical in clinical research to conduct secondary or exploratory analyses, especially when the hypothesized outcome was not achieved, as in this case. Researchers can explore the trial data and strive to understand why the hoped-for outcome did not materialize. Analyses can explore whether the intervention shows more promise for specific subgroups. Unlike the original RCT, such analyses cannot be considered definitive and care must be taken never to declare success based on exploratory analysis. Any promising results from such analyses should be subjected to rigorous confirmatory research.
It is also important to test the model in other settings and among different populations. Our earlier four-site adaptation of the coalition model showed great promise for readmission reduction; we do not know why that did not translate to the J-PAL trial. Given the importance of the problem we are seeking to address, researchers across the country should also add to the complex care management research literature with new studies.
My colleagues and I at Rutgers in collaboration with colleagues at J-PAL and the Camden Coalition are exploring one such opportunity for additional analysis. Working with the New Jersey Medicaid agency, we would link J-PAL’s RCT data to Medicaid claims and other services data. This will enable us to study a broader range of outcomes, including emergency department visits and use of prescription drugs, community-based primary care, mental health, and substance use treatment services over a longer period. The planned work will be hypothesis generating, not hypothesis testing, but it is nevertheless vitally important.
We must continue to study how committed professionals and paradigm-breaking strategies can help improve the lives of some of the most vulnerable in society.