We spend many billions of dollars each year on the discovery and development of new drugs, but almost none of it addresses two crucial questions: How do these new therapies compare with already known ones? What are the relative benefits and harms in a particular situation, for a person like you?

Such questions can best be answered by comparative effectiveness research.

To get approval from the Food and Drug Administration, drugs must be proved both effective and safe. The costs of doing this are significant, and they are most often borne by the pharmaceutical industry.

But the F.D.A.’s bar, while meaningful, often isn’t very useful for what physicians and patients really care about every day: how effective and safe drugs are compared with one another.

Real-World Questions

Consider antibiotics. In my work as a pediatrician, questions about their use come up a lot. Which drug is the best first-line therapy for which common illnesses? We don’t know. How long should we treat for different infections? We don’t know. What are the relative trade-offs between benefits and side effects in different patients in different circumstances? We don’t know.

The questions we need answered are legion. All the guidelines and practices we have are best guesses.

Comparative effectiveness research can take on many forms and involve more than drugs.

• Because of a trial published in The American Journal of Preventive Medicine, we know that an intensive lifestyle intervention works better than metformin to promote weight loss.

• A study published in JAMA Pediatrics last year showed that adding individual health coaching to enhanced primary care reduced pediatric obesity, but no more than enhanced primary care alone.

• A comparison of different levels of insurance published in The New England Journal of Medicine showed that enhanced drug insurance coverage led to increased medication adherence, lower patient spending and lower rates for a first major vascular event (like a heart attack or stroke), with no increased overall health care spending.

A Blood Pressure Study

We know that high blood pressure is both terribly prevalent and a significant risk factor for cardiovascular disease. We also know that there are a lot of drugs out there, all F.D.A.-approved, that can help reduce this risk by better controlling blood pressure. But which is best?

This question isn’t new. In 2002, the results of the Antihypertensive and Lipid-Lowering Treatment to Prevent Heart Attack Trial — a comparative effectiveness trial — were published in JAMA.

Participants had to be at least 55, have hypertension and have at least one other risk factor for coronary heart disease. They were randomly assigned to take one of four drugs, each with an entirely different mechanism, representing a different class of drugs.

Image

Chlorthalidone is a diuretic, or a drug that increases urine output. Amlodipine, a calcium channel blocker, causes blood vessels to relax and widen, and lowers the heart rate. Doxazosin does the same by blocking the effects of adrenaline on muscles throughout the body. Lisinopril blocks the enzyme angiotensin, which tightens blood vessels, leading to lower blood pressure. All the patients were tracked for four to eight years.

All of these drugs had been proved safe and effective; we just didn’t know what worked best as a first-line therapy for the many people with high blood pressure.

The main outcomes of interest were death from coronary heart disease, or a heart attack that didn’t lead to death. By those measures, there was no difference between any of the four drugs in that period. But chlorthalidone outperformed two of the others in lowering systolic blood pressure. That drug also performed better in preventing heart failure (a gradual weakening of the heart) and stroke, and lowering rates of cardiovascular disease.

The take-home message was that the diuretic was better in preventing at least one of the major types of cardiovascular disease. It was also the least expensive.

As you can imagine, this is immensely valuable information. It tells us what drug is best to start if you have someone over 55 with high blood pressure and at least one risk factor for coronary heart disease. That’s exactly the kind of question that only a comparative effectiveness trial can answer.

This study was enormous; it took place in 623 centers in Canada and the United States between 1994 and 1998, and included over 33,000 participants. It also cost more than $100 million, and that was two decades ago.

So was the debate over? Far from it. A stunning number of papers have been written critiquing this study.

There are methodological concerns, in that the primary endpoint (death and heart attacks) was somewhat ignored in favor of secondary outcomes like strokes and blood pressure. Most of the patients had probably been on other therapies before starting the trial, so it’s not clear if prior therapy could have changed results. Many patients received more than one drug, and the stepwise addition (adding drugs one at a time) might have favored the diuretic. Significantly more people on the diuretic developed diabetes than those on other drugs. And in the many years since the trial, these drugs have all become cheaper and generic, and more drugs have appeared, making the answers somewhat murky once again.

Recent work in The Journal of the American Heart Association even improved on the huge 2002 study by showing that using a combination of drugs to treat hypertension initially is better than starting patients on one drug and then progressing to more.

A Small Budget and a Large Task

In the United States, comparative effectiveness trials are supported almost exclusively by the National Institutes of Health and the Patient Centered Outcomes Research Institute. The latter’s executive director, Joe Selby, wrote to me: “It’s essential that we learn to ask and answer practical questions about the comparative clinical effectiveness of therapy options in the course of everyday care.”

He said the research institute was the only organization dedicated primarily to supporting and expanding this kind of research in the United States “with a rigor and scale that match the importance of this relatively new approach to building knowledge and information.”

The research institute’s budget constitutes a small percentage of overall public research funding.

Basic science research is necessary to make breakthroughs in how treatments might be created. Randomized controlled trials are necessary to determine if they have efficacy. Pragmatic trials can tell us if and how they’re effective in real world settings. Health services research can improve the ways in which we deliver them. But without comparative effectiveness research, too many important questions that concern patients will remain unanswered.