The Washington Post
Tuesday, October 25, 2005; Page HE01

A Shot of Fear
Flu Death Risk Often Exaggerated; So Is Benefit of Vaccine

By Steven Woloshin, Lisa M. Schwartz and H. Gilbert Welch
Special to The Washington Post


Medical research often becomes news. But sometimes the news is made to
appear more definitive and dramatic than the research warrants. This series
dissects health news to highlight some common study interpretation problems
we see as physician-researchers and show how the research community, medical
journals and the media can do better.

For years, the public health community has used fear as one strategy to
promote the flu vaccine. A vaccination poster distributed by the U.S.
Centers for Disease Control and Prevention (CDC), for example, emphasizes
that "36,000 Americans die of flu-related illnesses each year," implying
that the vaccine could prevent many of these deaths.

When it became aware of the vaccine shortage last October, the federal
government changed course and tried to reassure Americans that going without
a shot was no big deal. "We all need to take a deep breath. This is not an
emergency," CDC director Julie Gerberding advised the public.

Instead of urging vaccination for everyone age 50 and older, as they had
been doing, government officials recommended shots only for people 65 and
older, and those in selected high risk groups. The public's response was
predictable: People were upset and confused. Our local television news
played a story in which a pharmacist was called "a murderer" when his
vaccine supply ran out. Ironically, the crisis mentality led some to engage
in behaviors that probably increased their risk. Frail elderly people, some
with oxygen tanks, stood in long lines in the cold, waiting for the vaccine.
Others crowded clinics and doctors' offices, increasing their chance of
exposure to flu and other infectious agents.

With uncertainties about this year's vaccine supply, the CDC again
recommended that highest-risk people get priority for flu shots, at least
until late October. But last year's flu season may have left people confused
about essential points: Just how risky is the flu? And just how effective is
the vaccine? The answers to these questions may surprise readers.

How Risky Is the Flu?

First, a caveat: The risk calculations we analyze here describe typical flu
seasons only. We don't consider here what the picture would be in the event
of a deadly flu pandemic -- a worldwide outbreak of a new, highly virulent
flu strain, the potential for which has recently drawn considerable media
attention. No one really knows how likely such an outbreak is, but the risk
profile would certainly change. A pandemic is a fundamentally different
situation: The risk of death would be substantially higher, and untested
strategies (including new treatments, quarantine and a new vaccine) would
need to be implemented rapidly.

We deal here with what is known about typical flu seasons, based on data
that form the basis for the federal government's flu-risk figures.

By choosing to highlight the annual number of flu deaths, the CDC employed
an attention-grabbing tactic often used by public health and disease
advocacy groups. It's a tactic readers should be inoculated against if they
want a clear picture of the risks they face. (See "Research Basics:
Understanding How Big a Risk Is," right.)

In fact, it is very difficult to know how many people die from any given
disease because there is often much uncertainty in determining the cause of
death. This is particularly true for the flu. That's because it shares
symptoms with so many other diseases, and because people most likely to die
a flu-related death are also at high risk for many other causes of death.

Flu deaths are probably undercounted because doctors do not routinely test
for the flu, and because some deaths that should be attributed to the flu
are given other diagnoses. For example, someone who dies from a heart attack
because they are debilitated by the flu might not get counted as a flu
death. Some overcounting of flu deaths also occurs: Clearly not all winter
pneumonia deaths are caused by the flu.

According to the CDC, 90 percent of flu-related deaths occur among people
age 65 years and older. Based on this information and the age distribution
of the population, the chance of a flu-related death for people in that age
group is about one in 1,000. Another way of saying this is that the chance
of not dying from flu for those 65 and older is about 999 out of 1,000. (For
context, the chance of a flu-related death is slightly lower than the chance
of dying from a fall or other accident.)

For people younger than 65 (including children), the chance of a flu-related
death is much smaller -- about one in 100,000. Of course, adults and
children might be concerned about flu-related problems besides death, such
as being hospitalized or just suffering with unpleasant symptoms (typically
three to seven days of fever, muscle aches, headache, weakness, dry cough
and runny nose). As you might guess, counting the number of flu-related
hospitalizations or the number of people experiencing symptoms from the flu
is even more difficult than counting flu deaths.

How Good Is the Vaccine?

Getting a shot does not guarantee you will not get sick from the flu or die
from it. Recently, the Cochrane Collaboration, an international group that
evaluates the evidence for various medical interventions, reviewed the
medical literature on the effectiveness of the flu vaccine in preventing
death.

Unfortunately, the evidence on how well the vaccine works to prevent death
in the elderly is limited. Few of the existing studies are randomized
trials -- considered the gold standard for medical evidence. Instead, most
data are from observational studies -- studies in which scientists simply
count up outcomes (here, the number of deaths that occur among people who
did or did not get the vaccine).

But drawing conclusions about cause and effect from such observations is
fraught with problems.

For example, a 2003 study published in the New England Journal of Medicine
observed that the flu vaccine was associated with a 50 percent reduction in
the overall death rate (that is, death from heart disease, stroke, cancer
and all other causes combined). To attribute an effect of this magnitude
solely to the flu vaccine is ludicrous: Flu-related deaths make up less than
2 percent of all deaths. If the claim were accurate, the vaccine's power
would dwarf that of any other medical intervention. There is, however, a
much more likely explanation: People who choose to get a flu shot are much
healthier -- and therefore already at much lower risk of death -- than
people who do not.

Only five randomized trials have examined the effectiveness of the flu
vaccine. In these studies, patients were randomly assigned -- a selection
technique equivalent to the flip of a coin -- to get either a flu vaccine or
a placebo injection. But none of these studies looked at whether the vaccine
prevents death. Instead, the scientists measured who developed a flu-like
illness. For a summary of the findings of these studies, see "How Well Does
the Vaccine Work in the Elderly?" below.

In the absence of good randomized trial data, it is still possible to gauge
the effectiveness of vaccination by looking at time trends in flu vaccine
rates compared with flu-related deaths in the elderly. As more people get
vaccinated, you would expect the flu-related death rate to decline -- if the
vaccine is effective. But, as the graph below, titled "A Windening Gap,"
shows, despite a dramatic increase in vaccination among the elderly, deaths
from the flu and pneumonia have hardly budged. (The calculations have taken
into account the aging of the population.)

For younger adults, flu-related death is so rare that it has not been
reliably studied: Doing so would require a trial of millions of people.

Of course, the flu shot may have benefits besides reducing the chance of
death. Many may get flu shots simply to avoid getting sick. The Cochrane
Collaboration identified more than 20 randomized trials addressing this
question. The overall chance of developing "clinical" flu (we'll explain in
a minute) was 19 percent in those chosen, again by chance, to receive the
recommended flu vaccine vs. 23 percent in the control groups.

The careful reader may notice that these percentages are substantially
higher than those reported for the elderly. (See "How Well Does the Vaccine
Work in the Elderly?") This is because clinical flu is defined as a set of
non-specific symptoms including fever, cough and muscle aches -- symptoms
shared by many non-flu illnesses like the common cold. These non-flu
illnesses may be especially common in younger adults because of their
exposure to other people, particularly children. To try to isolate the
effect of the vaccine, scientists sometimes use laboratory tests to confirm
the activity of flu virus in the blood. Using this measure, the chance of
flu in the vaccine group is 2 percent vs. 7 percent in the control group.

Studies have also measured another outcome: how vaccination affects days
lost from work. On average, there are about 0.16 fewer days lost from work
per person vaccinated. Another way of saying this is that about 5 percent of
those vaccinated avoid missing about three days of work because of the flu.
(That is, 0.16 days divided by the 5 percent who benefited from vaccination
equals 3.2 days.) The other 95 percent vaccinated got no benefit.

Take-Home Messages

To promote vaccine use, many in the public health community have overstated
the risk of flu-related death and the effectiveness of the vaccine in
preventing it. While the flu vaccine may have some important benefit (less
flu-related illness), we really do not know whether it reduces the risk of
death. For younger individuals -- for whom the chance of flu-related death
is extremely small -- any death-protection benefit can only be very modest
(and it is unlikely we will ever reliably know whether it even exists).
However, we do know that the vaccine reduces the risk of being sick and time
lost from work. But because the effect is small, individuals will have to
judge for themselves whether it's worth the bother.

We are not suggesting that Americans forgo flu vaccines. We simply want to
help people make informed decisions.

For many people, getting the vaccine is a reasonable choice. And many may
reasonably choose not to get it. (Consequently, the use of flu vaccination
rates by Medicare and others to measure health care quality probably does
not make sense.)

Regardless, public health officials should not exaggerate risks or benefits
to promote vaccination. Exaggeration carries a price: Not only do some
people get scared and engage in behaviors that increase their risk (like
waiting in a crowded clinic for a flu shot). They may also grow cynical and
end up ignoring health messages that really matter.

Steven Woloshin, Lisa Schwartz and Gilbert Welch are physician-researchers
in the VA Outcomes Group in White River Junction, Vt., and faculty members
at the Dartmouth Medical School. They conduct regular seminars on how to
interpret medical studies. (Seehttp://www.vaoutcomes.org.) The views
expressed do not necessarily represent the views of the Department of
Veterans Affairs or the United States Government.

http://www.washingtonpost.com/wp-dyn/content/article/2005/10/22/AR2005102200