...despite decades of underfunding and a “motley of surveillance systems” (as characterized by The Star), public health units and healthcare providers have still managed, by and large, to protect Ontarians against vaccine preventable diseases through promoting and tracking high rates of compliance and coverage...
By Dr. Lawrence C. Loh
Adjunct Professor, Dalla Lana School of Public Health, University of Toronto
The opinions he expresses here are his own and do not represent any other institution with which he is presently or has previously affiliated
Stats, Data and the Popular Media: a Closer Look at The Toronto Star’s Stats on Vaccine Coverage in Ontario
There is a part of the world where measles outbreaks are incredibly common, owing to a relatively lower measles-vaccine coverage rate of 65%. Between 2017 and the middle of 2018 there were nearly 1000 cases of measles reported, with just under a hundred cases being reported monthly and a major outbreak to kick off 2017 with nearly 500 cases seen between January and February.
A recent article in the Toronto Star, Canada’s largest daily circulating newspaper, has reported a similarly low “coverage rate” in Toronto and other health units in Ontario, which might have you imagining that this is the case right here at home.
But it’s not. In fact, the jurisdiction in question (with the mid-sixties measles coverage rate) is actually Guinea, in West Africa – also infamous for being the epicenter of the 2014 Ebola outbreak.
The flawed statistics and analyses are presented in the Toronto Star article, along with a suggestion by an academic at the University of California, Berkley that the “coverage rates” presented are “nowhere near the level required for herd immunity.”
If true, however, why isn’t Ontario and the Greater Toronto Area seeing endemic transmission, perennial outbreaks, and hundreds of cases being reported every single month?
The answer lies in in echoes of a similar misstep by the newspaper around a since-retracted article on the safety of the quadrivalent-HPV vaccine.
How vaccine records and coverage rates are determined in Ontario
It’s important to understand a few basics: firstly, how vaccine coverage data is collected in Ontario, and secondly, how coverage rates are calculated.
In Ontario, vaccinations are delivered by a variety of providers, most typically physicians, and students are typically screened annually throughout the school year for compliance with the Immunization for School Pupils Act, which requires up to date records of vaccination or a filed exemption for school entry.
This data is constantly evolving as the school year goes on, and can’t necessarily be calculated in real-time, since it relies on the health unit obtaining records from parents that they themselves have obtained from their provider.
To that end, parents are typically sent several requests, followed by warnings, before being issued with a notice of suspension, all with the goal of obtaining either up to date records, or a filed exemption (either medical or philosophic.)
Once that data is obtained, it can be used to calculate coverage rates, which are specific to disease, not to “vaccines given”. Someone up to date for ‘pentacel’, for example, is up to date for five diseases and shores up overall coverage rates for diphtheria, pertussis, polio, tetanus and haemophilus influenzae B.
So does The Star’s metric actually represent “coverage rates?”
Where did it go wrong with the statistic used by the Star? In its investigative article, presented as “coverage rates” is the summary measure: “all seven-year olds who are up to date for all antigens (diseases) required by law for school entry as of September 2018.”
While one can agree that being up-to-date with vaccines is a great individual health asset for anyone, there are several reasons why using this statistic is a significant misinterpretation:
Coverage rates are disease specific
The Star’s metric does not represent a coverage rate; coverage rates are calculated specific to disease. There is no disease called “up to date for all vaccines.”
Take, for example, a student who received every vaccine except their Men-C. They would in fact be covered for every disease save for Men-C but would still show up as “not covered” in the Toronto Star’s summary measure.
Coverage rates should answer the question: “How well protected are we against (disease x)?” That means digging into the disease specific coverage rates for the population in question.
On this, incidentally, appropriate coverage data do exist and are released annually by Public Health Ontario at the close of the enforcement year. Using measles as an example, coverage rates in 2016-17 among 7-year old students are closer to the 85-90% level across the province and much closer to required levels for community immunity. This is why Ontario’s measles picture does not look like Guinea, despite cases of measles passing through our communities with remarkable regularity.
The metric is drawn at the wrong point in time
Hockey fans know that a winning percentage at the start of the season changes dramatically as the season goes on. Similarly, the data reported by the Star was pulled at the beginning of the school year, before enforcement activities began. As work progresses over the school year, more data on the vaccination status of children (reflected as well in higher coverage rates!) becomes available, meaning The Star’s numbers largely reflect a reporting delay rather than a doomsday scenario.
Compliance rates matter more and 7-year olds are only part of the population
While 7-year olds are the age used for provincial coverage reporting, public health units also enforce legislative compliance for all students of all ages; “compliance” means having either updated records or an exemption filed. This is most useful in outbreaks, and where The Star’s metric falls short yet again – because a summary measure says nothing about which students have filed records or exemptions for a specific disease. When pertussis comes to call, 98-99% compliance rates in most health units allow determination around which students are vaccinated and which students should be excluded until the outbreak is over.
Different health units, different realities
Given thirty-five different health units in Ontario with vastly different socioeconomic and demographic realities, one can imagine that some health units will have not that many students, while others contend with a large, diverse, and growing number. Despite some things being easier in rural areas than in the metropolis, though, health units still manage to get the data they need over the course of a year – to ensure that coverage rates don’t resemble that of Guinea.
What does it really mean?
The simplest test of whether The Star’s numbers hold muster is thus in the overall infectious disease reality observed in Ontario. Our picture is not the picture of Guinea, or many other settings worldwide, where many vaccine-preventable diseases remain an ever-present threat.
While they did get the metric wrong, The Star in an accompanying editorial used their findings to helpfully highlight the potential perils posed by vaccine hesitancy. Certainly, more could be done to reassure parents that vaccinations remain one of the best ways to protect the health of their children.
More likely, however, the numbers reflected in their summary measure from the start of the school year simply reflect how data moves in a system where vaccines are delivered by one stakeholder, held by another, and tracked by a third; they also do not reflect what is commonly understood as “coverage rates.”
While real-time coverage rate tracking would be the holy-grail, such work would require providers to be on board with the mandatory reporting of the vaccines they give and significant additional investment in electronic data capture, which looms heavy against a backdrop of significant proposed funding cuts to public health in Ontario.
Perhaps the real story here is thus a foreshadowing: despite decades of underfunding and a “motley of surveillance systems” (as characterized by The Star), public health units and healthcare providers have still managed, by and large, to protect Ontarians against vaccine preventable diseases through promoting and tracking high rates of compliance and coverage. Where might further cuts lead?