(This article is now part of the 'Long Reads' series of our Drug Safety Matters podcast. Listen to the article and a discussion with the author here.)
With the successful rollout of vaccines, starting barely a year into the pandemic, rigorous monitoring of their safe use immediately followed. Both regulators and WHO advocated for close monitoring and reporting of any potential adverse drug reaction (ADR), and this call was certainly adhered to: 2.9 million reports flooded into VigiBase during 2021, and an additional 1.7 million reports were entered during the first 10 months of 2022.
However, while these reports form a crucial asset in supporting the safe use of the vaccines, such a massive influx of reports does create an imbalance within VigiBase. “COVID-19 vaccine” is now, by far, the most commonly reported medicinal product type in VigiBase with 4.6 million reports (13% of the entire database). In a modest second place comes adalimumab with 650,000 reports, followed by etanercept taking the third spot on the podium with 570,000 reports. (And both adalimumab and etanercept have been on the market for over 20 years.)
Overall, the response to the call for reporting of COVID-19 ADRs must be considered a success. Nevertheless, when a single medicinal product makes up such a large proportion of the entire database, it will affect statistical signal detection, even when performed for other drugs. Let’s take Bell’s palsy as an example, where 87% of the 13,000 entries in VigiBase come from COVID-19 vaccine reports. It doesn’t matter which measure of disproportionality you use, the expected counts for Bell’s palsy will be inflated massively by the COVID-19 vaccine reports, which means that it will not appear as disproportionate for other drugs as long as the COVID-19 vaccine reports are in the way and hide the statistical signal. This effect is called masking, also known as cloaking, or competition bias.
Masking is not a new problem in pharmacovigilance. In 2013, UMC developed a method to uncover masking by identifying influential outliers. Earlier this year, we applied this method to all ADRs that have been reported for COVID-19 vaccine reports. What it basically does is compare the frequencies of an ADR with and without COVID-19 vaccine report. If we see an increase by at least 50%, we consider the ADR masked. The method also applies statistical shrinkage to the calculation to get a more robust measure, limit the effects of random variability, and shift focus away from ADRs that are overall very rarely reported in the database.
Using this threshold, we identified 132 ADRs as masked by COVID-19 vaccine reports. Of these, 21 are on the EMA’s list of Important Medical Events. Examples of reactions we see masked are sweating fever, Bell’s palsy, vaccine breakthrough infection, hemiparaesthesia, myocarditis, and pericarditis. And notably, for some of these ADRs, a majority of the reports in VigiBase now relate to COVID-19 vaccine reports, as is the case with our example, Bell’s palsy.
In a very interesting study made in VAERS, published in July 2022 in Drug Safety by Harpaz et al, we can see the opposite problem, how statistical detection of early COVID-19 vaccine signals in the very beginning of the vaccine rollout were delayed because of masking due to other drugs. For example, that study found that the statistical signals of myocarditis with Pfizer and Moderna vaccines were masked by smallpox vaccine. As more data accumulated, the Pfizer and Moderna vaccines then started to mask the signal from each other. This is also a very interesting piece of the puzzle, understanding that masking is not a static phenomenon, but something that evolves with the data. This study also teaches us that it might be especially important to consider masking when a new drug is introduced, or as long as there are few reports on that drug. Now the seesaw has flipped for the COVID-19 vaccines – signals are no longer masked from them but due to them.
So where do we stand now, knowing that 13% of VigiBase is made up of COVID-19 vaccine reports?
The large proportion of COVID-19 vaccine reports may hide or delay the detection of statistical signals of many ADRs when calculating disproportionality for other drugs. This is a problem that must be addressed. One way is to simply remove all COVID-19 vaccine reports when doing statistical signal detection for other drugs. Another way is to perform a method of unmasking, like the simple unmasking suggested by Juhlin et al 2013, which identifies any outliers and carries out a parallel analysis after they have been omitted. Regression analysis has also been proposed as a method to deal with the issue.
In the end, though, whichever method we choose is not that important. What is important is that we stay aware of the problem of masking (due to COVID-19 vaccines as well as due to other drugs) and do something.
Read more
R Harpaz, et al, “Signaling COVID‑19 Vaccine Adverse Events”, Drug Safety, 2022.
New VigiLyze Therapeutic Scope setting enables unmasking
A recent update to VigiLyze lets users select drug- or vaccine-specific search and analysis. By default, all reports are included in the dataset, including drugs and vaccines. But with the new “Therapeutic Scope” setting, users can easily limit the scope to either vaccines or drugs. By controlling which reports are available for searches, users gain options for calculations and results, such as disproportionality values and report counts in the Qualitative and Quantitative areas.