Why were Covid vaccine trials so fast?
The timeline to develop coronavirus vaccines blew many predictions out of the water.
Something I remember fairly vividly from the first few months of the pandemic was a sense of hopelessness that any effective drug or vaccine would become available anytime soon.
Most people didn’t believe it was possible to develop vaccines in the timeframe needed for them to be useful. Some looked at past vaccine timelines, which had averaged roughly 8 to 12 years, and thought this one would be similar. Others thought that, even though it was an emergency situation, it would still take at least a year and a half, or two years, or maybe even four.
“The grim truth behind this rosy forecast is that a vaccine probably won’t arrive any time soon. Clinical trials almost never succeed. We’ve never released a coronavirus vaccine for humans before. Our record for developing an entirely new vaccine is at least four years.”
— Stuart Thompson, New York Times, April 2020.
I had a different conclusion. In a piece I wrote in the summer of 2020, I explained why I believed that vaccines would most likely arrive within a year of the beginning of the pandemic (placing a 58% probability on enough doses for 25 million Americans being approved and available between October 2020 and March 2021, with my central estimate landing around February 2021).
We now know how the timeline panned out and my forecast was, if anything, slightly too pessimistic, since vaccines first became available three months earlier, in December 2020. In this post, I want to go into more detail and take a look back at what happened. Why were Covid vaccine trials so fast?
The scientific foundation
One of the most surprising things about Covid vaccines is how many of them there are: mRNA vaccines (Pfizer and Moderna), viral vector vaccines (Oxford-AstraZeneca and Johnson & Johnson), protein subunit vaccines (Novavax), inactivated whole virus vaccines (Sinovac and Sinopharm), and others. Although their protection dropped as the virus evolved into new strains, I take this bounty of options as a result of the coronavirus being relatively easy to develop vaccines for.
Part of the reason is the disease itself. With some infections, like HIV, no one clears the virus naturally or develops lasting immunity to it, so it’s hard to know what a vaccine should mimic. Covid-19 was very different: it was evident early on that most people recovered and developed antibodies that could neutralize the virus. This suggested it was possible to prime the immune system with a vaccine.
Another reason was prior research into coronaviruses. Work on SARS and MERS from earlier coronavirus outbreaks meant scientists already understood some of their features. They had identified the key antigen the immune system reacted against – the spike protein – and had animal models and laboratory assays ready to go. Some vaccines against animal coronaviruses had already been developed. Candidate vaccines for the earlier SARS virus had also been developed for humans (and shelved after the 2003 SARS epidemic was contained). When SARS-CoV-2 arrived, lots of this groundwork could be picked up.
There’s another feature that explains why some vaccines ended up highly effective: a stabilized spike protein. The spike protein gives the coronavirus its crown-like appearance, and is the main ingredient in most Covid vaccines. But interestingly, it changes shape during an infection. Before it fuses with a cell, it sits in a compact “pre-fusion” form (on the left, below); afterwards, it springs into an elongated “post-fusion” form (on the right). The former boosts immunity most because it’s what antibodies generally encounter, before the virus has entered cells. But when the spike protein is isolated for a vaccine, it tends to collapse into the later, post-fusion form.
Over the past decade, advances in structural biology, especially in cryo-electron microscopy, have helped scientists see the pre-fusion shape of coronaviruses directly. With that knowledge, they introduced a few mutations to stabilize the spike protein in its pre-fusion form, better for the immune system to recognize. This is a big reason why vaccines with the stabilized pre-fusion protein (including the mRNA vaccines and Novavax) generated much stronger neutralizing antibody responses and were likely much more effective than vaccines without the stabilized form (including the AstraZeneca and Sputnik V vaccines).
All of these features meant the chances of a vaccine succeeding were somewhat high. They’re also why they could be designed very fast – of course, with a lot of work on the part of scientists. It took two days to design the Moderna vaccine, which then spent 63 days in preclinical testing before entering clinical trials, versus a weekend and about 100 days for the Oxford/AstraZeneca vaccine. But the vaccines would still need to be tested in clinical trials and gain regulatory approval afterwards, and this process usually takes several years.
Parallel trials
The timeline for vaccine development typically goes like this. A candidate vaccine is first designed and tested in the lab and in animals, as ‘preclinical testing’, which takes around 1.5 years on average. Next, in ‘phase 1 trials’, it’s tested in a small number of participants for basic data on safety and the immune response, taking around 2.5 years. Phase 2 trials expand this to hundreds of participants to refine the dose and gather more data on immunogenicity and safety, and take around 3 years. Then phase 3 trials test efficacy and safety in hundreds or thousands of people, and take around 2.5 years. After all these stages, all the data is submitted to the regulator as a package, which reviews it, usually taking another year.
Historically, successful vaccines have taken around 10.7 years to make it through the whole pipeline. Each phase happens sequentially and functions as a ‘go / no-go’ system, where candidates are dropped if they flop at a stage.
Covid vaccine trials took place quite differently. Rather than running sequentially, the phases ran in parallel, as you can see in the diagram below. For example, phase 1 and 2 trials were combined, collecting the relevant data for each at once, or phase 2 and 3 were combined in the same way. Ultimately, the same amount of data was collected in a shorter timeframe, by overlapping the phases rather than doing them one by one.
This was a major reason it was possible to condense the whole timeline so much. But even then, the trials still ran much faster than usual. For example, Pfizer’s combined phase 2/3 trial then took about 4.5 months, but it usually takes an average around 2.5 years for phase 3 trials alone. What made that possible?
Easier recruitment
One of the biggest delays in clinical trials is the process of recruiting enough participants to run them at all. Only half of trials meet their original target for recruitment, and a fifth revise their target downward. But the Covid vaccine trials recruited huge numbers, fast. The phase 3 trials for Moderna and Pfizer enrolled around 30,000 and 44,000 participants respectively, making them some of the largest clinical trials in history.
I think there were a couple of reasons this was easier than usual. One was that people were simply more willing to volunteer. It was a pandemic after all, and everyone’s attention was on it, plus, many people were stuck at home without much to do anyway. Interest in volunteering for trials was very high. Dr Jim Kublin, who helped run a clinical trial network for Covid, recalled: “My colleagues were enrolling a phase one study for the Moderna vaccine, and they had like 10,000 people sign up on their website. They needed 40 people to enroll.”
The infrastructure to recruit participants was also more streamlined than usual. The National Institute of Allergy and Infectious Diseases (NIAID) created a clinical trial network with a shared volunteer registry called COVID-19 Prevention Network (CoVPN), from four existing NIAID trial networks, to supplement recruitment into trials. Volunteers could sign up with a short screening questionnaire to the network as a whole and be routed to whichever trial fit. The registry helped support recruitment for the late-stage trials of Moderna, AstraZeneca, Janssen, and Novavax, and by autumn 2020, almost half a million Americans had signed up through it to take part in Covid vaccine trials. Meanwhile in the UK, a similar shared volunteer registry recruited participants through the NHS. Over half a million volunteers signed up in total, feeding into trials including those for Novavax and Oxford/AstraZeneca.
Covid was Everywhere All At Once
It also took less time to test new drugs and vaccines because the disease was spreading rapidly. It’s challenging to run trials fast when diseases are rare or slow to develop. Taking a simple example, imagine a trial where 100 people are given a placebo and 100 people given a vaccine. If only 1 or 2% of them (so 1 or 2 people) caught Covid during the trial as the base rate, it would be very hard to spot a reduction from that already-low number. Even if vaccines reduced infections by half, you’d hardly be able to tell in this example, not with much confidence anyway. Without a much larger sample size, a difference between the two groups could simply reflect noise.
This reflects the ‘statistical power’ of the trial, and that it’s easier to figure out whether a vaccine is protective if new cases are arising quickly (i.e. the disease has a high incidence rate) or if the sample size is very large. You can see this in the model I made below. It’s also why it’s much harder to develop new vaccines for diseases we’ve reduced massively: it would be a struggle to test new polio vaccines in rich countries that have eliminated the virus, for this reason.
Fortunately for trial statisticians1 – but unfortunately for everyone else – Covid was spreading quickly. During Pfizer’s trial, the monthly incidence rate (the % of the population infected for the first time, per month) was around 0.5%. By September 2020, using seroprevalence data, the CDC estimated that up to 23% of people had already been infected in the hardest-hit areas, and just below 10% across most of the US.2 In this situation, what matters most is how fast new cases are accumulating while a trial is running: during the autumn and winter of 2020, outbreaks were developing fast, and cases were racking up quickly in the placebo group, which made it easier to see a reduction in the vaccine group. Within a few months, Moderna’s trial recorded 185 Covid cases in the placebo group versus 11 in the vaccinated group, and Pfizer’s trial recorded 162 versus 8.
Emergency Use Authorization
People often point to the ‘Emergency Use Authorization’ pathway as the main reason Covid vaccines were introduced so fast – that they weren’t approved the regular way, and faced lower standards of evidence than usual. Let’s dig into this a little.
In general, an emergency use authorization (EUA) is a conditional authorization that lets the FDA allow a product to be used during a declared public health emergency before it has gone through full approval, and which can be revised or revoked at any time. And the bar is intentionally low: the FDA only has to find that the product ‘may be effective’ and that its known and potential benefits outweigh its known risks, given that there is no alternative that’s approved.
The standard of a product that ‘may be effective’ is much weaker than the evidence of effectiveness required for full approval. One example was the emergency authorization of hydroxychloroquine as a treatment for Covid in March 2020, which relied on observational data rather than efficacy trials. A few months later, when the RECOVERY trial showed, in a randomized study, that it had no benefit, its authorization was pulled.
Does this mean that the standards were so relaxed for vaccines? Not really. The vaccines faced a stricter EUA, called ‘EUA plus’. In its June 2020 guidance, the FDA demanded large randomized controlled trials demonstrating an efficacy of at least 50% before a vaccine could be licensed. Then in October, it added more conditions an EUA would have to meet: a median of at least two months of safety follow-up after the final dose, and review by an independent advisory committee in a public meeting.
This meant that the standards for Covid vaccines were similar to a full approval, with two notable exceptions. 1) While full approval required a complete manufacturing dossier, with validation and facility inspections, the EUA accepted a less complete package, and deferred the validation and inspection until later. 2) While full approval required a longer safety and durability record, typically at least six months, the EUA accepted a two-month median follow up time after the final dose.
It’s worth noting that the vaccines did fulfill the requirements of a full approval later on, starting with Pfizer’s vaccine in August 2021.
Incidentally, the EUA’s shorter follow-up requirement was the subject of political controversy. When Trump complained that the vaccine was being delayed until after the election, it was because of the follow-up requirement, which wouldn’t be complete until later – Pfizer only reached the median two months of safety follow-up in the third week of November, over two weeks after the election, and filed its EUA request on November 20.
I don’t have strong views on the requirement, but it seems hard to strike a balance between collecting enough safety data to build public trust versus speed up the roll-out, and unlike all the other reasons for acceleration, this was the only way in which the safety threshold was laxer than a full approval, which was achieved later on. It seems like the crux is how important public trust would be in increasing vaccination rates and whether the requirement achieved that.3
Rolling regulatory review
Typically, when researchers finish their trials, they compile a complete dossier with lots of information about the trial, the product, the manufacturing process, and so on, and submit it to the regulator, which then reviews the complete package.
It usually takes a while to review the complete package. In the US, the FDA’s goal is to review the data and make a decision within 10 months of receiving the application, or 6 months if it qualifies for ‘priority review; in practice it takes an average of eleven months. Meanwhile in the EU, the European Medicines Agency’s goal is 210 days (roughly 7 months), but the clock only includes the days the agency is actively reviewing; it pauses whenever the regulator sends questions back to the company and waits for answers, so in reality, a review usually stretches well beyond a year.
The Covid vaccines faced a different process. In Europe, they faced a ‘rolling review’ process instead, where regulators assessed the data – on the manufacturing, preclinical research, trial results, etc. – as it arrived, instead of all together right at the end. So by the time the final data was in, most of the review had already been done, and it took much less time to reach a decision. Pfizer submitted its formal EU application on 30 November 2020 and received conditional authorization about three weeks later, on 21 December, because the rolling review had already evaluated the data as it came in.
In the US, the rolling review process applied to the full approval, rather than the initial authorization, since the vaccines went through the Emergency Use Authorization pathway instead, which began on 20th November for Pfizer and granted approval on 11th December 2020. The complete application, which did involve rolling review, was completed in May 2021 and received full approval on 23 August 2021, roughly three to four months, still much shorter than usual.
Can we apply some of these processes to speed up other drugs and vaccines?
By this point, you’re probably wondering why the same processes couldn’t apply to many other drugs and vaccines. I’ve wondered the same thing, and I think it’s true that various parts of this process could apply elsewhere too – such as clinical trial networks to recruit participants, more fundamental research into pathogens and candidate vaccines, and better trial design. (The Covid experience is a reason I became interested in clinical trial reform more broadly.)
But some parts would be challenging to scale up more widely. Rolling regulatory review, for example, is more demanding on the part of regulators. In the usual pipeline, most drugs or vaccines fail before they reach the finish line and are submitted for review, which means regulators are saved the time of reviewing lots of applications that would have been withdrawn anyway. (You could, of course, expand rolling reviews by hiring and training more regulatory staff, and reduce the trade-off!)
Parallelized trials are also challenging. When trials are run sequentially – with phase one, phase two, and then phase three – the sequence functions as a ‘go / no-go’ system, where developers can drop candidates they think are unlikely to succeed at the next stage. It saves time to run trials in parallel, but it also raises financial risk: instead of spending money on the next phase only once the previous one has paid off, a vaccine developer would have to commit to spending it up front.4 A similar problem applies to building production capacity during the trials, since that capacity would be wasted if the vaccine doesn’t work.
This would have been especially financially risky during a pandemic, because of the degree of uncertainty at the outset. What if the virus had actually been contained by the time the trials were complete, as was the case during the 2003 SARS outbreak? Or what if the fundamentals of the virus were misunderstood and the vaccines were designed incorrectly? By the time such large trials were complete, a lot of money might have been sunk into trials that would ultimately fail. This is where Operation Warp Speed comes in: in my view, it made it possible for a lot of the processes above to function at a lower financial risk, and in a more coordinated way.
Operation Warp Speed and its analogs
Operation Warp Speed was launched in May 2020 by the Trump administration, as a partnership between the Department of Health and Human Services and the Department of Defense. Its goal was to produce and deliver 300 million doses of safe and effective Covid vaccines, with the first doses becoming available by January 2021.
To make it happen, Operation Warp Speed funded the development of a range of candidates and their clinical trials, including the phase 3 trials of Moderna, Janssen, AstraZeneca, and Novavax.5 The CoVPN trial network was developed as part of Operation Warp Speed. Operation Warp Speed also paid for companies to start manufacturing their vaccines at industrial scale while trials were still running, so that millions of doses would be ready to ship once vaccines were authorized.
Importantly, it also committed in advance to buying hundreds of millions of doses of a vaccine that succeeded, meaning that the companies wouldn’t face a massive loss if the outbreak ended up petering out before vaccines were ready. This idea, called an advance market commitment, was originally developed by economists Michael Kremer and Rachel Glennerster to successfully speed up new pneumococcal vaccines in the 2000s, before it was adapted for the pandemic. As then-secretary of Health and Human Services, Alex Azar, described it, the government would invest more than 10 billion dollars to “de-risk vaccine companies’ development efforts and manufacture product in advance.” By the end of the year, it had committed around 13 billion dollars.
As I described earlier, running trials in parallel and manufacturing before trials are complete makes things faster but is financially risky, because a company commits money up front that would be wasted if the candidate vaccine failed. Operation Warp Speed absorbed much of this risk. (One thing that Operation Warp Speed didn’t do was change the regulatory bar: the EUA and rolling review procedures were set by the FDA separately.)
Other countries also used analogs to Operation Warp Speed to accelerate vaccine development and secure doses for their populations. The UK, for example, ran a Vaccine Taskforce, set up in spring 2020 and led by Kate Bingham, which also backed a spread of candidates across different technologies, made advance purchase agreements, and funded manufacturing. The EU ran a Vaccines Strategy, launched in June 2020, which used advance purchase agreements as well: the EU Commission financed part of companies’ upfront costs in return for the right to buy doses. Since it negotiated for all 27 member states, it had a lot of bargaining power, but it started much later; its first major agreement was signed months after the UK and US. And lastly, COVAX created a pooled procurement mechanism for countries around the world, aimed to secure doses for lower-income countries that couldn’t make their own deals (though they fell below their target of later delivering vaccines, delivering half of their target of 2 billion doses by the end of 2021 as high income countries struck bilateral deals quicker).
Operation Warp Speed-type efforts probably work best for large problems that need coordination across many actors, where the goal is well-defined and commercial markets won’t deliver it at the speed required, because the private returns are too small relative to the social value. But it’s also possible to work faster across the board, by creating clinical trial networks and shared volunteer registries to speed up recruitment, innovative trial designs that collect data more efficiently, investment into early-stage research, more regulatory capacity to review evidence as it arrives, and many other ideas we’ve shared on this blog.
Faster trials saved millions of lives
The speed of vaccine development during the pandemic was unprecedented, but it wasn’t unforeseeable. It came from a coordinated effort that involved dedicated research funding, clinical trial networks, parallel trial phases, rolling regulatory review, and parts of the review being deferred rather than skipped. What eased the process were decisions to de-risk the process financially, with upfront funding and advance commitments to buy the vaccines if they worked.
The result was extraordinary. Millions of deaths globally were averted by Covid vaccines in just the first year of vaccination alone.6 We take it for granted today, but it was not a given that trials would be completed within a year. Nor was it a given that the vaccines would have around 95% efficacy against the original strain of the coronavirus, which was much higher than most scientists anticipated: the median forecast was 50%. Speeding up the process saved millions of people.
When it comes to clinical trials, it’s easy to assume the process is so bureaucratic that we couldn’t move quickly even if we wanted to. The pandemic shows that is not the case. When the need was obvious, thousands of people around the world – including scientists, operators, policymakers, regulators, economists and government officials – worked differently to make sure vaccines became available faster.
It also reveals how slow the current system is. If clinical trials can be sped up so much without compromising on safety, we should find ways to reform them outside of pandemics as well. Every year of delays in testing is a year patients go without potentially lifesaving treatment, risking their lives. We shouldn’t be dragging our feet for the sake of following the status quo. Rather than treating pandemic trials as exceptional, we should ask whether that speed should have been the norm all along.
And the virus.
Okay, fine, around 10% is not ‘everywhere all at once’. But it still means it was the disease with the highest incidence rate that year, with the potential exception of rhinovirus.
In case you’re interested, my friend and co-blogger Witold Wieçek co-authored an interesting paper in which he and his colleagues estimated the number of deaths that would have been avoided if vaccination started earlier, including modelling how that might have changed depending on vaccine hesitancy. They estimate that around 50,000 deaths would be avoided in the US if vaccination began 30 days earlier, or 130,000 deaths if it began 90 days earlier, assuming its later trajectory remained similar. With vaccine hesitancy, those figures would be reduced.
You could avoid this to some degree with interim analyses, where candidate vaccines are dropped if they appear to be failing midway through a trial.
Pfizer declined push funding, but still accepted a roughly $2 billion advance purchase commitment.
The range of estimates varies between single-digit millions and twenty million deaths averted by Covid vaccinations, within the first year of vaccines becoming available. This of course refers to premature deaths, since no one is immortal… that we know of. Demographic estimates suggest that Covid led to roughly 10 years of life lost per person who died from the disease, on average.