The Truth About the 2020 Sturgis Motorcycle Rally and COVID-19

Dec 15, 2021

Tags:industry-newsnewsothertravelrallies

Harley-Davidson motorcycles ridden toward Sturgis, South Dakota.

According to media outlets, the Center for Health Economics and Policy Studies at San Diego State University published a study stating that the 2020 Sturgis Motorcycle Rally caused 250,000 cases and over $12 billion in economic damage. This is the same organization that concluded Black Lives Matter protests did not contribute to the spread.

According to some, this study is biased for many reasons like not using control groups, and not being peer-reviewed. The lead South Dakota epidemiologist called this study "fiction".

Riders Share advocates for freedom and safety. Because our team includes data scientists, we took a closer look at the report (and the claims against it) in order to provide the most unbiased view.

For simplicity, we will focus on three claims from The Reason vs. other media outlets:

  • 260+ South Dakota Cases (True). The paper claims the event caused a 60%+ increase in cases, whereas The Reason argues there was no control group. However, both forgot to look at an even more important metric: deaths.
  • 250,000+ Non-South Dakota Cases (Inconclusive). The paper claims 19% of all new COVID cases between August 2nd and September 2nd were caused by the Sturgis rally. They estimate over 40 million cases prior to Sturgis. and 1.4 million post-Sturgis, which is about 7x confirmed cases. The Reasons says - no control group. While very well done, the paper also estimated social distancing mandates don't work. Clearly, this paper needs a lot of refining, or to revise its conclusions. It goes against prior research and likely is missing something.
  • The 12 Billion Cost (Blatantly False). The paper claims the rally cost $12 billion, The Reasons questions the researcher's assumption of a $46,000 cost per case. This one is easy. In reality, the hospital bills were likely $200 - $250 million.

South Dakota Cases

Verdict: True - But Where Are The Deaths?

This segment of the report was well done. The researchers used the latest techniques in statistical modeling. There was some sort of control group: counties in South Dakota that did not host the motorcycle rally. Using these counties, researchers can sort of control for the effects of school openings and other confounders that contribute to COVID spreading.

The researchers also accounted for increases in testing. Testing reveals more asymptomatic cases. Many COVID papers do not account for this variable, which drastically distorts results via selection bias. It is very evident from the paper, and also from a cursory glance at the evidence, that the rally had an impact in cases:

Where we are confused is why deaths weren’t populated, because the paper did account for increases in testing:

We don't know the answer, but it could be age-related. Per the South Dakota government, most cases were young people. As most of us know, young people are dramatically less susceptible to the virus, and the risk increases exponentially with age.

So, yes. The Sturgis Rally led to 260 or so additional cases in South Dakota. The paper is correct that there was a 60% - 80% increase in cases, but from a very low base. Media outlets failed to look at hospitalizations and deaths, which are more important. So far, there is no evidence of increased deaths and the South Dakota website does not publish trends on hospitalizations.

Non-South Dakota Cases over 250,000

Verdict: Study concludes Sturgis caused increases, but it also concluded social distancing laws don't work. It therefore needs refining.

The researchers used differences-in-differences to account for changes in other counties during the time period - in short, as close to a control group in real life as you can have. In layman terms, this means they looked at trends before and after the rally in counties with and without attendants, and compared the differences.

They tried two methods: number of travelers relative to the population of the city of origin, and number of travelers total before and after. The first one makes more sense and had more statistical significance. They segmented counties by five levels of attendant inflow and five time periods.

They found counties with few Sturgis attendants saw a decline in cases, whereas counties with many attendants had increases, with statistical significance 24 days from the rally (although not super strong for high inflow counties):

They also divided the study between counties with strict and weak social distancing measures. While the paper’s conclusion is that strict measures helped deter spread, the table shows in all counties except those with High Inflow, strict social distancing measures had no significant impact on case increases. What gives?

In short: if the same study showing Sturgis had a massive impact on spread also showed social distancing policies don't work... well, is the study sound? Is there an unknown confounder? There are many things going on at these counties like dates schools opened (if at all) that are different, and a follow on study using the same method for the impact of this policy would help elucidate the truth. It does seem to indicate the results are likely overstated, but definitely very real and large.

Another way to present conclusions are that a gathering of possibly 480,000 people resulted in 1,700 deaths, and social distancing mandates don't work. The latter goes against existing literature and against its own conclusions, so rather, this paper needs refining. Or maybe social distancing measures don't work, or aren't being followed.

The 12 Billion Cost

Verdict: Blatantly False

The authors quote one study to estimate the costs of a COVID case at $46,000. It is impossible for somebody without symptoms or mild symptoms to cost $46,000 to society, and most COVID cases are mild or asymptomatic.

Let's say a hospitalization costs $45,000. Per CDC, there are two hospitalizations for every COVID death. That's $230 million in hospital bills.

The authors quote Kniesner and Sullivan (2020). It is purely arbitrary; the authors made no effort to justify why a person with no symptoms has a $11,000 cost. More importantly, do you really want to put a dollar value to life ? If you do, don't call it "economic" impact. That's not what most people think of when they read economics. This one is on journalists for choosing sensationalist headlines over digesting science on behalf of readers.

This is how this study came up with the number:

"We first use the Department of Transportation (2016) guidance on value per statistical life (VSL) and severity/injury estimates as a basis for our non-fatal valuations by category. After updating the figures for earnings and inflation the DOT guidance recommends using a VSL of about $11 million in 2019 dollars. We use the severity classifications in the DOT guidance as a basis for our non-fatal valuations. DOT (2016) recommends using six different severity categories in benefit-cost analyses including Level 1 (minor), which corresponds to using a 0.3 percent amount of the VSL, Level 2 (moderate), which uses about a 5 percent amount, Level 3 (serious), which uses about a 10 percent amount, Level 4 (severe), which uses about a 27 percent amount fraction, Level 5 (critical), which uses about a 59 percent amount, and Level 6 (unsurvivable) which uses a 100 percent amount (the full VSL). We therefore value asymptomatic cases at about $11,000 (in 2019 dollars) each which corresponds to using a 0.1 percent amount of the VSL in DOT (2016).3 Symptomatic cases with no hospitalizations are assumed to line up in the minor category (about $33,000 each)."

Scientists: if you really want to go down the dreaded path of calculating a value for human life, the value should rely upon the cost to save a life. In this way, we can optimize limited resources to save the most lives. For example: if funding universal healthcare can save 350,000 lives at the cost of $5 trillion over five years, then it should be prioritized over a $5 trillion quarantine that may or may not save 250,000 lives. Masks are the next best policy.

We shouldn't be putting dollar values on lives, though. That's not how it works.

Conclusions

One could likely accuse this analysis of being right or left wing after reading this article. Note that it advocates for universal healthcare (left) while advocating against quarantines (right). It is impossible to be unbiased, but we really try:

Some journalists at prestigious institutions are too lazy to read academic papers. The paper clearly shows an increase in cases in South Dakota, but a few hundreds cases is peanuts given the huge numbers in attendance. The death figures for South Dakota have not meaningfully increased as of 9/10/2020. As of 9/10, the Sturgis rally had a net positive impact on the state's economy, without question.

The paper clearly shows an association between Sturgis attendance and increases in cases, and it is possible (likely) the event resulted in 250,000 additional cases; but the paper also shows social distancing measures did not work, so more refinement is needed - it could be the counties where these people come from share other characteristics that correlated with an increase in spread.

The paper is wrong about costs. This event did not have a -$12 billion impact on GDP, which is how most people understand the headlines. 

In short, another headline for this paper could be: 480,000-person rally causes only 260 cases in South Dakota and strict social distancing measures weren't being followed elsewhere either, so you are just as bad as bikers.

Would you like us to crunch data on more motorcycle related news for you? Email support@riders-share.com for help.