When Dr. Farzad Mostashari was the assistant commissioner for the New York City Department of Health in the early 2000s, he did something unprecedented.

To keep tabs on the spread of disease in the region, Mostashari asked New York hospitals for access to a feed of their data, including the symptoms reported by some of the sickest patients. His team put together a website that collected anonymized information from emergency rooms across the state, and made it open for anyone to query.

Nearly two decades later, on March 11, 2020, his work suddenly gained new relevance. The World Health Organization declared the novel coronavirus a global pandemic, and predicted that the SARS-CoV-2 virus could kill more than a million people worldwide.

Mostashari, who left his government role in 2013 to work in the health-tech industry, was concerned about the lack of information flowing out of the U.S. Centers for Disease Control and Prevention. Its departments at all levels of government have seen waves of staffing cuts over the past decade. State and local preparedness projects at the CDC received $940 million in 2002, but funding levels decreased by 31% in the subsequent fifteen years, one study found.

But Mostashari’s website was still around, pulling in a feed of data from emergency rooms every day. So in early March, he started looking for incidents of patients complaining of flu-like symptoms that were outside the normal range for early spring. On March 4, he saw a spike in the data from New York that concerned him. For the next three days, he checked and rechecked the website to make sure it wasn’t a blip. By the fourth day, he knew something was wrong.

“Holy s---,” he recalled thinking. “Flu was going down, but patients were starting to come into the emergency department with a ferocity I hadn’t seen in 15 or 20 years.”

Mostashari sounded the alarm on Twitter on March 7, sharing several screenshots of the data, and stating “we need to be working URGENTLY on expanding and protecting healthcare capacity” in New York.

Official reports said there were only 100 confirmed cases of COVID-19 in New York hospitals at that time, which seemed less concerning than in other early hotspots like Washington state. But Mostashari implored his followers to take the data he was seeing seriously. He said that with the detectable increase in symptomatic emergency room visits, New York’s health systems should reasonably expect a “20x that burden of illness” in the next month if drastic steps weren’t taken to stem the virus.

Those steps came, but not quickly enough. It would be another two weeks — March 20 — before Gov. Andrew Cuomo ordered New Yorkers to stay home and for all non-essential businesses to close. By then, the disease had spread to tens of thousands in the New York area and had started to overwhelm emergency rooms. Mostashari couldn’t wrap his head around why state officials seemed to be reacting so slowly. The website he developed for New York was just one of several systems that researchers at a state and federal level had built in the past few decades to predict disease outbreaks.

“The data is sitting there,” he said. “I’ve been pulling my hair out thinking this needs to be investigated.”

It started with anthrax

The idea of developing an early warning system for public health dates back to the late 1990s, when the federal government was exploring ways to combat bioterrorism.

Back then, Dr. Kenneth Mandl was a junior faculty member at Boston Children’s Hospital treating sick kids coming into the emergency department. Mandl had background in bioinformatics, which refers to the collection and analysis of biological data, as well as a medical degree. Because of that background, a colleague invited him along to a meeting with the Defense Advanced Research Projects Agency, or DARPA, which oversees emerging technologies for the military.

The meeting, which took place in a lounge at the Massachusetts Institute for Technology, quickly took a dark turn. Representatives from DARPA shared their concerns about a terrorist unleashing deadly bacteria on the New York subway. They described to Mandl how they had created some computer-based simulations to analyze the potential impact, including the number of deaths. “They were like the men in black,” he recalled.

Through the models, DARPA determined it could reduce the rate of people getting sick if it had a real-time system to monitor hospitals for flu-like illness. After a person is exposed to anthrax, symptoms often include a cough, fever, body aches and fatigue. DARPA was afraid that doctors would chalk it up to a seasonal flu, and not order any further tests.

Mandl and his colleagues started working on a system to perform the kind of surveillance that DARPA had outlined. Preventing bioterrorism was the major focus, but from the early days they had a hunch that the methodology could also be used to track seasonal flu, chronic disease, suicidal ideation and drug addiction, and even pandemics.

In 2004, Mandl and a small team of researchers produced a seminal paper on “syndromic surveillance.” It outlined how to use bioinformatics to detect a surge in flu-like illness at hospitals, which could point to anthrax or other deadly infectious diseases.

“We knew that every patient going into the hospital would be asked about their chief complaint,” said Mandl, who is now the director of the computational health informatics program at Boston Children’s Hospital. “Through our research, we realized that that alone was good enough for us to run entire surveillance systems off of.”

It was a heady time for the researchers. In the years after the 9/11 terror attacks, funding for syndromic surveillance increased dramatically. No one could forget the antrhrax-laced letters sent to media companies and congressional offices following the attacks.

“We got a big bolus of funding after the anthrax scare,” said Janet Hamilton, executive director of the Council of State and Territorial Epidemiologists (CSTE). “When it comes to funding public health, money typically pours in after a crisis, and then it stops, and then starts again. It’s been a piecemeal approach.”

As the idea of syndromic surveillance gained credibility, Mandl and his colleagues were invited to large sporting events and other gatherings to help them protect against bioterror attacks. In 2003, the Greek government asked researchers from Mandl’s lab to fly over for the Summer Olympics scheduled for the following year.

Ben Reis, a colleague of Mandl’s, went to Athens ahead of the games to get a sense of the “baseline,” meaning what a typical Tuesday afternoon in August looked like at an emergency room. He knew that the overall number of emergency room visits would be higher during the Olympics, so looking at the total number of people with a flu-like illness wasn’t enough. The more meaningful metric would be a spike in the proportion of total hospital visits attributed to flu-like symptoms.

“We learned that the ratios were more robust than the total number,” he said.

The following year, a team of local graduate students from the epidemiology department biked around to all the hospitals in Athens to get a daily record on an Excel file, sometimes on a floppy disc, and shared it with the Hellenic Centre for Disease Control and Prevention. In the end, their efforts weren’t needed -- the Olympics went down without a hitch, and with 9/11 fresh on everybody’s mind, many locals left town and tourists ended up staying home.

But the research group began to realize their research could help public health departments respond to pandemics, another pressing concern at the time.

In the aftermath of the SARS outbreak in 2005, Reis was invited to Hong Kong, where almost 300 people had died. Hong Kong had been a pioneer in rapidly scaling “contact tracing,” where officials tracked people down who’d been in contact with an infected person, then actively quarantined them. It took a lot of resources, but it worked to stem the spread of the virus.

When Reis arrived, Hong Kong were still reeling from the recent outbreak, and its public health officials were willing to fund the system they wished they’d had during SARS.

Reis argued that the methods they had used to monitor large events like the Olympics were applicable to pandemics. He described the concept of “an epidemiological network,” and suggested it wasn’t enough to estimate the total number of possible cases. Instead, governments had to look at the whole picture.

“There’s something called homeostasis, which is a fancy medical term that essentially refers to the balance of different vital processes in your body,” Reis explained. “We used that analogy to say that the public health system also has a normal balance between hospitals and disease categories that can be tracked. It’s not a matter of who’s in town, or if one hospital is overloaded at a single point in time, but it’s about understanding whether the relationship between those categories was out of whack.”

A public health upgrade

Back in the U.S. under President George W. Bush, the funding continued to flow to syndromic surveillance efforts.

In 2003, staff at the CDC built out a federal system called Biosense to monitor emergency departments across the country. One of the early areas of focus involved tracking the seasonal flu, but eventually the database was expanded to survey a broader set of public health concerns.

Other key projects that furthered the field came out of the International Society of Syndromic Surveillance, which was formed in 2005. The group, which counted Mandl and Mostashari among its members, hosted regular conventions to discuss ideas and share research. One of the better-known initiatives was Distribute, which involved asking states to share data on flu-like illness and other syndromes on a daily basis publicly for anyone to query.

Some members of the group jokingly referred to themselves as “the disease hunters.”

Not everyone was comfortable with the federal government’s oversight of these programs. Some hospital executives did not want to share information so directly with the federal government and bypass the states, and in the end fewer than 10% agreed to contribute emergency department data to the Biosense program. Epidemiologists at the CDC had to rely on data from government sources, including the Department of Veterans Affairs and the U.S. Department of Defense’s hospitals. Officials at the agency acknowledged these shortcomings by 2007, noting in an interview with “The Scientist” that the program lacked real-time capability and was built more for a bioterror attack than for public health.

Then, in 2009, Biosense got a big boost. Taha Kass-Hout, an energetic cardiologist and data scientist, joined the CDC as a director of health informatics.

Kass-Hout’s big idea, according to former colleagues, was to take a bottoms-up approach to the problem by working with local and state health departments, which could better react to the data. He also looked for federal incentives to spur hospitals to share data without requiring them to build anything new. As he frequently described it, he wanted to help create a “catcher’s mitt” of all sorts of relevant data that could flow up to the CDC.

Within a few years, more than 70% of hospitals agreed to share data with the system, which was called Biosense 2.0. That system is now called the National Syndromic Surveillance Program (NSSP), and was one of the first government projects to be hosted on Amazon’s cloud computing service, AWS, where Kass-Hout now works.

“A lot of health systems weren’t reporting,” recalled Aneesh Chopra, the first chief technology of the White House, appointed in 2009 by President Barack Obama. “But that changed when we got this champion called Taha Kass-Hout, alongside a young whippersnapper over in New York City called Farzad Mostashari, who with a band of brothers and sisters set up a network on a local and state level and they got lots of hospitals to participate.”

NSSP still runs off of Amazon Web Services, and the company hired Kass-Hout in 2018 as its chief medical officer. An Amazon spokesperson did not make Kass-Hout available for an interview.

Kass-Hout has spoken publicly over the years about the potential to use data from non-traditional sources for tracking disease. The Biosense 2.0 system tracked news reports, absenteeism, weather reports, and social media. It also relied heavily on the symptoms that patients reported themselves, and not just their doctors’ interpretations or the test results. Kass-Hout co-authored research papers showing the value in asking patients whether they had a fever, versus checking it with a thermometer -- if they’d noticed a fever earlier in the day, they may simply have popped an aspirin, lowering their temperature before they arrived at the doctor’s office.

“In those days, we knew the next major pandemic would look like an atypical flu,” said Arien Malec, who worked in the White House at the Department of Health and Human Services at the same time as Kass-Hout. “Taha, when he was at the CDC, put in place a true bio-surveillance network that took all kinds of feeds and monitored them centrally. It was ‘big data’ before it was cool.”

The idea of extending the surveillance beyond health data to other sources was gaining ground outside of the CDC as well.

In the mid-2000s, John Brownstein, an epidemiologist at Boston Children’s Hospital, started to reach out to Silicon Valley with some ideas about how to leverage data from Google search queries, social media feeds and step counts via some of the earliest wearable trackers. In 2008, Google agreed to supply estimates for influenza for 25 countries with public health departments through a project called Google Flu Trends.

“I got tired of relying solely on health data,” he said. “I saw that with Google and Twitter, I could get to global data immediately and in real-time, and I saw the potential to use it as part of a broader biosurveillance system.”

When the H1N1 swine flu pandemic came in 2009, the epidemiologists and data scientists advocating for syndromic surveillance had their moment in the sun.

Public health officials leaned heavily on the Biosense program to help them assess the extent of the illness, learn where there were gaps in testing, and guide the experts at the federal level in making decisions about immunization recommendations, school and building closures, and other steps.

The agency routinely shared updates with the public about the regions that were hardest hit by the virus. The data went far deeper than simply reporting the number of confirmed cases of the disease, because the CDC knew that the true numbers were difficult to gauge when there are lags in testing.

“At that point, national syndromic surveillance really took off,” recalled Chopra, the former U.S. chief technology officer.

Government disinterest and calls to the private sector

As of Friday, more than 1 million people have been infected and more than 57,000 have died from COVID-19. The pandemic has moved well beyond the containment phase, and there are reports of community-spread throughout the United States and in many other countries.

It’s a perfect situation for syndromic surveillance.

Instead of reacting to outbreaks like a whack-a-mole game, syndromic surveillance could help officials send resources where they’re needed most -- before hospitals in hot spots like New York and New Orleans get overwhelmed. As shelter-in-place orders are relaxed in the coming months, public health experts say it could be used to inform local officials about potential outbreaks so they could send people home even if the rest of the county or state is back to work.

“The beauty of syndromic surveillance is that it can be used when there are limited lab reports,” said Isaac Bogoch, an associate professor of Infectious Diseases at the University of Toronto. “And right now, there’s a shortage of lab tests in almost every country.”

“It’s one of the best tools we have,” added Hamilton, the epidemiologist running CSTE. “It can also be helpful in tracking the severity of cases, and assess the proportion of patients that require more critical care and attention.”

But some of the key organizations and projects have lost funding in the past decade, as the memory of 9/11 and SARS grew dimmer.

The International Society for Syndromic Surveillance shut its doors in June of 2019, six months before the coronavirus started spreading like wildfire in Wuhan, China. Its last executive director, Shandy Dearth, who has spent her career in infectious disease surveillance, said the society struggled because of cutbacks to the CDC.

“A lot of the money that sustained us came in after 9/11 with all the emergency preparedness funding, but it just kept dwindling every year,” she said. “Public health gets an influx of interest right after something bad has happened, but we don’t put enough emphasis on prevention.”

The Biosense system, now NSSP, still exists and continues to pull in information every day. But even now, only about 70% of hospitals share data, and many in the public health say it could use an upgrade.

Some experts, frustrated with the lack of coordinated public support, are turning to the private sector — and the tech industry in particular — to help.

A philanthropic group with ties to the tech industry called Resolve to Save Lives views symptomatic surveillance as a key piece to get Americans back to regular life.

“If you look at some of the first known cases of this, like the patient at UC Davis, it took up to a week for them to get tested,” noted Cyrus Shahpar, a medical epidemiologist with Resolve to Save Lives and a former epidemic intelligence service officer with the CDC. “So we need to learn from the early signals.”

The group, run by former CDC chief Tom Friedan, has raised more than $225 million from tech-connected organizations like the Bill and Melinda Gates Foundation, Bloomberg Philanthropies, and the Chan Zuckerberg Initiative.

Amazon Web Services is currently working with the CDC to help the NSSP system scale and meet the increase in demand, according to one person familiar with the initiative, who was not authorized to speak to press about internal matters. AWS is providing cloud credits, they said, so it’s essentially being done at no cost.

Silicon Valley’s technology companies can also help out with surveillance efforts, as long as privacy standards can be maintained. Google, for instance, is now helping public health officials to track whether social distancing mandates are being followed, including at parks and at office buildings. Google has stressed that it would not share data about any individual’s movements, and the information is only available at an aggregate level.

Smaller tech players are getting involved as well. For instance, a start-up called Kinsa, which makes smart thermometers, is hoping to make its temperature data available to help the CDC detect coronavirus hot spots.

But the need for help from the private sector highlights the lack of funding for public health, experts say.

“Overall, our public infrastructure is fragmented and underfunded, both on a national and state level. Increases in funding for both emergency preparedness and syndromic surveillance — for acute pandemics like COVID19 as well as for ongoing epidemics like suicide — are desperately needed,” said Megan Ranney, an emergency physician and public health researcher at Brown University.

“The current system is broken,” she added. “COVID-19 lays bare the long-term poor planning for emergency care.”

Some of the longtime former members of the society say they regret their former band of disease-hunters had to disband.

“These concepts have been around for 20 years and were built for this very purpose,” said Brownstein, the epidemiologist in Boston. “It’s a big disappointment that it went belly-up because of a lack of funding right before a global pandemic.”