But the field is new, and the team's findings suggest that standardized guidelines for health care crowdsourcing ventures are needed so that data can be collected, reported, and replicated most efficiently.

"While the concept of 'citizen science' has been in existence for more than a century and crowdsourcing has been used in science for at least a decade, it has been utilized primarily by non-medical fields and little is known about its potential in health research," said the study's senior author Raina Merchant, MD, an assistant professor of Emergency Medicine at Penn.

Merchant and colleagues successfully utilized crowdsourcing in a recent study to locate and catalog the locations of lifesaving automated external defibrillators (AEDs) throughout Philadelphia in the MyHeartMap Challenge. Their study led to the identification of more than 1,400 AEDs in public places, and they hope to replicate the study in other major cities across the U.S.

For the current review, in addition to a traditional database search, her team employed crowdsourcing again to perform a literature search for health and medical research articles using two free websites: Yahoo! Answers and Quora. Through this approach, they were able to collect and analyze 21 health-related studies that include crowdsourcing techniques. The studies collectively engaged a crowd of over 136,000 people, ranging in focus from tracking H1N1 influenza outbreaks in near real time to classifying different types of polyps in the colon.

"There is understandably some apprehension about letting the lay public in on medical research or even assisting with making medical diagnoses because the stakes are so high in medicine. However, studies we reviewed showed that the crowd can be very successful, such as solving novel complex protein structure problems or identifying malaria infected red blood cells with a similar accuracy as a medical professional," said the study's first author Benjamin Ranard, a third year medical student in the Perelman School of Medicine.

The research team found that the studies centered around four main categories of tasks: problem solving, data processing, surveillance/monitoring and surveying.

However, they found considerable variability in the amount and type of data reported about the crowd and the experimental set up, which would make it difficult for other researchers to replicate or model their work for their own research. For instance, the articles rarely reported data about the demographics of the crowd participating, including information standard to most clinical trials such as the size of the cohort, age, gender, and geographic location. They also noted that the limited amount of studies they found is surprising given the potential benefits of this approach.

The authors recommend that other health and medical investigators should look at their own research projects and consider involving the public through crowdsourcing. Whenever research requires human processing that computers alone cannot do, such as visually sorting pictures or other data, they say there is a potential to involve the crowd. Crowdsourcing can also be used to take advantage of problem solving skills members of the public may have (such as solving three-dimensional puzzles), or to employ the crowd to act as human sensors reporting data about the environment (for example, reporting cases of influenza-like symptoms).

"Every health field from studying chronic diseases to global health has a potential need for human computing power that crowdsourcing could fill to accelerate research. Prior work has heralded crowdsourcing as a feasible method for data collection, but a clear roadmap for the types of questions crowdsourcing could answer and the ways it could be applied has been lacking," said Merchant. "This review points to the need for streamlining the process and implementing more rigorous guidelines for this approach."

They call for continued study of the scope of crowdsourcing to determine where it might be as useful as traditional data. To further explore the power of crowdsourcing and other research approaches via social media, Merchant was recently appointed director of the Social Media Lab at the Penn Medicine Center for Health Care Innovation. In this role, she will lead a program exploring ways in which new communication channels can enhance Penn's ability to understand and improve the health and health care of patients and other populations.

Other Penn Medicine authors include Yoonhee P. Ha, MSc , Zachary F. Meisel, MD, MPH, MS, David A. Asch, MD, MBA and, Lance B. Becker, MD.

The study was funded in part by the National Institutes of Health (K23 grant 10714038).