Internet New Frontier In Collecting Mind Data For Research?

Apple launches new health tracking tools for the iPhone. Medical researchers foray into Facebook to recruit clinical trial volunteers. Is it any wonder that web and mobile apps are increasingly seen as a new source for health data?

But psychologists are also looking to the Internet as a new source of information about the mind.

And an Indiana University researcher is on the forefront of those developing the tools to make it happen. Josh de Leeuw, a graduate student in the IU Department of Psychological and Brain Sciences, is the creator of jsPsych, a free open-source software platform that employs a common Web technology to conduct psychology experiments over the Internet.

The program allows psychologists without significant programming skills to deliver tasks common to research on the mind through a Web browser. Said de Leeuw:

“Conducting psychology research online is appealing for a number of reasons: faster data collection, lower costs and improved anonymity of subjects and experimenters. Internet users are more demographically diverse than the population conventionally sampled for behavioral research in psychology—a fact that could contribute to results that better reflect the population at large.”

This means the current pool of psychology research volunteers—generally, college students—could expand since participants would no longer need to travel to a researcher’s lab to perform tasks common in psychological research.

Addressing the Replication Crisis

Using jsPsych, a psychologist could present anyone with Internet access with on-screen stimuli and then collect responses. Study instructions and consent forms could also be delivered electronically.

Ben Motz, senior lecturer and director of undergraduate instruction in the Department of Psychological and Brain Sciences, who has collaborated with de Leeuw, explains:

“Online experimentation is the bleeding edge of psychology research, and many believe online studies will play an important role in addressing the so-called ‘replication crisis’ in psychology. Josh’s work is at the epicenter of this—and it’s all the more impressive that he’s developed this platform as a graduate student.”

The replication crisis refers to a growing concern in experimental psychology, and the larger scientific community, about the drop in studies able to confirm previous work with experiments that achieve the same results using the same methods, as well as the increased risk of data manipulation in studies with small sample sizes.

Reducing costs and other barriers to psychological research could increase the number of replication studies, as well as the overall number of volunteers and data produced, in turn creating stronger results.

A few psychology researchers are already using online services to conduct research. One of those tools is Amazon Mechanical Turk, a service from the retail giant that enables anyone to hire online workers to perform short, simple tasks.

Data Quality Concerns

Others remain hesitant to employ these methods due to worries about the quality of data collected outside the highly controlled environment of a research lab.

To address this concern, de Leeuw recently published a study in the journal Behavior Research Methods investigating whether one specific issue among brain researchers—the potential delays in response introduced by the use of online technology—represents a significant barrier to conducting research online.

That study looked at differences between Matlab’s Psychophysics Toolbox, one of the most popular software systems for psychological experiments, and jsPsych.

To measure the speed that information is displayed on-screen and its effect on reaction time, de Leeuw asked 30 volunteers to sit in front of a projection screen and hold a thumb-activated switch in each hand.

If volunteers saw the letter “N,” they clicked a button with their dominant hand. If they did not see the letter, they clicked with their other hand. Volunteers repeated this experiment 400 times, with only a 45-second break in the middle.

To prevent weariness from affecting the results, de Leeuw set up the screen to project two images simultaneously—one generated by jsPsych, the other by the traditional system—enabling response times to both systems to be measured simultaneously.

In psychology research, response times may be used to discover which mental operations are more difficult than others, or to measure how uncertain people are about a decision.

They can help distinguish between different theories of perception, memory, decision-making and social cognition. Psychologists refer to the study of the relationships between stimulus and response as psychophysics.

Measuring response times is only one of many applications of jsPsych.

Small Lag Time

The study revealed that jsPsych produced delays of 10 to 40 milliseconds compared to laboratory-based software. Three other statistical parameters measured showed no reliable difference between the systems.

“This is an extremely small lag time—it’s roughly the same as might be caused by other variations in experimental setup, such as a change in keyboard, mouse or monitor—and it is constant between different experimental conditions,” de Leeuw said. “In the context of a real human experiment, both systems are nearly identical.”

He added that results are also applicable to other software packages that use JavaScript, the only programming language supported by all major Web browsers that does not require additional software to run:

“Hopefully this study will convince more people that you can, in fact, conduct psychophysics in a Web browser. We’re looking at technology with potential to transform how research on the brain gets done.”

A tutorial on jsPsych is available here.

Reference:

Joshua R. de Leeuw, Benjamin A. Motz
Psychophysics in a Web browser? Comparing response times collected with JavaScript and Psychophysics Toolbox in a visual search task
Behavior Research Methods DOI 10.3758/s13428-015-0567-2

Illustration Credit: Josh de Leeuw and Benjamin Motz