Study Finds “Surprises” in Browser Warning Effectiveness

By | July 19, 2013

Posted in: Network Security Trends

Computer users generally take note of browser warnings about unsafe websites, but up to 70 percent of Google Chrome’s SSL (secure socket layer) warnings fail to dissuade users from visiting a site, according to a recent study.

The research on Chrome and Mozilla Firefox was conducted in May and June this year by Devdatta Akhawe of the University of California, Berkeley – a former Mozilla intern – and Adrienne Porter Felt, a research scientist at Google.

They based their findings on telemetry data on 25.4 million warning impressions, and said their study tended to contradict the popular opinion that security warnings are usually ignored.

“We find that browser security warnings can be successful: users clicked through (i.e., by-passed) fewer than a quarter of both browsers’ malware and phishing warnings, and a third of Mozilla Firefox’s SSL warnings,” their study said.

But they added: “We also find click through rates as high as 70.2% for Google Chrome SSL warnings, indicating that the user experience of a warning can have a tremendous impact on user behavior.  Such a high click through rate is undesirable: either users are not heeding valid warnings, or the browser is annoying users with invalid warnings and possibly causing warning fatigue.”

The researchers said more work was needed on making browser security warnings more effective. “At Google, we have begun experimenting with new warning designs to further improve our warnings.”

The researchers found wide variations in how users reacted to warnings on Chrome and Firefox.

“We found click through rates of 18% and 23.2% for Google Chrome’s phishing and malware warnings, respectively, and 31.6% for Firefox’s SSL warning. These warnings prevent 70% (or more) of attempted visits to potentially dangerous websites. Although these warnings could be improved, we consider these warnings successful at persuading and protecting users.”

The researchers suggested that higher technical skill, as indicated by use of Linux and prerelease channels, may predispose users to click through some types of warnings. “Technically advanced users might feel more confident in the security of their computers, be more curious about locked websites, or feel patronized by warnings.”

They said they were surprised that the amount of effort, i.e., the number of clicks, required to bypass a warning did not always have a major impact on user behavior. “To bypass Google Chrome’s malware and phishing warnings, the user must click twice: once on a small ‘advanced’ link, and then again to ‘proceed.’ Despite the hidden button, users click through Google Chrome’s malware/phishing warning at a higher rate than Mozilla Firefox’s simpler warning. Furthermore, 84% of users who open Mozilla Firefox’s ‘add exception’ dialog proceed through it.”

The researchers said the engineers who designed Chrome’s warning had introduced the extra step in the malware/phishing warning because they expected it to serve as a strong deterrent. “One possible explanation is that users make a single cognitive decision when faced with a warning,” the study said. “The decision might be based on the URL, warning appearance, or warning message. Once the user has decided to proceed, additional clicks or information are unlikely to change his or her decision.”

Symantec threat researcher Candid Wueest commented on the paper in a blog, saying that ignoring malware warnings can be foolish. “Symantec’s Internet Security Threat Report (ISTR) showed that 61% of the infected websites were hijacked legitimate websites,” he said. “Therefore, knowing the site does not prove that it is clean, even if you visited it before. It may have been compromised since your last visit and is now serving up malware through exploits.”

You May Also Be Interested In: