Chrome Has a Better Bug Catcher Than Mozilla

By | August 23, 2013

Posted in: Network Security Trends

A study comparing two leading vulnerability rewards programs (VRPs) has found that the one offered by Google Chrome is working better than that run by Mozilla.

The study, by Matthew Finifter, Devdatta Akhawe and David Wagner of the University of California, Berkeley, was presented this month at the USENIX Security Symposium in Washington, D.C.

The researchers said the Chrome VRP had cost about $580,000 over three years and had resulted in 501 bounties, while the Firefox VRP had cost approximately $570,000 in the same period and had yielded 190 bounties.

Their paper said that  28% of Chrome’s patched vulnerabilities appearing in security advisories over this period, and 24% of Firefox’s, were the result of VRP contributions.

“Both programs appear economically efficient, comparing favorably to the cost of hiring full-time security researchers. The Chrome VRP features low expected payouts accompanied by high potential payouts, while the Firefox VRP features fixed payouts.”

The researchers noted that  despite the fact that the two programs cost approximately the same,  “the Chrome VRP has identified more than three times as many bugs, is more popular and shows similar participation from repeat and first-time participants.”

They said there was “a stark difference” between the levels of external participation in the two VRPs. “Despite having the oldest bounty program, external contributions lag far behind internal contributions to Firefox’s security advisories. In contrast, external contributions to Chrome’s security advisories closely rival internal contributions.”

The paper said there were three key differences between the two programs:

Mozilla’s had a fixed payout of $3,000, which was about the same as the normal maximum payout for Chrome. “Nonetheless, Chrome’s tiered structure, with even higher payouts (e.g., $10,000) possible for clever bugs and special cases appears to be far more effective in encouraging participation. This makes sense with an understanding of incentives in lotteries: the larger the potential prize amount, the more willing participants are to accept a lower expected return, which, for VRPs, means the program can expect more participants.”

There was a far higher variance in the time-to-release-patch metric for critical vulnerabilities in Mozilla Firefox. “It is generally accepted that the viability of responsible disclosure depends on a reasonable vendor response time. Thus, the high variance in Mozilla’s response time could affect responsible disclosure through the VRP.”

And, the researchers said, Chrome’s VRP had a higher profile, with annual competitions like Pwnium providing particularly high rewards of up to $150,000. “Chrome authors also provide extra reward top-ups for ‘interesting’ bugs. We believe this sort of ‘gamification’ leads to a higher profile for the Chrome VRP, which may help encourage participation, particularly from researchers interested in wider recognition.”

The researchers said they recommended that Mozilla should change their reward structure to a tiered system like that of Chrome, and they urged Mozilla to reduce the variance in time to release a patch for critical vulnerabilities. “Mozilla can also consider holding its own annual competitions or otherwise increasing the PR surrounding its VRP,” they said.

You May Also Be Interested In: