Project Zero Prize Conclusion

Posted by Natalie Silvanovich, Project Zero

On September 13, 2016 we announced the Project Zero Prize. It concluded last week with no prizes awarded. The purpose of this post is to discuss what happened and what we learned about hacking contest design.

Throughout the contest, we did not receive any valid entries or bugs (everything we received was either spam, or did not remotely resemble a contest entry as described in the rules). We did hear from some teams and individuals who said they were working on the contest, but they did not submit any bugs or entries. Based on our discussions with them, as well as our general observations during the contest, we suspect that the following factors led to the lack of entries.

Entry Point Difficulty

It is rare for fully remote Android bugs to be reported, and it is likely that this was a sticking point for participants. The majority of Android bug chains begin with some user interaction, especially clicking a link, which was not allowed in this contest. While this type of bug is not unheard of, it is likely difficult to find quality bugs in this area. This means that the timeframe of the contest or prize amount may not have been adequate to elicit this type of bug.

Competing Contests

The Project Zero Prize rules were intended to encourage participants to file partial bug chains in the Android bug tracker during the contest, even if a full chain was not complete. In designing these rules, we underestimated the impact of other contests on participants’ incentives. The contest rules allowed for bugs that had already been filed to be used by the first filer at any point during the contest, and receive Android Security Rewards if they were not used as a part of a chain.  We expected these rules to encourage participants to file any bugs they found immediately, as only the first finder could use a specific bug, and multiple reports of the same Android bug are fairly common. Instead, some participants chose to save their bugs for other contests that had lower prize amounts but allowed user interaction, and accept the risk that someone else might report them in the meantime.

Prize Amount

It’s difficult to determine the right prize amount for this type of contest, and the fact that we did not receive any entries suggests that the prize amount might have been too low considering the type of bugs required to win this contest.

Overall, this contest was a learning experience, and we hope to put what we’ve learned to use in Google’s rewards programs and future contests. Stay tuned!

Also, if there were any aspects of the Project Zero Prize that affected your participation that we could improve, we would like to hear from you, either in the comments, or at project-zero-prize@google.com.