AWS launches BugBust contest: Help fix a $100m problem for a $12 tshirt

June 25, 2021 TH Author

AWS has set up a competition for its customers’ developers to find and fix one million bugs.

AWS CTO Werner Vogels on Friday introduced BugBust, which he described as “the world’s largest bug bashing challenge.”

“Eliminate software errors and save millions of dollars using Amazon CodeGuru, and win prizes and glory in the first annual AWS BugBust Challenge,” he declared. “Let the bug busting begin.”

But it’s always worth reading the fine print. Under the rules of the contest [PDF], AWS customers appoint a BugBust administrator who creates a BugBust event with a customer-controlled Python or Java code repository (GitHub, GitHub Enterprise, Bitbucket, and AWS CodeCommit repos only). The bugs flagged by the automated detection tool are then made available to participating developers, or players, to fix.

After players commit their fixes and their code gets merged into the repo, the admin uses Amazon CodeGuru Reviewer and Amazon CodeGuru Profiler (at no cost – initially) to re-analyze the code. If the flagged bugs are no longer present, the player who made the repairs gets a certain number of points based on the significance of the fix.

A code change to implement Java or Python best practices would be worth one point while an input validation bug would be worth three points, and the resolution of a security or concurrency bug would be worth five points. Resulting CPU usage improvements – a cost savings to AWS customers – bring additional point awards.

AWS maintains a global leaderboard that aggregates the scores of BugBust contests across participating organizations and will offer prizes to participants as follows:

100 points wins an AWS BugBust T-shirt (retail value $12, 4,000 winners)
2,000 points wins an AWS BugBust hoodie sweatshirt (retail value $27, 2,000 winners)
10,000 points wins an AWS BugBust trophy (retail value $60, 200 winners)
The top 10 of the AWS BugBust global leaderboard as of September 30th win an expenses paid trip to AWS re:Invent 2021 (retail value $4,000, 30 winners)

Incidentally, in its last financial quarter AWS banked $13.5bn in revenues, while Amazon as a whole reported profits of around $90m a day.

The goal of BugBust, according to Vogels, is to fix one million bugs representing an estimated $100m in technical debt for the companies running said code.

“It is dedicated to improving the code quality using the power of machine learning,” explained Vogels via video, “with a mission to bust one million bugs worldwide, all while helping AWS customers reduce costs.”

How much?

Developers discussing the contest have taken issue with the disparity between the theoretical cost savings to AWS clients and the rewards given to programmers for their work. AWS calculates that one point represents $769 of corporate value, based on bug fixing cost analysis from a 2015 research paper [PDF].

So those saving their companies $76,900 through their efforts – 100 points – get a $12 T-shirt for their labor, a barely noticeable pat on the back for those already paid for bug fixing and not a fair exchange for developers who might be tempted to take part in the BugBust contest on their own time.

Participants may even end up paying AWS for running its tools if their BugBust runs beyond 30 days or if they’re already using CodeGuru. AWS offers a 30-day free trial period for CodeGuru Reviewer and CodeGuru Profiler. After that, “you will be charged for CodeGuru Reviewer and CodeGuru Profiler according to standard usage rates,” according to AWS.

“It’s a testament to the bountification of how people treat any kind of software bug hunting and fixing,” said Katie Moussouris, CEO of Luta Security and the creator of Microsoft’s bug bounty program, in a phone interview with The Register. “And I think that’s misplaced.”

Moussouris said there’s a time and a place for bug bounties, though it’s more important that people are adequately supported to address security issues.

BugBust, she said, seems like a fine way to make developers aware of the tools AWS is offering for finding and fixing bugs. “Organizations that are serious about security need to get off the gimmick train,” she said. “This needs to be built into your culture.” ®