Source: Paul Holland, Principal Research Analyst, ISF
13 Sep 2019

Bug bounties and crowd sourced penetration testing, why might you need them?

Many organisations have implemented a Penetration Testing programme of some sort. This usually takes the form of performing a test on any application and then re-checking it on a regular basis. This is then followed by a defined grace period that an application owner needs to remediate identified vulnerabilities, concentrating on internet facing critical or high risks first. Whilst this is a good starting point, as it does at least mean organisations are thinking about security and resolving known issues, it is a point in time solution and can mean that vulnerabilities may exist for months within an application before they are remediated.

Regular penetration testing programmes leave an organisation in a precarious position. Constant penetration testing would be expensive and likely be prohibitive for organisations. By not performing constant testing there is a possibility of missing critical vulnerabilities within an application that could have far reaching consequences for the whole infrastructure and could affect the viability of that organisation to continue operating. EternalBlue (the vulnerability exploited by WannaCry, stopped some organisations for days and estimated global costs of recover and lost business at $4 billion). So how can these issues be addressed?

A bug bounty programme is a different way to look at vulnerabilities (also known as a vulnerability reward program (VRP)). Many large multi-national corporations already operate these programmes (e.g. Apple, Google, Facebook and Uber). A bug bounty is where an organisation allows individuals not directly associated with the organisation to try and hack into their systems and discover vulnerabilities. The idea is that many enthusiasts enjoy this type of challenge, and by offering a bug bounty it would mean they can gain a financial reward for their efforts and honesty, rather than using this knowledge for nefarious purposes. It encourages the rationale of preventing breaches. If a bug bounty programme is initiated, then there is a need to ensure that all legal and contractual considerations are in place before you begin your journey. Without these an ethical hacker may become unethical if you cannot respond in a timely and positive way.

Rewards are based on the severity of the vulnerability and the validity of the discovery. Once it has been authenticated the finder will receive their bug bounty. Bounties are becoming more popular with the first known bug bounty millionaire being made this year, even featuring in main stream news like the BBC. More and more enthusiasts are turning their skills to finding bugs and collecting bounties. With more organisations offering a bounty for disclosures, this is a win-win for both the white hat enthusiasts and the organisations with the forethought to give them that chance.

An alternative if the risks of allowing anyone to potentially attack your infrastructure seems too great, is to purchase a crowdsourced penetration testing solution. You contract a company that offers continuous testing. This is nearly identical to a bug bounty programme in that individuals still attempt to discover vulnerabilities. They are still paid on a discovery basis, but they will always use a specific IP address to connect to your network. This allows for a SOC to know that this traffic is known and expected. Both solutions offer the ability to have applications and infrastructure tested with longer term attack vectors utilised as part of the testing. Traditional penetration testing is time box and therefore some attack vectors are not achievable and are therefore ignored. These newer methods allow for a low and slow password attack to take place, there is not normally enough time in a penetration test to attempt this, as it will likely lock out the accounts that are used. It also benefits organisations when/if they are making constant changes to their applications and websites on a regular basis. These changes are tested as part of the normal process without the need to wait for the next cycle of testing to take place.

By utilising bug bounties (either through a programme or via crowd sourced testing) it allows organisations to improve their development processes and to resolve vulnerabilities in a timely manner and before they become public knowledge. The primary adopters of a bug bounty programme appear to be mainly technology focussed organisations but even more conservative organisations like the Pentagon are now offering bounties to the ethical hackers. If a contracted crowd sourced solution is purchased, the benefits of the wide variety of people who are out in the public domain will also be gained. The diverse skillsets and testing approaches from all over the world will see every part of an application being tested, whether that be the core operating system or a unique piece of JavaScript code.

This means that anyone at any time can attempt to discover a vulnerability within an application. The intended outcome is a steady stream of different tests that would therefore give an organisation better coverage of vulnerabilities that might exist across their infrastructure in a timelier manner. Penetration testing is likely to still be around for some time but even the organisations that offer this a penetration testing service are already moving to offer a continuous testing service. Although continuous testing seems to be more automated with tools and scripts doing the main bulk of the testing rather than a human performing the test. Which can mean that the continuous testing is not as in depth or detailed as the current penetration test, it therefore still leaves a gap in the testing performed when compared to the bug bounty option.

Organisations should consider offering bug bounties or purchasing a crowd sourced penetration testing solution as these provide better coverage of vulnerabilities that an attacker may try to exploit. Your solutions need to evolve as attackers are always evolving their attack methods.

Paul Holland is a Principal Research Analyst for the Information Security Forum.
He is currently a contributing author to the research report, Using Cloud Services Securely.
Paul has also provided subject matter expertise in support of recent research projects, including Building a successful SOC, Human-Centred Security (coming soon) and Demystifying Artificial Intelligence in Information Security (coming soon)