Security Reviews in a SaaS World

Rajan Kapoor, Director of Security, Dropbox

As organizations undergo digital transformation, they're storing more data than ever before on a myriad of SaaS platforms. In fact, a recent study found large companies utilize 163 apps on average. These apps contain sensitive information like customer data and even the data of customer’s customers. SaaS providers are now an integral part of an organization’s security perimeter, but the current process for evaluating them is insufficient.

To assess the security of SaaS providers, organizations will follow a typical Request for Information (RFI) process: ask providers questions about security, evaluate their answers, look at compliance documents, and read penetration test results. Breach after breach tells us that this method for assessing a provider gives an incomplete picture.

Assessing the security of SaaS providers requires a meaningful shift in how we evaluate their security posture. These guiding principles will help you get a better handle on a provider’s security:

Find controls that are externally verifiable.

Many of the security questions asked today cannot be verified without putting SaaS providers through an audit that gives access to their internal workings. Identifying controls that don’t require you to work in this way but still give you good visibility into how well the provider is protecting their services is key.

Look for continuous testing.

The assessments done today are largely limited to one point in time. You’ll get a good snapshot of how a SaaS provider was doing when the tests were run, but won’t have a sense of how the provider has been doing since then. Continuous testing is important because a provider’s security posture constantly evolves and there’s no guarantee it won’t degrade.

So how can you leverage both of these concepts to better understand the security of your providers? And how can you do it in a scalable way that will keep pace with the increasing rate of adoption of SaaS by your organization?

Enter the Security Research Community

The security research community is active, vibrant, and growing. A security researcher is anyone who takes an interest in testing security in the interest of public good. They’re driven by curiosity, compensation, and recognition. They’ll work to find vulnerabilities on the Internet and then partner with impacted organizations to report and mitigate these vulnerabilities.

Done right, engaging with the security research community will give you controls that are externally verifiable and demonstrate a commitment to undergoing continuous testing. When evaluating vendors, it is helpful to ask the following three questions:

Does the company embrace security research?

A good sign that you’re working with a mature security team is they welcome security research and are transparent about vulnerabilities identified on their services. These organizations embrace security researchers testing their services. They recognize that working with the security research community is one of the best ways to ensure they are making the right investments to secure their services and products.

A Vulnerability Disclosure Policy (VDP) demonstrates the organizations’ commitment to working with security researchers and protects them from legal action against research that falls within this scope. Organizations friendly to security research will have publicly published a VDP committed to protecting security researchers.

Mature security organizations also embrace transparency. They know that helping others learn from their security lessons makes the Internet more broadly a safer place.

Some organizations take a hostile approach to security research and this should be a red flag. In the past, we’ve seen companies take legal action when approached by researchers who have found vulnerabilities and, in extreme cases, try to pursue criminal cases. This is a sign that the organization is not serious about securing its platform and will go to great lengths to prevent disclosures.

Does the company make it easy to report vulnerabilities?

The second signal to look for is how easy an organization makes it to report a vulnerability. Do they have a publicly available way for researchers to contact them when they find a vulnerability? At a minimum, there should be an email address published on their websites with instructions on how to submit a bug.

Does the company have a bug bounty program?

Bug Bounty programs are an emerging method for providing incentives and recognition to security researchers who report a bug. A researcher will receive compensation (the bounty) for reporting a vulnerability and will also be allowed to publish their work publicly once the vulnerability has been resolved. Bug Bounty programs allow organizations to set reasonable parameters around the scope of what can be tested. For example, if a security researcher finds a way to access user data, the Bug Bounty program can restrict the researcher from probing the vulnerability any further and prevent them from accessing the user data to demonstrate the bug. Mature security organizations will have a very broad scope for testing, allowing researchers to test almost any surface of their product or services.

Another indicator of the maturity of the organization is the amount of money they will pay out. High bounties indicate that the organization is confident in their security practices and also that it’s becoming harder for researchers to find vulnerabilities. A low bounty indicates that the organization might not be yet confident in their security practices or that a high number of vulnerabilities are being reported.

A final indicator from a Bug Bounty program is how quickly an organization responds to and mitigates a vulnerability. Again, embracing the principle of transparency, organizations should disclose in their Bug Bounty program when a report was received, when it was resolved, and how much compensation the reporting researcher received.

Look for third parties that have a Bug Bounty program, preferably through a trusted provider like HackerOne, that includes most of their services in a wide scope for testing, and offers competitive bounties to researchers who participate in their program.

Pulling this all together: Questions to ask SaaS Providers

Now that we have a better idea of what signs to look for to evaluate the security maturity of third parties, here are some questions you can ask as part of your security review:

Do you have a published Vulnerability Disclosure Policy?
How would a security researcher report a vulnerability to you?
How quickly do you commit to resolving critical bug reports?
Are these reports publicly available and, if not, are you willing to share them?
Do you allow security researchers to publish their findings publicly?
Do you participate in a Bug Bounty program? Is it run by an independent third party?

On a final note, if you are a provider of services to your own customers, you should also look into how you would answer the questions above. Another sign of a mature security program is a willingness to commit to the same expectations you have of your third parties. You can get started by checking out Dropbox’s VDP which we published and open sourced last year. We encourage you to adapt it to your own organization’s needs.