Adventures in vulnerability reporting
Posted by Natalie Silvanovich, Project Zero
At Project Zero, we spend a lot of time reporting security bugs to vendors. Most of the time, this is a fairly straightforward process, but we occasionally encounter challenges getting information about vulnerabilities into the hands of vendors. Since it is important to user security that software vendors fix reported vulnerabilities in a timely matter, and vendors need to actually receive the report for this to happen, we have decided to share some of our experiences. We hope to show that good practices by software vendors can avoid delays in vulnerability reporting.
Effective Vulnerability Reporting Processes
There are several aspects of a bug reporting process that make reporting vulnerabilities easier from the bug reporter’s perspective. To start off, it’s important for a bug reporting process to be easy to find and use. We sometimes have difficulty figuring out how to report a vulnerability in a piece of software if the vulnerability reporting process is not documented on the project or vendor’s website, or if outdated material is not removed and instructions for reporting vulnerabilities are inconsistent. This can lead to delays in reporting. Effective vulnerability reporting processes are clearly documented, and the documentation is easy to find.
We also appreciate when the process for reporting a vulnerability is short and straightforward. Occasionally, we report dozens of vulnerabilities in a vendor’s products, and it is helpful when reporting does not require a lot of clicks and reading. Reporting processes that use email or bug trackers are usually the easiest, though webforms can be easy if they are not excessively long. While Project Zero will always report a vulnerability, even if reporting it is very time consuming, this is not necessarily the case for other bug reporters. Long bug reporting processes can cause bug reporters to report bugs more slowly, spend less time working on a piece of software or even give up on reporting a bug. The easier a bug reporting process is, the more likely it is that someone will go through with it.
It’s also important for bug reporting processes to be well-tested. While the majority we encounter are, we’ve occasionally had bug reporting email addresses bounce, webforms reject necessary information (like the reporter’s name) and security issues go unnoticed in bug trackers for months despite following the documented process. Vendors with good processes usually test that their process and any systems it involves works correctly on a regular basis.
Mandatory legal agreements in the reporting process are another problem that we encounter every so often. If a legal agreement contains language about disclosure or any other subject we don’t feel comfortable entering an agreement about on behalf of our company, deciding whether to enter the agreement can require a lengthy discussion, delaying the bug report. While legal agreements are sometimes necessary for rewards programs and code contributions, good vulnerability reporting processes allow bug reporters to report bugs without them.
It is also helpful when vendors confirm that vulnerability reports have been received in a timely manner. Since bug reports can get lost for a number of reasons, including bugs in the reporting interface and human error, it is a good idea to let reporters know that their report has been received, even if it won’t be processed right away. This lets the reporter know that they’ve reported the bug correctly, and don’t need to spend any more time reporting it, and makes it more likely that bug reporters will reach out if a bug report gets lost, as they will be expecting a confirmation.
Finally, even if good practices are followed in creating the bug reporting process, it is still possible that a bug reporting process has problems, so it is very helpful if vendors provide a way to give feedback on the process. It’s very rare for vendors to intentionally make bug reporting difficult, but unexpected problems happen fairly frequently, so it is good to provide a way bug reporters can reach out for help as a last resort if a reporting a bug fails for any reason.
Examples
One example of a bug we had difficulty reporting due to a vendor not following the practices described above is CVE-2018-10751. CVE-2018-10751 is a remote memory corruption vulnerability in OMACP affecting the Samsung S7 Edge. The issue can be triggered by sending a single SMS to the target device, and does not require any user interaction. The payload can be sent from an app on an Android device without root access or any special equipment. It is similar to CVE-2016-7990, which is described in detail here.
Samsung’s Vulnerability Reporting Process
CVE-2018-10751 is a serious vulnerability, and I wanted to report it immediately. I started off by reading Samsung Mobile’s Security Reporting page. This page has a button to create a bug report.
Pressing the button led to a sign-up page. I didn’t have a Samsung account, so I tried to sign up. Unfortunately, it led to this page:
https://security.samsungmobile.com/securityReporting.smsb Accessed February 22, 2018
Not speaking Korean, I wasn’t sure what to do here. I eventually went back to the previous page and tried the ‘Sign-in’ button.
This brought me to an English sign-up page, which then brought me to the account creation page. According to this page, I had to read and agree to some terms. Clicking the links led to over twenty separate agreements, most of which had nothing to do with vulnerability reporting.
https://account.samsung.com Accessed February 22, 2018
That’s a lot of text to read and review. Let’s just say I skimmed a bit. Once I clicked ‘Agree’, I was taken to a page where I could enter account information. The page required my birthdate and zip code, which I wasn’t thrilled to have to provide to report a vulnerability, but I wanted to get the issue reported, so I entered them. Finally, my account was created! I logged in, hoping to start reporting the bug, only to be greeted with more conditions.
These ones were in Korean, and I couldn’t figure out how to change the language. Eventually, I just selected confirm. Finally, I got to the form where I could report bugs!
I filled out the vulnerability information, and scrolled down, and there was one more set of terms to agree to:
https://security.samsungmobile.com/securityReporting.smsb Accessed February 22, 2018
These terms included:
- You MUST hold off disclosing the vulnerability in reasonable time, and you MUST get Samsung’s consent or inform Samsung about the date before disclosing the vulnerability.
- In some cases, Samsung may request not to disclose the vulnerability at all.
I was not able to submit this form without agreeing to allow Samsung some level of control over disclosure of reported vulnerability. I looked around Samsung’s security page to see if they provided an email address I could report the issue to, but they did not provide one. I was not comfortable reporting this bug through the mechanisms Samsung provides for vulnerability reporting on their website.
Problems with Vulnerability Reporting Processes
I encountered several problems while trying to report the above vulnerability—most of which have been since resolved by Samsung.
To start off, Samsung’s bug reporting process did not seem adequately tested. The many times that Korean text showed up while attempting to report this vulnerability suggests that it was not tested in English. As described above, is important for vendors to test vulnerability reporting processes, including for internationalization issues. The workflow is also excessively long, and requires the reporter to agree to a very large number of agreements, many of which have nothing to do with vulnerability reports. I suspect that the people testing this interface might have already had accounts, and not seen how long the process is for someone who just wants to report a bug.
This isn’t an uncommon problem. The Android security reporting template requires creating a GMail account, which can require clicking through many screens and verification via SMS in some circumstances. As a result of our feedback, the Android Security team has improved the documentation that vulnerability reports can be filed via email (security@android.com), although using the web form is still required to participate in the Android Security rewards program.
Another problem was that in order to report a bug, a reporter had to agree to the terms of the rewards program. This is an issue that Project Zero has been seeing increasingly often. When software vendors start rewards programs, they often remove existing mechanisms for reporting vulnerabilities, leaving bug reporters with no way to report vulnerabilities without entering into agreements.
This also occurred when Tavis Ormandy attempted to report the vulnerability he reluctantly dubbed CloudBleed. Cloudflare’s vulnerability reporting process is tied to its rewards program with HackerOne, and there is no clear way to report a vulnerability without creating a HackerOne account in their Vulnerability Disclosure Policy. The policy even states “We agree with their disclosure philosophy, and if you do too, please submit your vulnerability reports here” without providing an alternative for vulnerability reporters who don’t agree or don’t want to participate in the program for whatever reason. In Project Zero’s case, our disclosure deadline is 90 days meanwhile HackerOne’s deadline is 180 days. This vulnerability was also very urgent as it was actively leaking user data onto the Internet, and we didn’t want to delay reporting the issue while we read through HackerOne’s terms to determine whether they were compatible with our disclosure policy.
We find that vendors generally don’t intend to prevent bug reports from anyone who won’t agree to their disclosure rules, but this was the end result of Samsung and Cloudflare replacing their bug reporting process with a rewards program.
The specific terms of Samsung’s agreement were also fairly vague. In particular, it wasn’t clear what the consequences of breaking the terms would be. For example:
- You MUST hold off disclosing the vulnerability in reasonable time, and you MUST get Samsung’s consent or inform Samsung about the date before disclosing the vulnerability.
Does this mean that if someone discloses a vulnerability without permission, they are not eligible for a reward? Does it mean that if someone discloses the vulnerability without permission, Samsung can take legal action against them? While requiring that bug reporters not disclose vulnerabilities to receive rewards is a policy with debatable benefit, I would have been much more comfortable agreeing to these terms if they had spelled out that violating them would simply mean I would not receive a reward, as opposed to other legal consequences.
Overall, the issues of poorly tested bug reporting interfaces and requiring legal agreements to report vulnerabilities have come up multiple times, and led to delays of Project Zero reporting vulnerabilities. We recommend that vendors test their vulnerability reporting interfaces from the perspective of someone who’s never reported a bug from outside of their corporate network, and make sure to do localized testing. It is also important to allow bug reports without requiring the reporter to enter into excessive legal agreements.
While only accepting vulnerability reports via web forms can reduce the number of invalid reports, which is a major challenge for teams accepting vulnerability reports, they can also be unreliable and prevent vulnerability reporting in situations that were not expected by those designing them, unless they are very well tested. Having an alternate email address that vulnerability reporters can use to report bugs if they encounter problems is a good way to prevent this type of problem.
Reporting the Bug
I eventually contacted some members of the Knox security team at Samsung that I had worked with on previous bugs and they recommended reporting the issue to mobile.security@samsung.com. This email is not documented on the Samsung website, except for a single blog post from 2015.
The difficulty I encountered reporting this serious vulnerability delayed my report one week. It might have caused a longer delay if I did not have contacts at Samsung who could help.
Samsung started rolling out updates for CVE-2018-10751 (Samsung’s identifier SVE-2018-11463) in their April maintenance release.
Samsung has updated their account creation page so that it always displays English text if the language is set to English. Also, the vulnerability report form can now be submitted without agreeing to the terms for the Samsung’s rewards program, though the user still has to agree to two other agreements. They have also updated their bug reporting page to provide an email address as well as a webform. We appreciate the changes they have made to make reporting vulnerabilities in Samsung products easier for everyone.
Conclusion
Project Zero has occasionally had difficulty reporting vulnerabilities, leading to delays in reporting the bug. Usually, these are due to problems in the reporting process that were not intended or expected by the vendor. A difficult vulnerability reporting process can have a negative impact on user security due to delays in vulnerability reports, lost vulnerability reports and even bug reporters choosing not to report a vulnerability. We appreciate when vendors do the following to make their bug reporting processes easier for bug reporters:
Vendors should regularly test their vulnerability reporting interfaces in all supported languages
Vendors should streamline their vulnerability reporting processing as much as possible, and remove excessive clicks and legal agreements
Vendors should regularly solicit feedback on their vulnerability reporting mechanisms from vulnerability reporters and people they think are likely to report vulnerabilities
https://security.samsungmobile.com/securityReporting.smsb Accessed February 22, 2018
Admitting that one's product can have vulnerabilities seems to be the biggest blocker to me.
ReplyDeleteSo Project Zero reports a vulnerability to the vendor and doesn't request a CVE-ID if the vendor doesn't want one?
ReplyDeleteThe example about the SMS exploit is really interesting. People always think there's isn't any potential for harm if they don't interact anything. Wondering if a similar thing is happening/possible through voicemail, maybe visual voicemail.
Vendors probably fear the entire process..but still need to to be more transparent about these type things so the average person isn't walking around thinking their phone is magical. If everyone was more realistic about all this companies wouldn't worry their stock price/sales will be impacted and consumers wouldn't be as careless if they weren't mislead in such an extremely unethical way.
I was also wondering if there's any way to request that someone from Project Zero look into something unusual? Sort of related to code signing, ocsp requests, javascript, and some other persistence features (I think).