Vulnerability Management
SOC 2 Criteria: CC1.2, CC3.1, CC3.3, CC3.4, CC4.1, CC4.2, CC5.1, CC5.2, CC7.1, CC7.2
ISO 27001 Annex A: A.12.1.1, A.12.7.1, A.18.2.3
Keywords: Penetration testing, Pen testing, Vulnerability scans, Vulnerability priority levels, Security SLAs
Purpose
Bramble policy requires that:
- All product systems must be scanned for vulnerabilities at least monthly.
- All vulnerability findings must be reported, tagged, and tracked to resolution in accordance with the SLAs defined herein. Records of findings must be retained per our Record Retention Policy.
Roles and Responsibilities
Bramble’s Security Officer is responsible for updating, reviewing, and maintaining this policy.
Policy
Vulnerability Management is the recurring process of identifying, classifying, prioritizing, mitigating, and remediating vulnerabilities. This overview will focus on infrastructure vulnerabilities and the operational vulnerability management process. This process is designed to provide insight into our environments, leverage GitLab for vulnerability workflows, promote healthy patch management among other preventative best-practices, and remediate risk; all with the end goal to better secure our environments.
Information Systems Audit
The following guidelines will be observed for setting information systems audit controls:
- Audit requirements for access to systems and data should be agreed with appropriate management.
- Scope of technical audit tests should be agreed and controlled.
- Audit tests should be limited to read-only access to software and data.
- Access other than read-only should only be allowed for isolated copies of system files, which should be erased when the audit is completed, or given appropriate protection if there is an obligation to keep such files under audit documentation requirements.
- Requirements for special or additional processing should be identified and agreed.
- Audit tests that could affect system availability should be run outside business hours.
- All access should be monitored and logged to produce a reference trail.
Penetration Testing
Penetration testing is performed regularly by either a certified penetration tester on Bramble’s security team or an independent third party.
Findings from a vulnerability scan and/or penetration test are analyzed by the Security Officer, together with IT and Engineering as needed, and reported through the process defined in the next section.
Vulnerability Scanning and Infrastructure Security Testing
The scanning and identification of Bramble’s system vulnerabilities is performed by:
- Automated Drata security agent installed on all employees’ machines.
- Snyk.io
Additionally, periodic security scans of Bramble systems are done using a combination of external open-source and commercial vulnerability testing tools, including:
- Breachlock
- Tenable.io
- Nessus
The Vulnerability Management process
Arguably the most important step for a successful vulnerability management process is defining the scope that the process will cover. Security and Infrastructure partnered to come up with a scope that would make sure all of our critical environments and systems were covered during deployment. The following projects are currently in-scope:
| Environment | Project/Account | Production | Deployed |
|---|---|---|---|
| AWS | bramble-app | yes | yes |
| AWS | infra | yes | yes |
Note: If you believe a system you are responsible for should be included in the vulnerability management process, please contact the Security Incident Response Team.
With these environments scoped out and Snyk scanners deployed, we can begin the vulnerability management process. Keep in mind that vulnerability management is a feedback loop - vulnerability scanners provide the vulnerability data which is analyzed and ingested to mitigate and remediate found vulnerabilities. Feedback from this process feeds into preventative initiatives that further secure our environments.
Currently, we break down vulnerability management into the following steps:
1. Vulnerability Scanning
This step is where we scan resources in our environments to identify vulnerabilities. Once setup, scans run on regular cadences that meet or exceed our compliance framework requirements.
2. Reporting/Analysis
Vulnerability scan data is exported and analyzed to provide consolidated vulnerability data we can ingest into GitLab for vulnerability remediation tracking. This is currently a manual process where we export vulnerability data into a spreadsheet and pull out pertinent information.
Snyk also provides reporting functionality that is used by our Compliance team to run reports for audits.
3. Ingestion
Once the data is prepared in a format that we can pull out the most important information, we can ingest into GitLab. Issues are opened in the Vulnerability Management tracker to track the remediation process of the vulnerability. Another issue is opened in the Infrastructure issue trackerlinking to the vulnerability management tracker issue; these are so that the work can properly get prioritized and scheduled according to the Infrastructure team’s workflow.
Currently, we group vulnerabilities into a single remediation issue on a monthly basis as to consolidate the work required to remediate. From here, the SIRT team can work with Infra to prioritize and open additional remediation issues which are linked to the monthly remediation issue.
Vulnerability remediation issues should be tagged with the vulnerability type label. These leverage GtiLab’s scoped label capability. The following labels exist to track the vulnerability remediation workflow:
-
~vulnerability::vulnerable: This label identifies that the vulnerability has been opened, but not validated and is considered impactful to our environments per the assigned priority label. With this label a vulnerability issue should not be closed. -
~vulnerability::validated: This label identifies that the vulnerability has been validated as legitimate and is scheduled for mitigation or remediation. With this label a vulnerability issue should not be closed. -
~vulnerability::falsepositive: This label identifies that the vulnerability has been validated as a false positive and is no longer impactful to our environments. With this label a vulnerability issue can be closed. -
~vulnerability::exception: This label identifies that the vulnerability has been validated as legitimate and has an approved exception issue to account for a business need. In extreme circumstances, a vulnerability issue can be closed with an exception. -
~vulnerability::mitigated: This label identifies that the vulnerability has been validated and triaged. The impact has been reduced through compensating controls, but not remediated (it is still actively identified on vulnerability scans). With this label a vulnerability issue should not be closed. -
~vulnerability::remediated: This label identifies that the vulnerability has been remediated and the remediation has been validated. With this label a vulnerability issue can be closed.
We also add the VM label to all Vulnerability issues to scope the issues in the Vulnerability Management issue board.
4. Validation
Validation is an important part of vulnerability management. This is where we investigate to ensure that the vulnerability being reported has properly been identified.
Vulnerabilities can sometimes be identified during a scan, but are not actually on the system. This can happen for a number of reasons, but most commonly is the result of misflagged ports or services. These are classified as false positives and would go through the process to be closed as a false positive.
5. Remediation
Remediation is the part of the process in which a validated vulnerability is fixed. The remediation process would be tracked in the corresponding vulnerability issue in the Vulnerability Management issue tracker. SLAs are in place to help prioritize vulnerability based on severity. Once a vulnerability is remediated, we will run followup scans on the impacted systems to validate that the vulnerability is indeed remediated.
We’ve implementing an escalation path for remediation issues in the Infrastructure issue tracker that automatically tags the VM DRI and backup when remediation issues are approaching/past SLAs.
For improved tracking of remediation issues, we are using GitLab Epics. The remediation epic includes monthly subepics that track remediation progress for that month. If remediation SLAs do not require a vulnerability to be remediated in a month, it will be rolled over into the following subepic until remediated or its due date passes.
6. Feedback
The last step is for the Security Incident Response Team and Infrastructure to determine what we can learn from each vulnerability remediated. This may be an improvement on the vulnerability management process itself or establishing preventive mechanisms for a repetitive vulnerability type. This feedback will be documented in the vulnerability issue and could result in additional issues being opened.
As stated above, this process is a cyclical loop. Vulnerability scans are recurring, providing new vulnerability data that feed new vulnerability remediation and exception issues which then help update/escalate open issues/processes.
Remediation SLAs
Security and Infrastructure use these remediated SLAs based on a multitude of factors, such as severity, scope, impact, etc. All of these factors will be considered when mapping the priority to an Issue priority label. The SLAs are as follows:
| Priority | Severity Mapping | Time to mitigate | Time to remediate |
|---|---|---|---|
| severity::1/priority::1 | Zero-day | Within 24 hours | Within 72 hours (when technically feasible) |
| severity::2/priority::2 | Critical | N/A | Within 30 days |
| severity::3/priority::3 | High | N/A | Within 60 days |
| severity::4/priority::4 | Medium | N/A | Within 90 days |
| severity::4/priority::4 | Low | N/A | Best effort. |
severity::1/priority::1 vulnerabilities discovered in scans would be worked on immediately through the incident management process and adhere to any timelines determined as such. This includes a 15-minute engagement, 24-hour mitigation, and 72-hour remediation SLA (from time of reporting).
Note: Mitigation SLAs only apply to severity::1/priority::1 vulnerabilities. These types of vulnerabilities often coincide with broad industry-impacting zero-day vulnerabilities and in the event of these types of events the 72 hour target would be impossible to meet or exceed. These exceptions will be documented and noted as they occur.
Exception process
We understand that it is not always technologically feasible to keep all packages up-to-date due to application conflicts, or that a business decision may be made to not remediate a vulnerability because remediation would impact performance too greatly.
Low risk vulnerabilities that may not get prioritized within the remediation SLAs should have an exception approved for them, documenting the low likelihood of exploit due to layered security, other compensating controls, mean of exploitation, etc.
With this in mind we have a vulnerability exception process; If you’ve identified a vulnerability that is a candidate for an exception, please open a vulnerability exception issue in the vulnerability management issue tracker.
Please fill all out the pertinent information requested in the template. For reference, the information required is as follows:
- Vulnerability title
- Snyk ID
- Priority/Severity of Vulnerability
- Original remediation issue due date
- Length of requested exception
- List of applicable hosts
You will also need to describe the business need for the exception, document any existing/implemented compensating controls, and link any ongoing remediation efforts.
Exception length restrictions
We currently allow exception lengths based on priority/severity as follows:
| P/S | 30-days | 60-days | 90-days | 365-days |
|---|---|---|---|---|
| priority::1/severity::1 | Yes |
No |
No |
No |
| priority::2/severity::2 | Yes |
Yes |
Yes |
No |
| priority::3/severity::3 | Yes |
Yes |
Yes |
Yes |
| priority::4/severity::4 | Yes |
Yes |
Yes |
Yes |
Exception approval matrix
After the issue is open, the requestor should assign the due date to match that of the associated remediation issue and assign to the proper approver. The severity and priority of the vulnerability will dictate the approval process. This is documented below:
| P/S | Approver |
|---|---|
| priority::1/severity::1 | CTO |
| priority::2/severity::2 | VP of Security or Infrastructure |
| priority::3/severity::3 | Security Manager, Security Incident Response Team |
| priority::4/severity::4 | Security Engineer, Security Incident Response Team |
Vulnerability Scanning Schedule
Vulnerability scans occur on a daily in our scoped projects.
Contact
If you have any questions or concerns related to vulnerability management please contact the Security Incident Response Team in #team-security channel on slack, by tagging @team-sirt in slack or @brmbl-io/security/sirt in a Bramble issue, or finally you can open an issue in the Security Incident Response Team issue tracker. All work being done to improve this process is also tracked in the issue tracker.
Any questions regarding ownership around vulnerability management can be answered in Bramble’s tech stack documentation.