Extend tracker documentation

Include answers to questions that have come up on the mailing list
Include descriptions of the helper scripts in ./bin/

1 files changed, 148 insertions, 6 deletions

diff --git a/doc/security-team.d.o/security_tracker b/doc/security-team.d.o/security_tracker
index 7a42cbb61a..e7a5e079bb 100644
--- a/doc/security-team.d.o/security_tracker
+++ b/doc/security-team.d.o/security_tracker
@@ -16,6 +16,22 @@ online. Everything is designed to be very simple to use, transparent and
 easy to see what other people are working on so you can work on other
 things.
 
+The Debian Security Tracker is only concerned with how specific vulnerabilities affect
+Debian. Many vulnerabilities are triaged as NFU (`NOT-FOR-US`) simply because the
+vulnerable software is not (yet) packaged for Debian. Triage comments on any specific
+vulnerability only reflect the possible impact on a system running Debian.
+
+For example, systems with some additional or modified packages compared to Debian need
+a separate triage process for every NFU to find ones which are relevant to what has
+been added as well as a triage on packages which differ from Debian.
+
+When a vulnerability relates to a package, the triage will need to include an
+assessment of the severity of the vulnerability as it affects Debian. See [Severity
+levels](#security-levels).
+
+Entries in the Debian Security Tracker do not imply anything about how a vulnerability
+may affect systems other than Debian.
+
 Gentle Introduction
 -------------------
 
@@ -421,6 +437,13 @@ assess these levels.
 Certain packages may get higher or lower rating than usual, based on
 their importance.
 
+Assessments of severity are made against the binaries as provided by Debian. A
+vulnerability where an exploit would rely on changing configuration in a non-standard
+way or rebuilding the binary from source to enable|disable some feature is not
+considered to be of high severity. For each vulnerability, the severity assigned within
+the Debian Security Tracker only relates to how Debian views that vulnerability and how
+quickly the fix may need to be applied to the specified package(s) within Debian.
+
 ### Vulnerabilities without an assigned CVE id
 
 If you learn of a vulnerability to which no CVE id has been assigned yet, you can
@@ -541,22 +564,59 @@ cross-reference will be added automatically by the cron job. However,
 you do need to add `[lenny]` or `[squeeze]` entries to `CVE/list` when there
 is a `no-dsa` or `not-affected` condition.
 
+Summary of tracker syntax
+-------------------------
+
+For a vulnerability in a package in Debian or proposed for introduction into Debian,
+the syntax should contain at least the `PKG_NAME` tabbed line and a `NOTE:` providing a
+URL to the fixing commit. Other lines are added, where relevant, within the general
+syntax.
+
+    CVE-YYYY-NNNNNN [(description)]
+     \t RESERVED
+     \t - PKG_NAME [PKG_TAG | PKG_FIX_VERSION] SEVERITY_LEVEL (free text comment)
+     \t [codename] PKG_NAME [PKG_TAG | PKG_FIX_VERSION] (free text comment)
+     \t NOTE:
+     \t TODO:
+
+- Each tabbed line, except `RESERVED`, can be repeated, e.g. for code embedded in
+  multiple packages and/or to cover multiple suites. Codenames are listed in order of
+  the release date.
+- PKG_NAME is the source package name in the archive.
+- PKG_TAG : `<no-dsa>` | `<unfixed>` | `<undetermined>` | `<not-affected>` | `<itp>`
+- SEVERITY_LEVEL : `(unimportant)` | `(low)` | `(medium)` | `(high)`
+- The pre-commit hook will check the syntax of each entry.
+
+The description of the CVE is not edited in the security tracker but it will be
+shortened in the tracker page for the vulnerability.
+
+For `<itp>`, the comment needs to include the bug number as `(bug #NNNNNNNNNN)`.
+
+`NOTE:` annotations are often used for URLs for more information but can also be
+used for descriptive comments.
+
 Checking in your changes
 ------------------------
 
 After thoroughly researching each issue (as described above) and editing
 the relevant files, commit your changes. Peer review is (hopefully) done via the
 mailing list and IRC notifications (see [Automatic issue updates](#automatic-issue-updates) above).
-However, changes to the tracker website itself (e.g., the files in lib/*
-and bin/tracker_service.py) should be vetted and approved before being
+However, changes to the tracker website itself (e.g., the files in `lib/*`
+and `bin/tracker_service.py`) should be vetted and approved before being
 committed. The preferred way to do this is to send a patch to the
-debian-security-tracker@lists.debian.org mailing list.
+`debian-security-tracker@lists.debian.org` mailing list.
+
+- [https://lists.debian.org/debian-security-tracker/](https://lists.debian.org/debian-security-tracker/)
 
 Commits are checked for syntax errors before they are actually committed,
 and you'll receive an error and your commit is aborted if it is in error.
 To check your changes yourself beforehand, use `make check-syntax` from
 the root of the Git directory.
 
+Note: It can be useful to use `git worktree` support for merging changes to master and
+ease issues that can occur when someone else has committed in between. See [git
+worktree (1)](https://manpages.debian.org/unstable/git-man/git-worktree.1.en.html).
+
 Following up on security issues
 -------------------------------
 
@@ -573,7 +633,7 @@ Tracking of security bugs in the BTS and linking them to a user tag by CVE
 --------------------------------------------------------------------------
 
 There's an automated tagging of security-related bugs to CVE IDs through
-the user tag security for the user debian-security@lists.debian.org.
+the user tag security for the user `debian-security@lists.debian.org`.
 
 All bugs added to the tracker are automatically tagged. You can use
 the search
@@ -594,11 +654,93 @@ with the following content:
 Contributing with the security tracker code
 -------------------------------------------
 
-Either fill a bug against the security-tracker pseudo-package attaching the patch
-to be reviewed or create a merge request for the security-tracker project.
+Either file a bug against the `security-tracker` pseudo-package attaching the patch
+to be reviewed or create a merge request for the security-tracker project in Salsa.
+
+### Helper scripts for one-off updates
+
+On success, scripts output a snippet of the main CVE list showing the new CVE
+information. Make sure to check for warnings and errors reported by the script. The
+output file needs to be manually reviewed and can then be merged using
+`./bin/merge-cve-files` or sent for review by the security team by email.
+
+##### Updating a vulnerability
+
+* Mark a given released suite as not affected for a specific CVE and source package:
+
+    `./bin/update-vuln --cve CVE --src SRC --suite SUITE`
+
+* Add a bug number to an existing CVE entry
+
+    `./bin/update-vuln --cve CVE --number 1000000`
+
+* Add a note to a specific CVE entry
+
+    `./bin/update-vuln --cve CVE --note "quoted note string"`
+
+Example workflow:
+
+    ./bin/update-vuln --cve CVE-YYYY-NNNNN ...
+
+check for error and warning messages & merge into the main CVE list:
+
+    ./bin/merge-cve-files ./CVE-YYYY-NNNNN.list
+
+review change to data/CVE/list
+
+    git diff data/CVE/list
+    rm ./CVE-YYYY-NNNNN.list
+
+.. repeat for additional entries to this or other CVEs.
+
+    git add data/CVE/list
+    git commit
+
+#### Retrieve fixes in uploads to unstable
+
+`./bin/grab-cve-in-fix` supports different ways to retrieve one or more CVEs as fixed in unstable:
+
+- Using information directly from the upload into unstable:
+
+    `cat changes | ./bin/grab-cve-in-fix --input`
+
+- Using information in the lists.debian.org archive:
+
+    `./bin/grab-cve-in-fix --archive https://lists.debian.org/debian-devel-changes/2021/12/msg01280.html`
+
+- Using information in the package tracker:
+
+    `./bin/grab-cve-in-fix --tracker https://tracker.debian.org/news/1285227/accepted-freerdp2-241dfsg1-1-source-into-unstable/`
+
+- Using local caches in the security-tracker:
+
+    `./bin/grab-cve-in-fix --src SRC --cves [CVES...]`
+
+Note: to use `STDIN` with the --input option, the changes content must be signed - i.e.
+as it would appear in notifications after the upload. This can be used to double-check
+your CVE list before uploading to ftp-master. `./bin/grab-cve-in-fix` will report if a
+CVE does not exist or if the CVE is attributed to a different package.
 
 **TODO** (further details)
 
+### Contributing ongoing triage work
+
+Some familiarity with the tooling and syntax will be needed for this, as with any development
+project.
+
+* `./bin/check-new-issues` - use the -h option to see the help output.
+
+* `./bin/report-vuln` - generate the correct email body to report a bug against a source package
+  relating to an unfixed CVE(s).
+
+### Useful search support for checking new CVES
+
+- [https://www.debian.org/distrib/packages#search_packages](https://www.debian.org/distrib/packages#search_packages)
+- [https://wnpp.debian.net/](https://wnpp.debian.net/) (Be aware, forwarded ITPs might
+  not be found, so check the [WNPP bug list](https://bugs.debian.org/cgi-bin/pkgreport.cgi?pkg=wnpp;dist=unstable) also)
+- [https://tracker.debian.org/](https://tracker.debian.org/)
+- [https://codesearch.debian.net/](https://codesearch.debian.net/)
+
 Setting up a local testing instance
 -----------------------------------