pulling in other vulnerability databases

[Antoine Beaupré]

So picking one thing from this thread and adding the security tracker
people in the loop, so we can focus on *one* topic here. :)

https://snyk.io/vuln/npm:jquery

https://nodesecurity.io/advisories?search=jquery

Finally, I wanted to bring Snyk.io to the teams' attention. I'm a little
disturbed by that new service - I feel there's significant overlap
between their vulnerability reporting process and Mitre's DWF/DNA
process, even down to using Google forms to welcome submissions, in the
case of DWF (!!). The Snyk (closed) database tracks vulnerabilities in
web apps, mostly, covering the following languages: Golang, Java
(maven), Javascript (npm), .NET (nuget), PHP (composer), Python (pip),
and Ruby (rubygems). I haven't done a formal study, but a quick glance
at the latest issues show that only a small fraction of the issues
reported there have CVE IDs at all.
This connects with the question of how to track issues without CVEs. In
general, that is a problem we have in the security tracker because it's
so bound to CVE identifiers. But this is a new problem as well: by
opening a new process for submitting vulnerabilities, this system
potentially bypasses the CVE system altogether, using a
commercial/proprietary backend. I am worried about the impact this will
have on our triaging efforts and wonder where we should go from here.
Food for thought?

Competition for Mitre & CVEs (Snyk)
Language communities (NodeSecurity)
OS vendors (RH/SUSE)
Upstream projects (Xen, Linux etc)
Security community (oss-sec, fulldisclosure, conferences etc)
Each of them have their own identifiers and possibly also link to CVEs.
I'd suggest we need (semi-)automated ingestion of all of the above,
like we currently have for CVEs.

Okay, so this is a broader, recurring problem we have with the security
tracker right now... From my perspective, I've always and only used CVEs
as unique identifiers for vulnerabilities in my work in the security
tracker. When that was not possible, say because a CVE wasn't issued
yet, CVE-YYYY-XXXX templates were used, which leads to ugly TMP-FOO
markers in the web interface.

Are you saying there are ways, right now, in the security tracker, to
use non-CVE identifiers as unique markers in a meaningful way? I
remember we discussed using BTS bug numbers in the past, but right now
those are, as far as I know, bound to entries in data/CVE/list only.

Where should we start to implement the above? Any pointers to where a
Python dev might look in the security-tracker source to actually work on
this? It seems this would also affect the DSA/DLA release process - i
think it *is* possible to actually release a CVE-less DSA right now, but
I wonder if the process would stall in other places...

And are we admitting here that we should just give up on the CVE process
in those cases? Or should we make an effort (say like we do in reporting
bugs against packages in teh BTS) to actively seek CVE identifiers for
vulnerabilities that lack them, like I did for the jquery issues?

Sorry to bring so many questions and so few answers, but I figured it is
useful to actually scope out the problem better before starting an
implementation. :)

A.

[Paul Wise]

Post by Antoine Beaupré
Okay, so this is a broader, recurring problem we have with the security
tracker right now... From my perspective, I've always and only used CVEs
as unique identifiers for vulnerabilities in my work in the security
tracker. When that was not possible, say because a CVE wasn't issued
yet, CVE-YYYY-XXXX templates were used, which leads to ugly TMP-FOO
markers in the web interface.

That is how things are supposed to work indeed.

Post by Antoine Beaupré
Are you saying there are ways, right now, in the security tracker, to
use non-CVE identifiers as unique markers in a meaningful way?

Not AFAIK. I think that having non-CVE identifiers on hand is a useful
but secondary goal. Achieving the primary goal also makes it easier to
achieve this too.

I'm mainly pointing out we need better automated ingestion of
vulnerability data for multiple reasons:

Mitre is *very* slow at unrestricting CVE information after a project
has published advisories, just look in CVE/list for things that are
still RESERVED but we have data for.

There are more vulnerability databases than just CVEs and this problem
is not going to go away. Some of these may (partially) overlap with
the CVE database.

The manual ingestion of vulnerability information into the security
tracker currently falls mainly on just one person (carnil). Having
just one person do all of that work isn't fair on them and does not
scale. If we automate more ingestion then there will be less latency
and he will have less work to do and be able to focus more on other
things like vulnerability feeds that aren't automatable (like mailing
lists).

Post by Antoine Beaupré
we discussed using BTS bug numbers in the past, but right now
those are, as far as I know, bound to entries in data/CVE/list only.

Correct.

Post by Antoine Beaupré
Where should we start to implement the above? Any pointers to where a
Python dev might look in the security-tracker source to actually work on
this? It seems this would also affect the DSA/DLA release process - i
think it *is* possible to actually release a CVE-less DSA right now, but
I wonder if the process would stall in other places...

First we should decide if we want non-CVE identifiers and define how
we want to represent non-CVE identifiers in data/CVE/list. I would go
for yes and something like this:

CVE-2017-17565 (An issue was discovered in Xen through ...)
...
ID: XSA-251

Next the database would need an additional identifier mapping table.

The CVE page could then also list XSA-251 in the Name field, linked to
the canonical location for XSAs.

The search field could be used to lookup XSAs and redirect to the
right CVE page, just like we do for BTS entries.

After that I'm not sure, but I think I would look at how the
'automatic updates' commits happen and then extend that process to
include other types of data feeds than just the CVE one. I started a
list of possible feeds in check-external/sources.ini

I think the GSoC or Outreachy would be an ideal way to achieve this
and wrote a proposal a couple of years ago, but couldn't find anyone
interested in mentoring it. ISTR last time I brought up this idea it
was dismissed as a bad idea.

https://wiki.debian.org/SummerOfCode2015/ProjectProposals/SecurityTrackerCheckExternal

Post by Antoine Beaupré
And are we admitting here that we should just give up on the CVE process
in those cases? Or should we make an effort (say like we do in reporting
bugs against packages in teh BTS) to actively seek CVE identifiers for
vulnerabilities that lack them, like I did for the jquery issues?

I don't think it is a good idea to give up on the CVE process and I
think doing so would be a violation of the Debian Social Contract. In
my opinion it would be a good idea (modulo workload issues) to seek
CVE identifiers for vulnerabilities reported in all vulnerability
databases, just like is done for the BTS (tag=security is kind of a
vuln db) and other locations.

Post by Antoine Beaupré
Sorry to bring so many questions and so few answers, but I figured it is
useful to actually scope out the problem better before starting an
implementation. :)

Quite.

[Salvatore Bonaccorso]

Hi

Post by Paul Wise

Post by Antoine Beaupré
Okay, so this is a broader, recurring problem we have with the security
tracker right now... From my perspective, I've always and only used CVEs
as unique identifiers for vulnerabilities in my work in the security
tracker. When that was not possible, say because a CVE wasn't issued
yet, CVE-YYYY-XXXX templates were used, which leads to ugly TMP-FOO
markers in the web interface.

That is how things are supposed to work indeed.

Post by Antoine Beaupré
Are you saying there are ways, right now, in the security tracker, to
use non-CVE identifiers as unique markers in a meaningful way?

Not AFAIK. I think that having non-CVE identifiers on hand is a useful
but secondary goal. Achieving the primary goal also makes it easier to
achieve this too.
I'm mainly pointing out we need better automated ingestion of
Mitre is *very* slow at unrestricting CVE information after a project
has published advisories, just look in CVE/list for things that are
still RESERVED but we have data for.

This is IMHO only part of the truth. Actually MITRE is becoming faster
recently, I think that should be acknowledged as well. One point with
those RESERVED entries is that the (sub-)CNA assigning such should
notify the parent/top CNA about the publication. As you will note with
all the recent CVEs which got assigned by Debian, after publication a
team member did notify MITRE (if they do not notify themself via the
released DSA) and the state changed.

Cf. CVE-2018-0486, CVE-2017-16995 to CVE-2017-16997, CVE-2017-8805 to
CVE-2017-8824.

So it's as well on the responsibility of a project which is a CNA or got
assigned a CVE under "embargo" to notify then MITRE about the
publication.

And good news is that there is work on improving that workflow.

*But* you are right as well, that there are still way to many RESERVED
entries, for published CVEs which apparently never reached back to MITRE
database. I have not done any analysis. So this is only an example:
For 2018 there are currently 46 RESERVED, but already public CVEs. Out
of those at least 3 were assigned by DWF, 2x curl, 1x glibc and I
expect Kurt Seifried of DWF to merge them back soonish. One big hunck
is from the recent Mozilla advisory. All of those I would have expected
that they will merged back by DWF and mozilla in the upcoming few days.

Post by Paul Wise
There are more vulnerability databases than just CVEs and this problem
is not going to go away. Some of these may (partially) overlap with
the CVE database.
The manual ingestion of vulnerability information into the security
tracker currently falls mainly on just one person (carnil). Having
just one person do all of that work isn't fair on them and does not
scale. If we automate more ingestion then there will be less latency
and he will have less work to do and be able to focus more on other
things like vulnerability feeds that aren't automatable (like mailing
lists).

I'm actually not super-excited if non-MITRE feeds are automatically
merged. MITRE CVE database is our master-reference. Those can be
updated automatically. Every other external query should IMHO be
subject to a check for sanity. We have seen quite often discrepancies,
e.g. typos in CVE ids.

Maybe though I'm missunderstanding the proposal :)

Regards,
Salvatore

[Moritz Mühlenhoff]

Post by Salvatore Bonaccorso
I'm actually not super-excited if non-MITRE feeds are automatically
merged. MITRE CVE database is our master-reference.

Yeah, let's not do that.

Cheers,
Moritz

[Antoine Beaupré]

Post by Paul Wise

Post by Antoine Beaupré
Okay, so this is a broader, recurring problem we have with the security
tracker right now... From my perspective, I've always and only used CVEs
as unique identifiers for vulnerabilities in my work in the security
tracker. When that was not possible, say because a CVE wasn't issued
yet, CVE-YYYY-XXXX templates were used, which leads to ugly TMP-FOO
markers in the web interface.

That is how things are supposed to work indeed.

Post by Antoine Beaupré
Are you saying there are ways, right now, in the security tracker, to
use non-CVE identifiers as unique markers in a meaningful way?

Not AFAIK. I think that having non-CVE identifiers on hand is a useful
but secondary goal. Achieving the primary goal also makes it easier to
achieve this too.
I'm mainly pointing out we need better automated ingestion of
Mitre is *very* slow at unrestricting CVE information after a project
has published advisories, just look in CVE/list for things that are
still RESERVED but we have data for.

It was said by others, but I also want to stress that I've witnessed a
welcome improvement in the processing speed at Mitre. They are getting
better at this - it was this or complete failure, so I guess they had no
choice. :)

Post by Paul Wise
There are more vulnerability databases than just CVEs and this problem
is not going to go away. Some of these may (partially) overlap with
the CVE database.

I think we should *try* to make other databases sync up with the CVE
process. The security tracker itself is such a database, and we *do*
that process. We're not special: others, especially when they are
corporations like Snyk, should follow established community
standards. We should hold them up to this publicly.

Post by Paul Wise
The manual ingestion of vulnerability information into the security
tracker currently falls mainly on just one person (carnil). Having
just one person do all of that work isn't fair on them and does not
scale. If we automate more ingestion then there will be less latency
and he will have less work to do and be able to focus more on other
things like vulnerability feeds that aren't automatable (like mailing
lists).

Yeah, that's tricky business. I am constantly impressed by the amount of
work done by the security team, so I can certainly appreciate this! So I
think it would be interesting to have semi-automated tools to inspect
external databases and make sure we're not missing anything. We
shouldn't inject the data automatically like we do for authoritative
sources like Mitre, however, because that could become really chaotic;
it could actually *increase* the load on the team, because of the
increase in (possibly) unreviewed material coming in.

[...]

Post by Paul Wise
First we should decide if we want non-CVE identifiers and define how
we want to represent non-CVE identifiers in data/CVE/list. I would go
CVE-2017-17565 (An issue was discovered in Xen through ...)
...
ID: XSA-251
Next the database would need an additional identifier mapping table.

See, I'm lost here. Why would we have non-CVE-identified issues in
data/CVE/list? Shouldn't this be something like data/XSA/list in this
case?

The problem that you quickly run into is that you *will* have
duplication between those lists. Right now, CVE ids are a "primary key"
in data/CVE/list - if we add entries *without* CVE ids there, things get
confusing fast (i know we already do, but it's a temporary measure). And
if we add non-CVE identifiers *elsewhere*, we add the extra load of
filtering out duplicates...

So I guess I'm answering my own question: injesting other databases is
not really feasible.

[...]

Post by Paul Wise
I think the GSoC or Outreachy would be an ideal way to achieve this
and wrote a proposal a couple of years ago, but couldn't find anyone
interested in mentoring it. ISTR last time I brought up this idea it
was dismissed as a bad idea.
https://wiki.debian.org/SummerOfCode2015/ProjectProposals/SecurityTrackerCheckExternal

I think it would be a good idea if it would be restricted to a simple
triage / audit mechanism, not an automated ingestion mechanism.

Post by Paul Wise

Post by Antoine Beaupré
And are we admitting here that we should just give up on the CVE process
in those cases? Or should we make an effort (say like we do in reporting
bugs against packages in teh BTS) to actively seek CVE identifiers for
vulnerabilities that lack them, like I did for the jquery issues?

I don't think it is a good idea to give up on the CVE process and I
think doing so would be a violation of the Debian Social Contract. In
my opinion it would be a good idea (modulo workload issues) to seek
CVE identifiers for vulnerabilities reported in all vulnerability
databases, just like is done for the BTS (tag=security is kind of a
vuln db) and other locations.

I think it's fair to say we all seem to be in agreement that we want to
keep going with the CVE process. :)

A.

[Salvatore Bonaccorso]

Hi Antoine,

Post by Antoine Beaupré
So picking one thing from this thread and adding the security tracker
people in the loop, so we can focus on *one* topic here. :)

https://snyk.io/vuln/npm:jquery

https://nodesecurity.io/advisories?search=jquery

Finally, I wanted to bring Snyk.io to the teams' attention. I'm a little
disturbed by that new service - I feel there's significant overlap
between their vulnerability reporting process and Mitre's DWF/DNA
process, even down to using Google forms to welcome submissions, in the
case of DWF (!!). The Snyk (closed) database tracks vulnerabilities in
web apps, mostly, covering the following languages: Golang, Java
(maven), Javascript (npm), .NET (nuget), PHP (composer), Python (pip),
and Ruby (rubygems). I haven't done a formal study, but a quick glance
at the latest issues show that only a small fraction of the issues
reported there have CVE IDs at all.
This connects with the question of how to track issues without CVEs. In
general, that is a problem we have in the security tracker because it's
so bound to CVE identifiers. But this is a new problem as well: by
opening a new process for submitting vulnerabilities, this system
potentially bypasses the CVE system altogether, using a
commercial/proprietary backend. I am worried about the impact this will
have on our triaging efforts and wonder where we should go from here.
Food for thought?

Competition for Mitre & CVEs (Snyk)
Language communities (NodeSecurity)
OS vendors (RH/SUSE)
Upstream projects (Xen, Linux etc)
Security community (oss-sec, fulldisclosure, conferences etc)
Each of them have their own identifiers and possibly also link to CVEs.
I'd suggest we need (semi-)automated ingestion of all of the above,
like we currently have for CVEs.

Okay, so this is a broader, recurring problem we have with the security
tracker right now... From my perspective, I've always and only used CVEs
as unique identifiers for vulnerabilities in my work in the security
tracker. When that was not possible, say because a CVE wasn't issued
yet, CVE-YYYY-XXXX templates were used, which leads to ugly TMP-FOO
markers in the web interface.
Are you saying there are ways, right now, in the security tracker, to
use non-CVE identifiers as unique markers in a meaningful way? I
remember we discussed using BTS bug numbers in the past, but right now
those are, as far as I know, bound to entries in data/CVE/list only.

We can use CVEs and as fallback Debian Bugs as unique identifier. But
for releasing a security update we do not have arequirement that a CVE
has already to be assigned (ideally it is, because it makes
identifying and tracking it easier in the long term). If no CVE is
assigned, and no bug is present, just fill one for the tracking
purpose.

And btw, MITRE had admittely some problems in past to timely assign a
CVE to issues, but this is becoming increasingly better, CVEs are
assigned (on proper requests) quite fast in meanwhile.

Post by Antoine Beaupré
And are we admitting here that we should just give up on the CVE process
in those cases? Or should we make an effort (say like we do in reporting
bugs against packages in teh BTS) to actively seek CVE identifiers for
vulnerabilities that lack them, like I did for the jquery issues?

Please do not give up on the CVE process, when there is need for an
CVE identifer, please continue to request such.

Regards,
Salvatore

[Moritz Mühlenhoff]

So, regarding the first two (and similar), someone needs to teach those
folks about proper security tracking here... ;) Should I contact them
directly?

Who in particular? Node and Snyk? Sure, go ahead.

Cheers,
Moritz

[Antoine Beaupré]

Post by Moritz Mühlenhoff

So, regarding the first two (and similar), someone needs to teach those
folks about proper security tracking here... ;) Should I contact them
directly?

Who in particular? Node and Snyk? Sure, go ahead.

I contacted snyk.io with mitre and the secteam in cc.

I'm not sure what to say to nodesecurity.io folks: there's a lot of
stuff in there, and I'm not sure it affects us so much. I couldn't find
a package without a CVE that is also in Debian in the first few pages of
their advosiry pages...

pabs, did you have any issues in mind that were problematic here
specifically?

in any case, feel free to reuse the message I sent to snyk as a template
for others...

a.

[Paul Wise]

Post by Antoine Beaupré
I'm not sure what to say to nodesecurity.io folks

I've already contacted them multiple times in 2014 and once in 2016,
about incorporating CVEs into their workflow. The responses were
positive but didn't result in much change, except when the issues were
sent to oss-sec or Mitre by the Debian security team or myself or
others. Most of their recent advisories have CVEs but some don't. I'm
guessing the researchers who discovered the issues are getting CVEs.
I think the best outcome would be if NodeSecurity could become a CNA so
they could issue CVEs immediately with each advisory they send out.

https://marc.info/?i=***@chianamo
https://marc.info/?i=***@bonedaddy.net
https://marc.info/?l=oss-security&m=139757263925026&w=2
http://www.openwall.com/lists/oss-security/2016/02/20/2
http://www.openwall.com/lists/oss-security/2016/01/12/2

Post by Antoine Beaupré
pabs, did you have any issues in mind that were problematic here
specifically?

Here is one example culled from my email archive:

http://bugs.debian.org/862712
https://nodesecurity.io/advisories/338
https://security-tracker.debian.org/tracker/862712

It didn't end up getting added to the security tracker, didn't get a
CVE and only got fixed in Debian after I filed a bug about it.

[Ben Hutchings]

On Thu, 2018-01-25 at 10:17 -0500, Antoine Beaupré wrote:
[...]

OS vendors (RH/SUSE)
Upstream projects (Xen, Linux etc)

I believe those already follow the CVE process and eventually converge
over doing the right thing. So I am not really concerned about those
people.

[...]

Linux has a security contact (***@kernel.org), but this is only
used for reporting bugs and discussing how to fix them; CVE assignments
are left to distributions, DWF, etc. Many security fixes don't get
discussed there anyway.

I would estimate that less than half of security fixes in Linux
actually get CVE IDs.

Ben.

[Antoine Beaupré]

Post by Ben Hutchings
[...]

OS vendors (RH/SUSE)
Upstream projects (Xen, Linux etc)

I believe those already follow the CVE process and eventually converge
over doing the right thing. So I am not really concerned about those
people.

[...]
used for reporting bugs and discussing how to fix them; CVE assignments
are left to distributions, DWF, etc. Many security fixes don't get
discussed there anyway.
I would estimate that less than half of security fixes in Linux
actually get CVE IDs.

Well that's just disturbing. I am not sure, however, that I can
meaningfully change this by ... er... say writing the kernel mailing
lists, unfortunately.

I haven't got a reply from Snyk.io (yet?) by the way. I suspect I may
not get anything at all... Any other ideas as to the next steps in
general here?

a.