pub needs a dependency security audit feature

[kibantony]

pub needs a feature similar to npm audit that will scan a given project for known vulnerabilities and security issues by checking package versions against a vulnerability tracking database.

[jonasfj]

Wouldn't we also have to build such a database?

note. I'm not saying we shouldn't do this, only that it seems like we have lot of other things to do first.

[kibantony]

Yes. I'm not sure how pub works internally, but presumably you could add a table to the existing package database to track vulnerabilities per package. It would be nice to easily view and report vulnerabilities at pub.dartlang.org.

Alternatively you could rely on the something like the NVD, but that is problematic for a variety of other reasons.

In this age of constant security breaches, this is a critical feature. It's very unsettling to security minded engineers (all engineers?) to include packages from unknown sources with no means to report or audit for vulnerabilities. We can do this for nearly every other part of our stack, (Java, Node, etc.) but not Dart/Flutter. This will only get more critical as Flutter takes off and Dart becomes more mainstream, as I'm sure it will in the coming months.

[jonasfj]

I agree, this would be pretty neat to do. It's just not obvious to me how it would work.

Who can report a vulnerability? Who should review reported security issues? Who flags a package-version as containing a vulnerability?

I've been thinking a bit on how we could do a review system... I'm not sure what it would look like, maybe it's just starring favorit packages, or rating 1-5 stars or writing a text review in markdown.. inspired by how apps are rating the Android Play Store. Maybe, a vulnerability report is a just a particularly bad rating :)

Would vulnerability reports need to private? If so how would we warn users? How long we would keep vulnerabilities secret? Would they need independent reviews/verification?

It's not obvious to me how to do this. I would personally like to avoid supporting a pub audit thing that gives a false sense of impression, if the vulnerability reporting system isn't robust enough.

Note. I do think a pub audit command that could give insights into where the dependent packages originate from would be useful. But we likely need more metadata server side to see that two packages are published by the same author.

---

Note. You can report security issues to package uploaders, their emails are listed on the package page. It's not much, but you can contact the authors this way.

[kibantony]

These are good questions. I can give you specific answers (see below) which may or may not be entirely useful. I'm sure we can agree that once a platform reaches a certain threshold of maturity, explicit vulnerability reporting/tracking/auditing isn't just a neat feature, it's absolutely vital. I would argue that Dart/Flutter has already blown well past this point and will continue to accelerate.

I have no real insight into the organization, but the fact that you're asking questions of this nature leads me to believe there isn't much ancillary support coming from Google. In my answers below I cite npm as the standard example. It's not clear to me how well that applies since npm is backed by a single corporate entity that provides deeply dedicated support and resources. I would hope that Google would provide similar levels of support to Dart/Flutter given the direction it's headed. If that's the case then the short answer to all of your questions is: Escalate up to the larger team/management the need for first class security team and ancillary support to address the need for direct vulnerability reporting/tracking/auditing for Dart packages.

As for specific answers to your questions, this sort of thing is already being done by other language and dev tool teams, so one thing we can do is take a look around and see what's working for them. The closest example to pub/Dart would probably be npm. Looking at npm's policy we can consider some possible answers to your questions.

Who can report a vulnerability?

Anyone.

Who should review reported security issues? Who flags a package-version as containing a vulnerability?

Not the package maintainer. Presumably Google's security team. Or, if that's not feasible, a new security team formed as part of the Dart team. Are there any security engineers already working on Dart? (I hope so!)

Would vulnerability reports need to private? If so how would we warn users? How long we would keep vulnerabilities secret? Would they need independent reviews/verification?

Again npm's policy seems perfectly reasonable here. Reports are reviewed by the security team, independent of the package maintainer. They are kept private between the security team and the package maintainer for some reasonable amount of time (45 days in npm's case), after they are publicly disclosed (presumably at pub.dartlang.org).

[jonasfj]

...explicit vulnerability reporting/tracking/auditing isn't just a neat feature, it's absolutely vital. I would argue that Dart/Flutter has already blown well past this point and will continue to accelerate.

I think we're all in agreement :)

My point was that developing package moderation and vulnerability management policies for pub is a non-trivial process that involves many stakeholders. And it's not obvious that we should simply mirror what other package managers have done.

---

I guess you're suggesting that we simply have an email you can send vulnerabilities to, which we would review and forward to package owner. And if not fixed within 45 days we would publish the report.
(which perhaps seems easier than I initially thought)

I think that mostly involves publishing a policy, ensuring we have the right team receiving emails, and setting up some way of associating notices to specific package-versions. I filed dart-lang/pub-dev#2160, and closing this as it's not obvious what the CLI tool aspect will look like before the data is present serverside.

[kibantony]

Thank you @jonasfj. 👍

[bach942]

@jonasfj and @kibantony you might be able to use Snyk's database or have them create a plugin to scan the dependencies. They do this for other languages but is similar to npm audit, npm audit fix. Docs.

[oravecz]

Google now has the reporting process.
https://dart.dev/security

Google has the data reporting format.
https://github.com/ossf/osv-schema

Google has an open source database for this purpose.
https://github.com/google/osv

Google doesn’t have the priority to surface Dart into these processes or tools.
google/osv.dev#62

Google has started using GitHub Security advisories (per repo?) to publish security issues.
https://github.com/dart-lang/sdk/security/advisories

Google may be embracing GitHub’s advisory reporting, but not showing in Github’s centralized database.
https://github.com/advisories

Google has posted a handful of vulnerabilities to CVE (same 5 as shown in dart/sdk repo)
https://cve.report/software/dart/dart_software_development_kit

Google (and no other commercial or open source vulnerability scanners) exist for Dart at the time of writing. (pub.dev package vulnerabilities, or unsafe coding practices)

Google has no mechanism for tracking vulnerabilities across the thousands of libraries in pub.dev.

[stefanschaller]

@kibantony Very good points. Thanks!
There is just one thing that I personally not agree or let's say don't understand.
Why should we keep issues private for 45 days? Just to give the maintainer time to fix it? But maybe another developer could also fix it. That's the power of open source, right?

@oravecz Do you know if there is any progress since your last comment? I need something like the vulnerability scanner that works similar to npm audit .

[oravecz]

Last week this article on supply-chain security was published.
https://medium.com/dartlang/partnering-with-github-on-an-supply-chain-security-485eed1fc388

I admit to not knowing much about GitHub's Advisory Database, but it seems that it won't prevent bad actors, but rather people who opt-in, self-police and report their own vulnerabilities. While I think this will help for core Dart and Flutter code, the third-party libraries will be missed.

It would be good of Google to publish their GoogleDoc on a complete supply-chain security roadmap if such a thing has been considered.

[zmoshansky]

I'm very happy to see Github integration and improvements in the supply chain security.

I think it's still important to have an npm audit esque tool for those whose code is in a private cloud/non-github host. However, I don't see a reason a tool like that couldn't rely on GitHub's Advisory Database.

Without some of these tools, ISO27001 and similar enterprise standards will be much tougher to manage.

https://www.iso.org/isoiec-27001-information-security.html

[oravecz]

The main problem with GitHub Advisory database is that the package authors themselves have to "opt-in" and self-report vulnerabilities. Google does not require the advisory database as a criteria for listing in pub.dev. There is currently no carrot and no stick, so most package authors will ignore the capability.