Libp2p testing overhaul

[vasco-santos]

This issue will track the work pipeline regarding libp2p testing. The two main points here are the integration testing ownership and thorough System Testing + Interop Testing

Integration Testing

Currently, we have integration tests for libp2p modules, both on libp2p modules and libp2p itself. Despite causing an obvious testing overlap, this creates a problem on Node.js@15 with mismatch versions when a libp2p peerDependency is installed on a module. See more

The best solution to solve the above problem is to delegate libp2p modules to completely handle their integration tests. Libp2p would focus entirely on testing its API, configuration and core components, while the modules would be responsible for the integration and test the subsystem in depth.

Libp2p interfaces should be expanded to export all integration tests needed per type of module (pubsub, peerDiscovery, peerRouting, ...). With this, libp2p modules can require and use these tests by simply requiring them and providing instances of the module. This enables us to have less replicated code as the boilerplate to test different pubsub/peerDiscovery modules, while keeping a guarantee that these tests will run.

Before releasing a stable release of libp2p, we should have visibility of possible side effects of a new release within the context of each libp2p module, in order to guarantee that no regressions are introduced on the modules before release. We can achieve this by using Github Actions Worflow actions dispatch. We can leverage dispatch-action to let other repos know of the changes and run their workflow. To run additional dependency versions, we can use cross-env and test them similarly to dev.to/joshx/test-your-npm-package-against-multiple-versions-of-its-peer-dependency-34j4. The easier solution is probably to create nightly releases with commit in master and verify each libp2p module before cutting the final release.

With the above changes in practise, libp2p package json would only include as devDep the essencial modules to test its basic APIs (transports, streamMuxer and connEncryption). On the other side, libp2p modules would install libp2p as a peerDependency and devDependency using semver ranges.

To sum up, the above proposal includes:

• libp2p tests its basic API and core component
• libp2p-interfaces exports integration tests per module type
• libp2p modules are responsible for running their integration tests with libp2p
• ?libp2p should have CI visibility on side effects affecting its modules

Other ideas:

• In the future, we can have a js-libp2p-integration module with the integration tests, if we go with a lerna repository Consider libp2p mono repo #824
• We can do master commit release builds in js-libp2p as a way to test modules.

Thorough System Testing and Interop Testing

With the testground js sdk on its way, we can start to specify what tests plans we would like for a Thorough System Testing.

Other ideas:

• Perhaps a module js-libp2p-testing within the Lerna setup, which would include integration tests and test plans.

Related:

• [testing] fix node 15/16 install due to npm 7 changes #790

[vasco-santos]

@jacobheun let me know your thoughts on this proposal, so that I can iterate on a more actionable plan with tasks + milestones

[vasco-santos]

I did a PoC with pubsub for the integration testing proposal. There is also one point in libp2p/js-libp2p-interfaces#81 description that would be good to consider. Shortly, interface tests should receive an instance of libp2p instead of an instance of the module?

@achingbrain @hugomrdias what do you think about this testing structure?

[hugomrdias]

I did a PoC with pubsub for the integration testing proposal. There is also one point in libp2p/js-libp2p-interfaces#81 description that would be good to consider. Shortly, interface tests should receive an instance of libp2p instead of an instance of the module?

@achingbrain @hugomrdias what do you think about this testing structure?

with this setup an instance so that libp2p-interfaces doesnt depend on libp2p, only the package do.

Can we publish nightlies here dispatch to the others repos CI and there receive or fetch the latest version, force a npm install libp2p@version or npm i libp2p@next and avoid the cross-env/env vars part ?

[vasco-santos]

The initial part of this work with pubsub was done. We still need to do the same for the remaining libp2p modules

[maschad]

Closing as completed