Set up advanced CI tests

[ethanfrey]

This builds on #15 but also adds full-stack tests using real nodes.

This is done with docker and ts-relayer for testing.
Or maybe with ibctesting and pure go??

(Update: First version is done. I moved this issue to v1 as we will need to improve the situation)

[JakeHartnell]

@0xekez used interchaintest with Polytone: https://github.com/DA0-DA0/polytone/tree/main/tests

One advantage is that it can easily spin up multiple validator nodes per chain. Not a huge fan of Go, but IMO interchaintest is probably the best out there.

What do you think @0xekez?

[0xekez]

I'd recommend using ibctesting and using interchaintest to test that your code can work with real relayers. I think ibctesting is great because the entire Polytone ibctesting suite takes ten seconds to run, which makes iteration really fast, which is important when working with SDK code.

Interchaintest is a really cool idea, but their CosmWasm support was quite 🤷 when I was writing those tests. I made a lot of issues and they were very responsive, so I think it would be great to keep working with them, but if you just want to write tests with a real relayer, ts-relayer is much more likely to work at the moment.

[0xekez]

If I was making a CosmWasm IBC protocol today, I would use Polytone and skip the SDK / IBC / go entirely. There's no need to handshake, as you can instantiate your counterparty, and get their address back. Then for tests, I could write a loopback polytone contract that would work with multi-test, and you would be free of IBC, go, and the SDK entirely 🙏

Also, when a frontend is made for this we can re-use the relayer infrastructure we're making for DAO DAO.

[alpe]

Not sure if I got this correct. But polytone are contracts and the simtests package that contains tests and a test environment with wasmd ibctesting under the hood, which uses ibc-go ibctesting. The simtests env seems tailor made for the polytone contracts using vanilla wasmd app. Not really an option for the consumer side.
As a quick solution, it may be possible to provide the mesh-security-sdk e2e tests as binary so that the tests can be executed on CI here as well.
This would be a predefined test set on wasmd ibctesting package

[ethanfrey]

We are going with ibctesting for this not ts-relayer

[pyramation]

Hey guys! We already got mesh security working with starship for this, I'll find the demo video and send it so I can show you guys. it's based on cosmjs and kubernetes, and can run in github actions

We were able to get all the tests to pass, this was actually a few months ago.

Additionally OGP has a grant for Anmol to build this already for Mesh Security, we'll be adding him to the slack soon

[Anmol1696]

It might make more sense if we can note down the requirements we need from the E2E framework, then go from there, would help us figure out how we can try and solve it. I think we can divide the stack as well with tools that best support the usecases. What exact features the framework needs to have, and the kind of tests we would be writing here?

With Starship the idea is for ability to spin up the infra with a simple config file and then write test cases against it. For the hackathon version of MeshSecurity we did a quick port of the e2e tests to run against 2 chain that we spin up using this config file. Here is the old video of running this in the CI as well. FYI: this is older version, we have come a far way since then too.

Since you have decided to not run ts-relayer, we can additionally run hermes relayer as well as a chain registry with meta information about the infra, so the tests dont have to be hardcoded with that information.

Also it flexible enough to have the test cases to be written in any language we want, Go and CosmJS (we already have examples), would be really simple to have this in Rust as well, since the client is a light wrapper around the infra.

[ethanfrey]

This is being done in mesh-security-sdk right now with ibctesting.

There is a desire in the future for both:

• Proper native rust ibc multitest coverage
• Full-stack integration with starship, simulating multiple multi-node chains and real interactions between them, to test the full stack (including tendermint and relayers) and to provide a great integration point for dApp, explorers, and other clients.

[maurolacy]

For documentation, this is now done with ibctesting, and there are a couple open PRs to test using starship:

• Starship integration mesh-security-sdk#55
• Starship integration mesh-security-sdk#56