Test Canary Releases

[lforst]

We want to continuously test the latest or even release candidate versions of major frameworks with our SDK. We're gonna start off by doing this for react.

[AbhiPrasad]

Should the approach be we first allow for testing multiple versions of the same app with the e2e tests (that’s config through Env variables of some kind)? Then we can make the canary release just run the e2e test but with the single canary version (latest?)

[lforst]

Should the approach be we first allow for testing multiple versions of the same app with the e2e tests (that’s config through Env variables of some kind)? Then we can make the canary release just run the e2e test but with the single canary version (latest?)

That's the plan!

[lforst]

Should the approach be we first allow for testing multiple versions of the same app with the e2e tests (that’s config through Env variables of some kind)? Then we can make the canary release just run the e2e test but with the single canary version (latest?)

That's the plan!

On a second thought. Doing it with env vars is a bit complicated since we can have many different test projects and tests. Lemme see if we can make this configurable via the recipe.

[AbhiPrasad]

Thinking about this more, we can have two modes

a) canary mode that just tests the latest release (defaults to npmPackage@latest, but config through the recipe)
b) regular mode that tests the array of possible versions

We can then have a test:e2e:canary and a test:e2e that uses the two different modes.

This means we can do the canary stuff first, and then come back and add the array of stable major versions afterwards. What do you think?

[lforst]

Thinking about this more, we can have two modes

a) canary mode that just tests the latest release (defaults to npmPackage@latest, but config through the recipe) b) regular mode that tests the array of possible versions

We can then have a test:e2e:canary and a test:e2e that uses the two different modes.

This means we can do the canary stuff first, and then come back and add the array of stable major versions afterwards. What do you think?

Yeah this is actually what I was thinking. Something like this:

Or two different arrays versions and canaryVersions. Doesn't really matter I guess.