test(e2e): Add E2E Test recipe framework #5836
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
size-limit report 📦
|
AbhiPrasad
approved these changes
Sep 28, 2022
| "type": "string", | ||
| "description": "Name displayed in test output" | ||
| }, | ||
| "buildCommand": { |
There was a problem hiding this comment.
we should also add a timeout for build commands
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Ref: #5506
Adds the functionality to the E2E tests package to configure test recipes alongside very simple examples. E2E tests can be configured by adding standalone test applications to the
test-applicationsfolder containing atest-recipe.jsonfile.Via the
test-recipe.jsonwe can configure abuildCommand(that is run once, and is supposed to build the test application) and any number of tests that each have atestCommand. This command is supposed to actually conduct the test and verify the behavior of apps or SDKs.These commands are left very generic intentionally so that we stay flexible on how we can conduct tests. (e.g. some may want to run Playwright, some only run a node script, some may start a fake server, others might want to send events to prod sentry, etc.)
The important part is, that in the context of these commands, there is a test registry available at
http://localhost:4873that hosts all of the current Sentry packages. This allows us for example to simply doyarn install --registry http://localhost:4873in thebuildCommandin order to install our packages as they would look like if we did a release on the current branch.Visit this PR to see what failing E2E tests look like.