Skip to content

Add a way to mark a failed test case #58468

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
asashour opened this issue Jul 27, 2021 · 0 comments
Closed

Add a way to mark a failed test case #58468

asashour opened this issue Jul 27, 2021 · 0 comments
Assignees
Labels
devexp-linter Issues with the analyzer's support for the linter package legacy-area-analyzer Use area-devexp instead. P3 A lower priority bug or feature request type-enhancement A request for a change that isn't a bug type-test

Comments

@asashour
Copy link
Contributor

Use case

Sometimes there is a need to commit a test case, before actually implementing it.

As part of Test-driven development, this has the benefit of:

  • Agreeing first if a scenario is a correct with expected behavior, before starting the implementation.
  • One may lack the implementation knowledge, but a test case could be easier to contribute.
  • Having a single place of all reported issues (in one file), instead of searching in github issues.
  • When an implementation is (accidentally) fixed, a failing test would point to the tracked issue to be closed.

Some other projects use the same concept (notYetImplemented), e.g. Groovy and Selenium.

Needles to say, this doesn't mean implementation is done without testing. But it means tests can be committed, and implementation can later follow.

A similar request in test in dart-lang/test#1178

Proposal

An optional FAILING to be added to the lints comment (optionally having an issue number), and when set, the result of the test is reversed.

  • If the test fails (as expected) then the build will be green
  • If the implementation succeeds, then the build is red complaining Test XYZ is marked as failing but already succeeds.
void false_negative() {
  noLintTriggered(); // LINT FAILING
}
void false_posivite() {
  incorrectLintTriggered(); // OK FAILING dart-lang/linter#1234
}

As a side note, it seems Annotation.message is not used by any lint, except in annotation_test, and so it could be removed.

@pq pq added type-test type-enhancement A request for a change that isn't a bug labels Jul 28, 2021
@pq pq added the P3 A lower priority bug or feature request label Nov 14, 2022
@devoncarew devoncarew added devexp-linter Issues with the analyzer's support for the linter package legacy-area-analyzer Use area-devexp instead. labels Nov 19, 2024
@devoncarew devoncarew transferred this issue from dart-archive/linter Nov 19, 2024
@srawlins srawlins self-assigned this Dec 9, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
devexp-linter Issues with the analyzer's support for the linter package legacy-area-analyzer Use area-devexp instead. P3 A lower priority bug or feature request type-enhancement A request for a change that isn't a bug type-test
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants