E2e Test Fixes #25058

kdumontnu · 2023-06-02T19:59:20Z

Run e2e tests in series so that DB + fixtures can reinitialize between each test
Run only a single test each test loop iteration 🤦‍♂️
Reenable firefox test (Closes Firefox E2E tests are failing with "Page closed" #21355)
Use locator instead of query for playwright functions

To Do:

Add more tests?
~~Bump playwright~~ and review test setup

silverwind · 2023-06-02T20:08:21Z

review test setup

should check if Firefox can be re-enabled or if it's still failing.

tests/e2e/README.md

Co-authored-by: delvh <[email protected]>

kdumontnu · 2023-06-05T21:29:24Z

review test setup

should check if Firefox can be re-enabled or if it's still failing.

I've reenabled. I've been running it on my fork for a while and haven't seen it fail yet. 👍

As far as other tests go, do you have any recommendations for pieces of the front-end we know are fragile or we've run into issues repeatedly?

silverwind · 2023-06-08T10:49:50Z

As far as other tests go, do you have any recommendations for pieces of the front-end we know are fragile or we've run into issues repeatedly?

Nothing really comes to mind, but if you are looking for something to test, I'd say the PR flow is something that would be nice to have, e.g.

Create branch
Edit a file in Monaco, commit it
Create PR
Review the PR on a second account
Merge the PR
Delete the branch

Part of this flow is already covered by backend integration tests, but it should really be moved to E2E instead and out of backend integration tests.

silverwind · 2023-06-20T22:56:32Z

As far as other tests go, do you have any recommendations for pieces of the front-end we know are fragile or we've run into issues repeatedly?

Nothing really comes to mind, but if you are looking for something to test, I'd say the PR flow is something that would be nice to have, e.g.
* Create branch

* Edit a file in Monaco, commit it

* Create PR

* Review the PR on a second account

* Merge the PR

* Delete the branch
Part of this flow is already covered by backend integration tests, but it should really be moved to E2E instead and out of backend integration tests.

Maybe also start simpler:

Open issue
Comment on it
Close it

silverwind · 2023-07-07T20:27:18Z

tests/e2e/e2e_test.go

+						fmt.Printf("%v", stderr.String())
+						log.Fatal("Playwright Failed: %s", err)
+					} else {
+						fmt.Printf("%v", stdout.String())


I'm surposed ths fmt.Printf passes the linter.

silverwind · 2023-07-07T20:27:42Z

tests/e2e/issue.test.e2e.js

@@ -0,0 +1,56 @@
+// @ts-check


What's this for?

silverwind · 2023-07-07T20:28:47Z

tests/e2e/utils_e2e.js

@@ -53,7 +47,7 @@ export async function save_visual(page) {
      timeout: 20000,
      mask: [
        page.locator('.dashboard-navbar span>img.ui.avatar'),
-        page.locator('.ui.dropdown.jump.item span>img.ui.avatar'),
+        page.locator('.ui.dropdown.jump.item.tooltip span>img.ui.avatar'),


No better selector available? Maybe add a class name? This will break too easily.

silverwind · 2023-07-07T20:29:30Z

tests/e2e/example.test.e2e.js

-  await page.waitForLoadState('networkidle');
+  await page.locator('input#user_name').fill('user2');
+  await page.locator('input#password').fill('password');
+  await page.locator('form button.ui.green.button:visible').click();


Also a rather suboptimal selector. Afraid we have to add some class names to make tests more robust against class name changes.

silverwind · 2023-07-07T20:29:46Z

tests/e2e/example.test.e2e.js

+  await page.locator('input#email').fill(`e2e-test-${workerInfo.workerIndex}@test.com`);
+  await page.locator('input#password').fill('test123');
+  await page.locator('input#retype').fill('test123');
+  await page.locator('form button.ui.green.button:visible').click();


wxiaoguang · 2025-03-13T02:55:34Z

Stale for long time.

Continue or close?

wxiaoguang · 2025-03-19T11:54:00Z

@kdumontnu The same question is still in my mind .... is the e2e testing system really used or useful?

After years, I can see it only consumes some CI resources but no test is really written.

So, maybe removing the e2e testing system is also a choice?

kdumontnu · 2025-03-19T13:20:36Z

@kdumontnu The same question is still in my mind .... is the e2e testing system really used or useful?

After years, I can see it only consumes some CI resources but no test is really written.

So, maybe removing the e2e testing system is also a choice?

I think the decision must be:

Remove e2e tests
Improve and add tests

The e2e tests have certainly not been very helpful as written, which has certainly surprised me. There are some improvements that can be made to make them run more efficiently & reliably, but I don’t think that’s the major issue. Rather, it seems we’re not finding value in it.

I use them extensively in my fork to ensure I don’t regress on front-end features.

@wxiaoguang @silverwind @kerwin612 you do a lot of the front-end development recently. I think we should reflect on:

where do we run into the most bugs?
Where do we spend the most time reviewing?
What will help new contributors or currently prevents us from touching some code?

Some categories of “issues” I can think of are:

CSS leaking (changing one style and breaking a style on another page)
template errors, like nil de ref
Component initialization
breaking mobile layout or touch support
Cross-browser support
Performance
…

There are certainly different solutions to these. For instance, we could have integration tests request templates to catch 500 errors. Some of them I think are still best solved with e2e tests. I think migrating to typescript is an incredible help.

One thing I like to mention is that you are able to make contributions to the code base because you understand a lot of the nuances, but it’s difficult to scale that, which is why I suggest we focus on reviews and new contributions in the evaluation.

Happy to get everyone’s opinion though because I agree that e2e tests are not working right now

silverwind · 2025-03-19T14:07:01Z

I use playwright testing in other applications and find those tests to be valuable to assert that the UI is working on a very basic level. log in, log out, assert certain elements are present/absent before/after login state change. I'm only also only using chrome in these tests, for the sake of having them as stable as possible.

I have not investigated how gitea's e2e tests work in detail or why they even include go code, but that go code seems superflous. e2e tests should just compile and start the app (./gitea web, with a basic .ini config and sqlite), and then run playwright suites against it.

lunny · 2025-03-19T15:54:35Z

Or maybe we can move these tests to a standalone repository.

silverwind · 2025-03-19T16:44:41Z

Or maybe we can move these tests to a standalone repository.

Absolute no from me. Test code needs to be in the same repo as the code being tested.

wxiaoguang · 2025-03-26T13:50:18Z

I will answer from my side:

you do a lot of the front-end development recently. I think we should reflect on:
* where do we run into the most bugs?

Various edge cases or design problems.

* Where do we spend the most time reviewing?

Same as above, various edge cases or design problems.

* What will help new contributors or currently prevents us from touching some code?

Improve the whole framework to make code easy to write and avoid low-level mistakes.

Some categories of “issues” I can think of are:
* CSS leaking (changing one style and breaking a style on another page)

There used to be a lot, most of them are caused by hacky patches/abuses/pollution.

After recent years refactoring, the whole situation is much better now.

* template errors, like nil de ref

It could be covered by backend tests (and it is easier to be covered by backend tests because backend code could mock data more easily)

* Component initialization

I guess it is still a framework problem. At least, my new proposal "global init" will resolve most problems.

* breaking mobile layout or touch support

E2e test could do better in this field

* Cross-browser support

E2e test could do better in this field. Since we do not use cutting edge browser features, most modern browsers behave the same with our code.

* Performance

Not really related to e2e test

The really problem is:

If the e2e tests are written simply (simple component check, page check), backend tests and frontend unit tests could do better.
If the e2e tests need to be written complexly, I do not see there is manpower/time for it at current stage. The fact is nobody ever wrote a useful e2e test.

silverwind · 2025-03-26T17:05:50Z

Keep those e2e tests as minimal as possible. Complex tests offer little value and are prone to introduce flakiness and if they are testing too specific things, they prevent progress as some maintainers may not be willing to even touch e2e tests.

I could for example see value in having e2e that just visit a few pages and assert that the browser console does not log any errors. Some recent problems would have been caught by this.

wxiaoguang · 2025-04-23T13:05:06Z

I will disable the e2e test in CI (#34262) since it fails with new playwright and there is no active maintainer for it. The e2e test related code is still in code base, feel free to maintain it if anyone uses it.

wxiaoguang · 2025-04-23T22:49:01Z

-> update go&js dependencies #34262

Feel free to reopen if there would be new progresses. Thank you very much!

Remove DISPLAY env setting and add debug option

8fd961c

GiteaBot added the lgtm/need 2 This PR needs two approvals by maintainers to be considered for merging. label Jun 2, 2023

pull-request-size bot added the size/M label Jun 2, 2023

delvh reviewed Jun 2, 2023

View reviewed changes

tests/e2e/README.md Outdated Show resolved Hide resolved

Update tests/e2e/README.md

7182dc9

Co-authored-by: delvh <[email protected]>

pull-request-size bot added size/L and removed size/M labels Jun 5, 2023

kdumontnu added 4 commits June 5, 2023 17:13

Use playwright locator and login form

eef1e5b

Reenable Firefox testing

40c02fc

Merge remote-tracking branch 'upstream/main' into kd/fix-e2e_tests

aedd0e1

Add firefox to e2e_test

9d5280d

kdumontnu marked this pull request as ready for review June 5, 2023 21:29

kdumontnu changed the title ~~Remove DISPLAY env setting and add debug option~~ E2e Test Fixes Jun 5, 2023

kdumontnu added 2 commits June 8, 2023 07:28

Merge branch 'main' into kd/fix-e2e_tests

14dc954

Merge branch 'main' into kd/fix-e2e_tests

6abecd1

kdumontnu added 3 commits July 7, 2023 14:51

Minor e2e cleanup

1ade3aa

Add issues test

daf8251

Merge branch 'main' into kd/fix-e2e_tests

ee2e1d4

kdumontnu requested a review from silverwind July 7, 2023 18:57

silverwind reviewed Jul 7, 2023

View reviewed changes

tests/e2e/issue.test.e2e.js

@@ -0,0 +1,56 @@

// @ts-check

Copy link

Member

silverwind Jul 7, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What's this for?

silverwind reviewed Jul 7, 2023

View reviewed changes

wxiaoguang added the issue/needs-feedback For bugs, we need more details. For features, the feature must be described in more detail label Mar 13, 2025

techknowlogick removed the size/L label Apr 10, 2025

wxiaoguang closed this Apr 23, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

E2e Test Fixes #25058

E2e Test Fixes #25058

kdumontnu commented Jun 2, 2023 •

edited

Loading

silverwind commented Jun 2, 2023

kdumontnu commented Jun 5, 2023

silverwind commented Jun 8, 2023 •

edited

Loading

silverwind commented Jun 20, 2023

silverwind Jul 7, 2023

silverwind Jul 7, 2023

silverwind Jul 7, 2023

silverwind Jul 7, 2023

silverwind Jul 7, 2023

wxiaoguang commented Mar 13, 2025

wxiaoguang commented Mar 19, 2025 •

edited

Loading

kdumontnu commented Mar 19, 2025

silverwind commented Mar 19, 2025 •

edited

Loading

lunny commented Mar 19, 2025

silverwind commented Mar 19, 2025

wxiaoguang commented Mar 26, 2025 •

edited

Loading

silverwind commented Mar 26, 2025 •

edited

Loading

wxiaoguang commented Apr 23, 2025

wxiaoguang commented Apr 23, 2025

E2e Test Fixes #25058

E2e Test Fixes #25058

Conversation

kdumontnu commented Jun 2, 2023 • edited Loading

silverwind commented Jun 2, 2023

kdumontnu commented Jun 5, 2023

silverwind commented Jun 8, 2023 • edited Loading

silverwind commented Jun 20, 2023

silverwind Jul 7, 2023

Choose a reason for hiding this comment

silverwind Jul 7, 2023

Choose a reason for hiding this comment

silverwind Jul 7, 2023

Choose a reason for hiding this comment

silverwind Jul 7, 2023

Choose a reason for hiding this comment

silverwind Jul 7, 2023

Choose a reason for hiding this comment

wxiaoguang commented Mar 13, 2025

wxiaoguang commented Mar 19, 2025 • edited Loading

kdumontnu commented Mar 19, 2025

silverwind commented Mar 19, 2025 • edited Loading

lunny commented Mar 19, 2025

silverwind commented Mar 19, 2025

wxiaoguang commented Mar 26, 2025 • edited Loading

silverwind commented Mar 26, 2025 • edited Loading

wxiaoguang commented Apr 23, 2025

wxiaoguang commented Apr 23, 2025

kdumontnu commented Jun 2, 2023 •

edited

Loading

silverwind commented Jun 8, 2023 •

edited

Loading

wxiaoguang commented Mar 19, 2025 •

edited

Loading

silverwind commented Mar 19, 2025 •

edited

Loading

wxiaoguang commented Mar 26, 2025 •

edited

Loading

silverwind commented Mar 26, 2025 •

edited

Loading