Skip to content

Conversation

@davidsbailey
Copy link
Member

@davidsbailey davidsbailey commented Nov 6, 2025

Background

The impetus for this PR was to diagnose the following failure mode in drone ui tests: https://drone.cdn-code.org/code-dot-org/code-dot-org/43686/2/5

180 passed. 10 failed. Test count: 190. Duration: 11:21 minutes. Total successful reruns of flaky tests: 0.

many of the failures appear to have the same error message. here is one example: https://cucumber-logs.s3.amazonaws.com/circle/43686/Chrome_teacher_tools_authored_hints_output.html?versionId=qnbT33Vi_4tqllxkH1KPF82jCo8fBB3P

NoActiveTunnelForIdentifier - Misconfigured -- No active tunnel found for provided identifier (Selenium::WebDriver::Error::SessionNotCreatedError)

My hypothesis was that some saucelabs timeout was being exceeded and the tunnel was closing, due to the --first-run-local flag introducing a long delay between when the tunnel was set up and the firs time we tried to access it. I was unable to confirm this, so instead I'm resorting to turning on some debug logging in hopes of getting some more clues the next time this comes up.

docs: https://docs.saucelabs.com/dev/cli/sauce-connect-5/run/

Description

Done in this PR:

  • set sc debug log level to info
  • start dumping sc logs to drone output
  • increase open file limit, per this warning which is now showing up in the drone output
2025/11/07 16:44:21.958543 [control] [INFO] (server msg) The open file limit in your Operating System is too low. Sauce Labs recommends setting it to at least 16000. Please see https://docs.saucelabs.com/secure-connections/sauce-connect-5/system-requirements/ for more information.

Testing story

Follow-up work

  • At one point i tried setting --log-http proxy:error. On a test run with many failures, this produced about 20K lines of logging: https://drone.cdn-code.org/code-dot-org/code-dot-org/43795/2/5 . If the original problem recurs and the logging added in this PR doesn't turn up anything useful, we could consider turning on this http logging again, possibly grepping it to get just the url and status code:
2025/11/07 07:18:59.565444 [proxy] [INFO] [29-204cdf68] GET http://localhost-studio.code.org:3000/users/sign_in status=500 duration=1.030484046s

@davidsbailey davidsbailey changed the title extend sc tunnel idle timeout to avoid late failures in ui tests capture sauce connect debug logging in drone ui tests Nov 10, 2025
@davidsbailey davidsbailey marked this pull request as ready for review November 10, 2025 06:20
@davidsbailey davidsbailey requested a review from a team as a code owner November 10, 2025 06:20
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant