Skip to content

Conversation

@ycliu28
Copy link
Contributor

@ycliu28 ycliu28 commented Dec 11, 2022

Description

The test org.kohsuke.github.GHObjectTest.test_toString() is a flaky tests. It can pass mvn test but when run using the tool NonDex, it fails. NonDex is a tool that will introduce non-determinism in certain java collections. Here is the test result:

-------------------------------------------------------------------------------
Test set: org.kohsuke.github.GHObjectTest
-------------------------------------------------------------------------------
Tests run: 1, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.535 s <<< FAILURE! - in org.kohsuke.github.GHObjectTest
test_toString(org.kohsuke.github.GHObjectTest)  Time elapsed: 1.532 s  <<< FAILURE!
java.lang.AssertionError: 

Expected: a string containing "login=hub4j-test-org,location=<null>,blog=<null>,email=<null>,bio=<null>,name=<null>,company=<null>,type=Organization,followers=0,following=0,hireable=false"
     but: was "GHOrganization@7ce4de34[blog=<null>,hireable=false,company=<null>,login=hub4j-test-org,location=<null>,type=Organization,name=<null>,email=<null>,followers=0,bio=<null>,following=0,updatedAt=2015-04-20T00:42:30Z,createdAt=2014-05-10T19:39:11Z,url=http://localhost:42793/orgs/hub4j-test-org,nodeId=MDEyOk9yZ2FuaXphdGlvbjc1NDQ3Mzk=,id=7544739]"
	at org.hamcrest.MatcherAssert.assertThat(MatcherAssert.java:20)
	at org.kohsuke.github.AbstractGitHubWireMockTest.assertThat(AbstractGitHubWireMockTest.java:363)
	at org.kohsuke.github.GHObjectTest.test_toString(GHObjectTest.java:32)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
	at org.kohsuke.github.junit.WireMockMultiServerRule$1.evaluate(WireMockMultiServerRule.java:113)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.junit.runners.Suite.runChild(Suite.java:128)
	at org.junit.runners.Suite.runChild(Suite.java:27)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.apache.maven.surefire.junitcore.JUnitCore.run(JUnitCore.java:55)
	at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.createRequestAndRun(JUnitCoreWrapper.java:137)
	at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.executeEager(JUnitCoreWrapper.java:107)
	at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:83)
	at org.apache.maven.surefire.junitcore.JUnitCoreWrapper.execute(JUnitCoreWrapper.java:75)
	at org.apache.maven.surefire.junitcore.JUnitCoreProvider.invoke(JUnitCoreProvider.java:158)
	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:384)
	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:345)
	at org.apache.maven.surefire.booter.ForkedBooter.execute(ForkedBooter.java:126)
	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:418)

The cause of failure is that toString() method uses ReflectionToStringBuilder class, and some ToStringStyle will create strings where the fields are in a non-deterministic order. Since the goal of the test is not testing against a specific order, it can be fixed by separately assert the fields are in the result string.

Before submitting a PR:

  • Changes must not break binary backwards compatibility. If you are unclear on how to make the change you think is needed while maintaining backward compatibility, CONTRIBUTING.md for details.
  • Add JavaDocs and other comments as appropriate. Consider including links in comments to relevant documentation on https://docs.github.com/en/rest .
  • Add tests that cover any added or changed code. This generally requires capturing snapshot test data. See CONTRIBUTING.md for details.
  • Run mvn -D enable-ci clean install site locally. If this command doesn't succeed, your change will not pass CI.
  • Push your changes to a branch other than main. You will create your PR from that branch.

When creating a PR:

  • Fill in the "Description" above with clear summary of the changes. This includes:
    • If this PR fixes one or more issues, include "Fixes #" lines for each issue.
    • Provide links to relevant documentation on https://docs.github.com/en/rest where possible.
  • All lines of new code should be covered by tests as reported by code coverage. Any lines that are not covered must have PR comments explaining why they cannot be covered. For example, "Reaching this particular exception is hard and is not a particular common scenario."
  • Enable "Allow edits from maintainers".

@bitwiseman
Copy link
Member

@ycliu28
Please fix formatting, then we can apply.

@codecov
Copy link

codecov bot commented Jan 25, 2023

Codecov Report

Base: 79.22% // Head: 79.22% // No change to project coverage 👍

Coverage data is based on head (b61209e) compared to base (a222b97).
Patch has no changes to coverable lines.

Additional details and impacted files
@@            Coverage Diff            @@
##               main    #1576   +/-   ##
=========================================
  Coverage     79.22%   79.22%           
  Complexity     2141     2141           
=========================================
  Files           206      206           
  Lines          6576     6576           
  Branches        364      364           
=========================================
  Hits           5210     5210           
  Misses         1154     1154           
  Partials        212      212           

Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.

☔ View full report at Codecov.
📢 Do you have feedback about the report comment? Let us know in this issue.

@bitwiseman bitwiseman merged commit 3eb25e7 into hub4j:main Jan 25, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants