Non-deterministic test: BackpressureTest.testFlatMapAsync

[davidmoten]

Failed 2 out of 3 times when running full suite in Eclipse on slow 2 core laptop. Passes every time when run on its own.

java.lang.AssertionError
    at org.junit.Assert.fail(Assert.java:92)
    at org.junit.Assert.assertTrue(Assert.java:43)
    at org.junit.Assert.assertTrue(Assert.java:54)
    at rx.BackpressureTests.testFlatMapAsync(BackpressureTests.java:185)

[akarnokd]

That test annoys me as well because a slight threading delay makes it fail. You could post a PR which puts an @ignore on it.

[davidmoten]

Can do. I haven't looked at the test yet but might it benefit by starting with a Schedulers.reset() (revive #1769?) . I like the addition of reset() to the API for test isolation.

[akarnokd]

The problem is not the schedulers, but the non-deterministic thread stalls. On my Windows machine, a looped test fails 5% of the time. On travis, the same test fails 35% of the time. I was thinking about calling GC and sleeping for some milliseconds, but then the same issue might come back because processor C-states and threadpool spinups can still stall execution.

[davidmoten]

Interesting, I love how educational this project is. Tis for me anyway.

Erk, added @ignore, ran the suite on slow laptop and

java.lang.AssertionError: expected:<0> but was:<3>
    at org.junit.Assert.fail(Assert.java:93)
    at org.junit.Assert.failNotEquals(Assert.java:647)
    at org.junit.Assert.assertEquals(Assert.java:128)
    at org.junit.Assert.assertEquals(Assert.java:472)
    at org.junit.Assert.assertEquals(Assert.java:456)
    at rx.internal.operators.OperatorRetryTest.testRetryWithBackpressureParallel(OperatorRetryTest.java:759)

[davidmoten]

And

java.lang.Exception: test timed out after 10000 milliseconds
    at java.lang.Thread.interrupted(Thread.java:944)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.doAcquireSharedNanos(AbstractQueuedSynchronizer.java:1038)
    at java.util.concurrent.locks.AbstractQueuedSynchronizer.tryAcquireSharedNanos(AbstractQueuedSynchronizer.java:1328)
    at java.util.concurrent.CountDownLatch.await(CountDownLatch.java:277)
    at rx.observers.TestSubscriber.awaitTerminalEvent(TestSubscriber.java:229)
    at rx.internal.operators.OperatorRetryTest.testRetryWithBackpressure(OperatorRetryTest.java:705)

[akarnokd]

For the first failure, could you check PR #2563 and see what is the exact cause of failure.

[davidmoten]

Running PR got these for the record. I'll try again for flatmap async failure

java.lang.RuntimeException: Unexpected onError events: 1
    at rx.observers.TestSubscriber.assertNoErrors(TestSubscriber.java:195)
    at rx.internal.operators.OnSubscribeCombineLatestTest.testBackpressure(OnSubscribeCombineLatestTest.java:803)
    at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
    at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
    at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:483)
    at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:45)
    at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:15)
    at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:42)
    at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:20)
    at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:263)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:68)
    at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:47)
    at org.junit.runners.ParentRunner$3.run(ParentRunner.java:231)
    at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:60)
    at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:229)
    at org.junit.runners.ParentRunner.access$000(ParentRunner.java:50)
    at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:222)
    at org.junit.runners.ParentRunner.run(ParentRunner.java:300)
    at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:50)
    at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:459)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:675)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:382)
    at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:192)
Caused by: java.lang.IllegalArgumentException: number requested cannot be negative: -1
    at rx.Subscriber.request(Subscriber.java:100)
    at rx.internal.operators.OnSubscribeCombineLatest$MultiSourceRequestableSubscriber.requestUpTo(OnSubscribeCombineLatest.java:234)
    at rx.internal.operators.OnSubscribeCombineLatest$MultiSourceProducer.tick(OnSubscribeCombineLatest.java:161)
    at rx.internal.operators.OnSubscribeCombineLatest$MultiSourceProducer.onNext(OnSubscribeCombineLatest.java:209)
    at rx.internal.operators.OnSubscribeCombineLatest$MultiSourceRequestableSubscriber.onNext(OnSubscribeCombineLatest.java:252)
    at rx.internal.operators.OnSubscribeRange$RangeProducer.request(OnSubscribeRange.java:93)
    at rx.Subscriber.setProducer(Subscriber.java:152)
    at rx.internal.operators.OnSubscribeRange.call(OnSubscribeRange.java:39)
    at rx.internal.operators.OnSubscribeRange.call(OnSubscribeRange.java:1)
    at rx.Observable.unsafeSubscribe(Observable.java:7306)
    at rx.internal.operators.OnSubscribeCombineLatest$MultiSourceProducer.request(OnSubscribeCombineLatest.java:131)
    at rx.Subscriber.setProducer(Subscriber.java:152)
    at rx.internal.operators.OnSubscribeCombineLatest.call(OnSubscribeCombineLatest.java:68)
    at rx.internal.operators.OnSubscribeCombineLatest.call(OnSubscribeCombineLatest.java:1)
    at rx.Observable$1.call(Observable.java:146)
    at rx.Observable$1.call(Observable.java:1)
    at rx.Observable.subscribe(Observable.java:7396)
    at rx.internal.operators.OnSubscribeCombineLatestTest.testBackpressure(OnSubscribeCombineLatestTest.java:800)
    ... 23 more

and

java.lang.AssertionError
    at org.junit.Assert.fail(Assert.java:92)
    at org.junit.Assert.assertTrue(Assert.java:43)
    at org.junit.Assert.assertTrue(Assert.java:54)
    at rx.internal.operators.OperatorSerializeTest.testMultiThreadedWithNPEinMiddle(OperatorSerializeTest.java:134)

[akarnokd]

testMultiThreadedWithNPEinMiddle fails because sometimes instantiating an exception class takes longer and thus all regular events rush ahead.

Caused by: java.lang.IllegalArgumentException: number requested cannot be negative: -1
    at rx.internal.operators.OnSubscribeCombineLatest$MultiSourceRequestableSubscriber.requestUpTo(OnSubscribeCombineLatest.java:234)

in rx.internal.operators.OnSubscribeCombineLatestTest.testBackpressure looks like a genuine backpressure bug. I'll take a look at it.

[davidmoten]

The wind has changed and after ten suite runs in Eclipse with one Eclipse restart in the middle just for fun I can't repeat the flatMapAsync issue (although 3 suites failed with already mentioned problems).

[akarnokd]

I managed to get rx.internal.operators.OnSubscribeCombineLatestTest.testBackpressure fail once per 200k runs so I can't be sure fix #2566 works. Could you verify it?

[davidmoten]

Haven't seen it fail lately

[akarnokd]

Yep, because it has been @Ignored.