Describe quirks of JUL and Log4j2 in the javadoc of OutputCaptureExtension

[jianlinshi]

Hi, I ran into this problem below:

For each test, it passes. But if I run the test class, it always failed on the 2nd. Any ideas?

Thanks

import org.junit.jupiter.api.Test;
import org.junit.jupiter.api.extension.ExtendWith;
import org.springframework.boot.logging.AbstractLoggingSystemTests;
import org.springframework.boot.test.system.CapturedOutput;
import org.springframework.boot.test.system.OutputCaptureExtension;

import java.util.logging.Logger;

import static org.assertj.core.api.Assertions.assertThat;

@ExtendWith(OutputCaptureExtension.class)
public class TestCapture extends AbstractLoggingSystemTests {
    static Logger LOG=Logger.getLogger(TestCapture.class.getName());


    @Test
    void test(CapturedOutput output) {
        System.out.println("ok");
        LOG.warning("error");
        assertThat(output).contains("ok");
        assertThat(output).contains("error");

    }

    @Test
    void test2(CapturedOutput output) {
        LOG.info("not ok");
        LOG.warning("type 1 error");
        assertThat(output).contains("not");
        assertThat(output).contains("type 1");
    }

}

[philwebb]

We've seen issues relating to log configuration in the past, for example #27902 (comment).

Can you please share a sample application that replicates the issue so we can see exactly how your logs are being configured.

[jianlinshi]

Thanks for the quick response. I didn't use log4j. The code above is able to replicate the issue.

[philwebb]

Although the snippet above might replicate the issue, it would be best if we could have a sample application. That way we know that we're aligned on logging configurations and dependency versions.

[jianlinshi]

https://github.com/jianlinshi/nlp-etl-runner

[philwebb]

Thanks for the sample. The problem is with java.util.logging.StreamHandler which is caching the output OutputStream. With a ConsoleHandler the output is a OutputStreamWriter wrapping System.err.

Our OutputCaptureExtension works by replacing System.err when the test runs. If you run one test at a time things are fine, but when you run both tests the second one doesn't fetch System.err again so the captured output is sent to the wrong place.

To get things working, you can add the following to your tests:

@AfterEach
void reset() throws Exception {
	LogManager.getLogManager().readConfiguration();
}

That will force all handles to get closed so the next run will capture the correct System.err.

[philwebb]

Flagging for team attention to see if we should try and do this automatically in the OutputCaptureExtension. I'm not sure we should because the extension really shouldn't know about any specific loggers.