When running the tutorial, SELECT from the database happens before INSERT

[davidmoore-at-edge]

Hi there,

I'm brand new to Spring Batch, so I was following the tutorial at https://spring.io/guides/gs/batch-processing/ and it made sense for the most part. However, when I got to the point of actually running the code, I noticed that I was only seeing the output for when the data was processed, e.g.

Converting (firstName: Jill, lastName: Doe) into (firstName: JILL, lastName: DOE)

but not for when the data was read from the internal database, e.g.

Found <firstName: JILL, lastName: DOE> in the database.

This occurred regardless of whether I used "gradlew bootRun" or "gradlew build" followed by "java -jar build/libs/gs-batch-processing-0.1.0.jar." After adding in some debug statements, it appeared to me that the SELECT statement in Application.run() was getting called before the INSERT statements of BatchConfiguration.writer().

Do you have any suggestions for why this is happening?

In case it matters, I'm using Java version 1.8.0_20 on Windows 7 64-bit.

Thank you for any help you can provide!
David

[gregturn]

Thanks! Perhaps if I put a CountDownLatch, we can hold off checking the results until the batch is processes.

[davidmoore-at-edge]

That sounds like a good option! I figured it might have been an issue with the thread timing, but being new to Spring Batch (and also still pretty new to gradle) I wasn't sure. I'll see what I can do to implement that in my local version of the project and I'll keep an eye on this project in case you get it done before I do. Thanks!

[jjjesus]

Same thing. Because I'm a Spring n00b, I thought I missed something.

But, I configured logback for DEBUG on jdbc and get this as one of the first lines logged:

.10:16:38.854 [main] DEBUG o.s.jdbc.core.JdbcTemplate - Executing SQL query [SELECT first_name, last_name FROM people] 

followed much later by

.10:16:39.019 [main] DEBUG o.s.jdbc.core.JdbcTemplate - Executing SQL batch update [INSERT INTO people (first_name, last_name) VALUES (?, ?)] 

[gregturn]

@mminella Do you have time to update this guide with a latch?

[mminella]

@jjjesus or @davidmoore-at-edge Can you provide your code somewhere? By default, I believe SpringApplication#run will block until the batch job completes because the default JobLauncher provided executes jobs synchronously. When I run the code in the complete module of this guide it works without the CountDownLatch...

[davidmoore-at-edge]

I initially used the same code that's on the tutorial. What I wound up doing later based on another tutorial was to use a JobLauncher to run the job. Using the "@EnableBatchProcessing" annotation exposes the JobLauncher bean ("jobLauncher"), which you can then use to run the job. The code that we're using in our project has evolved quite a bit since I started looking at your tutorial, but I think setting up Application.java like this would allow the code to run without threading issues:

//packages and imports left out for brevity

//Note: might also need @ComponentScan, @EnableAutoConfiguration, and/or @Configuration
//I'm not sure what those do, but we have them in our project
@SpringBootApplication //we don't have this annotation in our project
public class Application {
// Note: we wound up not using the CommandLineRunner interface since doing so would have required us to implement the run() method.

    @Autowired
    JdbcTemplate jdbcTemplate;

    @Autowired
    JobLauncher jobLauncher;

    @Autowired
    Job importUserJob;

    public static void main(String[] args) {
        // Prevent auto-execution of batch jobs if you want to run these on demand via JobLauncher
        System.setProperty("spring.batch.job.enabled", "false");
        try {
            JobExecution execution = jobLauncher.run(importUserJob, new JobParameters());
            List<Person> results = jdbcTemplate.query("SELECT first_name, last_name FROM people", new RowMapper<Person>() {
                @Override
                public Person mapRow(ResultSet rs, int row) throws SQLException {
                    return new Person(rs.getString(1), rs.getString(2));
                }
            });

            if (results.isEmpty()) {
                System.out.println("SELECT first_name, last_name FROM people RETURNED 0 RESULTS");
            }

            for (Person person : results) {
                System.out.println("Found <" + person + "> in the database.");
            }
        } catch (Exception e) {
            //do exception handling here
        }
    }

Like I said, our project has evolved quite a bit, so this isn't exactly what we have, but it's close to it. I hope I haven't missed any details required to make this work, but give this a shot and see if it does the trick. I hope that helps!

[tsachev]

I hit the same issue.
The problem seems to be that CommandLineRunners are run depending on their order. But since JobLauncherCommandLineRunner neither implements Ordered or have @Order annotation it implicitly gets Ordered.LOWEST_PRECEDENCE. So does the Application which is a CommandLineRunner too.

There is no guarantee which one will run first in this case.

Created an issue about this spring-projects/spring-boot#2943.

I was able to workaround it be overriding the orders with this code

package hello;

import java.util.List;

import org.springframework.batch.core.explore.JobExplorer;
import org.springframework.batch.core.launch.JobLauncher;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.boot.CommandLineRunner;
import org.springframework.boot.SpringApplication;
import org.springframework.boot.autoconfigure.SpringBootApplication;
import org.springframework.boot.autoconfigure.batch.JobLauncherCommandLineRunner;
import org.springframework.context.annotation.Bean;
import org.springframework.core.Ordered;
import org.springframework.core.annotation.Order;
import org.springframework.jdbc.core.JdbcTemplate;

@SpringBootApplication
@Order(Ordered.LOWEST_PRECEDENCE)
public class Application implements CommandLineRunner {

    public static void main(String[] args) {
        SpringApplication.run(Application.class, args);
    }

    @Autowired
    JdbcTemplate jdbcTemplate;

    @Override
    public void run(String... args) throws Exception {
        List<Person> results = jdbcTemplate.query(
                "select first_name, last_name from people",
                (rs, rowNum) -> new Person(rs.getString(1), rs.getString(2)));
        results.forEach((person) -> System.out.println("Found <" + person + "> in the database."));
    }

    @Bean
    public JobLauncherCommandLineRunner jobLauncherCommandLineRunner(
            JobLauncher jobLauncher, JobExplorer jobExplorer) {
        class OrderedJobLauncherCommandLineRunner extends JobLauncherCommandLineRunner implements Ordered
        {

            public OrderedJobLauncherCommandLineRunner() {
                super(jobLauncher, jobExplorer);
            }

            @Override
            public int getOrder() {
                return Ordered.HIGHEST_PRECEDENCE;
            }

        }
        return new OrderedJobLauncherCommandLineRunner();
    }
}