Document how to make Spring Batch work with Spring Data JPA repositories [BATCH-2642]

[spring-projects-issues]

Petr Dvorak opened BATCH-2642 and commented

We are using Spring Boot and we were struggling with a following issue: We needed to use our Spring Data JPA repositories (interfaces extending 'CrudRepository') in a Spring Batch writer and for some reason, the 'save(s)' method did not work for us - method got called, no exception or error was in the log, but the object was not persisted. In principle, it was exactly the same issue as the one reported here:

https://stackoverflow.com/questions/38287298/persist-issue-with-a-spring-batch-itemwriter-using-a-jpa-repository

The offending part was this one:

@Component
public class MessagesDigestMailerItemWriter implements ItemWriter<UserAccount> {

    @Autowired
    private MessageRepository messageRepository;

    @Autowired
    private MailerService mailerService;

    @Override
    public void write(List<? extends UserAccount> userAccounts) throws Exception {

        for (UserAccount userAccount : userAccounts) {
            if (userAccount.isEmailNotification()) {
                mailerService.mailMessagesDigest(userAccount);
            }
            for (Message message : userAccount.getReceivedMessages()) {
                message.setNotificationSent(true);
                messageRepository.save(message); //NOT SAVING!!
            }
        }
    }
}

We finally fixed the issue by adding following code:

@Configuration
public class JpaConfig {

    private final DataSource dataSource;

    @Autowired
    public JpaConfig(@Qualifier("dataSource") DataSource dataSource) {
        this.dataSource = dataSource;
    }

    @Bean
    @Primary
    public JpaTransactionManager jpaTransactionManager() {
        final JpaTransactionManager transactionManager = new JpaTransactionManager();
        transactionManager.setDataSource(dataSource);
        return transactionManager;
    }

}

... and later, using autowired transaction manager for the step config:

@Autowired
private PlatformTransactionManager transactionManager;

private TaskletStep buildTaskletStep() {
        return stepBuilderFactory.get("SendCampaignStep")
                    .<UserAccount, UserAccount>chunk(pushServiceConfiguration.getCampaignBatchSize())
                    .reader(userAccountItemReader)
                    .processor(userAccountItemProcessor)
                    .writer(userAccountItemWriter)
                    .transactionManager(transactionManager)
                    .build();
    }
}

... which replaces the default DataSourceTransactionManager in our extension of DefaultBatchConfigurer - suddenly, the data is written in the repository correctly.

However, the whole fix feels a bit magical and we could use some official documentation on how to use Spring Boot Data JPA repositories inside Spring Batch writer (or other Spring Batch contexts).

---

Affects: 3.0.9, 4.0.1

Issue Links:

• BATCH-2294 Overriding transaction management
  ("depends on")

Referenced from: pull request #620

Backported to: 4.1.0.M3

[spring-projects-issues]

Michael Minella commented

Your "fix" isn't actually correct. We'd expect you to implement a BatchConfigurer that returns the JpaTransactionManager. We're in the process of updating the documentation so that it's more clear how this should behave.

[spring-projects-issues]

Petr Dvorak commented

Thank you for the pointer to BatchConfigurer. I was hoping I can save some time by using DefaultBatchConfigurer but of course I can just use if for inspiration and provide implementation that returns the JpaTransactionManager myself. The documentation would be nice, though...

[spring-projects-issues]

Mahmoud Ben Hassine commented

The issue here is the same as BATCH-2294 and BATCH-2724 : Not being able to provide a JpaTransactionManager results in unexpected results (like not saving objects here or incorrect behaviour in BATCH-2724).

This issue would be fixed once we are able to provide a JpaTransactionManager by subclassing DefaulBatchConfigurer and override getTransactionManager. There is an open PR to update the documentation: https://github.com/spring-projects/spring-batch/pull/620/files which would resolve this JIRA Ticket (of type "Documenation"). But the example would work only once BATCH-2294 is resolved.

I'm linking this to BATCH-2294 and adding it to v4.1.0.M3.

[spring-projects-issues]

Mahmoud Ben Hassine commented

The documentation has been updated with an example of how to provide a custom BatchConfigurer (See https://github.com/spring-projects/spring-batch/pull/620/files).

In the case of this issue, it would be something like:

@Bean
 public BatchConfigurer batchConfigurer() {
 	return new DefaultBatchConfigurer() {
 		@Override
 		public PlatformTransactionManager getTransactionManager() {
 			return jpaTransactionManager();
 		}
 	};
 }

[kdhindsa]

@benas
I am facing the exact same problem, but I am getting new kinds of errors now. I wasn't able to find documentation related to the following error messages.

I tried creating a BatchConfigurer, but I am getting an exception.

  @Bean
  public BatchConfigurer batchConfigurer(@Qualifier("dataSource") DataSource dataSource,
      EntityManagerFactory entityManagerFactory) {

    return new DefaultBatchConfigurer() {
      @Override
      public PlatformTransactionManager getTransactionManager() {
        JpaTransactionManager transactionManager = new JpaTransactionManager();
        transactionManager.setDataSource(dataSource);
        transactionManager.setEntityManagerFactory(entityManagerFactory);
        return transactionManager;
      }
    };
  }

java.lang.IllegalStateException: Failed to execute ApplicationRunner
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:789) [spring-boot-2.3.0.RELEASE.jar:2.3.0.RELEASE]
	at org.springframework.boot.SpringApplication.callRunners(SpringApplication.java:776) [spring-boot-2.3.0.RELEASE.jar:2.3.0.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:322) [spring-boot-2.3.0.RELEASE.jar:2.3.0.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1237) [spring-boot-2.3.0.RELEASE.jar:2.3.0.RELEASE]
	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1226) [spring-boot-2.3.0.RELEASE.jar:2.3.0.RELEASE]
	at com.linkedin.questingestor.QuestIngestorApplication.main(QuestIngestorApplication.java:13) [classes/:na]
Caused by: org.springframework.transaction.InvalidIsolationLevelException: DefaultJpaDialect does not support custom isolation levels due to limitations in standard JPA. Specific arrangements may be implemented in custom JpaDialect variants.
	at org.springframework.orm.jpa.DefaultJpaDialect.beginTransaction(DefaultJpaDialect.java:66) ~[spring-orm-5.2.6.RELEASE.jar:5.2.6.RELEASE]
	at org.springframework.orm.jpa.JpaTransactionManager.doBegin(JpaTransactionManager.java:402) ~[spring-orm-5.2.6.RELEASE.jar:5.2.6.RELEASE]
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.startTransaction(AbstractPlatformTransactionManager.java:400) ~[spring-tx-5.2.6.RELEASE.jar:5.2.6.RELEASE]
	at org.springframework.transaction.support.AbstractPlatformTransactionManager.getTransaction(AbstractPlatformTransactionManager.java:373) ~[spring-tx-5.2.6.RELEASE.jar:5.2.6.RELEASE]
	at org.springframework.transaction.interceptor.TransactionAspectSupport.createTransactionIfNecessary(TransactionAspectSupport.java:572) ~[spring-tx-5.2.6.RELEASE.jar:5.2.6.RELEASE]
	at org.springframework.transaction.interceptor.TransactionAspectSupport.invokeWithinTransaction(TransactionAspectSupport.java:360) ~[spring-tx-5.2.6.RELEASE.jar:5.2.6.RELEASE]
	at org.springframework.transaction.interceptor.TransactionInterceptor.invoke(TransactionInterceptor.java:118) ~[spring-tx-5.2.6.RELEASE.jar:5.2.6.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.6.RELEASE.jar:5.2.6.RELEASE]
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.6.RELEASE.jar:5.2.6.RELEASE]
	at com.sun.proxy.$Proxy60.getLastJobExecution(Unknown Source) ~[na:na]
	at org.springframework.batch.core.launch.support.SimpleJobLauncher.run(SimpleJobLauncher.java:104) ~[spring-batch-core-4.2.2.RELEASE.jar:4.2.2.RELEASE]
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) ~[na:1.8.0_172]
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) ~[na:1.8.0_172]
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) ~[na:1.8.0_172]
	at java.lang.reflect.Method.invoke(Method.java:498) ~[na:1.8.0_172]
	at org.springframework.aop.support.AopUtils.invokeJoinpointUsingReflection(AopUtils.java:344) ~[spring-aop-5.2.6.RELEASE.jar:5.2.6.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.invokeJoinpoint(ReflectiveMethodInvocation.java:198) ~[spring-aop-5.2.6.RELEASE.jar:5.2.6.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163) ~[spring-aop-5.2.6.RELEASE.jar:5.2.6.RELEASE]
	at org.springframework.batch.core.configuration.annotation.SimpleBatchConfiguration$PassthruAdvice.invoke(SimpleBatchConfiguration.java:127) ~[spring-batch-core-4.2.2.RELEASE.jar:4.2.2.RELEASE]
	at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186) ~[spring-aop-5.2.6.RELEASE.jar:5.2.6.RELEASE]
	at org.springframework.aop.framework.JdkDynamicAopProxy.invoke(JdkDynamicAopProxy.java:212) ~[spring-aop-5.2.6.RELEASE.jar:5.2.6.RELEASE]
	at com.sun.proxy.$Proxy68.run(Unknown Source) ~[na:na]
	at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.execute(JobLauncherApplicationRunner.java:199) ~[spring-boot-autoconfigure-2.3.0.RELEASE.jar:2.3.0.RELEASE]
	at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.executeLocalJobs(JobLauncherApplicationRunner.java:173) ~[spring-boot-autoconfigure-2.3.0.RELEASE.jar:2.3.0.RELEASE]
	at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.launchJobFromProperties(JobLauncherApplicationRunner.java:160) ~[spring-boot-autoconfigure-2.3.0.RELEASE.jar:2.3.0.RELEASE]
	at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:155) ~[spring-boot-autoconfigure-2.3.0.RELEASE.jar:2.3.0.RELEASE]
	at org.springframework.boot.autoconfigure.batch.JobLauncherApplicationRunner.run(JobLauncherApplicationRunner.java:150) ~[spring-boot-autoconfigure-2.3.0.RELEASE.jar:2.3.0.RELEASE]
	at org.springframework.boot.SpringApplication.callRunner(SpringApplication.java:786) [spring-boot-2.3.0.RELEASE.jar:2.3.0.RELEASE]
	... 5 common frames omitted

2020-07-03 13:21:11.331  INFO 51400 --- [           main] j.LocalContainerEntityManagerFactoryBean : Closing JPA EntityManagerFactory for persistence unit 'default'
2020-07-03 13:21:11.332  INFO 51400 --- [           main] o.s.s.concurrent.ThreadPoolTaskExecutor  : Shutting down ExecutorService 'applicationTaskExecutor'
2020-07-03 13:21:11.333  INFO 51400 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown initiated...
2020-07-03 13:21:11.821  INFO 51400 --- [           main] com.zaxxer.hikari.HikariDataSource       : HikariPool-1 - Shutdown completed.

[fmbenhassine]

@kdhindsa Please ask your question on SO. The error DefaultJpaDialect does not support custom isolation levels due to limitations in standard JPA. Specific arrangements may be implemented in custom JpaDialect variants. does not seem to be related to this issue here.