배경 : 스프링배치 7주차 스터디 6주차 미흡한 과제 이행으로 인한 정리를 진행함. (7주차 = 좀 더 열심히 하는 주)
참고 : 7주차 교재 (+ 6주차 교재)
(아래 글은 한국 스프링 사용자 모임(KSUG)에서 진행된 스프링 배치 스터디 내용을 정리한 게시글입니다.
DEVOCEAN에 연재 중인 KIDO님의 글을 참고하여 실습한 내용을 기록했습니다.)
[SpringBatch 연재 07] MyBatisPagingItemReader로 DB내용을 읽고, MyBatisItemWriter로 DB에 쓰기
devocean.sk.com
[SpringBatch 연재 06] JpaPagingItemReader로 DB내용을 읽고, JpaItemWriter로 DB에 쓰기
devocean.sk.com
내용
미진한 6주 간단 정리
[요약] - ChatGPT
`JpaPagingItemReader`는 Spring Batch에서 JPA를 사용하여 데이터베이스로부터 데이터를 페이지 단위로 읽는 기능을 제공하는 요소입니다. JPA의 다양한 기능을 활용하여 최적화된 데이터 읽기, 객체 매핑 자동화, 커서 제어 등을 수행하며, 주요 구성 요소로는 `EntityManagerFactory`, `JpaQueryProvider`, `PageSize` 등이 있습니다. 또한, 로그 출력을 위한 단순 `ItemProcessor`와 함께 사용되어 입력받은 데이터를 처리하지 않고 단순 출력만 수행하는 예제도 포함되어 있습니다. 반면, `JpaItemWriter`는 JPA를 통해 데이터베이스에 데이터를 저장하는 역할을 하며, 설정 복잡성 및 데이터베이스 종속성을 고려해야 하지만, ORM 연동의 장점이 있습니다.
[혼잣말]
1.아 6주차에서는 5주차에 배운 Paging 관련 작업들을 JPA 환경에서 사용하는 법을 배웠었지..
2.JPA를 사용하기 위해서 build.gradle에 추가해야하는 항목도 공유해주셨구나 ... 샘플코드도..
spring-batch-tutorials/06.03.JpaPagingItemReaderSample/build.gradle at 10.01.FlowControl · schooldevops/spring-batch-tutorials
Contribute to schooldevops/spring-batch-tutorials development by creating an account on GitHub.
github.com
spring-batch-tutorials/08.03.JpaItemWriterSample/build.gradle at 10.01.FlowControl · schooldevops/spring-batch-tutorials
Contribute to schooldevops/spring-batch-tutorials development by creating an account on GitHub.
github.com
일단... 6주차 내용 더 잡다보면 7주차 못할 것 같아서, 우선 7주차에 집중해보자.
7주차 메인 내용 정리
[요약] - ChatGPT
MyBatis는 여전히 Spring Java 프로젝트에서 인기 있는 ORM 도구로, 직관적인 사용법과 DB Tool을 통한 쿼리 실행이 장점입니다. MyBatis를 Spring Batch에서 사용하는 방법으로 MyBatisPagingItemReader를 통해 데이터를 읽고, MyBatisBatchItemWriter를 통해 데이터를 저장합니다. MyBatisPagingItemReader는 설정이 간편하며 쿼리 최적화와 동적 쿼리 생성 장점을 갖고 있지만, MyBatis 의존성과 커스터마이징의 복잡성이 단점입니다.
Mybatis 란?
[참고]
https://velog.io/@cyseok123/MyBatis
[MyBatis] MyBatis란? 개념 및 구조
Mybatis는 자바 오브젝트와 SQL사이의 자동 매핑 기능을 지원하는 ORM(Object relational Mapping)프레임워크이다.
velog.io
위 블로그가 너무 자세하게 잘 설명되어 있다.
기존 java 단에서 JDBC 연결을 위해서는
1. connection 맺고 --> 2. select 명령 전달 실행 --> 3. resultSet 객체에 저장 --> 4. rs.next()를 이용, resultSet 처리행을 하나씩 이동시켜 받아오는 구조
위 복잡한 프로세스로 구현하는 것 대신 프로그래밍 코드와 SQL문 코드의 분리로 이식성이 좋다.
파라미터와 결과를 객체(DTO등) 으로 자동 매핑 지원, Spring 설정 간단
요 멘트정도로 정리하려 한다.
특히.. 수도없이 많이 Mybatis 코드를 만들어왔지만, 항상 복사 붙여넣기 형태로 해왔다. 위 블로그에는 구현 순서가 적혀 있어서, 나도 공부할겸 같이 적어본다.
Mybatis 구현 순서 (코드 작성 순서)
1. DB 테이블 생성 및 설정
2. 도메인 객체 (ex) DTO) 설계 및 클래스 작성
3. DAO 인터페이스 / 실행 가능 인터페이스 정의
4. XML Mapper 생성 및 SQL문 작성
5. Mybatis에 작성한 XML Mapper 인식 설정 (Spring과 연동)
6. DAO 인터페이스 구현한 클래스 작성
7. 스프링에 DAO 등록
MyBatisPagingItemReader
MyBatisPagingItemReader는 Spring Batch와 MyBatis를 연동하여 대용량 데이터를 효율적으로 처리하기 위한 ItemReader 구현체입니다. 이 클래스는 페이징 기법을 사용하여 데이터베이스에서 데이터를 읽어오는 역할을 합니다
특징
1. 페이징 처리: MyBatisPagingItemReader는 데이터를 페이지 단위로 읽어옵니다. 이는 대규모 데이터 집합을 처리할 때 메모리 사용을 최적화하고 성능을 향상시킵니다
2. MyBatis 통합: MyBatis의 Object Relation Mapper를 활용하여 데이터베이스 쿼리 결과를 Java 객체로 매핑합니다
3. 간편한 설정: MyBatis 쿼리 매퍼를 직접 활용할 수 있어 설정이 비교적 간단합니다
4. 쿼리 최적화: MyBatis의 다양한 기능을 활용하여 최적화된 쿼리를 작성할 수 있습니다
5. 동적 쿼리 지원: 런타임 시 조건에 따라 동적으로 쿼리를 생성할 수 있습니다
구성요소
(원본 교제 그대로 타이핑해서 쓰고 정독하였음)
- SqlSessionFactory: MyBatis 설정 정보 및 SQL 쿼리 매퍼 정보를 담고 있는 객체이다.
- QueryId: 데이터를 읽을 MyBatis 쿼리 ID이다.
- PageSize: 페이징 쿼리를 위한 페이지 크기를 지정한다.
1. SqlSessionFactory
- MyBatisPagingItemReader SqlSessionFactory 객체를 통해 MyBatis와 연동된다.
- SqlSessionFactory는 다음과 같은 방법으로 설정할 수 있다.
- @Bean 어노테이션을 사용하여 직접 생성
- Spring Batch XML 설정 파일에서 설정
- Java 코드에서 직접 설정
2. QueryId
- MyBatisPagingItemReader setQueryId() 메소드를 통해 데이터를 읽을 MyBatis 쿼리 ID를 설정한다.
- 쿼리 ID는 com.example.mapper.CustomerMapper.selectCustomers 와 같은 형식으로 지정된다.
3. PageSize
- MyBatisItemReader는 pageSize를 이용하여 offset, limit 을 이용하는 기준을 설정할 수 있다.
4. 추가 구성요소
- SkippableItemReader: 오류 발생 시 해당 Item을 건너뛸 수 있도록 한다.
- ReadListener: 읽기 시작, 종료, 오류 발생 등의 이벤트를 처리할 수 있도록 한다.
- SaveStateCallback: 잡의 중단 시 현재 상태를 저장하여 재시작 시 이어서 처리할 수 있도록 한다.
샘플코드
build.gradle (mybatis 사용 위한 사전 필수)
dependencies {
...
//mybatis
implementation 'org.mybatis.spring.boot:mybatis-spring-boot-starter:3.0.3'
implementation 'org.mybatis:mybatis:3.5.13'
implementation 'org.mybatis:mybatis-spring:2.1.0'
}
Customer 작성
Customer.java (기존 대비 id (Seq) 추가 )
package com.ksko.spring_batch.batch_sample.jobs.models;
import lombok.Data;
@Data
public class Customer {
private Long id;
private String name;
private int age;
private String gender;
private String grade; //준수님 DB 깔 맞추기위해 추가
}
query 작성
customer.xml
<?xml version="1.0" encoding="UTF-8" ?>
<!DOCTYPE mapper PUBLIC "-//mybatis.org//DTD Mapper 3.0//EN"
"http://mybatis.org/dtd/mybatis-3-mapper.dtd">
<mapper namespace="com.ksko.spring_batch.batch_sample.jobs">
<resultMap id="customerResult" type="com.ksko.spring_batch.batch_sample.jobs.models.Customer">
<result property="id" column="id"></result>
<result property="name" column="name"></result>
<result property="age" column="age"></result>
<result property="gender" column="gender"></result>
</resultMap>
<select id="selectCustomers" resultMap="customerResult">
SELECT ID
, NAME
, AGE
, GENDER
FROM CUSTOMER
LIMIT #{_skiprows}, #{_pagesize}
<!-- #{_skiprows}: MyBatis가 제공하는 매개변수로, 건너뛸 행(row)의 수를 지정
#{_pagesize}: 한 페이지에 가져올 행(row)의 수를 지정합니다. -->
</select>
</mapper>
(DB 내 생성된 테이블 명이 대문자일 경우 인식을 못하기도 했다. 준수님 공용 테스트 DB 에 맞춰서 했다.)
실행 결과 로그
2024-11-19T16:56:59.110+09:00 INFO 35984 --- [ main] c.k.s.b.BatchSampleApplication : Starting BatchSampleApplication using Java 21.0.4 with PID 35984 (D:\ksko\workspace\spring-batch-study\build\classes\java\main started by 202010011 in D:\ksko\workspace\spring-batch-study)
2024-11-19T16:56:59.112+09:00 INFO 35984 --- [ main] c.k.s.b.BatchSampleApplication : No active profile set, falling back to 1 default profile: "default"
2024-11-19T16:56:59.428+09:00 WARN 35984 --- [ main] o.m.s.mapper.ClassPathMapperScanner : No MyBatis mapper was found in '[com.ksko.spring_batch.batch_sample]' package. Please check your configuration.
2024-11-19T16:56:59.525+09:00 WARN 35984 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.boot.autoconfigure.jdbc.DataSourceConfiguration$Hikari' of type [org.springframework.boot.autoconfigure.jdbc.DataSourceConfiguration$Hikari] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-19T16:56:59.546+09:00 WARN 35984 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'spring.datasource-org.springframework.boot.autoconfigure.jdbc.DataSourceProperties' of type [org.springframework.boot.autoconfigure.jdbc.DataSourceProperties] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-19T16:56:59.547+09:00 WARN 35984 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration$PooledDataSourceConfiguration' of type [org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration$PooledDataSourceConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-19T16:56:59.547+09:00 WARN 35984 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'jdbcConnectionDetails' of type [org.springframework.boot.autoconfigure.jdbc.PropertiesJdbcConnectionDetails] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-19T16:56:59.559+09:00 WARN 35984 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'dataSource' of type [com.zaxxer.hikari.HikariDataSource] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-19T16:56:59.562+09:00 WARN 35984 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.boot.autoconfigure.jdbc.DataSourceTransactionManagerAutoConfiguration$JdbcTransactionManagerConfiguration' of type [org.springframework.boot.autoconfigure.jdbc.DataSourceTransactionManagerAutoConfiguration$JdbcTransactionManagerConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-19T16:56:59.565+09:00 WARN 35984 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.boot.autoconfigure.transaction.TransactionManagerCustomizationAutoConfiguration' of type [org.springframework.boot.autoconfigure.transaction.TransactionManagerCustomizationAutoConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-19T16:56:59.567+09:00 WARN 35984 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'transactionExecutionListeners' of type [org.springframework.boot.autoconfigure.transaction.ExecutionListenersTransactionManagerCustomizer] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-19T16:56:59.569+09:00 WARN 35984 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'spring.transaction-org.springframework.boot.autoconfigure.transaction.TransactionProperties' of type [org.springframework.boot.autoconfigure.transaction.TransactionProperties] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-19T16:56:59.569+09:00 WARN 35984 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'platformTransactionManagerCustomizers' of type [org.springframework.boot.autoconfigure.transaction.TransactionManagerCustomizers] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-19T16:56:59.572+09:00 WARN 35984 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'transactionManager' of type [org.springframework.jdbc.support.JdbcTransactionManager] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-19T16:56:59.573+09:00 WARN 35984 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'spring.batch-org.springframework.boot.autoconfigure.batch.BatchProperties' of type [org.springframework.boot.autoconfigure.batch.BatchProperties] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-19T16:56:59.578+09:00 WARN 35984 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.boot.autoconfigure.batch.BatchAutoConfiguration$SpringBootBatchConfiguration' of type [org.springframework.boot.autoconfigure.batch.BatchAutoConfiguration$SpringBootBatchConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). The currently created BeanPostProcessor [jobRegistryBeanPostProcessor] is declared through a non-static factory method on that class; consider declaring it as static instead.
2024-11-19T16:56:59.714+09:00 INFO 35984 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2024-11-19T16:57:00.069+09:00 INFO 35984 --- [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Added connection com.mysql.cj.jdbc.ConnectionImpl@1d4fb213
2024-11-19T16:57:00.070+09:00 INFO 35984 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2024-11-19T16:57:00.100+09:00 INFO 35984 --- [ main] c.k.s.b.j.m.MyBatisReaderJobConfig : ------------------ Init customerJdbcCursorStep -----------------
2024-11-19T16:57:00.124+09:00 INFO 35984 --- [ main] c.k.s.b.j.m.MyBatisReaderJobConfig : ------------------ Init customerJdbcCursorPagingJob -----------------
2024-11-19T16:57:00.225+09:00 INFO 35984 --- [ main] c.k.s.b.BatchSampleApplication : Started BatchSampleApplication in 1.378 seconds (process running for 1.851)
2024-11-19T16:57:00.227+09:00 INFO 35984 --- [ main] o.s.b.a.b.JobLauncherApplicationRunner : Running default command line with: []
2024-11-19T16:57:00.800+09:00 INFO 35984 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=MYBATIS_CHUNK_JOB]] launched with the following parameters: [{'run.id':'{value=2, type=class java.lang.Long, identifying=true}'}]
2024-11-19T16:57:01.103+09:00 INFO 35984 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [customerJdbcCursorStep]
2024-11-19T16:57:01.271+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=1, name=Alice, age=43, gender=F)
2024-11-19T16:57:01.273+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=2, name=Bob, age=48, gender=M)
2024-11-19T16:57:01.381+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=3, name=Charlie, age=40, gender=M)
2024-11-19T16:57:01.381+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=4, name=Diana, age=45, gender=F)
2024-11-19T16:57:01.454+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=5, name=Eve, age=47, gender=F)
2024-11-19T16:57:01.454+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=6, name=Frank, age=53, gender=M)
2024-11-19T16:57:01.563+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=7, name=Grace, age=42, gender=F)
2024-11-19T16:57:01.564+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=8, name=Hank, age=46, gender=M)
2024-11-19T16:57:01.647+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=9, name=Ivy, age=44, gender=F)
2024-11-19T16:57:01.647+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=10, name=Jack, age=49, gender=M)
2024-11-19T16:57:01.716+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=11, name=Kate, age=47, gender=F)
2024-11-19T16:57:01.716+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=12, name=Leo, age=57, gender=M)
2024-11-19T16:57:01.803+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=13, name=Mia, age=45, gender=F)
2024-11-19T16:57:01.803+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=14, name=Nate, age=53, gender=M)
2024-11-19T16:57:01.892+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=15, name=Olivia, age=51, gender=F)
2024-11-19T16:57:01.892+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=16, name=Paul, age=58, gender=M)
2024-11-19T16:57:01.970+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=17, name=Quinn, age=49, gender=M)
2024-11-19T16:57:01.970+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=18, name=Rachel, age=52, gender=F)
2024-11-19T16:57:02.043+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=19, name=Sam, age=56, gender=M)
2024-11-19T16:57:02.043+09:00 INFO 35984 --- [ main] c.k.s.b.j.mybatis.CustomerItemProcessor : Item Processor ------------------- Customer(id=20, name=Tina, age=50, gender=F)
2024-11-19T16:57:02.241+09:00 INFO 35984 --- [ main] o.s.batch.core.step.AbstractStep : Step: [customerJdbcCursorStep] executed in 1s138ms
2024-11-19T16:57:02.475+09:00 INFO 35984 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=MYBATIS_CHUNK_JOB]] completed with the following parameters: [{'run.id':'{value=2, type=class java.lang.Long, identifying=true}'}] and the following status: [COMPLETED] in 1s586ms
2024-11-19T16:57:02.479+09:00 INFO 35984 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
2024-11-19T16:57:04.753+09:00 INFO 35984 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.
일단 다행이 클리어.
MyBatisItemWriter
MyBatisBatchItemWriter는 Spring Batch에서 MyBatis를 사용하여 데이터베이스에 대량의 데이터를 효율적으로 쓰기 위한 ItemWriter 구현체입니다.
특징
1. MyBatis 통합: MyBatis의 SqlSession을 사용하여 데이터베이스 작업을 수행합니다.
2. 배치 처리: 여러 아이템을 한 번에 처리하여 성능을 향상시킵니다.
3. 설정 용이성: XML 매퍼 파일을 사용하여 SQL 쿼리를 관리할 수 있어 유지보수가 쉽습니다.
4. 트랜잭션 지원: Spring의 트랜잭션 관리와 통합되어 있습니다.
5. 유연성: 동적 SQL을 지원하여 복잡한 쓰기 작업도 처리할 수 있습니다.
구성요소
1. SqlSessionTemplate: MyBatis SqlSession 생성 및 관리를 위한 템플릿 객체이다.
2. SqlSessionFactory: SqlSessionTemplate 생성을 위한 팩토리 객체이다
3. StatementId: 실행할 MyBatis SQL 맵퍼의 Statement ID이다.
4. ItemToParameterConverter: 객체를 ParameterMap으로 변경할수 있다.
장점
1. ORM 연동: MyBatis를 통해 다양한 데이터베이스에 데이터를 저장할 수 있다.
2. SQL 쿼리 분리: SQL 쿼리를 Java 코드로부터 분리하여 관리 및 유지 보수가 용이하다.
3. 유연성: 다양한 설정을 통해 원하는 방식으로 데이터를 저장할 수 있다.
단점
1. 설정 복잡성: MyBatis 설정 및 SQL 맵퍼 작성이 복잡할 수 있다.
2. 데이터베이스 종속: 특정 데이터베이스에 종속적이다.
3. 오류 가능성: 설정 오류 시 데이터 손상 가능성이 있다.
(내용 정리가 어떤 다른 블로그보다 잘되어있어서 데보션 블로그 그대로 가져옴)
(DB에 write 하기 위한) customer.xml 파일 작성
customer.xml - 준수님 개발DB에 맞춰서 넣으려다보니 GRADE 컬럼 추가하였다.
<insert id="insertCustomers" parameterType="com.ksko.spring_batch.batch_sample.jobs.models.Customer">
INSERT
INTO CUSTOMER (NAME, AGE, GENDER, GRADE)
VALUES (
#{name}
, #{age}
, #{gender}
, #{grade}
)
</insert>
MyBatisBatchItemWriter 작성
하려고 했는데.. 이거 뭘 어떻게 작성해야하나 막막해서 ㅠㅠ
아래 기도님 샘플 소스를 보고 따라쳤다..
(아직도 멀었음)
MybatisItemWriterJobConfig.Java
package com.ksko.spring_batch.batch_sample.jobs.mybatis;
import com.ksko.spring_batch.batch_sample.jobs.models.Customer;
import lombok.extern.slf4j.Slf4j;
import org.apache.ibatis.session.SqlSessionFactory;
import org.mybatis.spring.batch.MyBatisBatchItemWriter;
import org.mybatis.spring.batch.builder.MyBatisBatchItemWriterBuilder;
import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.job.builder.JobBuilder;
import org.springframework.batch.core.launch.support.RunIdIncrementer;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.core.step.builder.StepBuilder;
import org.springframework.batch.item.file.FlatFileItemReader;
import org.springframework.batch.item.file.builder.FlatFileItemReaderBuilder;
import org.springframework.beans.factory.annotation.Autowired;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.ClassPathResource;
import org.springframework.transaction.PlatformTransactionManager;
import javax.sql.DataSource;
import java.util.HashMap;
import java.util.Map;
@Slf4j
@Configuration
public class MybatisItemWriterJobConfig {
// CHUNK 크기를 지정한다.
public static final int CHUNK_SIZE = 100;
public static final String ENCODING = "UTF-8";
public static final String MY_BATIS_ITEM_WRITER = "MY_BATIS_ITEM_WRITER";
@Autowired
DataSource dataSource;
@Autowired
SqlSessionFactory sqlSessionFactory;
@Bean
public FlatFileItemReader<Customer> flatFileItemReader() {
return new FlatFileItemReaderBuilder<Customer>()
.name("FlatFileItemReader")
.resource(new ClassPathResource("./customer.csv"))
.encoding(ENCODING)
.delimited().delimiter(",")
.names("name", "age", "gender", "grade") //준수님 블로그에 컬럼에 있는 GRADE 추가
.targetType(Customer.class)
.build();
}
//@Bean
public MyBatisBatchItemWriter<Customer> myBatisBatchItemWriter2() {
return new MyBatisBatchItemWriterBuilder<Customer>()
.sqlSessionFactory(sqlSessionFactory) // MyBatis의 SqlSessionFactory 설정. 데이터베이스 연결 및 SQL 실행을 관리
.statementId("com.ksko.spring_batch.batch_sample.jobs.insertCustomers") // 실행할 SQL 문의 ID 설정. 여기서는 Customer 삽입 SQL을 가리키는 완전한 경로 지정
.itemToParameterConverter(item -> {
Map<String, Object> parameter = new HashMap<>();
parameter.put("name", item.getName());
parameter.put("age", item.getAge());
parameter.put("gender", item.getGender());
return parameter;
})
.build();
}
@Bean
public Step flatFileStep(JobRepository jobRepository, PlatformTransactionManager transactionManager) {
log.info("------------------ Init flatFileStep -----------------");
return new StepBuilder("flatFileStep", jobRepository)
.<Customer, Customer>chunk(CHUNK_SIZE, transactionManager)
.reader(flatFileItemReader())
.writer(myBatisBatchItemWriter())
.build();
}
@Bean
public Job flatFileJob(Step flatFileStep, JobRepository jobRepository) {
log.info("------------------ Init flatFileJob -----------------");
return new JobBuilder(MY_BATIS_ITEM_WRITER, jobRepository)
.incrementer(new RunIdIncrementer())
.start(flatFileStep)
.build();
}
}
결과조회
2024-11-20T17:21:19.784+09:00 INFO 36648 --- [ main] c.k.s.b.BatchSampleApplication : Starting BatchSampleApplication using Java 21.0.4 with PID 36648 (D:\ksko\workspace\spring-batch-study\build\classes\java\main started by 202010011 in D:\ksko\workspace\spring-batch-study)
2024-11-20T17:21:19.788+09:00 INFO 36648 --- [ main] c.k.s.b.BatchSampleApplication : No active profile set, falling back to 1 default profile: "default"
2024-11-20T17:21:20.331+09:00 WARN 36648 --- [ main] o.m.s.mapper.ClassPathMapperScanner : No MyBatis mapper was found in '[com.ksko.spring_batch.batch_sample]' package. Please check your configuration.
2024-11-20T17:21:20.439+09:00 WARN 36648 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.boot.autoconfigure.jdbc.DataSourceConfiguration$Hikari' of type [org.springframework.boot.autoconfigure.jdbc.DataSourceConfiguration$Hikari] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-20T17:21:20.466+09:00 WARN 36648 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'spring.datasource-org.springframework.boot.autoconfigure.jdbc.DataSourceProperties' of type [org.springframework.boot.autoconfigure.jdbc.DataSourceProperties] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-20T17:21:20.467+09:00 WARN 36648 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration$PooledDataSourceConfiguration' of type [org.springframework.boot.autoconfigure.jdbc.DataSourceAutoConfiguration$PooledDataSourceConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-20T17:21:20.468+09:00 WARN 36648 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'jdbcConnectionDetails' of type [org.springframework.boot.autoconfigure.jdbc.PropertiesJdbcConnectionDetails] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-20T17:21:20.488+09:00 WARN 36648 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'dataSource' of type [com.zaxxer.hikari.HikariDataSource] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-20T17:21:20.491+09:00 WARN 36648 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.boot.autoconfigure.jdbc.DataSourceTransactionManagerAutoConfiguration$JdbcTransactionManagerConfiguration' of type [org.springframework.boot.autoconfigure.jdbc.DataSourceTransactionManagerAutoConfiguration$JdbcTransactionManagerConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-20T17:21:20.496+09:00 WARN 36648 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.boot.autoconfigure.transaction.TransactionManagerCustomizationAutoConfiguration' of type [org.springframework.boot.autoconfigure.transaction.TransactionManagerCustomizationAutoConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-20T17:21:20.500+09:00 WARN 36648 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'transactionExecutionListeners' of type [org.springframework.boot.autoconfigure.transaction.ExecutionListenersTransactionManagerCustomizer] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-20T17:21:20.503+09:00 WARN 36648 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'spring.transaction-org.springframework.boot.autoconfigure.transaction.TransactionProperties' of type [org.springframework.boot.autoconfigure.transaction.TransactionProperties] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-20T17:21:20.504+09:00 WARN 36648 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'platformTransactionManagerCustomizers' of type [org.springframework.boot.autoconfigure.transaction.TransactionManagerCustomizers] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-20T17:21:20.508+09:00 WARN 36648 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'transactionManager' of type [org.springframework.jdbc.support.JdbcTransactionManager] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-20T17:21:20.510+09:00 WARN 36648 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'spring.batch-org.springframework.boot.autoconfigure.batch.BatchProperties' of type [org.springframework.boot.autoconfigure.batch.BatchProperties] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). Is this bean getting eagerly injected into a currently created BeanPostProcessor [jobRegistryBeanPostProcessor]? Check the corresponding BeanPostProcessor declaration and its dependencies.
2024-11-20T17:21:20.515+09:00 WARN 36648 --- [ main] trationDelegate$BeanPostProcessorChecker : Bean 'org.springframework.boot.autoconfigure.batch.BatchAutoConfiguration$SpringBootBatchConfiguration' of type [org.springframework.boot.autoconfigure.batch.BatchAutoConfiguration$SpringBootBatchConfiguration] is not eligible for getting processed by all BeanPostProcessors (for example: not eligible for auto-proxying). The currently created BeanPostProcessor [jobRegistryBeanPostProcessor] is declared through a non-static factory method on that class; consider declaring it as static instead.
2024-11-20T17:21:20.695+09:00 INFO 36648 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Starting...
2024-11-20T17:21:21.600+09:00 INFO 36648 --- [ main] com.zaxxer.hikari.pool.HikariPool : HikariPool-1 - Added connection com.mysql.cj.jdbc.ConnectionImpl@7b61bf11
2024-11-20T17:21:21.607+09:00 INFO 36648 --- [ main] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Start completed.
2024-11-20T17:21:21.639+09:00 INFO 36648 --- [ main] c.k.s.b.j.m.MybatisItemWriterJobConfig : ------------------ Init flatFileStep -----------------
2024-11-20T17:21:21.667+09:00 INFO 36648 --- [ main] c.k.s.b.j.m.MybatisItemWriterJobConfig : ------------------ Init flatFileJob -----------------
2024-11-20T17:21:21.771+09:00 INFO 36648 --- [ main] c.k.s.b.BatchSampleApplication : Started BatchSampleApplication in 2.36 seconds (process running for 3.011)
2024-11-20T17:21:21.773+09:00 INFO 36648 --- [ main] o.s.b.a.b.JobLauncherApplicationRunner : Running default command line with: []
2024-11-20T17:21:23.957+09:00 INFO 36648 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=MY_BATIS_ITEM_WRITER]] launched with the following parameters: [{'run.id':'{value=18, type=class java.lang.Long, identifying=true}'}]
2024-11-20T17:21:24.524+09:00 INFO 36648 --- [ main] o.s.batch.core.job.SimpleStepHandler : Executing step: [flatFileStep]
2024-11-20T17:21:25.045+09:00 INFO 36648 --- [ main] o.s.batch.core.step.AbstractStep : Step: [flatFileStep] executed in 520ms
2024-11-20T17:21:25.387+09:00 INFO 36648 --- [ main] o.s.b.c.l.support.SimpleJobLauncher : Job: [SimpleJob: [name=MY_BATIS_ITEM_WRITER]] completed with the following parameters: [{'run.id':'{value=18, type=class java.lang.Long, identifying=true}'}] and the following status: [COMPLETED] in 1s272ms
2024-11-20T17:21:25.391+09:00 INFO 36648 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown initiated...
2024-11-20T17:21:27.230+09:00 INFO 36648 --- [ionShutdownHook] com.zaxxer.hikari.HikariDataSource : HikariPool-1 - Shutdown completed.
겨우겨우 클리어 ㅠㅠ
DB 에 INSERT 되엇나 확인
초심으로 돌아가서,
초기 스프링 배치 DB 구조에 데이터 잘 들어갔나 확인까지 해보자.
[정리]
1. 무언가 배치를 실행 하기 위한 정형화된 틀 같은게 있는 것 같다. 이 실습에서 하는 틀 같은게 있나
2. mybatis 통해서 읽고 쓰는 것은 그래도 따라 해볼만 했다. mybatis 관련 경력 이직 면접 질문도 받아본것 같다. 원리들도 알아두는게 좋을 것 같다.
3. 다음주는 오프라인 모임이다.. 가자.
[참고]
마이바티스 공식 사이트에 오늘 해야할 내용이 정리가 되어있어서 우선 남김
mybatis-spring
Spring Batch As of version 1.1.0 MyBatis-Spring provides three beans for building Spring Batch applications: the MyBatisPagingItemReader, the MyBatisCursorItemReader and the MyBatisBatchItemWriter. Also, As of version 2.0.0 provides three builder classes f
mybatis.org
728x90
