Spring Batch | Microservices #7
Ғылым және технология
In this video I will show how to create a complete Spring Batch project. I will show how to configure a Job and the Steps. I will implement a Step with a reader, processor and Writer. I will implement another Step with a Tasklet.
This is the seventh video of the playlist where I will build a microservices architecture for a webpage, how to create microservices with Spring Boot and Spring Cloud, and how to handle the microservices with Docker and Kubernetes: • Microservices
Content:
* Explain the main components of Spring Batch. What are the Jobs and the Steps. What. are the components of a Step. How to link multiple Steps in a Job;
* Create a Spring Batch project from zero;
* Create a Step with a Reader, Processor and Writer;
* Create a Step with a Tasklet;
* Create a Job with two Steps;
* Configure Spring Batch to always create the follow-up tables;
* Run a Job at the startup of the application;
* Trigger a Job from an endpoint.
Repository: github.com/ser...
My NEW eBook: sergiolema.dev...
Blog: bit.ly/47ornJL
LinkedIn: bit.ly/41Nn61q
Facebook: bit.ly/47rc9nh
Boost your APIs with API Monitor: bit.ly/4d7C0ny
Пікірлер: 24
Awesome !!! Just a simple demo but it helps a lot
@TheDevWorldbySergioLema
2 жыл бұрын
Thank you!!
Your videos are so helpful and easy to fallow. Great great job!
@TheDevWorldbySergioLema
Жыл бұрын
Thanks Andjela
Just perfect. Thank you.
@TheDevWorldbySergioLema
Жыл бұрын
Thanks for watching!
You are awesome love from india 🙋♂️🙋♂️
@TheDevWorldbySergioLema
2 жыл бұрын
Thank you so much!
Empiezo a verlo, pero las gracias por delante!!
@TheDevWorldbySergioLema
2 жыл бұрын
Gracias a ti por tu interés!
You've mention that this will run in an async way, but other resources use a SimpleAsyncTaskExecutor somewhere in their configs. How do I know then if I need that extra configuration step or not? Thanks for the video. It actually helps a lot to understand that, in reality, Spring Batch is really not as complicates as it sounds!
@TheDevWorldbySergioLema
8 ай бұрын
This is up to you. It will depend on how big the operation is. How do you manage the memory consumption, database pool, pagination... If for you, a simple SimpleAsyncTaskExecutor is enough, don't go into Spring Batch complex configurations. But if you need proper execution, exceptions handling, pagination and take care of the memory consumption, I recommend you to use a Spring Batch process. Hope this answers your question.
so what is the diffrance between spring batch and Spring Cloud Data Flow
@TheDevWorldbySergioLema
5 ай бұрын
Spring Batch and Spring Cloud Dataflow are not at the same level of abstraction to be compared to each other. Spring Batch is a batch processing framework that provides features to create and run batch jobs. Spring Cloud Dataflow on the other hand is a tool that allows you to run streaming and ephemeral apps, including, but not limited to Spring Batch jobs. Source: stackoverflow.com/questions/74553943/benefits-of-using-spring-data-flow-over-spring-batching
it was really helpful, could you please do a video to demonstrate how to read multiple different structure files in one spring microservice? Will that be multi-job or multi-task approach? Thanks in Advance!
@TheDevWorldbySergioLema
Жыл бұрын
It will depend. If your goal is to aggregate all the files into a single target (a database, another file...) then I multi-task is best. Having multiple readers a single hander and a single writer. If the files are used for different purpose, have a multi-job. Different readers with different handlers and different writers.
OK couple of questions. So JPA has poor support for batching so can you ingest data using Spring Batch to make up for it? It isn't clear in your demo as you don't show the SQL as to whether it's actually batching the transaction to the DB. Specifically does the writer actually support JDBC Batching? Also, what if you need to ensure that your writer writes out a chunk even when enough time has past but you haven't yet reached your chunk size?
@TheDevWorldbySergioLema
Жыл бұрын
You can perform some batch operations with JPA, but with Spring Batch, you can do them without saturating the database connections. With Spring Batch I can separate the operations of read, transform and write. And I can do them in chunks. I don't use SQL queries, but Spring Data repositories. I'm not sure you can setup a timeout to trigger a Writer before the previous steps haven't finished yet.
what do you think about running multiple batches on one application server?
@TheDevWorldbySergioLema
Жыл бұрын
If the machine has enough resources, why not. They must be clearly separate (not one batch to do all). Maybe run them at different time to avoid concurrency problems or consume too much resources.
Can you please write Junits for spring batch, for job and steps and reader /writer
@TheDevWorldbySergioLema
Жыл бұрын
You should find all the necessary information in the following link about how to create unit tests for a Spring Batch application: docs.spring.io/spring-batch/docs/current/reference/html/testing.html
HI...how do I create my meta tables. they don't seem to be created automatically in spring boot ?
@TheDevWorldbySergioLema
Жыл бұрын
The meta tables should be created at the first run of Spring Batch.