PricingLoginStart free
Spring Boot 4 Complete Guide · Lektion

Partitionierung und parallele Step-Ausführung

Steigern Sie den Durchsatz mit mehrthreadigen Steps, Partitionierung und Strategien für Remote Chunking.

Lektion 4 von 413 Schritte

Partitionierung und parallele Step-Ausführung ist eine kostenlose Spring Boot 4 Complete Guide-Lektion auf CoddyKit. Dies ist Lektion 4 von 4. Du kannst die komplette Lektion unten kostenlos lesen – dann übst du sie direkt im Browser mit einem integrierten Code-Editor und einem KI-Tutor rund um die Uhr. Sie ist Teil des Spring Boot 4 Complete Guide-Lernpfads, und dein Fortschritt wird über Web und CoddyKit-App synchronisiert. Der Spring Boot 4 Complete Guide-Kurs umfasst insgesamt 4 Lektionen.

Teile dieser Lektion wurden noch nicht übersetzt und werden auf Englisch angezeigt.

Why Scale Spring Batch?

Single-threaded Spring Batch jobs process one chunk at a time — fine for small datasets, but too slow for millions of records. When batch throughput becomes a bottleneck, Spring Batch offers four scaling strategies:

  • Multi-threaded Step — parallel threads within a single JVM step
  • Parallel Steps — independent steps run concurrently in a flow
  • Partitioning — divide data into partitions, each processed by a worker step
  • Remote Chunking — offload chunk processing to remote workers over messaging middleware

Each strategy has different complexity/throughput trade-offs. This lesson covers all four, starting with the simplest.

Multi-Threaded Steps with TaskExecutor

The easiest way to add parallelism is to inject a TaskExecutor into your Step. Spring Batch will execute chunks concurrently on a thread pool. Important: the ItemReader must be thread-safe (e.g. stateless, or use SynchronizedItemStreamReader).

Configure a multi-threaded step like this:

import org.springframework.batch.core.Step;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.core.step.builder.StepBuilder;
import org.springframework.batch.item.file.FlatFileItemReader;
import org.springframework.batch.item.file.builder.FlatFileItemReaderBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.ClassPathResource;
import org.springframework.core.task.SimpleAsyncTaskExecutor;
import org.springframework.transaction.PlatformTransactionManager;

@Configuration
public class MultiThreadedStepConfig {

    @Bean
    public Step multiThreadedStep(JobRepository jobRepository,
                                   PlatformTransactionManager txManager,
                                   FlatFileItemReader<String> reader) {
        return new StepBuilder("multiThreadedStep", jobRepository)
                .<String, String>chunk(100, txManager)
                .reader(reader)
                .writer(items -> items.forEach(System.out::println))
                .taskExecutor(new SimpleAsyncTaskExecutor())
                .throttleLimit(4)          // max concurrent threads
                .build();
    }
}

Thread-Safe Readers with SynchronizedItemStreamReader

Standard FlatFileItemReader is not thread-safe because it maintains internal state (current line position). Wrapping it in SynchronizedItemStreamReader serialises read() calls, making it safe for multi-threaded steps without altering processing or writing parallelism.

import org.springframework.batch.item.file.FlatFileItemReader;
import org.springframework.batch.item.file.builder.FlatFileItemReaderBuilder;
import org.springframework.batch.item.support.SynchronizedItemStreamReader;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.io.ClassPathResource;

@Configuration
public class SafeReaderConfig {

    @Bean
    public SynchronizedItemStreamReader<String> synchronizedReader() {
        FlatFileItemReader<String> delegate = new FlatFileItemReaderBuilder<String>()
                .name("lineReader")
                .resource(new ClassPathResource("data/input.csv"))
                .lineMapper((line, lineNumber) -> line)
                .build();

        SynchronizedItemStreamReader<String> reader = new SynchronizedItemStreamReader<>();
        reader.setDelegate(delegate);
        return reader;
    }
}

Parallel Steps with Split Flows

When you have independent steps (e.g. loading products and loading customers simultaneously), use a split() flow so both steps run concurrently. Spring Batch's FlowBuilder supports this natively.

Steps inside a split share no data — they must operate on separate resources to avoid contention.

import org.springframework.batch.core.Job;
import org.springframework.batch.core.Step;
import org.springframework.batch.core.job.builder.FlowBuilder;
import org.springframework.batch.core.job.builder.JobBuilder;
import org.springframework.batch.core.job.flow.Flow;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.task.SimpleAsyncTaskExecutor;

@Configuration
public class ParallelStepsConfig {

    @Bean
    public Flow productFlow(Step loadProductsStep) {
        return new FlowBuilder<Flow>("productFlow")
                .start(loadProductsStep)
                .build();
    }

    @Bean
    public Flow customerFlow(Step loadCustomersStep) {
        return new FlowBuilder<Flow>("customerFlow")
                .start(loadCustomersStep)
                .build();
    }

    @Bean
    public Job parallelJob(JobRepository jobRepository,
                           Flow productFlow,
                           Flow customerFlow) {
        return new JobBuilder("parallelJob", jobRepository)
                .start(productFlow)
                .split(new SimpleAsyncTaskExecutor())
                .add(customerFlow)
                .end()
                .build();
    }
}

Introduction to Partitioning

Partitioning divides a dataset into non-overlapping partitions, each processed independently by a worker step. A manager step (formerly called master) creates the partitions and delegates them.

  • Partitioner — creates ExecutionContext maps, one per partition
  • PartitionHandler — decides how worker steps are launched (local or remote)
  • TaskExecutorPartitionHandler — runs workers locally in a thread pool

Each worker step receives its own ExecutionContext with partition-specific parameters (e.g. row range, file path).

Implementing a Custom Partitioner

A Partitioner returns a Map<String, ExecutionContext> where each entry represents one partition. The map keys become the partition names visible in the Job Repository.

This example partitions a table by ID range — each worker handles a slice of rows:

import org.springframework.batch.core.partition.support.Partitioner;
import org.springframework.batch.item.ExecutionContext;
import java.util.HashMap;
import java.util.Map;

public class RangePartitioner implements Partitioner {

    private final long totalRows;
    private final int gridSize;

    public RangePartitioner(long totalRows, int gridSize) {
        this.totalRows = totalRows;
        this.gridSize = gridSize;
    }

    @Override
    public Map<String, ExecutionContext> partition(int gridSize) {
        long partitionSize = totalRows / gridSize;
        Map<String, ExecutionContext> partitions = new HashMap<>();

        for (int i = 0; i < gridSize; i++) {
            ExecutionContext ctx = new ExecutionContext();
            long minId = i * partitionSize + 1;
            long maxId = (i == gridSize - 1) ? totalRows : (i + 1) * partitionSize;
            ctx.putLong("minId", minId);
            ctx.putLong("maxId", maxId);
            ctx.putString("name", "partition" + i);
            partitions.put("partition" + i, ctx);
        }
        return partitions;
    }
}

Wiring the Partitioned Step

Once you have a Partitioner, wire it into a manager step using StepBuilder.partitioner(). The TaskExecutorPartitionHandler runs each worker step in a thread pool within the same JVM.

The worker step reads using @StepScope beans that pull minId/maxId from the partition's ExecutionContext.

import org.springframework.batch.core.Step;
import org.springframework.batch.core.partition.support.TaskExecutorPartitionHandler;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.core.step.builder.StepBuilder;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.core.task.SimpleAsyncTaskExecutor;
import org.springframework.transaction.PlatformTransactionManager;

@Configuration
public class PartitionedStepConfig {

    @Bean
    public Step managerStep(JobRepository jobRepository,
                             Step workerStep,
                             RangePartitioner partitioner) {
        TaskExecutorPartitionHandler handler = new TaskExecutorPartitionHandler();
        handler.setStep(workerStep);
        handler.setTaskExecutor(new SimpleAsyncTaskExecutor());
        handler.setGridSize(8);  // 8 parallel workers

        return new StepBuilder("managerStep", jobRepository)
                .partitioner("workerStep", partitioner)
                .partitionHandler(handler)
                .build();
    }
}

Step-Scoped Worker Beans

Worker step beans must be declared with @StepScope so Spring creates a new instance per partition, injecting each partition's ExecutionContext values via @Value("#{stepExecutionContext['minId']}").

This pattern ensures each thread reads a completely independent row range with no shared state:

import org.springframework.batch.core.Step;
import org.springframework.batch.core.repository.JobRepository;
import org.springframework.batch.core.scope.context.StepSynchronizationManager;
import org.springframework.batch.core.step.builder.StepBuilder;
import org.springframework.batch.item.database.JdbcPagingItemReader;
import org.springframework.batch.item.database.Order;
import org.springframework.batch.item.database.support.SqlPagingQueryProviderFactoryBean;
import org.springframework.beans.factory.annotation.Value;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.context.annotation.Scope;
import org.springframework.context.annotation.ScopedProxyMode;
import javax.sql.DataSource;
import java.util.Map;

@Configuration
public class WorkerStepConfig {

    @Bean
    @Scope(value = "step", proxyMode = ScopedProxyMode.TARGET_CLASS)
    public JdbcPagingItemReader<Order> workerReader(
            DataSource dataSource,
            @Value("#{stepExecutionContext['minId']}") Long minId,
            @Value("#{stepExecutionContext['maxId']}") Long maxId) throws Exception {

        JdbcPagingItemReader<Order> reader = new JdbcPagingItemReader<>();
        reader.setDataSource(dataSource);
        reader.setPageSize(100);
        reader.setRowMapper((rs, i) -> new Order(rs.getLong("id"), rs.getString("status")));
        reader.setSelectClause("SELECT id, status");
        reader.setFromClause("FROM orders");
        reader.setWhereClause("WHERE id BETWEEN " + minId + " AND " + maxId);
        reader.setSortKeys(Map.of("id", org.springframework.batch.item.database.Order.ASCENDING));
        reader.afterPropertiesSet();
        return reader;
    }
}

Remote Partitioning with Spring Integration

Remote Partitioning moves worker steps to separate JVM processes (or Kubernetes pods). The manager sends partition StepExecutionRequest messages over a message broker (RabbitMQ, Kafka, etc.) and workers respond with results.

  • Add spring-batch-integration dependency
  • Manager uses MessageChannelPartitionHandler
  • Workers listen on an input channel with StepExecutionRequestHandler

This scales horizontally — add more worker pods to increase throughput without redeploying the manager.

Remote Chunking Architecture

Remote Chunking differs from partitioning: the manager reads all data and sends individual chunks to remote workers for processing and writing. Workers don't access the datasource directly.

  • Lower latency per item (manager controls read order)
  • Network becomes the bottleneck at high throughput
  • Guaranteed delivery requires a durable broker (no data loss on worker crash)

Use remote chunking when processing/writing is the CPU bottleneck, not reading. Use remote partitioning when reading is also slow.

// Remote Chunking manager configuration (spring-batch-integration)
import org.springframework.batch.integration.chunk.RemoteChunkingManagerStepBuilderFactory;
import org.springframework.batch.core.Step;
import org.springframework.context.annotation.Bean;
import org.springframework.context.annotation.Configuration;
import org.springframework.integration.channel.DirectChannel;
import org.springframework.integration.channel.QueueChannel;

@Configuration
public class RemoteChunkingConfig {

    private final RemoteChunkingManagerStepBuilderFactory managerStepBuilderFactory;

    public RemoteChunkingConfig(RemoteChunkingManagerStepBuilderFactory factory) {
        this.managerStepBuilderFactory = factory;
    }

    @Bean
    public DirectChannel requests() { return new DirectChannel(); }

    @Bean
    public QueueChannel replies() { return new QueueChannel(); }

    @Bean
    public Step remoteChunkingManagerStep() {
        return managerStepBuilderFactory
                .get("remoteChunkingManager")
                .<String, String>chunk(200)
                .reader(flatFileReader())       // manager reads
                .outputChannel(requests())      // sends chunks to workers
                .inputChannel(replies())        // receives ack from workers
                .build();
    }

    private org.springframework.batch.item.ItemReader<String> flatFileReader() {
        // returns a configured FlatFileItemReader
        return null; // replace with actual reader bean
    }
}

Choosing the Right Strategy

Picking the wrong strategy leads to either under-utilization or unnecessary complexity. Use this decision guide:

  • Multi-threaded step — data fits in one source, reader can be made thread-safe, simplest option
  • Parallel steps — independent data loads that don't share state
  • Local partitioning — large single-source dataset, single JVM, ID/date ranges easy to define
  • Remote partitioning — data is too large for one JVM, horizontal scaling needed, workers can reach the data source
  • Remote chunking — processing/writing is the bottleneck, workers are stateless, a message broker is already in your stack

Always prefer local strategies first — they are easier to monitor, debug, and restart after failure.

Knowledge Check: Partitioning vs Remote Chunking

Test your understanding of when to apply each Spring Batch scaling strategy.

Beantworte es in der interaktiven Lektion

Lesson Recap: Partitioning and Parallel Execution

In this lesson you learned how Spring Batch scales throughput beyond single-threaded processing:

  • Multi-threaded steps add parallelism with minimal config — use SimpleAsyncTaskExecutor and wrap stateful readers in SynchronizedItemStreamReader
  • Parallel steps via split() flows run independent steps concurrently within one job
  • Local partitioning divides a dataset into ID/date ranges; a TaskExecutorPartitionHandler runs worker steps in a thread pool; worker beans use @StepScope + @Value("#{stepExecutionContext[...]}")"
  • Remote partitioning distributes workers across JVMs/pods over a message broker — best when reading is the bottleneck and workers can reach the data source
  • Remote chunking ships pre-read chunks to stateless remote workers — best when processing/writing is the bottleneck

Always start with the simplest strategy that meets your throughput requirements, and prefer local strategies to keep observability and failure recovery straightforward.

Kostenlos starten

Lerne Java mit einem KI-Tutor — kostenlos

Schreibe und führe echten Code in deinem Browser aus, bekomme sofortige Hilfe von einem 24/7 KI-Tutor und setze dein Lernen im Web oder in der App fort.

Kurse
21
Lektionen
84
Kostenlos lernenDen vollständigen Kurs ansehen

Häufig gestellte Fragen

Ist die Lektion „Partitionierung und parallele Step-Ausführung“ kostenlos?

Ja — der vollständige Text von „Partitionierung und parallele Step-Ausführung“ ist hier im Web kostenlos zu lesen. Um sie interaktiv zu üben (integrierter Code-Editor und 24/7 KI-Tutor) und den Rest des Spring Boot 4 Complete Guide-Kurses freizuschalten, upgrade auf CoddyKit PRO. Der Spring Boot 4 Complete Guide-Kurs umfasst insgesamt 4 Lektionen.

Was lerne ich in „Partitionierung und parallele Step-Ausführung“?

Steigern Sie den Durchsatz mit mehrthreadigen Steps, Partitionierung und Strategien für Remote Chunking. Du übst Spring Boot 4 Complete Guide mit praktischem Code, den du direkt im Browser ausführst, und ein 24/7 KI-Tutor beantwortet deine Fragen während du die Lektion bearbeitest.

Brauche ich Erfahrung, um Spring Boot 4 Complete Guide zu starten?

Keine Vorkenntnisse erforderlich. Spring Boot 4 Complete Guide auf CoddyKit ist für Anfänger bis fortgeschrittene Lernende strukturiert, sodass du hier starten oder von Anfang an beginnen und in deinem eigenen Tempo voranschreiten kannst. Dies ist Lektion 4 von 4.

Wie lange dauert die Lektion „Partitionierung und parallele Step-Ausführung“?

Die meisten CoddyKit-Lektionen dauern etwa 5–10 Minuten. Jede ist kompakt und interaktiv, sodass du stetig Fortschritte machst und genau dort weitermachst, wo du aufgehört hast – im Web und in der App.

Kann ich in dieser Spring Boot 4 Complete Guide-Lektion Code schreiben und ausführen?

Ja. Jede Spring Boot 4 Complete Guide-Lektion enthält einen integrierten Code-Editor, sodass du echten Code direkt in deinem Browser schreibst und ausführst und sofort KI-Feedback erhältst — ohne lokale Einrichtung erforderlich.

Alle Lektionen in diesem Kurs

  1. Jobs, Steps und das JobRepository-Modell
  2. Chunk-orientierte Reader-Processor-Writer-Abläufe
  3. Fehlertoleranz, Skip- und Retry-Richtlinien
  4. Partitionierung und parallele Step-Ausführung
← Zurück zu Spring Boot 4 Complete Guide