Avro для определения схем

Welcome! In this lesson, we'll explore Apache Avro, a powerful data serialization system. It's widely used in event-driven architectures, especially with Kafka.

Avro helps define the structure of your data (events) using a schema. This schema acts like a blueprint, ensuring consistency and enabling seamless communication between different applications.

Avro's schema-driven approach offers several benefits:

• Data Compactness: Avro serializes data efficiently, leading to smaller message sizes.
• Schema Evolution: It provides robust rules for how schemas can change over time without breaking old applications.
• Language Agnostic: Schemas are language-independent, allowing different programming languages to read and write the same data.
• Strong Typing: Ensures data types are consistent, reducing runtime errors.

Avro schemas are defined using JSON. Each schema describes a data structure, most commonly a record type.

Key elements of an Avro schema include:

• type: The kind of schema (e.g., "record", "enum", "array").
• name: The name of the schema (e.g., "User", "Order").
• namespace: An optional string to qualify the name.
• fields: For record types, an array of field definitions.

Avro supports a set of basic, primitive data types:

• null: No value.
• boolean: true or false.
• int: 32-bit signed integer.
• long: 64-bit signed integer.
• float: Single precision (32-bit) floating-point number.
• double: Double precision (64-bit) floating-point number.
• bytes: Sequence of 8-bit unsigned bytes.
• string: Unicode character sequence.

Let's define a simple Avro record schema for a User. Each field needs a name and a type.

Here, id is an int and name is a string.

You can define complex data structures by nesting records. For example, an Order record might contain a Customer record within one of its fields.

Avro also supports collection types:

• array: For a list of items of the same type. The items property specifies the type of elements.
• map: For key-value pairs, where keys are always strings. The values property specifies the type of values.

An enum defines a fixed set of named values. This is useful for fields with a limited, predefined set of options.

The symbols property lists all allowed string values for the enum.

A union allows a field to be one of several specified types. This is crucial for defining optional or nullable fields.

To make a field nullable, you define its type as a union of "null" and its actual type, e.g., ["null", "string"]. The first type in the union is often the default if not explicitly set.

You can specify a default value for a field. This is particularly important for schema evolution, as it ensures that older data (without the new field) can still be read without error, using the default value.

The default value must be a valid instance of the field's type.

Consider an Avro record for an event. We need a field named comment that can contain a string but is optional (can be null) and defaults to null if not provided.

Great job! You've learned the fundamentals of defining schemas with Apache Avro.

• Avro schemas are JSON-based blueprints for your data.
• They support primitive types (int, string, etc.) and complex types (records, arrays, maps, enums, unions).
• Unions are key for defining nullable or optional fields.
• Default values are crucial for backward compatibility and schema evolution.

Next, we'll see how to integrate these schemas with Spring Boot and Schema Registry.