Security: Authentication & Authorization
Implement fundamental security measures in Kafka, including client authentication and authorization for data access.
Security: Authentication & Authorization is a free Apache Kafka & Stream Processing Fundamentals lesson on CoddyKit — lesson 3 of 4. You can read the complete lesson below for free — then practise it hands-on in the browser with a built-in code editor and a 24/7 AI tutor. It is part of the Apache Kafka & Stream Processing Fundamentals learning path, one of 4 lessons in the course, and your progress syncs across the web and the CoddyKit app.
Why Secure Kafka?
In today's data-driven world, securing your data is paramount. Kafka, often the backbone for critical data streams, needs robust security measures.
Without security, sensitive information could be exposed, data integrity compromised, and compliance regulations violated. This lesson covers the fundamental ways to protect your Kafka cluster.
Authentication: Who Are You?
Authentication is the process of verifying the identity of a client (like a producer or consumer) trying to connect to a Kafka broker.
Think of it like showing your ID at an airport. Kafka needs to confirm that you are who you claim to be before allowing any interaction. This prevents unauthorized users from even connecting.
Authorization: What Can You Do?
Once a client is authenticated (their identity is confirmed), authorization determines what actions they are permitted to perform.
This is like having a boarding pass after showing your ID. The pass dictates which gate you can access and which flight you can board. Kafka uses authorization to control access to specific topics, consumer groups, and other resources.
Kafka's Security Toolkit
Kafka offers several mechanisms to implement both authentication and authorization:
- Authentication: Primarily handled by SASL (Simple Authentication and Security Layer) or SSL/TLS.
- Authorization: Managed through Access Control Lists (ACLs), which define permissions for authenticated users on specific resources.
These layers work together to create a secure data streaming environment.
SASL/PLAIN Broker Setup
SASL (Simple Authentication and Security Layer) provides a framework for authentication. One common mechanism is SASL/PLAIN, which uses a simple username and password.
To enable SASL/PLAIN on a Kafka broker, you need to add security configurations to its server.properties file. Here's a basic example:
# server.properties
listeners=PLAINTEXT://:9092,SASL_PLAINTEXT://:9093
sasl.enabled.mechanisms=PLAIN
sasl.mechanism.inter.broker.protocol=PLAIN
authorizer.class.name=kafka.security.auth.SimpleAclAuthorizer
supers.users=User:admin
listener.name.sasl_plaintext.plain.sasl.jaas.config=
org.apache.kafka.common.security.plain.PlainLoginModule required
username="admin" password="admin-secret";
Client Authentication with SASL/PLAIN
Once the broker is configured for SASL/PLAIN, clients (producers or consumers) must provide valid credentials to connect. You specify these details in the client's configuration.
Try running this simple Java producer example, configured to use SASL/PLAIN:
import org.apache.kafka.clients.producer.*;
import java.util.Properties;
public class SecureProducer {
public static void main(String[] args) {
Properties props = new Properties();
props.put("bootstrap.servers", "localhost:9093");
props.put("key.serializer", "org.apache.kafka.common.serialization.StringSerializer");
props.put("value.serializer", "org.apache.kafka.common.serialization.StringSerializer");
// SASL_PLAINTEXT configuration
props.put("security.protocol", "SASL_PLAINTEXT");
props.put("sasl.mechanism", "PLAIN");
props.put("sasl.jaas.config",
"org.apache.kafka.common.security.plain.PlainLoginModule required " +
"username=\"admin\" password=\"admin-secret\";");
Producer<String, String> producer = new KafkaProducer<>(props);
try {
for (int i = 0; i < 5; i++) {
String message = "Hello Secure Kafka " + i;
producer.send(new ProducerRecord<>("my-secure-topic", "key" + i, message));
System.out.println("Sent: " + message);
}
} catch (Exception e) {
e.printStackTrace();
} finally {
producer.close();
}
}
}Authorization with Access Control Lists
After a client authenticates, Kafka uses Access Control Lists (ACLs) to decide if they are authorized to perform a specific action on a resource.
An ACL is a rule that specifies who (a user principal), from where (host), can do what (operation like READ, WRITE), on which resource (topic, group, broker).
Managing ACLs with the CLI
Kafka provides a command-line tool, kafka-acls.sh, to manage ACLs. You can grant or revoke permissions for users on various Kafka resources.
Here are some common commands:
- Grant write access to a topic:
kafka-acls.sh --authorizer-properties authorizer.properties --add --allow-principal User:admin --producer --topic my-secure-topic - Grant read access to a consumer group:
kafka-acls.sh --authorizer-properties authorizer.properties --add --allow-principal User:consumerUser --consumer --group my-group - List all ACLs:
kafka-acls.sh --authorizer-properties authorizer.properties --list --topic my-secure-topic
Security Checkpoint
You've learned about the fundamental security concepts in Kafka. Let's test your understanding!
Which of the following best describes the purpose of Authorization in Kafka?
Secure Streams: A Summary
Great job! You've covered the essentials of Kafka security.
- Authentication verifies who a client is (e.g., via SASL/PLAIN).
- Authorization determines what an authenticated client can do (e.g., via ACLs).
- Implementing these measures protects your data streams from unauthorized access and ensures data integrity.
Securing your Kafka cluster is a critical step for any production deployment!
Frequently asked questions
Is the “Security: Authentication & Authorization” lesson free?
Yes — the full text of “Security: Authentication & Authorization” is free to read here on the web, and the Apache Kafka & Stream Processing Fundamentals course includes 4 lessons in total. To practise it interactively (a built-in code editor and a 24/7 AI tutor) and unlock the rest of the Apache Kafka & Stream Processing Fundamentals course, upgrade to CoddyKit PRO.
What will I learn in “Security: Authentication & Authorization”?
Implement fundamental security measures in Kafka, including client authentication and authorization for data access. You practise Apache Kafka & Stream Processing Fundamentals with hands-on code you run directly in the browser, and a 24/7 AI tutor answers your questions as you work through the lesson.
Do I need any experience to start Apache Kafka & Stream Processing Fundamentals?
No prior experience is required. Apache Kafka & Stream Processing Fundamentals on CoddyKit is structured for beginners through advanced learners; this is — lesson 3 of 4, so you can start here or from the beginning and move at your own pace.
How long does the “Security: Authentication & Authorization” lesson take?
Most CoddyKit lessons take about 5–10 minutes. Each one is bite-sized and interactive, so you make steady progress and pick up exactly where you left off across the web and the app.
Can I write and run code in this Apache Kafka & Stream Processing Fundamentals lesson?
Yes. Every Apache Kafka & Stream Processing Fundamentals lesson includes a built-in code editor, so you write and run real code right in your browser and get instant AI feedback — no local setup required.
All lessons in this course
- Command-Line Tools for Kafka
- Monitoring Kafka with JMX & Tools
- Security: Authentication & Authorization
- Consumer Lag Tracking & Alerting