# Kafka Producer Design Apache Kafka® is an open-source distributed streaming system used for stream processing, real-time data pipelines, and data integration at scale. This topic describes how Kafka producers are designed to enable load balancing and asynchronous send operations. ## What is a producer? A producer is a client application that publishes data to a Kafka cluster. A Kafka producer works by writing messages directly to topics in a Kafka broker. The sections in this topic describe how producers are designed to enable load balancing and asynchronous send operations. ## Producer load balancing The client producer controls which partition it publishes messages to. Also, the producer sends data directly to the broker that is the partition leader without any intervening routing tier. To help a producer do this, all Kafka nodes provide metadata that specifies the brokers that are alive, and the brokers that are leaders for the partitions of a topic. This enables a producer to appropriately route its requests. Load balancing can be random, or you can apply a semantic partitioning function. You can specify a key to partition by, and Kafka uses that key to hash to a partition. There is also an option to override the partition function if needed. For example, if the specified key is `user id` then all data for a given user would be sent to the same partition. This in turn will enables consumers to make locality assumptions about their consumption. This style of partitioning is explicitly designed to allow locality-sensitive processing in consumers. ## Batching Batching enables efficiency, and to enable batching the Kafka producer tries to accumulate data in memory, and sends larger batches in a single request. The batching can be configured: - By batch size (example: 64 kb). Use the [batch.size](/platform/current/installation/configuration/producer-configs.html#batch-size) property to set this. - By wait time (example: 10 ms). Use the [linger.ms](/platform/current/installation/configuration/producer-configs.html#linger-ms) property to set this. This batching enable the accumulation of more bytes to send, and a few larger I/O operations on the servers. It also provides a mechanism to trade a small amount of additional latency for better throughput. ## Related content - [Kafka Producer](/platform/current/clients/producer.html) - [In-sync replicas and producer acks](replication.md#producer-acks) - [Producer configuration reference](/platform/current/installation/configuration/producer-configs.html) - [Kafka Consumer Design: Consumers, Consumer Groups, and Offsets](consumer-design.md#consumer-design) - [Kafka Message Delivery Guarantees](delivery-semantics.md#delivery-semantics) - [Kafka Log Compaction](log_compaction.md#log-compaction) - [Kafka Replication and Committed Messages](replication.md#replication) - [Kafka Quotas](quotas.md#quotas) - [Producer API](/platform/current/clients/javadocs/javadoc/org/apache/kafka/clients/producer/KafkaProducer.html) #### NOTE This website includes content developed at the [Apache Software Foundation](https://www.apache.org/) under the terms of the [Apache License v2](https://www.apache.org/licenses/LICENSE-2.0.html).