The table below offers a rough idea of the total monthly cost of running a cluster. The table compares the cost of running open source code yourself with the cost of the managed service.
Producer bandwidth | Estimated monthly price (USD) | |
---|---|---|
Apache Kafka on Compute Engine | Managed Service for Apache Kafka (see pricing calculator) | |
10 MiB/s | $0.9K | $1.1K |
100 MiB/s | $9.1K | $11K |
This estimate assumes the following:
In general, the total cost of running a Managed Service for Apache Kafka cluster depends on the size of the cluster, the volume of data you publish and consume, as well as the duration for which you store the data. As a rough guide, you can assume that a single vCPU can handle 20MiB/s of publish traffic and 80MiB/s of consumer traffic.
In addition, an important parameter to consider is the average utilization of RAM and vCPU capacity. Unpredictable and variable traffic requires low average utilization to accommodate the peaks in producer or consumer bandwidth.
For clusters with utilization above 20%, the cost of data transfer between zones is the largest component of the total cost. This cost comes from replication of data between leader and follower brokers as well as data transfer traffic between clients and brokers. You can reduce this cost by configuring your consumer clients to use local replicas, compressing your data. If high availability and durability of storage is not critical for a topic, you can also reduce the number of replicas stored for a given topic to reduce replication charges.
The cost of operating a Managed Service for Apache Kafka consists of charges related to running Kafka clusters, Connect clusters, and Private Service Connect (PSC) fees for access to the cluster. You can calculate your Managed Service for Apache Kafka costs using the pricing calculator.
To learn how you can save 40% with a three-year commitment or 20% with a one-year commitment on compute, review our committed use discounts (CUDs) page. CUDs do not apply to Connect clusters.
To size or scale a Managed Service for Apache Kafka cluster, you must set the total vCPU count and RAM size for the Kafka cluster. To do so, you must first understand the throughput that your Kafka cluster can handle. For more information on how to estimate the throughput, see Estimate vCPUs and memory for your cluster.
Cluster charges are billed using the three SKUs in the region where you run the cluster:
Component | Default* (USD) | Managed Service for Apache Kafka CUD - 1 Year* (USD) | Managed Service for Apache Kafka CUD - 3 Year* (USD) |
---|---|---|---|
CPU+RAM | $0.09 / 1 hour | $0.072 / 1 hour | $0.054 / 1 hour |
Connect CPU+RAM | $0.12 / 1 hour | $0.096 / 1 hour | $0.072 / 1 hour |
Local Storage | $0.000232877 / 1 gibibyte hour | - | - |
Long term storage | $0.000136986 / 1 gibibyte hour | - | - |
Component | Price (USD) |
---|---|
Inter-zone data transfer | $0.01 / 1 gibibyte, per 1 month / account |
Compute charges
The vCPU and RAM usage is reported in terms of abstract units called Data Compute Units (DCUs). A simple way to think about this unit is that a single vCPU with 4 GiB of RAM corresponds to 1 DCU.
For example, a cluster with 6 vCPUs and 4 GiB of RAM per vCPU consumes 6 DCU. The cost of this cluster is 6 DCU * 9 cents / hour / DCU = 54 cents / hour in us-central1.
You can, however, use different amounts of RAM per vCPU in your clusters. The conversion factors from compute resources to DCUs are described in the following table:
Compute resources | Equivalent DCU amounts |
---|---|
1 vCPU | 0.6 |
1 GiB RAM | 0.1 |
When you use 1 GiB of RAM for an hour, you pay for 0.1 DCUs for that hour.
See the create cluster page for estimating your cluster size.
Storage charges
You are billed for local persistent disk storage provision for every broker as well as long term storage used by the tiered storage system.
You are billed for 100 GB of local storage per CPU in each cluster.
Long term storage is used when the tiered storage system moves data from local disk. You pay for storage of a single replica of each topic in long term storage.
For example, if you have a cluster with 3 vCPU and your producer clients produce 50 GB data to it, you will pay for
Connect clusters do not incur storage costs because connectors are stateless.
Inter-zone network charges
A Managed Service for Apache Kafka cluster replicates data across multiple zones. You are charged for inter-zone data transfer within the cluster. The cost is $0.01 per 1 gibibyte. By default, the replication factor for a cluster is 3. This means you pay for inter-zone data transfer for replicating data to 2 out of the 3 zones.
Let's say you have a cluster with a replication factor of 3, which is the default. You write 10 GiB of data within a single zone. With a replication factor of 3, your 10 GiB of data is copied to 2 other zones. A total of 20 GiB of data (10 GiB x 2 copies) is transferred. The cost for this inter-zone data transfer is $0.20 (20 GiB x $0.01/GiB).
Connect cluster workers do not exchange data between each other.
Inter-region network charges
If you deploy a connector that reads or writes data across different regions, you are billed at the standard inter-region data transfer rates. This can range anywhere between $0.02 and $0.14 per GiB, depending on the source-destination region pair. For the most up-to-date information about inter-region data transfer pricing, see the subsection on “VM-VM data transfer pricing within Google Cloud” in the VPC peering guide. Connect clusters use the same endpoints provisioned by your Managed Service for Apache Kafka cluster to exchange data.
Managed Service for Apache Kafka service sets up all Private Service Connect (PSC) endpoints for your service. These endpoints are resources in a project you own and are billed separately. You can read more about how PSC is used here. The PSC charges include the following components:
Workers belonging to a Connect cluster act like clients with respect to your Managed Service for Apache Kafka cluster. When estimating the cost of your Connect deployment, remember to account for the increased consumer data processing charges.
For the most up-to-date information about VPC pricing, see the subsection on "Using a Private Service Connect endpoint (forwarding rule) to access a published service" in the "Private Service Connect" section of the VPC pricing guide.