Apache Flink

Editor’s note: This Apache Flink review is based on my hands-on test of Flink 2.3.0 (the June 2026 release) as a standalone cluster on my own machine, plus PyFlink 2.3.0 from pip, in October 2026; the managed Flink services from Confluent, AWS, Decodable, Alibaba and Ververica were evaluated from their published rates and documented behavior, not run. I weigh pricing transparency, real-world adoption signals, development momentum, openness and exit costs, practitioner sentiment and my own verdict.

Quick verdict: I recommend Apache Flink for teams whose pipelines are stateful (joins across streams, per-key windows, CDC materialization) and who either have platform engineers to run it or will buy it managed. Go in knowing the software is free but the cluster never scales to zero, connectors still lag the 2.x line, and the 1.x to 2.x upgrade carries no state compatibility guarantee.

Key Takeaways

• Apache 2.0 and free; managed Flink starts at $0.11 per KPU-hour on AWS (plus one orchestration KPU), $0.21 per CFU-hour on Confluent and $0.12 per credit on Decodable.
• My RocksDB-backed SQL window job checkpointed every 5 s in about 0.8 s, stopped with an 11 MB savepoint in 2.9 s and restarted at double parallelism from two SET lines.
• The adaptive scheduler rescaled my running job from parallelism 1 to 3 over REST in 15.4 s, with no resubmit.
• The 605 MB download bundles no Kafka connector, and Cassandra, HBase, Hive, MongoDB, Pulsar and RabbitMQ still list only 1.x releases.
• PyFlink’s pinned apache-beam (2.54 to 2.61) had to compile from source on my Mac, and a Python UDF added about 50% wall time to the same SQL aggregation.

Among the stream processing tools in the Panoply directory, Flink is the engine underneath Confluent‘s managed Flink and the one Apache Kafka, Redpanda and Debezium feed.

In this review, I’ll cover Apache Flink’s managed pricing, my savepoint and rescale tests, its state model, the 2.x SQL line, and how it compares with Spark Structured Streaming, Kafka Streams and RisingWave.

Apache Flink Pros and Cons

How Much Does Apache Flink Cost?

Apache Flink is Apache 2.0 and costs nothing; you pay for compute that never stops, either as your own cluster or by the unit from a managed vendor.

• Self-managed ($0 software): for teams with platform engineers; Zalando’s ad-event join runs on about EUR 30 a day of EC2 after moving to Flink, down from EUR 80.
• Amazon Managed Service for Apache Flink ($0.11 per KPU-hour): for AWS shops; a KPU is 1 vCPU and 4 GB, every app bills one extra orchestration KPU, storage is $0.10 per GB-month (50 GB per KPU), and Studio notebooks bill 2 extra KPUs.
• Confluent Cloud for Apache Flink ($0.21 per CFU-hour): for Confluent Kafka users; billed at $0.0035 per CFU-minute with a 1 CFU-minute minimum per statement, a MAX_CFU cap, a regional multiplier and a 99.99% SLA.
• Decodable ($0.12 per credit on demand, $0.10 Enterprise): for CDC-first teams; Small, Medium and Large tasks burn 1, 2 and 4 credits an hour, and the free tier never expires, with 4 concurrent tasks and 24 hours or 10 GiB of retention.
• Alibaba Realtime Compute for Apache Flink ($0.05 to $0.10 per CU-hour): for Alibaba Cloud users; $0.075 in US (Virginia), plus a fixed 2-CU management fee every hour, even when idle.
• Ververica and Google (no published rate): Ververica is “Talk to Sales” with three years of support per minor version (1.20 until August 2, 2027); Google’s BigQuery Engine for Apache Flink is in Preview with no rates.

Route	Unit	Rate	Minimum or overhead	SLA
Self-managed Apache Flink	Your VMs or Kubernetes nodes	$0 software	Always-on cluster plus recovery headroom	None, you run it
Amazon Managed Service for Apache Flink	KPU (1 vCPU, 4 GB)	$0.11 per KPU-hour (US East)	1 orchestration KPU per app, $0.10 per GB-month storage	AWS service terms
Confluent Cloud for Apache Flink	CFU	$0.21 per CFU-hour, $0.0035 per CFU-minute	1 CFU-minute per statement, regional multiplier	99.99%
Decodable	Credit per task-hour	$0.12 on demand, $0.10 Enterprise	Small 1, Medium 2, Large 4 credits an hour	None on Free, 99.9% On Demand, 99.99% Enterprise
Alibaba Realtime Compute for Apache Flink	CU	$0.05 to $0.10 per CU-hour by region	2-CU management fee every hour	Alibaba Cloud service terms
Ververica / Google BigQuery Engine	Contract / Preview	Not published	Talk to Sales / Preview	Contract / none in Preview

What a Real Workload Costs

My arithmetic for one always-on job sized at 4 units, running 730 hours a month:

• AWS: (4 KPUs + 1 orchestration) x $0.11 x 730 = about $402, plus 200 GB of running storage x $0.10 = $20, so about $422.
• Confluent: 4 CFUs x $0.21 x 730 = about $613.
• Decodable: one Large task (4 credits) x $0.12 x 730 = about $350.
• Alibaba (US Virginia): (4 CUs + 2 management) x $0.075 x 730 = about $329.

A CFU is not a KPU, and a Decodable Large task is not defined as 4 vCPUs, so treat these as list-price anchors, not a benchmark. A self-managed cluster carries its own overhead: it runs 24/7 and is over-provisioned to survive a restart, because recovery reloads all state.

Is Apache Flink Good Value for Money?

• Good fit: stateful jobs replacing a homegrown service; Zalando went from 20 pods to 5.
• Less convincing fit: one small job, where AWS’s orchestration KPU doubles a 1-KPU bill and Alibaba’s 2-CU fee triples a 1-CU bill.
• Budget priority: size for recovery and RocksDB native memory, not average load.

Reviewer’s Notes: I recommend starting the first stateful job on Decodable’s free tier, or on AWS at 1 KPU, which bills as 2 and comes to about $161 a month (2 x $0.11 x 730) before storage. Self-manage only once you run more than a handful of jobs and have someone who will own RocksDB tuning.

My Experience With Apache Flink

Flink 2.3.0 refused my Mac’s stock Java 8 outright; on Java 17 the 605 MB download (624 MB unpacked) brought a JobManager up in 6.1 s. Every timing below is a single-machine loopback number from one TaskManager on a 16 GB Apple Silicon iMac, not cluster performance.

Setting Up a Cluster and Running the First Job

The Java 8 error was UnsupportedClassVersionError (class file 55.0, so Java 11 is the floor). The single conf/config.yaml is strict YAML: a second top-level jobmanager: block I appended silently stopped the JobManager from starting until I merged the two.

The bundled WordCount ran in 755 ms, or 3.5 s including client JVM start. A 'connector'='kafka' table then failed at planning with “Could not find any factory for identifier ‘kafka'”, listing the seven bundled factories: blackhole, datagen, filesystem, legacy-csv, print, python-arrow-source and python-input-format.

Checkpoints, Savepoints and Restarting at Double Parallelism

My SQL job read datagen at a 50,000 rows/s target across 100,000 customer keys into a 1-minute tumbling window on RocksDB, with 5 s checkpoints. At parallelism 1 on one slot it sustained 17,478 rows/s, well short of the target, while the backpressure endpoint still reported “ok”.

After 45 s, 9 checkpoints had completed and 1 had failed, at a p50 size of 9.1 MB and 763 ms end to end. bin/flink stop with a savepoint path finished in 2.9 s and left an 11 MB savepoint. Two SET lines in front of the same script, execution.state-recovery.path and parallelism.default = 2, brought it back at parallelism 2, with restored.is_savepoint true and 4 checkpoints completed in the first 20 s.

Rescaling a Running Job Without a Resubmit

With the adaptive scheduler on, a PUT to the job’s resource-requirements endpoint with an upper bound of 3 moved both vertices from parallelism 1 to 3 with no savepoint, no resubmit and the same job ID. The new 2.3 Rescales tab logged INITIAL_SCHEDULE in 347 ms and UPDATE_REQUIREMENT in 15,356 ms. The rescale cost one checkpoint (“Checkpoint Coordinator is suspending”) and a restore from the last completed one, and the three source subtasks then summed to 34,704 rows/s.

The 2.3 SQL Additions, Tried

FROM_CHANGELOG rejected my first op mapping with “Upsert mode (without UPDATE_BEFORE) is not supported in this version”, so 2.3 handles retract changelogs only. With UPDATE_BEFORE added, five input rows came back as +I, +I, -U, +U, -D. A guessed on_time argument crashed the planner with a raw ClassCastException instead of a validation error.

An upsert query into a sink whose primary key differed from the upsert key failed planning and demanded ON CONFLICT DO DEDUPLICATE, DO ERROR or DO NOTHING. On 2.2 the same query would have silently added a SinkUpsertMaterializer, so a 2.2 job can stop planning after an upgrade to 2.3 until you add the clause.

PyFlink

The apache-flink wheel is prebuilt for arm64, but its pinned apache-beam (2.54 to 2.61) has no macOS arm64 wheel, so pip compiled Beam from source. The first build died on a missing pkg_resources until I pinned setuptools<81. The venv came to 619 MB, 348 MB of it pyflink with its own copy of the Flink jars.

On 200,000 datagen rows paced at 10 s, a Python scalar UDF job finished in 15.6 s against 10.7 s for the same aggregation in pure SQL, so the UDF cost about 5 s.

Reviewer’s Notes: The savepoint restore at a new parallelism and the REST rescale both worked on the first attempt, the part of Flink that earns its reputation. Budget setup time for connector jars, Java 17, strict YAML and, on a Mac, a Beam build. I did not run Kafka, Kubernetes, ForSt, a multi-node cluster or any managed service; those topics below are documented behavior.

State, Checkpoints and Exactly-Once: Where Flink Earns Its Keep

Flink’s exactly-once guarantee rests on checkpoints of its keyed state, and the state backend sets both speed and how large that state can grow:

• HashMap: the default, keeps state on the JVM heap and is the fastest.
• EmbeddedRocksDB: keeps state on local disk, snapshots asynchronously, is roughly an order of magnitude slower than heap and caps a single key or value at 2^31 bytes; incremental checkpoints turn on with execution.checkpointing.incremental: true.
• ForSt: keeps SST files on remote storage such as S3 and is the only backend with async State API V2, but it is experimental and supports neither canonical savepoints nor full snapshots.

Under heavy backpressure, barrier propagation dominates checkpoint time; the fixes are removing the bottleneck, buffer debloating or unaligned checkpoints. Version 2.3 adds opt-in checkpointing during recovery from an unaligned checkpoint (FLIP-547), off by default behind two flags, and a watermark-alignment catch-up buffer (default 3) that lifts the old cap of roughly 30 event-time seconds per real second; pipeline.watermark-alignment.buffer-size: 0 restores 2.2 behavior.

One fleak.ai practitioner watched GC pauses jump from about 200 ms to over 5 s after adding one field to a stored object, causing checkpoint timeouts and restart loops that took two days to diagnose. The same write-up says “you become a RocksDB tuning expert”, calls savepoint-stop-deploy-restart “incredibly fragile” and warns that schema changes “can, and will, ruin your week”.

Zalando’s engineering team found default compaction did not reclaim expired state, saw the kernel OOM-kill pods over native RocksDB memory and had Karpenter evict pods mid-rescale; with about 70M timer entries, its interval join lost to a hand-written KeyedCoProcessFunction. A job that constantly fails on checkpoint timeouts or sits under constant backpressure may need a redesign, not more tuning.

My 5 s checkpoints at 763 ms on 9 MB of state are the easy case; Netflix reported jobs with up to 4 TB of state in 2024. Kafka Streams also keeps local state in RocksDB but restores it by replaying changelog topics, and Spark Structured Streaming writes its checkpoint once per micro-batch.

Flink SQL, Process Table Functions and the 2.x Feature Line

Flink 2.0 (March 24, 2025) removed the DataSet API, the Scala DataStream and DataSet APIs, SourceFunction, SinkFunction and Sink V1, TableSource and TableSink, per-job mode, Java 8 and flink-conf.yaml (a migration tool converts it). The three releases since:

• 2.1 (July 31, 2025): Process Table Functions (FLIP-440) brought managed state and timers into SQL, alongside CREATE MODEL with ML_PREDICT, a VARIANT type, Delta Join (on by default, but it needs external infrastructure such as Apache Fluss) and an opt-in multi-join; 116 contributors shipped 16 FLIPs.
• 2.2 (December 4, 2025): VECTOR_SEARCH (FLIP-540) arrived with no open-source VectorSearchTableSource yet, plus FRESHNESS and DISTRIBUTED BY for materialized tables, SinkUpsertMaterializer V2 and async Python DataStream functions for LLM calls; 73 contributors, 9 FLIPs and 220+ resolved issues.
• 2.3 (June 25, 2026): FROM_CHANGELOG and TO_CHANGELOG (retract-only in my test), ALTER MATERIALIZED TABLE with START_MODE (default FROM_BEGINNING, so a changed query reprocesses history), ON CONFLICT, an experimental native S3 filesystem on AWS SDK v2, opt-in adaptive partition selection (up to about 3x throughput under skew) and an applications layer with its own REST API; about 91 contributors, 15 FLIPs.

The subprojects move on their own clocks:

• Flink CDC 3.7.0: works with Flink 1.20.x and 2.2.x; 3.6 added an Oracle source and a Hudi sink, 3.5 a PostgreSQL pipeline source and a Fluss sink.
• Flink Agents 0.3.1 (July 25, 2026): a preview with experimental APIs for Flink 1.20 to 2.3.
• Flink ML 2.3.0: still compatible with Flink 1.17 only.

RisingWave and Materialize sell SQL materialized views as the whole product; Flink’s materialized tables are the in-engine equivalent, but they need a catalog that supports them, so I did not run them on my standalone cluster.

Connectors, Deployment and the 2.x Upgrade Tax

Kafka connector 5.0.0 supports Flink 2.1.x and 2.2.x (4.0.1 covers 2.0.x, 3.4.0 covers 1.20.x), JDBC 4.1.0 covers 2.1 and 2.2, HTTP 1.0.0 covers 2.2, and Elasticsearch 4.0.0 and the AWS connector 6.0.1 stop at 2.0.x; none of them lists 2.3 yet. Still without any 2.x release: Cassandra 3.2.0, HBase 4.0.0, Hive 3.0.0, MongoDB 2.0.0, OpenSearch 2.0.0, Pulsar 4.2.0, RabbitMQ 3.0.1 and Prometheus 1.0.0. Paimon, Iceberg and Delta ship from their own projects, and my 2.3 download bundled only the seven factories listed above.

• Kubernetes Operator 1.16.1: supports Flink 1.19 to 2.3, with the autoscaler since 1.7 and autotuning since 1.8.
• Java: 17 is the default and recommended runtime, 11 is supported and 21 is experimental; AWS’s managed service dropped 11 and does not support 21.
• LTS: 1.20 got 1.20.5 on June 3, 2026 but has no official Apache end date (FLIP-458 proposed two years); Ververica supports it to August 2, 2027.
• Security: a remote-code-execution CVE drove a May 2026 patch wave (1.20.4, 2.0.2, 2.1.2, 2.2.1) in a quarter with about two dozen vulnerability reports.

To move from 1.20:

• Remove DataSet, Scala API and SourceFunction code, then move to Source and Sink V2 and the 2.x connector releases.
• Check state: Kryo moved from 2.24 to 5.6, and POJOs holding HashMap or ArrayList can break restores, while Avro and Protobuf state is unaffected.
• Migrate flink-conf.yaml to config.yaml, take a savepoint, deploy on 2.x and validate in shadow mode.
• On Kinesis, the EFO source can fail on resharding (FLINK-37648) and deadlock with the sink under backpressure (FLINK-34071); Zalando also wrote its own Protobuf deaggregation layer.

Flink has 128 committers and 59 PMC members (June 2026), but the user@ list fell about 80% to roughly 15 messages a month while dev@ hit about 580 in May 2026, a shift the PMC puts down to Slack, Stack Overflow and LLM assistants. The PMC still flags connector maintenance as a concern.

How Does Apache Flink Compare to Competitors?

Confluent, AWS and Decodable sell the Flink engine itself, so the real rivals are Spark Structured Streaming, Kafka Streams and the streaming databases, and each wins a narrower job than stateful, low-latency work on large keyed state.

Tool	Model	License or price anchor	Wins when
Apache Flink, self-managed	Distributed event-at-a-time engine	Apache 2.0, $0 software	Stateful joins and windows, sub-second latency, any cloud
Confluent Cloud for Apache Flink	Serverless SQL statements	$0.21 per CFU-hour, billed per minute	You already run Confluent Kafka
Amazon Managed Service for Apache Flink	Provisioned managed Flink	$0.11 per KPU-hour plus 1 orchestration KPU	AWS-centered stacks
Decodable	Managed Flink SaaS	$0.12 per credit, permanent free tier	CDC-first teams proving a first job
Spark Structured Streaming	Micro-batch engine	Apache 2.0; Databricks real-time mode	You already run Spark and tolerate seconds
Kafka Streams	Java library, no cluster	Apache 2.0	Kafka-to-Kafka microservices
RisingWave	Streaming database	Apache 2.0 self-managed, cloud from $0.227 per RWU-hour	SQL materialized views are the whole job
Materialize	Streaming database	Credit-priced	Incremental SQL views as the product
Arroyo	Rust SQL stream engine	Open source, Cloudflare-owned since April 2025	Teams that want a Rust engine

• Spark Structured Streaming: micro-batch with a latency floor in classic mode; streaming practitioner Kai Waehner (July 2026) points to a Databricks low-latency case at P50 44 ms and P99 near 500 ms, with Kafka, Aerospike and Redis still underneath. It wins when the team already runs Spark and seconds are acceptable.
• Kafka Streams: an embedded Java library under Apache 2.0 with no cluster to run, the pick for Kafka-to-Kafka microservices where state lives next to the service; it loses on SQL, non-JVM teams and any source or sink that is not Kafka.
• RisingWave: a SQL-first streaming database from $0.227 per RWU-hour in the cloud, the pick when SQL views are the whole job, with a smaller ecosystem than Flink and no permanent free tier.
• Materialize: incremental SQL views priced in credits, the pick when a consistent SQL view is the deliverable rather than a pipeline; check its current credit rate before comparing it with a CFU or KPU.
• Confluent Cloud for Apache Flink: serverless statements billed per minute with a 99.99% SLA, the default inside Confluent Kafka; MAX_CFU can be raised but never lowered.
• Amazon Managed Service for Apache Flink: the cheapest unit at $0.11 with in-place version upgrades, but its 2.2 runtime lacks materialized tables, ForSt, Java 21 and Studio notebooks, and several 2.x connectors are missing.
• Decodable: a free tier that never expires and 1-minute billing make it the cheapest place to prove a CDC job, with 99.99% reserved for annual Enterprise contracts.
• Arroyo: a Rust SQL engine Cloudflare acquired in April 2025 that stays open source and self-hostable, still a young project next to Flink.

Debezium and Estuary Flow feed any of these, so a CDC source does not settle the choice; state size and who runs the cluster do.

How I Reviewed Apache Flink

I ran Flink 2.3.0 standalone on a Temurin 17 JRE on my own machine: WordCount, a RocksDB-backed SQL window job with 5 s checkpoints, stop with savepoint and restore at parallelism 2, a live REST rescale from 1 to 3 under the adaptive scheduler, the 2.3 FROM_CHANGELOG and ON CONFLICT behavior, a missing-connector failure, and PyFlink 2.3.0 with a Python UDF.

I evaluated the managed rates and SLAs from Confluent, AWS, Decodable, Alibaba, Ververica and Google, release notes from 2.0 to 2.3, connector and operator compatibility, and Kubernetes and large-state behavior from Zalando and Netflix engineering material and practitioner posts.

I weighed pricing transparency, real-world adoption signals, development momentum, openness and exit costs, practitioner sentiment and my own verdict. Rivals are compared on list prices. Prices current as of October 2026.

Should You Build on Apache Flink?

I recommend Apache Flink for stateful joins, per-key windows, CDC materialization and sub-second latency, provided you either have platform engineers on staff or buy a managed route. In my test a savepoint restore at a new parallelism and a live REST rescale both worked on the first attempt, and the 2.x SQL line now covers Process Table Functions, ML_PREDICT and changelog conversion inside the engine.

Skip it for stateless or seconds-latency work where Spark Structured Streaming or Kafka Streams already fit, and skip it if nobody on the team will own RocksDB tuning and checkpoint timeouts. Shops that want everything as SQL views should price RisingWave first, and anyone depending on Cassandra, Hive, Pulsar or another connector still stuck on 1.x should stay on 1.20 LTS until a 2.x release ships.

Download 2.3.0 at flink.apache.org, run the SQL client against datagen for an afternoon as I did, then price the first real job on Decodable’s free tier or on AWS at 1 KPU before committing to a cluster.

FAQ

Is Apache Flink free?

Yes. Flink is Apache 2.0 open source with no license fee; you pay for the machines or a managed service. List rates are $0.11 per KPU-hour on AWS plus one orchestration KPU per app, $0.21 per CFU-hour on Confluent, $0.12 per credit on Decodable and $0.05 to $0.10 per CU-hour on Alibaba. See How Much Does Apache Flink Cost? above for a worked monthly bill.

Which Java and Python versions does Flink 2.3 need?

Java 11 is the minimum, Java 17 is the default and recommended runtime, and Java 21 is experimental. On Java 8, Flink 2.3 fails immediately with UnsupportedClassVersionError, which is what my Mac’s stock runtime produced. PyFlink 2.3 publishes wheels for Python 3.9 to 3.12, and on arm64 Macs its apache-beam dependency builds from source.

Does Flink need Kafka?

No. Flink has connectors for Kinesis, JDBC databases, files and CDC sources, and the download bundles datagen and filesystem; the Pulsar connector (4.2.0) still targets 1.20 only. Kafka is the most common source, but its connector (5.0.0 for Flink 2.1 and 2.2) is a separate jar you drop into lib/; without it, a Kafka table fails at planning, as it did in my test.

Is Flink better than Spark Structured Streaming?

It is different. Flink processes each event as it arrives in a long-running job, while Spark runs micro-batches, though Databricks markets a sub-second real-time mode. I pick Flink for stateful, sub-second work with large keyed state, and Spark for teams already on Spark that tolerate seconds of latency. See the comparison above for the full table.

Is Flink 1.20 still supported?

Yes for now. 1.20 is the LTS line, with 1.20.5 released June 3, 2026, but Apache has set no official end date; Ververica supports 1.20 until August 2, 2027. Plan the move to 2.x with a savepoint and a shadow run, because state compatibility between 1.x and 2.x is not guaranteed.