Quick answer:
A data lake is a big, cheap pool of raw files. Instead of forcing data into tidy tables before storing it (the way a warehouse does), a lake accepts everything as-is: CSVs, JSON logs, images, database exports. You decide what structure it has later, at the moment you read it.
The pitch is “store first, ask questions later”. Object storage like Amazon S3 costs a couple of cents per gigabyte per month, so keeping everything forever is finally affordable. Whether you can ever find anything again is a separate matter (more on that below).
What is a data lake?
Technically, a data lake is just files in object storage (S3, Google Cloud Storage, Azure Blob) plus some convention about how they’re organized. There’s no database engine built in. The engine shows up at query time: Spark, Trino, or a warehouse pointing at the files from outside.
The defining idea is schema-on-read. A warehouse enforces structure when data goes in; a lake applies structure when data comes out. That makes writing to a lake nearly effortless and reading from it the hard part.
In practice most lake data ends up in Parquet, a compressed columnar file format that query engines can scan fast. A well-run lake looks less like a junk drawer and more like a warehouse that happens to live in open files.
Who coined the term, and why?
James Dixon, then CTO of Pentaho, coined “data lake” in 2010. His image: a data mart (a small, purpose-built slice of a warehouse) is bottled water, cleaned and packaged for one use. A lake is the natural body of water it came from. Everyone can draw from it for their own purpose.
The timing wasn’t an accident. Hadoop was peaking, storage was getting cheap, and companies were generating log data that didn’t fit neatly into rows and columns. The lake gave that mess a home and a flattering name.
What’s a data swamp?
The failure mode. You dump everything in, skip documentation and ownership, and 2 years later nobody knows which of the 14 copies of users_final_v3 is real. The data exists but nobody trusts it, so nobody uses it, so nobody maintains it.
This happened often enough that it drove 2 whole product categories: data catalogs (so you can find things) and data observability (so you know when things break). If a vendor tells you their lake can’t swamp, they’re selling something.
When does a lake beat a warehouse?
Three honest cases:
- Volume with low value per byte. Raw clickstream, sensor data, logs. Paying warehouse prices to store data you’ll query twice a year is a waste.
- Data that isn’t tabular. Images, audio, model training sets. Warehouses have gotten better at semi-structured data, but files are files.
- Machine learning workloads. Data scientists want raw history, and training pipelines read files directly. This is the world Databricks grew up in.
For plain business analytics (revenue by month, retention by cohort), a warehouse is still the shorter path. Most mid-size companies run both: lake for raw and heavy, warehouse for clean and queryable.
What are the benefits and drawbacks of a data lake?
Benefits of a data lake
Storage costs that round to nothing
Object storage runs a couple of cents per gigabyte per month, with colder tiers cheaper still. “Should we keep this?” stops being a budget question, and years of raw history stay available for the analysis you haven’t thought of yet.
Nothing gets rejected at the door
Logs, images, exports, half-broken CSVs from a vendor: all of it lands without anyone designing a schema first. For fast-moving teams, capture-now-model-later is a real advantage over the warehouse’s paperwork.
Open files, no hostage situation
Parquet on S3 is readable by Spark, Trino, DuckDB, every warehouse, and whatever wins next year. Your storage layer stops being a bargaining chip in any vendor negotiation.
Machine learning’s native habitat
Training pipelines want raw files and lots of them. Data scientists work against the lake directly instead of begging for warehouse exports.
Drawbacks of a data lake
The swamp is the default outcome
Without enforced ownership and documentation, a lake decays into 10,000 files nobody trusts. Order requires ongoing human effort the technology does none of for you.
Reading is the hard part
Schema-on-read means every consumer re-solves the structure problem. A business analyst can’t self-serve against raw JSON the way they can against a modeled warehouse table.
Databases exist for reasons
Plain files have no transactions, no enforcement, no efficient updates. You either add a table format layer (which is the lakehouse, with its own maintenance duties) or live with the sharp edges.
Governance gets harder, not easier
Finding and deleting one customer’s personal data across a billion loose files is a genuinely nasty problem. Regulators don’t accept “it’s in there somewhere”.
Where does the lake stand now?
The lake and the warehouse are converging into the lakehouse. Open table formats like Apache Iceberg and Delta Lake bolt database behavior (transactions, schema enforcement, time travel) onto plain lake files. Meanwhile Snowflake and BigQuery learned to query lake files directly, so the same bytes can serve both worlds.
My take: the pure lake, as a standalone thing you build and defend, is fading. The lake as the cheap storage layer underneath everything else is winning completely. The files won; the swamp got a management layer.