ETL

Quick answer:

ETL stands for Extract, Transform, Load: pull data out of a source system, reshape and clean it, then load the finished result into a destination (usually a data warehouse). It’s the original pattern for moving data around, dating back to the mainframe era, and the word survives as shorthand for “data plumbing” even where the actual order of steps has changed.

If you hear a data engineer say “I spent all week fixing ETL”, they mean pipelines broke. The acronym long ago outgrew its literal meaning.

What is ETL?

Three steps, in order:

  • Extract: read data from the source. A production database, a SaaS API, a folder of files.
  • Transform: clean it in transit. Fix types, rename fields, join tables, mask sensitive columns, aggregate.
  • Load: write the polished result into the destination warehouse.

The defining trait is that transformation happens before loading, on a separate processing server that sits between source and destination. Only the finished, modeled tables ever reach the warehouse.

Why did transformation come first?

Economics. Through the 1990s and 2000s, warehouse compute was the most expensive resource in the building. A Teradata appliance cost millions, so you didn’t waste its cycles on cleanup work. You did the dirty work on cheaper middleware (Informatica, IBM DataStage, Talend, SSIS) and loaded only what analysts would query.

Storage was expensive too, which meant keeping raw copies of everything in the warehouse was off the table. ETL was the rational answer to 2 constraints that have since disappeared.

How is ETL different from ELT?

Same 3 steps, different order. ELT loads raw data into the warehouse first and transforms it there, using the warehouse’s own engine. Cheap cloud storage and elastic compute made this practical around the mid-2010s, and it’s now the default for analytics: Fivetran or Airbyte handle extract and load, dbt handles transform inside the warehouse.

The practical wins of flipping the order: you keep the raw history (so you can re-transform when logic changes without re-extracting), and transformations become SQL that analysts can read, instead of jobs locked inside a middleware tool.

When is classic ETL still the right call?

More often than the modern-stack marketing admits:

  • Compliance. If PII must never land in the warehouse (healthcare, banking), you have to mask or drop it in transit. That’s transform-before-load by definition.
  • Warehouse cost control. Transforming in Snowflake means paying Snowflake for every run. Heavy pre-aggregation upstream can be dramatically cheaper at scale.
  • Streaming. Real-time pipelines transform data as it flows because there’s no “later”. Nobody calls it ETL, but that’s what it is.
  • Destinations that can’t transform. If you’re loading into an operational system rather than a warehouse, the work has to happen en route.

What are the benefits and drawbacks of classic ETL?

Benefits of ETL

Sensitive data never lands where it shouldn’t

Masking, hashing, and dropping fields happens in transit, so the warehouse only ever holds what it’s allowed to hold. For regulated data, this property isn’t optional, and ELT has to reinvent it awkwardly.

The warehouse bill stays small

Transformation compute runs on your own middleware instead of metered warehouse credits. At heavy scale, pre-aggregating upstream is real money saved every month.

Destinations get finished goods

Only clean, modeled data arrives, so the warehouse never accumulates a raw layer that analysts stumble into by accident. Some governance teams count that as a feature, honestly.

Decades of tooling maturity

Informatica-class platforms have handled bank-grade volumes since the 1990s. Boring, proven, documented: qualities that matter more in year 5 than in the proof of concept.

Drawbacks of ETL

Raw history is gone

You kept only the transformed output, so when the business logic changes, you re-extract from sources.. if the sources still have the data. This is the single biggest reason ELT won.

Transformation lives with specialists

Logic locked inside middleware belongs to the engineers who run the middleware. Analysts wait in a ticket queue for changes they could have written themselves in SQL.

Coupled steps break together

Because extract and transform ship as one job, a change to either means retesting both. Pipelines get brittle exactly where they should be dumb.

Slower iteration, by design

Model the target, build the mapping, deploy the job, then see the data. ELT’s “load it, poke at it in SQL” loop runs circles around that in the exploratory phase.

Where does ETL stand now?

As an architecture, it lost the analytics mainstream to ELT and isn’t coming back. As a word, it won: job listings say “ETL engineer”, vendors sell “ETL tools” that are technically ELT tools, and everyone understands what’s meant. Even this directory files ingestion tools under ETL-adjacent labels, because that’s the term people search.

The honest summary: learn the acronym for the history and the interviews, then learn ELT for how the work is actually arranged today. And remember the pattern isn’t religion. The right order of T and L is whatever your constraints (compliance, cost, latency) say it is.

Avatar photo

Panoply

Panoply wrote for the Panoply blog.