Databases Intermediate

What is ETL (Extract, Transform, Load)?

A data pipeline process that extracts data from sources, transforms it into a suitable format, and loads it into a destination system.

ETL processes move data between systems. Extract pulls data from databases, APIs, files, or streams. Transform cleans, validates, enriches, and restructures the data. Load writes the processed data to a data warehouse or target system.

Modern variations include ELT (load raw data first, transform in the warehouse) and real-time streaming. Tools include Apache Airflow, dbt (transform layer), Apache Spark, and cloud services like AWS Glue. ETL is fundamental to data warehousing and analytics.

Learn More About This Topic

SQL for Absolute Beginners

Related reading

Redis Fundamentals

Related reading

MongoDB Fundamentals: Building Scalable NoSQL Applications with Document Databases

Related reading

Related Terms

Window Function

An SQL function that performs calculations across a set of rows related to the current row without collapsing the result set.

Stored Procedure

A precompiled collection of SQL statements stored in the database that can be executed as a single unit.

PostgreSQL's binary JSON data type that stores JSON documents with indexing, querying, and manipulation capabilities.

A database command that shows how the query planner will execute a SQL query, revealing join methods, scan types, and estimated costs.

A database object that enables row-by-row processing of query results, useful for operations that cannot be done in bulk.

Database Sharding

A horizontal scaling strategy that distributes data across multiple database servers based on a partition key.

View All Databases Terms →