Airflow Unit Tests and Integration Tests
-
Updated
Nov 16, 2022 - Python
#
airflow-dags
Here are 262 public repositories matching this topic...
Gerador de DAGs no Apache Airflow para fazer clipping do Diário Oficial da União.
-
Updated
May 28, 2025 - Python
Data Engineering examples for Airflow, Prefect; dbt for BigQuery, Redshift, ClickHouse, Postgres, DuckDB; PySpark for Batch processing; Kafka for Stream processing
airflow kafka spark pyspark prefect airflow-dags ksqldb typer-cli dbt-bigquery dbt-postgres dbt-clickhouse dbt-redshift
-
Updated
Jun 19, 2025 - Python
-
Updated
Aug 18, 2021 - Python
Zero configuration Airflow plugin that let you manage your DAG files.
-
Updated
Sep 26, 2021 - Python
Orchestration of data science and earth observation models in Apache Airflow, scale-up with Celery Executor, experiment with jupyter notebook using a docker containers composition
-
Updated
Aug 23, 2022 - Python
🎵 LyricWave – AI Music Composer (Proof of Concept) 🎶 A personal project exploring automatic generation of unique MP4 songs. LyricWave blends lyrics with AI-generated melodies and synthetic vocals to experiment with new forms of musical expression. A creative testbed to push your ideas into sound. 🚀🎧
docker flask airflow mongodb docker-compose pytorch celery music-generation flask-api airflow-docker apache-airflow airflow-dags huggingface airflow-operators huggingface-transformers music-processing stable-diffusion suno-ai musicgen audiocraft
-
Updated
Mar 13, 2025 - Python
My self-learning about Apache Airflow
-
Updated
Jul 13, 2022 - Python
This is my Apache Airflow Local development setup on Windows 10 WSL2/Mac using docker-compose. It will also include some sample DAGs and workflows.
python docker yaml airflow sql superset data-engineering workflows airflow-docker treasuredata apache-superset airflow-dags wsl2 airflow-testing sample-dags airflow-community
-
Updated
Feb 9, 2024 - Python
Apache Airflow Guide
python airflow awesome big-data distributed data-engineering business-intelligence awesome-list airflow-plugin airflow-docker awesome-resources business-analytics airflow-dags airflow-operators
-
Updated
May 7, 2024 - Python
Orchestrate your Databricks notebooks in Airflow and execute them as Databricks Workflows
-
Updated
Aug 8, 2024 - Python
An end-to-end Twitter Data Pipeline that extracts data from Twitter and loads it into AWS S3.
python airflow twitter etl twitter-api scheduler s3 data-engineering tweepy boto3 data-pipeline apache-airflow etl-pipeline etl-job data-engineering-pipeline airflow-dags twitter-data-pipeline
-
Updated
Aug 26, 2023 - Python
Here I added 9 projects which have been made by me during my apprenticeship in Yandex.Practicum as data engineer.
python kubernetes redis sql kafka mongodb hadoop docker-compose postgresql pyspark vertica airflow-dags
-
Updated
Dec 16, 2023 - Python
A starting point for a data stack using Python, Apache Airflow and Metabase.
python docker template data-science airflow stack sphinx metabase data-engineering airflow-docker data-stack airflow-dags doccano
-
Updated
Dec 8, 2022 - Python
This project creates a basic web service for solving image-based CAPTCHAs. Using the Flask framework, it allows users to upload CAPTCHA images and employs an Optical Character Recognition (OCR) pipeline to extract the embedded text.
-
Updated
Mar 31, 2024 - Python
End-to-end data engineering processes for the NIGERIA Health Facility Registry (HFR). The project leveraged Selenium, Pandas, PySpark, PostgreSQL and Airflow
python data-science airflow data-mining apache-spark pyspark postgresql-database elt data-engineer data-ingestion data-warehousing database-design selenium-python etl-pipeline erdiagram data-engineering-pipeline etl-automation airflow-dags nigerian-data data-warehouse-architecture
-
Updated
Sep 1, 2022 - Python
Automated pipeline for generating and processing Dock balance reports using Apache Airflow, SFTP, AWS S3, and Lambda.
-
Updated
Apr 2, 2025 - Python
An ETL Data Pipelines Project that uses AirFlow DAGs to extract employees' data from PostgreSQL Schemas, load it in AWS Data Lake, Transform it with Python script, and Finally load it into SnowFlake Data warehouse using SCD type 2.
-
Updated
May 26, 2023 - 6B5B Python
Airflow DAGs repository synced for the MLOps plattform of the repository mlops-eks-mlplatform
-
Updated
Sep 28, 2023 - Python
Improve this page
Add a description, image, and links to the airflow-dags topic page so that developers can more easily learn about it.
Add this topic to your repo
To associate your repository with the airflow-dags topic, visit your repo's landing page and select "manage topics."