# dataengineering

4 min read

Alex Merced

Nov 8

Intro to SQL using Apache Iceberg and Dremio

#database #sql #dataengineering #datascience

4

22 min read

Sourabh Gupta

Oct 28

5 Best ETL Tools: A Comprehensive Comparison Guide

#etl #datascience #dataengineering #learning

3 min read

Geazi Anc

Oct 18

Data Engineering with Scala: Mastering Real-Time Data Processing with Apache Flink and Google Pub/Sub

#dataengineering #scala #datascience #flink

15 min read

Hiromi

Nov 5

SAP S/4HANA Cloud

#sap #dataengineering #erp #cloud

1

2 min read

luminousmen

Sep 29

Why Apache Spark RDD is immutable?

#dataengineering #bigdata #data

3 min read

Alex Merced

Oct 30

Data Modeling - Entities and Events

#database #dataengineering #datamodeling #dataanalytics

6 min read

Emmanuel Awa

Sep 26

Data Engineering in Observability: The Backbone of Modern Monitoring

#dataengineering #observability #opensource #banking

5 min read

Geazi Anc

Oct 28

Análise de dados de tráfego aéreo em tempo real com Spark Structured Streaming e Apache Kafka

#dataengineering #python #braziliandevs #spark

8 min read

Sourabh Gupta

Oct 28

Oracle to Snowflake Migration: Steps, Challenges & Best Practices

#datascience #dataengineering #tutorial #learning

3 min read

MissMati

Oct 27

Data Engineering in 2024: Innovations and Trends Shaping the Future

#dataengineering #data #analytics

6

13 min read

Paulet Wairagu

Oct 26

My journey learning Apache Spark

#spark #sql #dataengineering

2 min read

Sajjad Rahman

Oct 24

AWS DATA ENGINEER - 101

#aws #dataengineering #awschallenge #awsbigdata

2 min read

Abdullah Haggag

Oct 24

The Journey From a CSV File to Apache Hive Table

#hadoop #hive #bigdata #dataengineering

5

6 min read

Gustavo

Sep 19

Achieving Clean and Scalable PySpark Code: A Guide to Avoiding Redundancy

#pyspark #dataengineering #cleancode #python

5 min read

Pablo Arango Ramirez

Oct 22

Capítulo 2 - Modelos de Datos y Lenguajes de Consulta

#data #sql #nosql #dataengineering

7 min read

Alex Merced

Oct 21

All About Parquet Part 01 - An Introduction

#database #dataengineering

4 min read

Alex Merced

Oct 21

All About Parquet Part 05 - Compression Techniques in Parquet

#database #datascience #dataengineering

5 min read

Alex Merced

Oct 21

All About Parquet Part 10 - Performance Tuning and Best Practices with Parquet

#database #datascience #dataengineering

8

6 min read

Alex Merced

Oct 21

All About Parquet Part 03 - Parquet File Structure | Pages, Row Groups, and Columns

#database #datascience #dataengineering

5 min read

Alex Merced

Oct 21

All About Parquet Part 08 - Reading and Writing Parquet Files in Python

#database #datascience #dataengineering #data

9

5 min read

Alex Merced

Oct 21

All About Parquet Part 07 - Metadata in Parquet | Improving Data Efficiency

#data #database #dataengineering #datascience

5 min read

Alex Merced

Oct 21

All About Parquet Part 04 - Schema Evolution in Parquet

#database #datascience #dataengineering

5 min read

Alex Merced

Oct 21

All About Parquet Part 06 - Encoding in Parquet | Optimizing for Storage

#database #datascience #dataengineering

6 min read

prakhyatkarri

Oct 20

From a Unified Bronze Layer to Multiple Silver Layers: Streamlining Data Transformation in Databricks Unity Catalog

#databricks #unitycatalog #medallionarchitecture #dataengineering

5 min read

Regnard Raquedan

Sep 13

Clear Link Between DevSecOps and Data Engineering

#dataengineering #devops #devsecops #cloud

1 min read

Farah Kim

Oct 17

Still Using SQL, Python, & Excel for Data Deduplication? Here's Why You Need Better Tools.

#algorithms #ai #dataengineering

5

4 min read

Abdullah Haggag

Oct 17

Building a Big Data Playground Sandbox for Learning

#dataengineering #bigdata #opensource

5

5 min read

Dendi Handian

Sep 12

Capture Browser XHR/Fetch API Response Automatically into JSON Files

#help #dataengineering #chrome #javascript

1 min read

Mark Yu

Oct 16

The True Cost of Poor Data Quality: Why It Matters and How to Improve It

#database #datascience #dataengineering #management

6 min read

Pavol Z. Kutaj

Oct 14

Explaining the History of Data Lakehouse

#lakehouse #dataengineering #warehouse

2 min read

Marco Porracin

Sep 8

Building a User-Friendly, Budget-Friendly Alternative to dbt Cloud

#dbt #dataengineering #opensource #datascience

1 min read

Norton Augusto Herrero dos Santos

Oct 12

O que é Engenharia de Dados?

#dataengineering #datascience

1 min read

Nuthan Kishore

Sep 7

How SQL Spatial Data Solves Real-World Problems

#firstpost #spatialdata #dataengineering

6 min read

Pavol Z. Kutaj

Oct 11

Explaining CDC (Change Data Capture)

#databricks #dataengineering

1 min read

allan-pg

Oct 10

Handling Outliers 101: Why the IQR Method is Your Go-To Tool

#python #datascience #dataengineering #data

3 min read

Nico Bistolfi

Oct 9

Go vs Python for File Processing: A Performance and Architecture Perspective

#python #go #performance #dataengineering

5 min read

Alex Merced

Oct 7

Exploring Data Operations with PySpark, Pandas, DuckDB, Polars, and DataFusion in a Python Notebook

#python #database #datascience #dataengineering

4

13 min read

Lulu Cheng for jarrid.xyz

Sep 3

Secure Data Stack: Navigating Adoption Challenges of Data Encryption

#security #dataengineering #encryption #infosec

5 min read

Gichuki Edwin

Oct 7

Python 101: Introduction to Python as a Data Analytics Tool

#python #analytics #datascience #dataengineering

3 min read

Alex Merced

Oct 5

Ultimate Directory of Apache Iceberg Resources

#database #dataengineering #datascience #elasticsearch

14 min read

Chetan Gupta

Oct 4

Understanding OLTP and Choosing the Right Database

#dataengineering #mongodb #postgressql #mysql

6 min read

Alex Merced

Oct 4

Change Data Capture (CDC) when there is no CDC

#database #dataengineering #postgres

11 min read

Milcah

Aug 27

The Ultimate Guide to Data Engineering

#dataengineering #data

2 min read

Apache Doris

Aug 27

Evolution of Data Sharding Towards Automation and Flexibility

#opensource #dataengineering #database #automation

15 min read

Chetan Gupta

Sep 29

Data Showdown: OLAP vs. OLTP – The Battle of Real-Time and Analytics Titans

#bigdata #dataengineering #understanding #database

5 min read

Olga Shabalina for AWS Community Builders

Sep 28

Serverless PDF Processing with AWS Lambda and Textract

#cloudcomputing #serverless #lambda #dataengineering

10

9 min read

Aram Panasenco

Sep 25

The Simplest Data Architecture

#data #architecture #dataengineering #analytics

21 min read

Danny Chan for MongoDB Builders

Aug 21

🌐 Get started: What is MongoDB Operational Data Layer? (Part 1)

#database #mongodb #data #dataengineering

2 min read

Arooba Aqeel

Aug 18

ETL Real Estate Data Engineering with Redfin: From Extraction to Visualization

#dataengineering #etlproject #aws

3 min read

Lulu Cheng for jarrid.xyz

Sep 10

End-to-End AWS KMS Encryption and Decryption Tutorial

#tutorial #aws #kms #dataengineering

3 min read

Stephen

Sep 20

Working with Gigantic Google BigQuery Partitioned Tables in DBT

#bigquery #dbt #dataengineering #googlecloud

3 min read

DeadPunnk

Aug 16

Cogumelos Mágicos: explorando e tratando dados nulos com Mage

#python #mage #etl #dataengineering

6 min read

Sandeep

Sep 18

Apache Airflow

#airflow #development #workflow #dataengineering

4 min read

Anshul Kichara

Aug 14

Building and Managing Production-Ready Apache Airflow: From Setup to Troubleshooting

#dataengineering #devops #technology #software

2 min read

Paula David

Aug 27

The Must-Have Features of Modern Data Transformation Tools

#datatransformation #datatransformationtools #dataengineering #bigdata

6 min read

Chetan Gupta

Sep 16

An End-to-End Guide to dbt (Data Build Tool) with a Use Case Example

#dataengineering #datatransformation #dbt #bigdata

4

4 min read

Sohil Shah

Aug 25

Data Pipeline Techniques in Action

#opensource #datascience #softwaredevelopment #dataengineering

1 min read

jinesh vora

Sep 13

From Data Lakes to Data Mesh: The Emerging Trends of Data Management and Analytics

#datascience #database #dataengineering #python