PinnedAlireza SadeghiOpen Source Data Engineering Landscape 2024Exploration of the open source software in data engineering ecosystem11 min read·Feb 4, 2024--15--15
Alireza SadeghiHow to build a dual Incremental + snapshot data ingestion pipelineA useful batch data ingestion pattern for maximum data correctness and reliability as well as providing low latency access6 min read·Oct 1, 2023----
Alireza SadeghiTechniques For Periodically Extracting Data From Relational DatabasesPresenting techniques for extracting data from relational databases when building ETL pipelines for a data lake, DWH or data lakehouse10 min read·Sep 19, 2023----
Alireza SadeghiTechniques for Managing Dependency Between Data PipelinesIt’s a common challenge to manage dependency between data pipelines on data-driven systems and analytical platforms which having data…7 min read·Aug 29, 2023----
Alireza SadeghiInternal Storage Design of Modern Key-value Database Engines [Part 1]Deep dive into physical storage design implemented by many modern popular key-value stores such as Amazon Dynamo DB, Apache Cassandra, Riak8 min read·Aug 14, 2023----
Alireza SadeghiAirflow callbacks to Slack notifications for DAG monitoring and alertingIn this post I’ll demonstrate the step by step guide to integrate Airflow workflows with Slack for notification and monitoring purpose. The…5 min read·Jul 23, 2023----
Alireza SadeghiinTowards DevAdding Custom Country Map to Apache SupersetIn this post I demonstrate the steps followed to add a custom country map to superset repository and rebuild the app.5 min read·Jul 12, 2023--1--1