Review: Building a Real Time Data Warehouse

Apr 11, 2020 · 4 min read

Many data engineers coming from traditional batch processing frameworks have questions about real time data processing systems, like

“What kind of data model did you implement, for real-time processing?”

3 Key Points to Help You Partition Late Arriving Events

Apr 5, 2020 · 5 min read

3 Techniques to process events arriving late in real time streaming systems

Scheduling a SQL script, using Apache Airflow, with an example

Mar 29, 2020 · 3 min read

One of the most common use cases for Apache Airflow is to run scheduled SQL scripts. Developers who start with Airflow often ask the following questions

“How to use airflow to orchestrate sql?”

10 Key skills, to help you become a data engineer

Mar 20, 2020 · 3 min read

Key skills required for a data engineer. If you are getting into data engineering, this post gives you a ordered list of topics you can start learning.