About This Tutorial
As data-driven organizations grow, life gets… complicated
- data and task pipeline requirement grow
- more data sources and sinks are added to the mix
- tasks become dependent on each other in complicated ways
- service level agreements need to be adhered to and monitored
- schedules need to be maintained
Apache Airflow, initially developed and open-sourced by those nice folks at Airbnb, solves these problems and more.
In this tutorial session, we’ll be starting off by getting to grips with Airflow as a stand-alone tool, and then we’ll see how we can get it to play nice with Django.
It would be useful if attendees were familiar with Django coming into this session. No prior experience with Airflow is needed.
What we’ll cover:
- What Airflow is all about and what kinds of problems you could use it to solve
- Airflow development setup
- Getting to grips with Dags and Tasks
- Integration with Django
- Advanced topics
Presenters
- I create syllabuses to teach critical tech skills
- I mentor learners and staff in technical and “soft” skills
- I help design and set up human/management systems (we have a LOT of people we need to keep on track)
- I write code so we can improve, automate, and modularise our interventions so we can impact more people in meaningful ways
Sheena O'Connell (she/her)
I’ve been programming in some form since my early teens. After high school, I went on to get an Honors degree in electrical engineering from the University of the Witwatersrand, South Africa. I’ve always been drawn to software development and to teaching. I worked for a string of startups and gained a lot of experience in all aspects of software development, from requirements gathering and UX design to implementation, deployment, and monitoring.
I am currently the CTO of Umuzi. Our purpose is to reduce social inequality in Africa through digital education.
My role at Umuzi is many-fold:
I spend my free time climbing, playing guitar, reading and writing.