Serverless Data Engineering Pipelines in Python

Photo by Kampus Production from Pexels


Including AWS best practices to avoid them

Photo by ThisIsEngineering from Pexels
· #1 Wrong logging configuration
· #2 Failing to enable “Auto-assign public…


Some activities we don’t spend enough time on

People in a meeting
Photo by Canva Studio from Pexels

1. Backups and Preventing Accidental Deletion


Amazon CTO Werner Vogels might be right — and here’s why

Code on female’s face
Photo by cottonbro from Pexels


Some use cases may surprise you.

Photo by Andrea Piacquadio from Pexels


Make better design choices by avoiding those pitfalls

Female writing on a whiteboard
Photo by ThisisEngineering RAEng on Unsplash

1. Treating Schema Design as a One-Off Project


A deep dive into boto3 and how AWS built it

Photo by Kindel Media from Pexels


Ensure data quality in your S3 data lake using Python, AWS Lambda, SNS, and Great Expectations

Chart, magnifying glass and eye glasses on table
Photo by Anna Nekrashevich from Pexels | Branded content disclosure.


And is it a good thing?

City view by the river
Photo by Lukas Hartmann from Pexels | Branded content disclosure.


Real-time technologies are powerful but they add significant complexity to your data architecture

Photo by Victor Wang from Pexels

Anna Geller

Data Engineer, M.Sc. in BI, AWS Certified Solution Architect: www.annageller.com. Get my articles via email: https://annageller.medium.com/subscribe

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store