Including AWS best practices to avoid them

Photo by ThisIsEngineering from Pexels
· #1 Wrong logging configuration
· #2 Failing to enable “Auto-assign public IP”
· #3 Storing credentials in plain text in the ECS task definition
#1. Using AWS Systems Manager Parameter Store
#2. Using AWS Secrets Manager
What’s the difference between AWS Systems Manager Parameter Store and AWS Secrets Manager?
· #4 Using the same IAM task…


Some activities we don’t spend enough time on

People in a meeting
Photo by Canva Studio from Pexels

1. Backups and Preventing Accidental Deletion

Have you ever deleted something prematurely only to figure out that there is no backup? A good rule of thumb is to check three times before deleting anything. This may involve cross-checking if we are in the right environment, region, database schema, or S3 bucket.


Amazon CTO Werner Vogels might be right — and here’s why

Code on female’s face
Photo by cottonbro from Pexels


Some use cases may surprise you.

Photo by Andrea Piacquadio from Pexels


Make better design choices by avoiding those pitfalls

Female writing on a whiteboard
Photo by ThisisEngineering RAEng on Unsplash

1. Treating Schema Design as a One-Off Project

Building data assets is an ongoing process. As your analytical needs change over time, the schema will have to be adjusted as well. Treating data modeling as a one-off activity is unrealistic. …


A deep dive into boto3 and how AWS built it

Photo by Kindel Media from Pexels


Ensure data quality in your S3 data lake using Python, AWS Lambda, SNS, and Great Expectations

Chart, magnifying glass and eye glasses on table
Photo by Anna Nekrashevich from Pexels | Branded content disclosure.

Table of Contents

· Python libraries for data quality
· Using Great Expectations
· Using…


And is it a good thing?

City view by the river
Photo by Lukas Hartmann from Pexels | Branded content disclosure.

1. Orchestrating Containers

If you want to run reproducible data pipelines and ML models that can run anywhere, you probably know that a Docker image is the way to go. …


Real-time technologies are powerful but they add significant complexity to your data architecture

Photo by Victor Wang from Pexels


How to choose a decoupling service that suits your use case

Man looking at mountain
Photo by Wil Stewart on Unsplash | Branded content disclosure.

AWS SQS

As one of the oldest AWS services, SQS has a track record of providing an extremely simple and effective decoupling mechanism. The entire service is based on sending messages to the queue and allowing for applications…

Anna Geller

Data Engineer, M.Sc. in BI, AWS Certified Solution Architect: www.annageller.com. Get my articles via email: https://annageller.medium.com/subscribe

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store