Data Engineering Community (DEC) (@data_dec) header
Data Engineering Community (DEC) (@data_dec) profile picture

A non-profit organisation providing Data Engineers a supportive and collaborative platform.

Nigeria
Joined April 2024
Photo by Data Engineering Community (DEC) (@data_dec)Photo by Data Engineering Community (DEC) (@data_dec)Photo by Data Engineering Community (DEC) (@data_dec)Photo by Data Engineering Community (DEC) (@data_dec)Photo by Data Engineering Community (DEC) (@data_dec)Photo by Data Engineering Community (DEC) (@data_dec)Photo by Data Engineering Community (DEC) (@data_dec)Photo by Data Engineering Community (DEC) (@data_dec)Photo by Data Engineering Community (DEC) (@data_dec)Photo by Data Engineering Community (DEC) (@data_dec)
Compute: Where Does Your Pipeline Actually Run? Here's something beginners sometimes mix up: Storage is not compute. S3 can store your data, but it isn't where you normally think of your Python application or Spark job executing.
1
2
120
📌Learn ◉ AWS — Compute Services ◉ AWS — EC2 Getting Started ◉ AWS — Lambda Developer Guide ◉ AWS — Amazon ECS ◉ AWS — Amazon EMR 📌Hands-on: Launch an EC2 instance and identify its AMI, instance type, EBS volume, subnet and security group.
1
10
💡Resource of the Day: AWS Compute documentation
4
IAM: AWS Doesn't Just Ask "Can You Run This?" Imagine you've written a Python script that uploads a file to S3. ◉ The code looks correct. ◉ Your bucket exists. ◉ The file exists. ◉ You run the script. ◉ And AWS responds: AccessDenied
1
1
125
💡Why this matters for Data Engineers A production data pipeline needs more than correct Python or SQL. It needs the right identity and permissions to interact with the rest of the platform. 📌Learn ◉ AWS — IAM User Guide ◉ AWS — IAM Policy Elements
1
11
◉ AWS Skill Builder — IAM fundamentals ◉ AWS — IAM Policy Simulator 📌Hands-on: Create an IAM role/policy that allows access to one S3 location but not another. Resource of the Day: AWS IAM documentation
10