AWS Containerization, Serverless Computing, and Simplified Infrastructure Management

Link to exam: aws.amazon.com/certification/certified-clou..

Containerization:

Docker

Definition:

Software development platform for deploying apps.
Apps are packaged in containers runnable on any OS.

Advantages:

Uniform app behavior across different environments.
Compatibility across machines.
Predictable behavior, less maintenance.
Works with any language, OS, or technology.
Quick scaling of containers.

Storage:

Docker images stored in Docker Repositories.
Public repositories like Docker Hub.
Private repositories like Amazon ECR.

Docker versus Virtual Machines

Docker is a form of virtualization but shares resources with the host.
Allows multiple containers on a single server, unlike traditional VMs.

Container orchestration and management:

ECS (Elastic Container Service)

ECS is a container orchestration service by AWS that manages Docker containers.
Requires provisioning and maintenance of infrastructure.

It provides cluster management capabilities, allowing users to run, stop, and manage containers within a cluster and integrates with load balancers.
ECS can be used with EC2 instances or AWS Fargate.

EKS (Elastic Kubernetes Service):

EKS is a Kubernetes orchestration service provided by AWS.
It manages Kubernetes-based containers on the AWS public cloud and provides cluster management capabilities through Kubernetes.
EKS automates load distribution and parallel processing using Kubernetes.

Fargate

Fargate is a serverless compute engine that supports both ECS and EKS.
It allows users to focus on their Docker containers without managing the underlying EC2 instances.
Fargate is a Container-as-a-Service (CaaS) solution, eliminating the need to provision, configure, or scale clusters of virtual machines or servers.
AWS manages container runtime based on resource needs.

ECR (Elastic Container Registry)

AWS-managed private Docker container registry.
Stores, manages, and deploys Docker container images.
Integrated with ECS, EKS, and Fargate.
Secure image storage with encryption.
Simplifies container deployment.

Serverless Computing:

What’s serverless?

Serverless computing, also known as Function as a Service (FaaS), is a cloud computing model where cloud providers dynamically manage the allocation and provisioning of resources needed to execute code.

No Server Management: Developers avoid server provisioning and management as the cloud provider handles it all, including scaling and maintenance
Event-Driven Execution: Functions respond to events like HTTP requests or database changes, enabling event-driven architectures.
Pay-per-Use Pricing: You pay only for resources used during function execution, with charges based on milliseconds of usage.
Microservices Architecture: Encourages modular, scalable applications composed of small, independent functions communicating via APIs.
Stateless Execution: Functions are stateless, requiring external management of state through databases or storage services.

AWS Lambda pioneered serverless with FaaS.

AWS Lambda

Serverless: AWS Lambda is a serverless compute service by AWS. It executes code in response to events without server provisioning. Allows developers to focus solely on writing code.
Event-Driven Execution: Developers upload code and define triggers for automatic execution.
Supports multiple languages: Node.js, Python, Java, Go, .NET Core.

Integrated with AWS Services: Build serverless applications that leverage the full capabilities of the AWS ecosystem.

Automatic Scaling: Lambda automatically scales to handle incoming requests, from a few requests per day to thousands per second, without any manual intervention.

Built-In High Availability: Functions are replicated across multiple Availability Zones within a region, ensuring built-in high availability and fault tolerance.

Pay-Per-Use Pricing: You only pay for number of requests and the compute time consumed by your functions executions, measured in milliseconds, with no charges for idle time.

Example:

Image Processing: Automatically resize or generate thumbnails for images uploaded to an S3 bucket triggered by S3 bucket events.
Real-Time Data Processing: Analyze streaming data from IoT devices triggered by IoT events or Kinesis data streams.
Backend for Mobile Apps: Serve as the backend for mobile applications, handling user authentication and data storage triggered by API Gateway requests.
Webhooks: Execute code in response to webhook events, such as GitHub commits or Slack messages, triggered by HTTP requests.
Serverless CRON Job: Execute a scheduled task (e.g., database backup) at regular intervals using AWS Lambda triggered by CloudWatch Events with a CRON expression.

Amazon API Gateway

Managed Service: AWS-managed platform for creating, publishing, and securing APIs without infrastructure management.
RESTful and WebSocket Support: Allows creation of RESTful and WebSocket APIs to suit different communication needs.
Backend Integration: Integrates with AWS Lambda, HTTP endpoints, AWS services, and third-party HTTP endpoints for flexible backend connectivity.
Key Features: Offers versioning, request/response transformations, throttling, caching, and access control.
AWS Lambda Integration: Seamless integration with AWS Lambda for easy creation of serverless APIs.
Access Control and Monetization: Supports usage plans and API keys for access control and monetization.

WebSocket Support: Enables real-time, bidirectional communication for interactive applications.

Microservices Architecture: Acts as a front door for microservices, facilitating seamless connectivity and integration.

Batch Processing: AWS Batch

Managed Batch Processing: AWS-managed service for batch processing of large-scale computing workloads.
Compute Environments: Users can set up compute environments within a VPC, allowing for the execution of batch jobs across multiple Availability Zones.
Job Definitions: AWS Batch enables the definition of job specifications, including the Docker container images to be utilized for job execution. These images can be stored in container registries within or outside of the AWS infrastructure.
Automatic Job Scheduling and Scaling: The service automates job scheduling and dynamically adjusts compute resources, including EC2 instances or Spot Instances, based on workload demand for optimal resource utilization and cost-efficiency.
Job Queues and Prioritization: Supports job queues for organizing and prioritizing batch jobs, ensuring efficient resource allocation.

Integrations

Integration with AWS Services: Seamlessly integrates with other AWS services such as Amazon S3, AWS Lambda, and Amazon DynamoDB for data storage, processing, and management.
Integration with Amazon ECS: AWS Batch utilizes Amazon ECS container instances for job execution within its compute environments.
AWS Batch on Amazon EKS: Users can leverage AWS Batch on Amazon EKS to schedule and scale batch workloads within existing Amazon EKS clusters. While AWS Batch manages the orchestration of nodes and pods, it does not perform lifecycle operations on EKS clusters.

Usage Scenarios:

Batch Processing: Ideal for scenarios requiring the execution of large-scale batch jobs, such as data processing, ETL (Extract, Transform, Load) tasks, and scientific simulations.
Custom Compute Environments: Users can define custom compute environments tailored to specific job requirements, allowing for versatile job execution.

Example: Running data processing jobs on large datasets stored in Amazon S3 using AWS Batch, where compute resources are dynamically provisioned based on workload demands, ensuring efficient processing and resource utilization.

AWS Batch Vs AWS Lambda

Feature	AWS Batch	AWS Lambda
Managed Service	✔️	✔️
Compute Resource Type	EC2 Instances, Spot Instances	N/A (Managed by AWS)
Scaling	Automatic scaling based on job demand	Automatic scaling based on event triggers
Fault Tolerance	Offers fault tolerance features like job retries and timeout settings	N/A (Each function invocation is independent)
Disk Space	Supports attaching EBS volumes to EC2 instances	Limited disk space (temporary storage) for function execution
Time Constraint	No inherent time limit	15 minutes
Serverless	❌	✔️
Use Cases	Batch processing of large-scale computing workloads	Event-driven and serverless computing for short-lived tasks

Simplified Cloud Computing: Amazon Lightsail

Simple Cloud Computing: Provides an easy-to-use cloud computing platform for developers, offering pre-configured virtual private servers (VPS) and other resources.

Pre-Configured Instances: Offers pre-configured instances with a choice of operating systems, development stacks, and applications, enabling rapid deployment of applications without the complexity of manual configuration.

Integrated Management Tools: Includes integrated management tools for monitoring, security, and networking, simplifying administration tasks for users. Simpler alternative to EC2, RDS, etc., suitable for beginners

Predictable Pricing: Offers predictable pricing with flat-rate billing, making it easy to budget for and manage costs.

Use Cases: Ideal for Dev-Environment, small businesses, and individuals looking for an affordable and straightforward cloud computing solution for hosting websites, applications, and development environments.

Summary

Containerization:

Docker: Platform for deploying apps in containers, offers compatibility and easy scaling.
Docker vs. Virtual Machines: Docker shares resources, supports multiple containers on one server.

Container Orchestration and Management:

ECS (Elastic Container Service): AWS-managed service for Docker container orchestration.
EKS (Elastic Kubernetes Service): AWS-managed Kubernetes service for container orchestration.
Fargate: Serverless compute engine for running containers without managing EC2 instances.
ECR (Elastic Container Registry): AWS-managed private Docker container registry.

Serverless Computing:

AWS Lambda: Serverless compute service executing code in response to events, offers automatic scaling and pay-per-use pricing.
Amazon API Gateway: Managed service for creating, publishing, and securing APIs, integrates seamlessly with AWS Lambda.

Batch Processing:

AWS Batch: Managed service for batch processing large-scale computing workloads, supports compute environments, job definitions, and automatic scaling.

AWS Batch Vs AWS Lambda:

AWS Batch utilizes EC2 instances and supports automatic scaling based on job demand,
AWS Lambda is serverless, managed by AWS, and automatically scales based on event triggers with a 15-minute execution time limit and limited disk space for function execution.

Simplified Cloud Computing-Amazon Lightsail:

Provides easy-to-use cloud computing platform with pre-configured instances, ideal for small businesses and beginners.