AWS Kinesis helps you move from batch to real-time. In this post, I’ll walk through how Kinesis works, the key concepts you really need to know, and a couple of real-world architectures using Kinesis for logs and IoT-style data.

I. Why real-time streaming matters

For many years, we were happy with batch:

• Every hour: export logs, run a cron job, push to S3
• Every night: run a big ETL job, load to a data warehouse
• Product teams wait until “tomorrow” for reports

Today, that’s often not enough:

Introduction

Automating the build and deployment of Docker images using GitHub Actions and AWS Elastic Container Registry (ECR) significantly streamlines your software delivery process.

By integrating these powerful tools, developers benefit from reduced manual intervention, minimized errors, and accelerated deployment cycles.

GitHub Actions automates your build workflow whenever code changes, while AWS ECR securely stores and manages your container images, ensuring reliability and scalability.

In this guide, you’ll learn step-by-step how to effortlessly set up continuous integration and continuous deployment (CI/CD) to AWS ECR, maximizing your development efficiency.

Ubuntu 22.04.3 LTS - Windows Subsystem for Linux - AMD64

1. Install & Config AWS CLI

sudo snap install aws-cli --classic
aws --version
which aws

Configure AWS account

aws configure
AWS Access Key ID [None]: xxxxxxxxxxxx
AWS Secret Access Key [None]: xxxxxxxxxxxxxxxx+xxxxxx
Default region name [None]: ap-southeast-1
Default output format [None]: json

You can verify your aws account:

cat ~/.aws/credentials
cat ~/.aws/config
aws ec2 describe-vpcs

2. Install kubectl CLI

kubectl is the command-line interface (CLI) tool used to interact with Kubernetes clusters. It allows developers and operators to manage cluster resources, inspect states, deploy applications, and perform administrative tasks.

1. Introduction

• Why exposing services matters in Kubernetes:

  • Your workloads are useless if no one can reach them. Service abstraction lets you decouple pod lifecycles from stable endpoints.

• Overview of Service types vs. Ingress:

  • Services give every workload an IP; Ingress turns a set of Services into a coherent, externally reachable API surface.

2. Kubernetes Service Types Explained

a. ClusterIP

- Exposes the Service on a cluster-internal IP
- Accessible **only inside** the cluster
- Typical use: 
    - Internal-only services 
    - Microservices talking to each other

b. NodePort

- Exposes the Service on each node’s IP at a static port (the NodePort)
- Accessible from outside the cluster: same VPC or the internet (depending on firewall/Security Group rules)
- Reachable via **NodeIP:NodePort** (port range 30000 - 32767)
- Typical use:
    - Simple setups, debugging, or in environments without cloud LBs. 
    - Quick test or direct access, not recommended for production internet-facing apps.

c. LoadBalancer

- Creates a cloud load balancer (cloud provider integration, e.g., AWS ALB).
- Assigns a public IP and routes traffic to your Service.
- Accessible from the internet.
- Typical use: exposing production workloads to the internet (websites, APIs).

d. ExternalName

- Maps the Service to an external DNS name.
- Used for routing cluster traffic to resources outside Kubernetes.
- Not used for exposing services to the internet.
- Typical use: referencing services outside the cluster by DNS name(database,SaaS endpoints).

3. Ingress

• Ingress is a Kubernetes resource that manages external HTTP/HTTPS traffic
  • Receives incoming requests from outside the cluster
  • Routes them to the right Service based on rules (URL path, host, headers…)
• Typical flow: User → ALB/NLB/NGINX (provisioned by the Ingress Controller) → Service → Pods.
• Ingress Controller:
  • A pod/deployment running inside your cluster
  • monitors your Ingress resources.
  • configures and manages actual routing rules on a cloud load balancer
• Ingress Class:
  • a way to specify which Ingress Controller is responsible for handling a particular Ingress resource

4. Curious question

Q: Why Do You Need Ingress? A: Without Ingress, to expose multiple applications or services externally, you would need many individual LoadBalancer services, which becomes expensive and complex quickly.