Parottasalna

AI, Backend Engineering & Architecture Guides

Mastering AWS VPC Endpoints: A Step-by-Step Manual Guide and Demo

Understanding how Amazon Virtual Private Cloud (VPC) Endpoints work is a crucial part of mastering AWS networking. VPC Endpoints allow your private instances to communicate with AWS services like S3 without needing a public IP address, an Internet Gateway (IGW), or a NAT Gateway. Traffic remains entirely within the AWS private backbone.

In this blog post, we will walk through the exact steps to manually build a demonstration environment from scratch using the AWS Management Console. We will then perform a live test to see what happens when a private server tries to reach S3 with and without a VPC Endpoint.

Architecture Overview

We will build the following architecture

  • A VPC with one Public Subnet and one Private Subnet.
  • An Internet Gateway attached to the Public Subnet.
  • A Bastion Host (Public EC2) in the Public Subnet to allow SSH access.
  • A Private EC2 instance in the Private Subnet with absolutely no internet access.
  • An IAM Role attached to the Private EC2 granting S3 Read-Only access.
  • A test S3 bucket.
  • A VPC Gateway Endpoint for S3.

Phase 1: Building the Infrastructure Manually

Follow these steps in the AWS Management Console to create the baseline environment.

1. Create the VPC and Subnets

  1. Navigate to the VPC Dashboard and select “Your VPCs”.
  2. Click “Create VPC”.
    • Name: vpc-endpoint-demo-vpc
    • IPv4 CIDR block: 10.0.0.0/16
  3. Navigate to “Subnets” and click “Create subnet”.
    • Create a Public Subnet (vpc-endpoint-demo-public-subnet) with CIDR 10.0.1.0/24 in Availability Zone A. Enable “Auto-assign public IPv4 address” in the subnet settings after creation.
    • Create a Private Subnet (vpc-endpoint-demo-private-subnet) with CIDR 10.0.2.0/24 in Availability Zone B.

2. Configure Internet Access for the Public Subnet

  1. Navigate to “Internet Gateways” and click “Create internet gateway”.
    • Name: vpc-endpoint-demo-igw
    • Once created, select it, click “Actions”, and attach it to your VPC.
  2. Navigate to “Route Tables”.
    • Select the main route table for your VPC and rename it to vpc-endpoint-demo-private-rt. Ensure it has no internet route (0.0.0.0/0). Associate this route table with your Private Subnet.
    • Click “Create route table” and name it vpc-endpoint-demo-public-rt. Select your VPC.
    • Edit the routes for vpc-endpoint-demo-public-rt and add a route for 0.0.0.0/0 pointing to the Internet Gateway.
    • Associate vpc-endpoint-demo-public-rt with your Public Subnet.

3. Set Up Security Groups

  1. Navigate to “Security Groups” in the EC2 Dashboard and click “Create security group”.
    • Name: vpc-endpoint-demo-public-sg
    • VPC: Select your VPC.
    • Inbound Rules: Add SSH (Port 22) from Anywhere (0.0.0.0/0).
  2. Create another security group.
    • Name: vpc-endpoint-demo-private-sg
    • VPC: Select your VPC.
    • Inbound Rules: Add SSH (Port 22) and set the source to the vpc-endpoint-demo-public-sg security group.

4. Create the IAM Role for the Private EC2

  1. Navigate to the IAM Dashboard.
  2. Select “Roles” and click “Create role”.
  3. Choose “AWS service” and select “EC2” as the use case.
  4. Attach the policy: AmazonS3ReadOnlyAccess.
  5. Name the role: vpc-endpoint-demo-private-ec2-role and create it.

5. Prepare the S3 Bucket

  1. Navigate to the S3 Dashboard.
  2. Click “Create bucket”.
    • Name: vpc-endpoint-demo-[your-unique-id]
    • Ensure “Block all public access” is checked.
  3. Once created, upload a simple text file named “hello.txt” containing a message like “Hello from VPC Endpoint! Traffic never left AWS.”

6. Launch the EC2 Instances

You will need an existing SSH key pair (e.g., server-pem). We will use Ubuntu 26.04 as our OS.

Launch the Bastion Host:

  1. Navigate to “Instances” and click “Launch instances”.
  2. Name: vpc-endpoint-demo-bastion
  3. AMI: Ubuntu Server 26.04 LTS.
  4. Instance type: t3.micro.
  5. Key pair: Select your existing key pair.
  6. Network settings: Select your VPC, choose the Public Subnet, and select the vpc-endpoint-demo-public-sg security group.
  7. Advanced details (User data): Provide the following script to download the AWS CLI installer. This instance has internet access, so it can download the file for us.#!/bin/bash apt-get update -y apt-get install -y unzip curl curl -sL "https://awscli.amazonaws.com/awscli-exe-linux-x86_64.zip" -o "/home/ubuntu/awscliv2.zip" chown ubuntu:ubuntu /home/ubuntu/awscliv2.zip
  8. Click “Launch instance”.

Launch the Private EC2:

  1. Name: vpc-endpoint-demo-private
  2. AMI: Ubuntu Server 26.04 LTS.
  3. Instance type: t3.micro.
  4. Key pair: Select the same key pair.
  5. Network settings: Select your VPC, choose the Private Subnet, and select the vpc-endpoint-demo-private-sg security group.
  6. Advanced details (IAM instance profile): Select the vpc-endpoint-demo-private-ec2-role.
  7. Advanced details (User data): Provide the following script. This instance has no internet, so it only installs unzip from its local cache.#!/bin/bash apt-get update -y apt-get install -y unzip
  8. Click “Launch instance”.

Phase 2: Installing AWS CLI on the Airgapped Private EC2

The private EC2 instance cannot connect to the internet to download the AWS CLI. We must use the Bastion host as a relay.

  1. SSH into the Bastion Host from your local machine, enabling SSH agent forwarding so you can jump to the private instance later: ssh -A -i server-pem -o IdentitiesOnly=yes ubuntu@<bastion_public_ip>
  2. Verify the AWS CLI zip file downloaded successfully during the boot process: ls -lh ~/awscliv2.zip
  3. Securely copy (SCP) the zip file from the Bastion directly to the Private EC2: scp ~/awscliv2.zip ubuntu@<private_ec2_private_ip>:~/
  4. SSH from the Bastion into the Private EC2: ssh -o IdentitiesOnly=yes ubuntu@<private_ec2_private_ip>
  5. Install the AWS CLI on the Private EC2 using the copied zip file: unzip ~/awscliv2.zip -d ~/aws-install sudo ~/aws-install/aws/install

Phase 3: The Demo – Accessing S3 Without a VPC Endpoint

While logged into the Private EC2, let us attempt to access S3. The instance has the correct IAM permissions, but let us observe the network behavior.

  1. Confirm the instance has no internet access. This command will hang and eventually time out: curl --max-time 5 https://www.google.com
  2. Try to list S3 buckets using the AWS CLI. This will also hang and time out: export AWS_DEFAULT_REGION=ap-south-1 aws s3 ls --cli-connect-timeout 5 --cli-read-timeout 5You will receive an error like “Could not connect to the endpoint URL”.

Why did this fail? The AWS CLI attempts to reach the public S3 endpoint (e.g., s3.ap-south-1.amazonaws.com) over the internet. Since this private subnet has no Internet Gateway or NAT Gateway, the TCP connection cannot be established. IAM permissions alone are not enough; a valid network route is required.

Exit the Private EC2 and Bastion sessions to return to your local machine.

Phase 4: Adding the VPC Endpoint Manually

Now, we will create the network route by adding a VPC Gateway Endpoint.

  1. In the AWS Management Console, navigate to the VPC Dashboard.
  2. Select “Endpoints” and click “Create endpoint”.
  3. Name: vpc-endpoint-demo-s3-endpoint
  4. Service category: AWS services.
  5. Services: Search for “s3” and select the “Gateway” type for your region (e.g., com.amazonaws.ap-south-1.s3).
  6. VPC: Select your VPC.
  7. Route tables: Select the vpc-endpoint-demo-private-rt route table. This automatically injects a route into your private subnet directing S3 traffic to the endpoint.
  8. Policy: Full access.
  9. Click “Create endpoint”.

Phase 5: The Demo – Accessing S3 With a VPC Endpoint

With the VPC Endpoint in place, let us re-test our access.

  1. SSH back into the Bastion, and then jump to the Private EC2. ssh -A -i server-pem -o IdentitiesOnly=yes ubuntu@<bastion_public_ip> ssh -o IdentitiesOnly=yes ubuntu@<private_ec2_private_ip>
  2. Confirm internet access is still blocked. The VPC Endpoint does not grant general internet access: curl --max-time 5 https://www.google.comThis will still time out as expected.
  3. Attempt to list S3 buckets again: export AWS_DEFAULT_REGION=ap-south-1 aws s3 lsThis time, the command succeeds instantly and lists your buckets.
  4. Download the test file from your bucket and read it: aws s3 cp s3://vpc-endpoint-demo-[your-unique-id]/hello.txt . cat hello.txtYou should see your message: “Hello from VPC Endpoint! Traffic never left AWS.”

Conclusion

This demonstration highlights a core principle of AWS security and networking. By using a VPC Gateway Endpoint, you established a secure, private connection to Amazon S3. The traffic never traversed the public internet, avoiding the cost and exposure of a NAT Gateway. Even without internet access, your private server was able to securely retrieve its data.

Discover more from Parottasalna

Subscribe now to keep reading and get access to the full archive.

Continue reading