Deep Architectural Analysis: The AWS NAT Gateway S3 Extortion Trap
In cloud engineering forums like r/aws and r/devops, NAT Gateway bill shock is one of the most frequently lamented cost traps. Developers routinely report monthly AWS bills jumping by $2,000 to $18,000 without any change in outward product traffic. The root cause is almost always unrouted internal S3 traffic passing through a NAT Gateway.
1. Why Does AWS Charge $0.045/GB on Internal Traffic?
By default, Amazon S3 IP addresses are public IPv4 addresses managed by AWS. When an EC2 instance, ECS container, or Kubernetes pod located inside a private subnet initiates an HTTPS connection to s3.us-east-1.amazonaws.com, the routing table looks for the 0.0.0.0/0 rule. This default route points straight to your VPC's NAT Gateway.
Even though the packets never leave Amazonβs global backbone network and travel only a few hundred meters between datacenter racks in the same AWS region, AWS classifies this as NAT Gateway egress data processing, billing you $0.045 for every gigabyte.
2. The Top 4 Hidden Culprits Behind S3 NAT Leaks
-
ECR Container Image Pulls: Docker images hosted in AWS Elastic Container Registry (ECR) store image layers directly in an AWS-managed S3 bucket behind the scenes. Every time a node scales up or a pod restarts, dozens of gigabytes of container layers route through the NAT Gateway unless an S3 Gateway Endpoint is active.
-
EMR, Spark, & Trino Big Data Clusters: Distributed analytics queries read and write hundreds of gigabytes of Parquet files from S3 buckets. If workers sit in private subnets, simple queries can cost $50-$200 in pure NAT Gateway data processing fees per execution!
-
Athena Query Result Spilling: Athena writes intermediate and final query results back to S3 staging buckets. Processing multiple gigabytes of CSV/JSON results through private Lambda functions triggers the NAT tax.
-
Automated Database & Backup Dumps: Nightly Postgres/MySQL
pg_dump scripts archiving snapshots to S3 buckets run continuously through NAT interfaces.
3. Gateway Endpoint vs. Interface Endpoint (PrivateLink)
AWS offers two types of VPC Endpoints for S3, and confusing them can cost thousands of dollars:
-
S3 Gateway Endpoint (Free - $0.00): Works entirely at the route table layer via AWS Prefix Lists. It costs $0.00 per hour and $0.00 per gigabyte processed. It cannot be accessed over Direct Connect or VPC Peering.
-
S3 Interface Endpoint (PrivateLink - Paid): Creates an Elastic Network Interface (ENI) with a private IP inside your subnet. AWS charges $0.01/hour per AZ plus $0.01/GB data processing. This is only necessary for hybrid clouds or Direct Connect connections originating from outside AWS.
4. Securing Gateway Endpoints with IAM Least-Privilege Policies
A common compliance concern is data exfiltration: if an instance in your VPC has direct access to the public S3 prefix list, could a malicious actor or vulnerable dependency exfiltrate data to an attacker-controlled AWS account's S3 bucket?
The answer is VPC Endpoint Policies. You can attach a strict JSON policy to the S3 Gateway Endpoint specifying that Action: "s3:*" is strictly denied unless the destination bucket matches your company's AWS Account ID (aws:PrincipalAccount) or ARN prefix.