Building the CI/CD of the Future, creating the VPC for EKS cluster
In this tutorial, I will share my experience as a DevOps engineer at Cloudify.co, this is the first post of the tutorial in which I will describe how to create VPC for the EKS cluster of our CI/CD based on Jenkins.
Building the CI/CD of the Future published posts:
- Creating the VPC for EKS cluster
- Creating the EKS cluster
- Adding the Cluster Autoscaler
- Add Ingress Nginx and Cert-Manager
- Install and configure Jenkins
- Create your first pipeline
What is Amazon VPC?
Amazon Virtual Private Cloud (Amazon VPC) enables you to launch AWS resources into a virtual network that you’ve defined. This virtual network closely resembles a traditional network that you’d operate in your own data center, with the benefits of using the scalable infrastructure of AWS.
Amazon VPC concepts
Subnet — A range of IP addresses in your VPC.
Route table — A set of rules, called routes, that are used to determine where network traffic is directed.
Internet gateway — A gateway that you attach to your VPC to enable communication between resources in your VPC and the internet.
VPC endpoint — Enables you to privately connect your VPC to supported AWS services and VPC endpoint services powered by PrivateLink without requiring an internet gateway, NAT device, VPN connection, or AWS Direct Connect connection. Instances in your VPC do not require public IP addresses to communicate with resources in the service. Traffic between your VPC and the other service does not leave the Amazon network.
Two options for creating VPC for EKS cluster
- Provision first your VPC using tools like CloudFormation or Terraform and then create an EKS cluster on top of it.
- Create a VPC using tools like ‘eksctl’ which creates automatically VPC for your EKS cluster.
The second option less preferred because you don’t have full control over all the processes, you can’t be sure about aspects like worker nodes you provision will be a part of public subnet with attached public IP or in a private subnet.
You can read on the AWS website more info about this:
If you deployed a VPC using
eksctlor by using either of the Amazon EKS AWS CloudFormation VPC templates:
On or after 03/26/2020 — Public IPv4 addresses are automatically assigned by public subnets to new worker nodes deployed to public subnets.
Before 03/26/2020 — Public IPv4 addresses are not automatically assigned by public subnets to new worker nodes deployed to public subnets.
Of course, if you do the provision of VPC using the ‘eksctl’ utility, you not completely understand what components created on AWS eventually, so I encourage you to do it manually, meaning first create the VPC and only then the EKS cluster.
Another important consideration if VPC have
- Private subnets only
- Public subnets only
- Private and Public subnets
For the CI/CD case, the third option will be the best, I need private and public subnets, our Jenkins master will have a public IP and will be reachable through webhooks of Github.
The workloads will be executed on worker nodes provisioned in private subnets that don’t have a public IP and SSH access, according to security best practices.
Recommended by AWS
We recommend a VPC with public and private subnets so that Kubernetes can create public load balancers in the public subnets that load balance traffic to pods running on worker nodes that are in private subnets.
I guess VPC with public subnets only used for demonstration purposes only, cause it does not feels very secure to have a public IP for each worker node you provision.
Amazon EKS requires subnets in at least two Availability Zones, for resiliency, it is advisable to always have 2 public and 2 private subnets and ensure they are both in different availability zones.
Of course, more availability zones event better and more suitable for the production cluster, but to simplify all the process I will create an EKS cluster with two availability zones.
Creating a VPC for EKS cluster
To create VPC for our EKS cluster I will use AWS CloudFormation template
VPC components of EKS cluster
- VPC: 192.168.0.0/16 (65534 hosts), IP calculator
- 2 public subnets: 192.168.0.0/18 and 192.168.64.0/18, (16382 hosts in each subnet), IP calculator
- 2 private subnets: 192.168.128.0/18 and 192.168.192.0/18 (16382 hosts in each subnet), IP calculator
- InternetGateway connected to VPC and public subnets
- 2 NAT gateways with 2 Elastic IPs in public subnets, one in each public subnet. Each private subnet connected to NAT gateway via a routing table
- Other components like RouteTables and RouteTableAssociation …
Creating VPC for EKS cluster
- You must have an AWS account
Creating IAM user on AWS with programmatic access
In AWS account go to Services -> IAM -> Users -> Add User
Attach ‘AdministratorAccess’ policy
I used ‘AdministratorAccess’ policy to simplify the process, otherwise, you need to attach a lot of different policies for VPC/EKS creation, and that not something I want to focus on in this tutorial.
Create the user and download .csv file with credentials for programmatic access to AWS
Install AWS Cli
$ aws configure
Set your AWS Access Key ID / AWS Secret Access Key / Default region name from CSV file you downloaded
Create the VPC using the cloud formation template and AWS Cli
Download CloudFormation template from here , I assume file in ‘downloads’ folder, name of the stack you creating is ‘eks-vpc’ and the region is ‘us-east-1’
$ aws cloudformation create-stack --stack-name eks-vpc --template-body file:///Users/igor/downloads/eks-cluster/eks-cluster/amazon-eks-vpc-private-subnets.yaml --region=eu-west-1
In AWS account go to Services -> CloudFormation -> Stacks, you must see
In AWS account go to Services -> VPC -> Your VPCs
In this post I tried to explain how to create VPC for EKS cluster, different options and considerations you must take in account before actually doing it, like creating the VPC first or create the VPC as a part of EKS cluster creation with ‘eksctl’ utility, use private/public subnets only or combination of both, how to make EKS cluster resilient and eventually I showed you how to create the cluster using the ‘awscli’ and CloudFormation template from AWS.
If you want to be notified when the next post of this tutorial is published, please follow me here on medium and on Twitter (@warolv).
For consulting gigs, you can reach me on Upwork
Thank you for reading, I hope you enjoyed, see you in next post.