Blog

Home / AWS / Interconnecting two AWS VPC regions

Interconnecting two AWS VPC regions

In AWS
4

Introduction

Amazon Web Services (AWS) Virtual Private Cloud (VPC) provides a good way to isolate your cloud servers from the shared data-center environment and to reduce public cloud security threats. For a more detailed discussion about security threats arising from IaaS consumption see our previous post on the need for VPC solutions.

There are several good reasons why one would want to deploy virtual servers (VMs) in more than a single AWS region. The first obvious reason is resiliency to data-center disasters, but there are others, for example, reducing network latency when serving customers on different continents may require deployments in NA, APAC and EU data-centers.

In many cases, servers deployed on one region need to exchange traffic with servers on another region. A very common case is Disaster Recovery (DR) scheme where a 2nd region is used as a passive replica of the first region, thus requiring a constant and stable network connectivity between the two regions.

The AWS VPC implementation, however, does not allow a VPC to extend across regions. VPCs deployed on multiple regions are like isolated ‘islands’ with private IP subnets. Communication between those VPC  islands is possible only using their Internet gateways, thus compromising security.

The following explains how to interconnect two VPC deployments on two AWS regions. The solution has the following benefits:

  1. The two VPC regions are securely interconnected using an IPSec connection, creating one big multi-region VPC.
  2. Networking specifics are transparent to the applications
  3. It is simple and pretty straightforward. Requires only 2 ‘software router’ instances, 2 Elastic IP (EIP) addresses and simple configuration.

Few words about us before diving into the technical explanations – 40Cloud solution provides a very comprehensive set of security features such as VPN, firewall, Access Control and identity integration that are combined with automation and monitoring capabilities, all delivered as-a-service. At any stage, if you are having trouble with the following, if you are facing more complicated cloud deployment scenarios or security challenges, or if you’d just like to get an off-the-shelf  security solution and free yourself to work on your core business, you are invited to use the 40Cloud AMI or BYOL available on the AWS Marketplace.

AWS Background

This section discusses the basics of the AWS data-center architecture to make sure we have a stable basis before proceeding to the slightly more complicated tasks. If you feel comfortable with AWS VPC architecture, you’re invited to jump to the next section.

AWS has few different modes of operation: EC2, EC2-VPC and Default-VPC.

EC2 is basically a shared network virtual datacenter in which one can allocate instances that can communicate with each other inside and outside AWS account boundaries. Instances are protected on the network level using EC2 Security Groups, which are stateless firewall rules.

EC2-VPC is very similar to EC2 but provides a private isolated (virtual) network. Instances within the virtual private network can intercommunicate using their private IP addressing.  An EC2 Internet Gateway allows VPC instances to communicate with external resources, on the EC2 ‘network’ and on the public Internet. Using EIP (Elastic IP address) the VPC instances can be reached from outside the VPC.

Default VPC is basically a predefined and pre-configured EC2-VPC setup. The Default VPC private network is configured for the 172.31.0.0/16 subnet, and has floating IPs assigned by default to the virtual servers. Note that Default VPC is at the moment not supported in all regions and all accounts.

Network Design

In this example, I’ll connect two AWS VPCs that are located in two different regions (see drawing below) by creating a site to site VPN. Once connected, all communication between the instances in the two VPCs will be allowed (using Security Groups). The connectivity between the VPCs will be established using two Linux instances that will function as simple software routers and software VPN. The software routers will be securing the cross region communication using software IPSec VPN. For the purpose of this example, I will be using an Ubuntu 12.04 instance running OpenSwan. Note that this example doesn’t take into account redundancy and High Availability considerations.

VPC-Interconnect-net-diagram

Configuring AWS VPCs

1. First Steps

Your first step is signing into the AWS administration console using your AWS credentials (your AWS credentials should either be ‘root admin’ or at least have permissions to configure all elements in a VPC and to launch instances).

Once you’re signed in, click on the “Services” pull down and then choose “VPC”.

AWS Admin Console

Screenshot 1 – AWS services selection panel

2. The VPC Toolbox

After selecting VPC from the services menu, you’ll see the VPC toolbox on the left side of the page. Make sure you select the right region from the list of regions in this example, I selected Ireland for my first VPC.

You will be using the following tools in the VPC toolbox to create and configure your VPC: “Your VPCs”, “Subnets”, “Security Groups”, “Elastic IPs”, “Route Tables” and “Internet Gateways”.

 

AWS Admin Console

Screenshot 2 – The VPC toolbox and region list

3. Building the VPC

Step 1 – Creating your VPC

  • Click on “Your VPCs” in the VPC toolbox – you should see a button on the top of the page titled “Create VPC”. Click on the “Create VPC” button
  • You should now see a “Create VPC” popup window with two fields.
    • In the CIDR Block field I’ve used “10.20.0.0/16” in this example as we will be connecting two /16 networks. You are free to pick any private subnet, just make sure the different subnets don’t have any overlapping IP addresses.
    • In Tenancy Choose “default”

 VPC Popup Dialog

 Screenshot 3 – Create VPC Popup Dialog

  • Click on the “Yes, Create” button to create your VPC

Step 2 – Creating the VPC’s Subnet

  • Click on  “Subnets” in the VPC toolbox – You should see a button on the top of the page titled “Create Subnet”. Click the “Create Subnet” button.
  • You should now see a “Create Subnet” Popup window with three fields.
    • In the VPC field make sure you select the VPC you just created (you should have only one)
    • In the Availability Zone field select “No Preference”
    • In the CIDR Block field please enter the VPC subnet again, e.g. 10.20.0.0/16 in this example. You can actually split your VPC into several subnets but in this example we will not need this capability.

Subnet popup

Screenshot 4 – Create Subnet Popup Dialog

Step 3 – Creating an Internet Gateway (GW)

An Internet Gateway (GW) is required to enable your instances (and the new software router) to reach any network outside the private VPC’s subnets (i.e. on the AWS network and the public Internet).

  • Click on  “Internet Gateways” in the VPC toolbox – You should see a button on the top of the page titled “Create Internet Gateway”.
  • Press on the “Create Internet Gateway” button.
  • You should now see a  “Create Internet Gateway” Popup window
  • Click on the “Yes, Create” Button

Internet Gateway popup

 Image 5 – Create Internet Gateway Popup Dialog

  • Check the checkbox near the newly created Internet Gateway from the list and click the “Attach to VPC” Button.
  • You should now see a new popup window titled “Attach To VPC” with a VPC selection box.
  • Select the VPC you’ve created from the list and click the “Yes, Attach” Button.

 

VPC popup dialog

 Screenshot 6 – Attach to VPC Popup Dialog

Now you have an Internet GW in your VPC, all is left is to add the relevant route to the routing table.

Step 4 – Configuring the Subnet’s Routing table

A Route Table is automatically created for every VPC that you create (also called the Main route table). By default, all subnets are associated to the Main Route Table. Also, by default, the main route table allows instances within the VPC  to communicate.  To allow instances to reach the ‘outside world’ we will need to add the following configuration:

  • Click on “Route Tables” in the VPC toolbox – You should already have a route table available and listed.
  • Check the checkbox near the route table line. Once checked you’ll see at the bottom of the screen your routes.
  • In the empty line do the following:
    • Enter “0.0.0.0/0” in the Destination column.
    • Select your Internet Gateway in the Target column (it’s name should start with igw-xxxxxx)
    • Click the “Add” button at the right side of the new line

Main route table

Screenshot 7 – the main route table

4. Building the Second VPC

Before creating the software routers you’ll need to configure the 2nd VPC located at a different region than the first one, in this example in the AWS Oregon data-center.

  • To do so, first select the 2nd target region from the region list (see, for example, screenshot #3 above, at the top right).
  • Repeat steps 1-4 above with the following exception:
    •  Use a different subnet in the CIDR block at the “Create Subnet” popup Dialog. I used 10.20.0.0/16 for the first VPC, so I am using 10.50.0.0/16 for the second VPC.

All the other steps are identical.

5.  Setting up the connection between the two VPC

Background

There are many techniques that can be used for connecting two private subnets over a public IP network. In this example I’ll be using OpenSwan’s IPSec implementation for creating a point-to-point connection and for routing the traffic between the private subnets.

To route traffic, OpenSwan uses policy-based routing that is dedicated for IPSec traffic. These routes will tell the IPSec layer that whenever it sees an IP destination address that belongs to the remote VPC subnet, it should send it over the IPSec connection.

Step 1 – create the first L3 Router Instance

  • Create a new instance – I’m assuming here that you know how to create and access a new instance on AWS. In my example, I created the first instance in the Ireland data-center.
  • Make sure you create the instance inside the VPC and not in the EC2-Classic.
  • You can start with a micro instance (you can always resize it later based on need).
  • I am using Ubuntu 12.04 LTS from the AWS OS selection.
  • When creating the instance, please make sure its security groups configuration allows the following ports to subnet 0.0.0.0/0 (we will tighten security once the setup is completed)
    • Allow UDP 4500 (IPSec/UDP) from 0.0.0.0/0
    • Allow UDP 500 (IKE protocol) from 0.0.0.0/0
    • Allow TCP 22  (SSH protocol) from 0.0.0.0/0
  • Please Associate an EIP to the instance you just created (make sure the EIP is allocated for VPC and not for EC2)
  • Select the Instance in the Instance List and then Click on the “Action button”. Select the “Change Source/Dest. Check”. Click on the “Yes disable” button (this is a critical step, without it the software routers will not accept or forward traffic that is not intended to the Routers themselves, hence they won’t function as software routers).
  • In the following, the new instance will be accessed using SSH (you should be sudo capable).

Step 2 – OpenSwan Installation and configuration

  • To Enable Packet Forwarding on your instance, run the following command:
     sudo sysctl -w net.ipv4.ip_forward=1
  • To install OpenSwan, run the following command:
     sudo apt-get install –y openswan

This should install OpensSwan and all its dependencies.

  • You’ll need to update the following 2 files to get OpenSwan working:

1)    /etc/ipsec.conf

2)    /etc/ipsec.secret

The following is an example for how to configure both files (just copy and paste the example and fill in the missing properties based on your instances data) – You can reach out to our support to better understand IPSec options and alternatives.
Download an example of /etc/ipsec.conf file

Note that you’ll need to complete both the left and right properties. Left should be filled with the private IP address of your instance (in the example, in the Ireland region) and right should be filled with the remote EIP of the remote GW in the second region (Oregon, in my example). Since you haven’t yet created the 2nd software router in the second region (Oregon) you can keep the right property empty until you’ll have all the data available.

To obtain the private IP address of the instance just type the following in the instance’s command line (using SSH).

sudo ip addr show

 

The output of the above command should be similar to the below (with different IP address associated to eth0). Just copy your IP address and paste it into the configuration file.

root@ip-10-128-1-231:/home/ubuntu# sudo ip addr show
1: lo: <LOOPBACK,UP,LOWER_UP> mtu 16436 qdisc noqueue state UNKNOWN
   link/loopback 00:00:00:00:00:00 brd 00:00:00:00:00:00
   inet 127.0.0.1/8 scope host lo
   inet6 ::1/128 scope host
   valid_lft forever preferred_lft forever
2: eth0: <BROADCAST,MULTICAST,UP,LOWER_UP> mtu 1500 qdisc pfifo_fast state UP qlen 1000
   link/ether 2a:31:d4:27:44:dd brd ff:ff:ff:ff:ff:ff
   inet 10.128.1.231/24 brd 10.128.1.255 scope global eth0
   inet6 fe80::2831:d4ff:fe27:44dd/64 scope link
   valid_lft forever preferred_lft forever

Download an example of /etc/ipsec.secrets file

Note that this it is very unsecure to use the above shared-secret (the shared secret is the “1234554321” string), please replace it with a longer and more random string or use asymmetric keys instead (you can reach out to our support for more help on how to set things up in a more secure manner).

Steps for setting up the 2nd software router

To create the 2nd software router, in the second region (Oregon in my example), just repeat the same steps you made for the first Router (in the Ireland data-center). The only difference should be the OpenSwan configuration as noted below.

Here is an example for the Oregon OpenSwan configuration files:
Download an example of /etc/ipsec.conf file

Note that you’ll need to complete both the left and right properties. Left should be filled with the private IP address of your instance (in Oregon) and right should be filled with the remote EIP of the remote software router (Ireland).
Download an example of /etc/ipsec.secrets file
You must use the exact same shared secret used in the Ireland software router’s /etc/ipsec.secrets file.

Now that both software routers are configured correctly, you should run the following set of commands in each one of them (it doesn’t matter with which server you start):

sudo service ipsec stop
sudo service ipsec start
sudo ipsec auto –add vpc2vpcConnection
sudo ipsec auto –up vpc2vpcConnection

One Last step before we’re done:

Configuring the VPC to route via the software routers

  • Go back to your AWS console, VPC service and select the 1st region (Ireland)
  • Click on “Route Tables” in the toolbox – You should already have a route table available to you in the list.
  • Check the checkbox near the route table line and you’ll see at the bottom of the screen your already configured routes.
  • In the empty line, do the following:
    • In the Destination column, enter the second VPC IP subnet (10.50.0.0/16 in the example).
    • In the Target column, Select “Enter Instance ID” and in the popup window select your newly created software router (in Ireland).
    • Click on the “Add” button at the right hand side of the new line.

4.8

Screenshot 8 – routing traffic via your software router

  • Now Select the 2nd region (Oregon)
  • Click on “Route Tables” in the toolbox – You should already have a route table available to you in the list.
  • Check the checkbox near the route table line and you’ll see at the bottom of the screen your already configured routes.
  • In the empty line do the following:
    • In the Destination column, enter the first VPC IP subnet (10.20.0.0/16 in the example).
    • In the Target column, Select “Enter Instance ID” and in the popup window select your newly created software router (in Oregon).
    • Click on the “Add” button at the right hand side of the new line.

6. Creating instances and testing traffic flow between the two VPCs

The next step is to validate that the setup is working and to understand how to use it.

First, create a new instance in each one of the VPCs (one in Ireland and one in Oregon). Make sure you place the instances in a different Security Group than the software router’s Security Group (So you’re instance will not allow direct communication from the public internet)

Last Step – Configuring Security Groups

In order to allow communication between the two VPCs, you’ll have to add rules to both the software routers’ Security Groups and the Instances Security Groups in the following manner (again, this is not the best practice from security point of view but it will get you started with the instances pinging  each other).

  • Ireland Instance Security Group
    • Add the following rule – Allow All Traffic from 10.50.0.0/16
  • Ireland Software Router Security Group
    • Add the following rule – Allow All Traffic from 10.20.0.0/16
  • Oregon Instance Security Group
    • Add the following rule – Allow All Traffic from 10.20.0.0/16
  • Oregon Software Router Security Group
    • Add the following rule – Allow All Traffic from 10.50.0.0/16

Here is how it should look like in the AWS Security Group Management page:

4.9

Screenshot 9 – Ireland Server’s Security Group

4.10

Screenshot 10 – Ireland Software Router’s Security Group

4.11

Screenshot 11 – Oregon Server’s Security Group

4.12

Screenshot 12 – Oregon Software Router’s Security Group

Few last IMPORTANT comments on Security

  • It is highly recommended that you’ll only open up UDP ports 500 and 4500 to the specific Elastic IP address of the peer Software Router’s (and not as I’m showing in this example to 0.0.0.0/0 which means everyone)
  • You should also change the source address for the SSH traffic to your office’s address or subnets.
  • Please remember to change the shared secret to something more secure than in the above example

Now the instances in the two regions should be able to communicate (test it by pinging from one instance to another)

Hope that helped.

We are happy to receive comments and questions on this post. You can leave them here at the Blog or by sending an email to our support team.

If you’d like us to do it for you, just join our service for free to get a glimpse of our capabilities. Good Luck!

Amir Naftali

Subscribe to newsletter

Recommended Posts

Showing 4 comments

  • Simon So
    Reply

    Thanks for the detailed guide! Helped me to set up mine!

    One minor thing: please correct the typo, /etc/ipsec.conf, not /etc/ipsec.config. Important for the unexperienced (like myself).

    • 40Cloud
      Reply

      Hi Simon, thanks for the comment.
      The post was updated.

  • Riaan
    Reply

    Hi, also a big thank you for going through the trouble of creating a detailed howto on how to configure ipsec between two VPCs.

    Just one arbitrary addition: in the docs you refer to “Download an example of ..” with a “download” image below it. The image however is a link to the article itself and not to an example config file.

pingbacks / trackbacks

Free Trial

Request a Demo