Part 7. How to Set Up Networking

Let’s walk through an overview of how physical networks work. Networking is a huge topic, so I’ve had to compress a lot of information, and what you’re seeing here is a simplified picture that skips over some of the nuances. Let’s start by thinking through how you’d connect computers together. Connecting two computers is easy: all it takes is a single cable, as shown in Figure 59.

Connecting n computers is more complicated. If you had to connect every computer to every other computer, you’d need n² cables, which would be messy and expensive. The solution is to connect all the computers to a single switch, a device that can forward data between computers, which only requires N cables, as shown in Figure 60.

These connected computers form a network. Connecting two networks is easy; you typically do it using routers, as shown in Figure 61.

The term "internet" is derived from interconnected networks: a network of networks. Many of those networks are private. For example, you might have a private network in your house or a private network in a data center. For your home network, you probably got a router from your ISP, which is actually both a switch and a router, and it creates a private network that allows the devices you have at home (e.g., your computer, laptop, phone, tablet, printer, TV) to talk to one another. For a data center network, the data center technicians set up various switches and routers, and this creates a private network that allows the servers in that data center to talk to one another.

Let’s take a look at each of these.

If you deploy into the cloud, the cloud provider has already taken care of all the physical networking for you: all the servers, switches, routers, and cables are already hooked up, largely in a way you can’t see or control. What you can control is a virtual network, which you configure entirely in software (which is why it’s sometimes referred to as software-defined networking). In this section, you’ll learn about virtual networks in the cloud, virtual networks in orchestration tools, and then go through an example of creating a virtual network in AWS.

The traditional approach used at many companies for managing access to private networks is the castle-and-moat model, based on the analogy to a castle with a secure perimeter (walls, moat, drawbridge, etc.), and a soft interior. It’s hard to get into the castle, but once you’re inside, you have free rein to move around. An equivalent private network is one that doesn’t allow you to access anything from outside the network, but once you’re "in" the network, you can access everything.

In a physical network, with the castle-and-moat model, merely being connected to the network means you’re "in." For example, with many corporate office networks, if you are plugged into the network via a physical cable, you can access everything in that network: all the wiki pages, the issue tracker, the IT help desk, and so on. However, if you’re outside the physical network, how do you connect to it? For example, if you’re working from home, how do you get access to your corporate office network? Or if you have infrastructure deployed in a VPC in the cloud, how do you get access to the private subnets of that VPC?

A common solution is to deploy a bastion host. In a fortress, a bastion is a structure designed to stick out of the wall, allowing for more reinforcement and extra armaments, so that it can better withstand attacks. In a network, a bastion host is a server designed to be visible outside the network (i.e., it’s in the DMZ), and this server has extra hardening and monitoring, so it can better withstand attacks. The idea is that you keep the vast majority of your servers private, with the network acting as a secure perimeter (like a wall and moat), and you use the bastion host as the sole entry point to that network. Since there’s just one bastion, you can put a lot of effort into making it secure. Users connect to the bastion host via protocols such as SSH, RDP, or VPN, as you’ll see later in this blog post. Since the bastion host is "in" the network, after you’ve successfully connected to the bastion host, you’re now also "in," and you can access everything else in the network, as shown in Figure 64.

For example, if you can connect to the bastion host in Figure 64, you can access everything in the private subnets of that VPC, including the private servers and database with IPs 10.0.0.20, 10.0.0.21, and 10.0.0.22. This approach worked in the past, but in the modern world, the castle-and-moat approach leads to security concerns, as discussed next.

The castle-and-moat approach originated in a world where:

In short, your location on the network mattered; some locations could be trusted, while others could not. This is increasingly not the world we live in, as these days:

As a result, for many companies, the idea of a secure perimeter and soft interior no longer makes sense. There’s no clear "perimeter" or "interior" anymore, and no location on the network can be implicitly trusted. This has led to the rise of zero trust architecture (ZTA), which is based on the concept of "never trust, always verify": you never trust a user or device just because they have access to a location on the network. The core principles of ZTA can be summarized as follows:

Some of the major publications on the ZTA model include "No More Chewy Centers: Introducing the Zero Trust Model of Information Security" by John Kindervag, where he coins the term "Zero Trust Model," Zero Trust by NIST, and "BeyondCorp: A New Approach to Enterprise Security" by Rory Ward and Betsy Beyer at Google. Google’s BeyondCorp paper is arguably what popularized ZTA, even though the paper doesn’t ever use that term.

A surprising principle in the BeyondCorp paper is that Google no longer requires employees working remotely to use a VPN to access internal resources. Instead, those resources are accessible directly via the public internet. This may seem like a paradox: how can exposing internal resources to the public be more secure? Google’s take is that exposing internal tools publicly forces you to put more effort into securing them than if you merely relied on the network perimeter for security. Figure 65 shows a simplified version of the architecture Google describes in BeyondCorp.

The idea is that you expose your internal resources to the public internet via an access proxy, which uses the user database, device registry, and access policies to authenticate, authorize, and encrypt every connection. From a quick glance, the zero trust approach in Figure 65 might not look all that different from the castle-and-moat approach in Figure 64: both rely on a single entry point to the network (a bastion host or an access proxy) that grants access to private resources. The key difference is that in the castle-and-moat approach, only the bastion host is protected, and all the private resources are open, so if you can get past the bastion, you get access to all the private resources, whereas with the zero trust approach, every single private resource is protected, and each one requires you to go through an authorization process with the access proxy. Instead of a single perimeter around all the resources in your network, the zero trust approach is like putting a separate perimeter around each individual resource.

Therefore, zero trust isn’t a single tool you adopt, but something you integrate into every part of your architecture, including the following:

Implementing a true ZTA is a tremendous amount of work, and few companies pull it off fully. It’s a good goal for all companies to strive for, but how far down the ZTA path you go depends on your company’s size. Smaller startups typically use the castle-and-moat approach; mid-sized companies often adopt a handful of ZTA principles, such as using SSO and securing microservice communication; large enterprises try to go for most of the ZTA principles. As you saw in Section 1.4, you need to adapt your architecture to the needs and capabilities of your company.

Now that you’ve seen the castle-and-moat and zero trust approaches, let’s look at some of the most common tools you use to access private networks: SSH, RDP, and VPN.

Secure Shell (SSH) is a client-server protocol that allows you to connect to a computer over the network to execute commands, as shown in Figure 66.

For example, the client could be the computer of a developer named Alice on your team, and the server could be the bastion host. When Alice connects to the bastion host over SSH, she gets a remote terminal where she can run commands and access the private network as if she were using the bastion host directly.

SSH is ubiquitous: just about all modern Linux, Unix, and macOS distributions support SSH natively, and there are multiple clients for Windows. SSH is also generally considered a mature and secure protocol, as it’s an open standard with open source implementations, it has been around for about 30 years, and it has a massive community around it.

Under the hood, SSH uses public-key cryptography for authentication and encryption; you’ll learn more about these topics in Part 8. For now, all you need to know is that SSH relies on a key pair, which consists of a public key and a private key. Configuring one server to accept one user’s public key is no problem, but at scale, this becomes a challenge. If you need to support a large number of servers and developers, key rotation and revocation (e.g., when a developer leaves the company), and different levels of permissions and access (including temporary access), things get a lot more complicated. One solution is to use managed services from cloud providers, such as Amazon EC2 Instance Connect in AWS or metadata-managed SSH connections in Google Cloud (full list). Another solution is to use the general-purpose connectivity tools I mentioned earlier, such as Teleport or Tailscale.

Let’s take a quick look at how to use SSH.

Remote Desktop Protocol (RDP) is a way to connect to a Windows server remotely and to manage it via the full Windows UI, as shown in Figure 68. It’s just like being at the computer: you can use the mouse, keyboard, and all the desktop apps.

Being able to use the full Windows UI makes RDP accessible to all roles at a company (not just developers), and it can be a nicer experience than being limited to a terminal (as with SSH). However, RDP works only with Windows servers, and it is somewhat notorious for security vulnerabilities, so you can’t expose it directly to the public internet (as you’ll see shortly).

Let’s take a quick look at how to use RDP.

Next, you configure the client:

Now that you’ve configured your clients and servers, you open up the RDP client, type in the IP address of the server to connect to (which might be an RD Gateway IP), enter the username and password when prompted, and after a minute or two, you’ll be logged in. This will give you access to the full Windows UI, as you saw in Figure 68, and from that UI, you’ll have access to the private network.

Being able to use a UI to access the private network is great, but it’s the UI of another computer. Sometimes you want to be able to access the private network directly from your own computer’s UI, as that’s where you have all your apps and data. This is one of the areas where VPN shines, as discussed next.

A virtual private network (VPN) is a way to extend a private network across multiple networks or devices. The fact that you’re on a VPN is transparent to the software running on those devices. The software can communicate with the private network as if the device were plugged physically into the network, without the software being aware of the VPN or having to do anything differently.

VPN clients are available for almost every OS (including smartphones), allowing you to access private networks from your own devices in a way that’s accessible to all roles at a company (and not just developers). Most VPN tools are built around either Internet Protocol Security (IPsec) or Transport Layer Security (TLS), two protocols that are generally considered mature and secure, as they have been around for more than 30 years, are ubiquitous, and have massive communities.

IPsec and TLS typically rely on certificates, which are based on public-key cryptography (like SSH) but allow for mutual authentication, where the client can verify that the VPN server is really who it says it is by using the server’s certificate, and the server can verify that the user is really who they say they are by using the client’s certificate. This is great for security, but managing certificates at scale can sometimes be challenging (you’ll learn more about IPsec, TLS, and certificates in Part 8). Another challenge with VPN is that routing all your network traffic through VPN servers can increase latency and degrade throughput.

These days, VPNs have three common use cases:

Let’s take a quick look at how to use VPN for the first two use cases.

For the use case of connecting remote employees to an office or data center network, you typically use a client/server architecture. First, you configure the VPN server:

Next, you configure the client:

For the use case of connecting two data centers together, the details depend on the devices you’re using, but at a high level, in each data center, you do the following:

Now that you’ve seen how to access a private network from the outside, let’s turn our attention to how services within a private network can communicate.

As soon as you have one service, A, that needs to talk to another service, B, you have to figure out service discovery: how does A figure out the right IP addresses to use to talk to B? This can be a challenging problem, as each service may have multiple replicas running on multiple servers, and the number of replicas and which servers they are running on may change frequently as you deploy new versions, replicas crash and are replaced, or you scale the number of replicas up or down in response to load.

Let’s go over some of the tools you can use for service discovery.

As you saw in Part 6, a big part of breaking your code into services is defining an API for the service and maintaining it over the long term. One of the key decisions you’ll have to make is the protocol you will use for that API, which consists of two primary choices:

Next, we’ll go over some of the most common protocols in use today and then explore the key factors to consider when picking a protocol.

A service mesh is a networking layer designed to help manage communication between applications in a microservice architecture by providing a single, unified solution to the following problems:

Almost all of these are problems of scale. If you have only two or three services, a small team, and not a lot of load, these problems are not likely to affect you, and a service mesh may be an unnecessary overhead. If you have hundreds of services owned by dozens of teams and high load, these are problems you’ll be dealing with every day. If you try to solve these problems one at a time, you’ll find that it is a huge amount of work and that the solution to one has an impact on the other (e.g., how you manage encryption affects your ability to do tracing and traffic mirroring). Moreover, the simple solutions you’re likely to try first may require you to make code changes to every single app, and as you learned in Part 6, rolling out global changes across many services can take a long time.

This is where a service mesh can be of use. It gives you an integrated, all-in-one solution to these problems, and just as important, it can solve most of these problems in a way that is transparent and does not require you to change your app code.

When things are working, a service mesh can feel like a magical way to upgrade the security and debuggability of your microservice architecture. However, when things aren’t working, the service mesh itself can be difficult to debug, as it introduces many new moving parts (encryption, authentication, authorization, routing, firewalls). Moreover, understanding, installing, configuring, and managing a service mesh can be a lot of overhead. If you’re at the scale where you need solutions to the problems listed earlier, a service mesh is worth it; if you’re a tiny startup, it’ll only slow you down.

Service mesh tools can be divided into three buckets. The first bucket is the service mesh tools designed for use with Kubernetes, which include Linkerd (the project that coined the term "service mesh"), Istio, and Cilium. The second bucket is managed service mesh tools from cloud providers, such as AWS App Mesh and Google Cloud Service Mesh. The third bucket is service mesh tools that can be used with any orchestration approach (e.g., Kubernetes, EC2, and on-prem servers), such as Consul service mesh and Kuma (full list).

The best way to get a feel for what a service mesh does is to try one out, so let’s go through an example of using Istio with Kubernetes.

Istio is a popular service mesh for Kubernetes that was originally created by Google, IBM, and Lyft, and released as open source in 2017. Let’s see how Istio can help you manage the two microservices you deployed with Kubernetes in Part 6. One of those microservices was a backend app that exposed a simple JSON-over-HTTP REST API. The other microservice was a frontend app that made service calls to the backend, using the service discovery mechanism built into Kubernetes, and then rendered the data it got back using HTML. First, make a copy of those two sample apps into the folder you’re using for this blog post’s examples:

Second, you’ll need a Kubernetes cluster. The easiest one to use for learning and testing is the one that comes with Docker Desktop, so just as you did in Part 3, fire up that cluster, and make sure you’re authenticated to it:

Third, download and install the latest Istio release (minimum version 1.22). The release will be in a folder called istio-<VERSION>, where <VERSION> is the version of Istio you installed. This folder will include a samples subfolder that has some useful sample code (which you’ll use shortly), and istioctl, a CLI tool that has useful helper functions for working with Istio (which you should add to your PATH). Use istioctl to install Istio in your Kubernetes cluster as follows:

This uses a minimal profile to install Istio, which is good enough for learning and testing (see the installation instructions for profiles you can use for production). The way Istio works is to inject its own sidecar into every Pod you deploy into Kubernetes. That sidecar provides all the security, observability, resiliency, and traffic management features, without you having to change your application code. To configure Istio to inject its sidecar into all Pods that you deploy into the default namespace, run the following command:

Istio supports a number of integrations with observability tools. For this example, let’s use the sample add-ons that come with the Istio release, which include a dashboard for Istio called Kiali, a database for monitoring data called Prometheus, a UI for visualizing monitoring data called Grafana, and a distributed tracing tool called Jaeger:

At this point, you can verify that everything is installed correctly by running the verify-install command:

If everything looks good, deploy the frontend and backend apps as you did before:

After a few seconds, you should be able to make a request to the frontend as follows:

At this point, everything should be working exactly as before. So is Istio doing anything? One way to find out is to open up the Kiali dashboard you installed earlier:

This command opens the dashboard in your web browser. Click Traffic Graph in the menu on the left, and you should see something similar to Figure 69.

If the traffic graph doesn’t show you anything, run curl localhost several more times, and then click the refresh button in the top right of the dashboard. You should see a visualization of the path that your requests take through your microservices, including through the Services and Pods. Right away, you see one of the key benefits of a service mesh: observability. You get not only this service mesh visualization, but also aggregated logs (click Workloads in the left menu, select sample-app-backend, and click the Logs tab), metrics (run istioctl dashboard grafana), and distributed traces (run istioctl dashboard jaeger). When you’re done experimenting with Kiali, press Ctrl-C to exit.

Another key benefit of service meshes is security, including support for automatically encrypting, authenticating, and authorizing all requests within the service mesh. By default, to make it possible to install Istio without breaking everything, Istio initially allows unencrypted, unauthenticated, and unauthorized requests to go through. However, you can change this by configuring policies in Istio. Create a new folder called istio, and within it, a file called istio-auth.yml, with the content shown in Example 132.

This code does the following:

Deploy these policies as follows:

Now, look what happens if you try to access the frontend app again:

Since your request to the frontend wasn’t using mTLS, Istio rejects the connection immediately. Enforcing mTLS makes sense for backends, as they should be accessible only to other services. However, your frontend should be accessible to users outside your company, so you can disable the mTLS requirement for the frontend as shown in Example 133.

This is an authentication policy that works as follows:

You can put the YAML in Example 133 into a new YAML file, but dealing with too many YAML files for the frontend is tedious and error prone. Let’s instead use the --- divider to combine the frontend’s sample-app-deployment.yml, sample-app-service.yml, and the YAML you just saw in Example 133 into a single file called kubernetes-config.yml, with the structure shown in Example 134.

With all your YAML in a single kubernetes-config.yml, you can delete the sample-app-deployment.yml and sample-app-service.yml files, and deploy changes to the frontend app with a single call to kubectl apply:

Try accessing the frontend again, adding the --write-out flag so that curl prints the HTTP response code after the response body:

You get an error again, but this time, it’s a different error. That’s because two policies are at play: an authentication policy and an authorization policy. You added an authentication policy that allows the frontend to be accessed without mTLS, so Istio is no longer blocking your request entirely, but you still get a 403 response code (Forbidden) because the allow-nothing authorization policy is still blocking all requests. To fix this, you need to add authorization policies to the backend and the frontend.

This requires that Istio has a way to identify the frontend and backend. Istio uses Kubernetes service accounts as identities, automatically providing a TLS certificate to each application based on its service account, and using mTLS to provide mutual authentication (i.e., the backend will verify that the request is coming from the frontend, and the frontend will verify that it is really talking to the backend). Istio will handle all the TLS details for you, so all you need to do is associate the frontend and backend with their own service accounts and add an authorization policy to each one.

Start with the frontend, updating its kubernetes-config.yml as shown in Example 135.

Here are the updates to make to the frontend:

Run apply to deploy these changes to the frontend:

Next, head over to the backend, and combine its Deployment and Service definitions into a single kubernetes-config.yml file, separated by ---, just as you did for the frontend (and then delete sample-app-deployment.yml and sample-app-service.yml). Once that’s done, update the backend’s kubernetes-config.yml as shown in Example 136.

Here are the updates to make to the backend:

Run apply to deploy these changes to the backend:

And now test the frontend one more time:

Congrats, you get a 200 (OK) response code and the expected HTML response body, which means you now have microservices running in Kubernetes, using service discovery, and communicating securely via a service mesh! With the authentication and authorization policies you have in place, you have significantly improved your security posture. All communication between services (such as the request the frontend successfully made to the backend) is now encrypted, authenticated, and authorized—all without you having to modify the Node.js source code of either app. Moreover, you have access to all the other service mesh benefits, too: observability, resiliency, and traffic management.

When you’re done testing, you can run delete on the kubernetes-config.yml files of the frontend and backend to clean up the apps. If you wish to uninstall Istio, first remove the global authorization and authentication policies:

Next, uninstall the add-ons:

And finally, uninstall Istio itself, including deleting its namespace and removing the default labeling behavior:

One of the benefits of software-defined networking is that it’s fast and easy to try different networking approaches. Instead of having to spend hours or days setting up physical routers, switches, and cables, you can try out a tool like Istio in minutes, and if it doesn’t work for you, it takes only a few more minutes to uninstall Istio and try something else.