Hyperscale companies realized horizontal scale is the only solution, the applications we experience are developed using may individual elements often running in fleets of containers. These individual elements are combined to create applications. Ingresses are the entry-point into these massive hyperscale systems and implement the logic constructing modern applications that include containers and other elements. These Ingresses resides at the system or network edge implementing the rules necessary to direct requests, thereby constructing the application.

Ingresses can be installed in the cluster, operate external to the cluster, or can be described as a service mesh, where all POD’s contain an…


Confused by Enhanced Berkeley Packet Filters, understandably so. There is lots of marketing material from companies using it to offer many different solutions and a lesser amount of terse technical material. Here we will try to split the difference and provide a high level view in an effort to assemble the jigsaw. We will focus on eBPF for networking, however the technology has many other uses.

The Linux Kernel has “hooks” where functionality can be added. At each of these hooks, information related to that hook is available. Using eBPF a small program running in a protected manner can be…


Have you ever wondered how networking inside the POD is constructed. I was forced to learn how this worked recently when I needed to figure out which veth-pair is associated with a POD without “execing” into the POD.

Each POD is a shared network namespace. The containers in the POD share that same namespace. The key to this operation is the /pause container. This container, hidden from the kubectl, is effectively a parent container and creates the network namespace. The pause container can be seen using docker or CRIO commands. …


In this article we discuss three open source load-balancer controllers that can be used with any distribution of Kubernetes.

  • MetalLB. The popular and most well know load-balancer controller
  • PureLB. The newest addition. (Full disclosure, I am involved in the development of PureLB)
  • OpenELB. A relatively recent addition, initially focused on routing only

Adding a controller that implements Service Type LoadBalancer functionality is a key networking component necessary for simple, scaleable cluster operation.

  • Enable controlled external access to cluster services/applications
  • External resources are pre-configured
  • Easy to integrate with automated workflows (CI/CD)

The first is obvious however the second two points are…


What is a k8s Load Balancer?

Load balancer has become a confusing term in Kubernetes. Its safe to assume that everyone reading this understands what a load-balancer does, it distributes packets/sessions to set of destinations. There is a lot of load-balancing going on in k8s and lots of components doing it kube-proxy, ingress controllers and of course the service resource type called LoadBalancer.

The service type LoadBalancer is the subject of this post, understanding how it works is simple. A Service is used to provide access to the network ports on a POD, setting the type to LoadBalancer triggers a controller to configure a k8s external…


A key component of your k8s deployment is the Container Network Interface (CNI) If you deploy Kubernetes services using providers such as Google or AWS, your CNI will be chosen by your cloud provider however understanding how CNI’s operates is important in having a complete understanding of how k8s is accessed. Cloud vendors have a unique environment, they use a combination of techniques to provide isolation and forwarding at scale in their multi-tenant network infrastructure, therefore their CNIs need to work closely with their infrastructure. If you’re using a downstream distribution it will often include a default CNI, and in…


This may seem a little obvious however one of the most important decisions you can make in the design of a k8s system is identifying the need for a system perimeter. All networks have perimeters, including public cloud services, and understanding how the k8s system fits will ensure that the level of security you require will be maintained.

The perimeter is the network boundary where access is controlled to the hosts that reside within. In Enterprises these are often called zones and access to and between these zones is controlled with firewalls. In a non-cloud native environment, those network devices…


Understanding the routing and packet forwarding is key to correct configuration and understanding of failure modes. The easiest way to get an understanding of how packets are getting forwarded is to run kubeproxy in iptables mode, although there are a lot of rules, they can be relatively easily followed.

From the address range defined in the metalb configuration, an address is allocated to the service and can be viewed displaying the service as “external ip”. This address block is a specific routed network allocated to Metallb. …


This design for an external k8s Gateway can provide a high level of security from outside access. It ensures that only the specific ip address and ports used by services are exposed to outside traffic and provides the option to add traffic rate limiting to services using additional annotations in the services.

In addition to your k8s cluster, the solution consists of three components.

  • Linux Gateway. A host running Free Range Routing (FRR) and configured to use NetFilter Tables.
  • Metallb. Metalb is a k8s Load Balancer manager. It uses the Loadbalancer API to allocate ip addresses and direct traffic to…


Still coming to terms with k8s networking, you are not alone? The automation simplicity offered by public cloud providers and workstation versions (minikube & microk8s) hide the network complexity that enables application developers to manage networking functions. Adding in the the redefinition and new terminology around “ingresses”, “load-balancers” and “service mesh”, makes building a private k8s infrastructure that leverages the potential benefits, and is sufficiently robust, a significant undertaking. …

Adam Dunstan

Tech enthusiast, infrastructure specialist, leader & engineer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store