Image for post
Image for post

Confused by Enhanced Berkeley Packet Filters, understandably so. There is lots of marketing material from companies using it to offer many different solutions and a lesser amount of terse technical material. Here we will try to split the difference and provide a high level view in an effort to assemble the jigsaw. We will focus on eBPF for networking, however the technology has many other uses.

The Linux Kernel has “hooks” where functionality can be added. At each of these hooks, information related to that hook is available. Using eBPF a small program running in a protected manner can be inserted at each of those hooks. …


Image for post
Image for post

Have you ever wondered how networking inside the POD is constructed. I was forced to learn how this worked recently when I needed to figure out which veth-pair is associated with a POD without “execing” into the POD.

Each POD is a shared network namespace. The containers in the POD share that same namespace. The key to this operation is the /pause container. This container, hidden from the kubectl, is effectively a parent container and creates the network namespace. The pause container can be seen using docker or CRIO commands. …


Image for post
Image for post

In this article we discuss three open source load-balancer controllers that can be used with any distribution of Kubernetes.

  • MetalLB. The popular and most well know load-balancer controller
  • PureLB. The newest addition. (Full disclosure, I am involved in the development of PureLB)
  • Porter. A relatively recent addition, initially focused on routing only

Adding a controller that implements Service Type LoadBalancer functionality is a key networking component necessary for simple, scaleable cluster operation.

  • Enable controlled external access to cluster services/applications
  • External resources are pre-configured
  • Easy to integrate with automated workflows (CI/CD)

The first is obvious however the second two points are just as critical in designing a load-balancer solution for your cluster. Depending on the deployment model, it’s common that the team responsible for ensuring reliable networking to the cluster is different from the team running the cluster(s). Pre-configuration allows the network team to help get things setup and leave operation to the cluster team. This facilitates easy integration with CI/CD because the use of load-balancer resources is now part of the standard k8s “application” definition and workflow. …


What is a k8s Load Balancer?

Image for post
Image for post

Load balancer has become a confusing term in Kubernetes. Its safe to assume that everyone reading this understands what a load-balancer does, it distributes packets/sessions to set of destinations. There is a lot of load-balancing going on in k8s and lots of components doing it kube-proxy, ingress controllers and of course the service resource type called LoadBalancer.

The service type LoadBalancer is the subject of this post, understanding how it works is simple. A Service is used to provide access to the network ports on a POD, setting the type to LoadBalancer triggers a controller to configure a k8s external resource to direct packets towards the cluster. The external resource can be anything that can make the IP address added to the service reachable and forward to a set of nodes. Load Balancers are not a native component of k8s, its easy to find out if you have one installed by attempting to use it. If there is a Load Balancer “controller” your service should get an external address allocated and shown in the service, if not the external address will remain “pending”. …


Image for post
Image for post

A key component of your k8s deployment is the Container Network Interface (CNI) If you deploy Kubernetes services using providers such as Google or AWS, your CNI will be chosen by your cloud provider however understanding how CNI’s operates is important in having a complete understanding of how k8s is accessed. Cloud vendors have a unique environment, they use a combination of techniques to provide isolation and forwarding at scale in their multi-tenant network infrastructure, therefore their CNIs need to work closely with their infrastructure. If you’re using a downstream distribution it will often include a default CNI, and in some cases allow you to use an alternative CNI. …


Image for post
Image for post

This may seem a little obvious however one of the most important decisions you can make in the design of a k8s system is identifying the need for a system perimeter. All networks have perimeters, including public cloud services, and understanding how the k8s system fits will ensure that the level of security you require will be maintained.

The perimeter is the network boundary where access is controlled to the hosts that reside within. In Enterprises these are often called zones and access to and between these zones is controlled with firewalls. In a non-cloud native environment, those network devices are independently managed, either manually or using network-specific orchestration systems. …


Understanding the routing and packet forwarding is key to correct configuration and understanding of failure modes. The easiest way to get an understanding of how packets are getting forwarded is to run kubeproxy in iptables mode, although there are a lot of rules, they can be relatively easily followed.

From the address range defined in the metalb configuration, an address is allocated to the service and can be viewed displaying the service as “external ip”. This address block is a specific routed network allocated to Metallb. …


This design for an external k8s Gateway can provide a high level of security from outside access. It ensures that only the specific ip address and ports used by services are exposed to outside traffic and provides the option to add traffic rate limiting to services using additional annotations in the services.

Image for post
Image for post

In addition to your k8s cluster, the solution consists of three components.

  • Linux Gateway. A host running Free Range Routing (FRR) and configured to use NetFilter Tables.
  • Metallb. Metalb is a k8s Load Balancer manager. It uses the Loadbalancer API to allocate ip addresses and direct traffic to PODs. …


Still coming to terms with k8s networking, you are not alone? The automation simplicity offered by public cloud providers and workstation versions (minikube & microk8s) hide the network complexity that enables application developers to manage networking functions. Adding in the the redefinition and new terminology around “ingresses”, “load-balancers” and “service mesh”, makes building a private k8s infrastructure that leverages the potential benefits, and is sufficiently robust, a significant undertaking. …


k8s Ansible Operators promise to provide a simple way to automate infrastructure management by bringing Ansible into the k8s complex and using k8s custom resources as well as other information contained in the k8s api.

Most of the examples in documentation and posts focus on using ansible inside the k8s complex, and while valuable, there are plenty of ways to solve those problems. I wanted to use ansible operators to manage infrastructure outside of the k8s complex. Here are a few things that I needed to figure out to achieve this.

Can I watch any resource?
You can watch any resource in the complex, just add them to the watches file and make sure that you have permission, the default roles do not include pod/status. Remember that when your executing a role from a resource other than the CR, variables (extra_vars) from the CR will not be available, it’s necessary to look them up independently. …

About

Adam Dunstan

Tech enthusiast, infrastructure specialist, leader & engineer

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store