Did you overlook platform when selecting API Security & Management?

The platform used to implement API Security and Management is arguably as important as the features Security and Management features offered. The platform provides the dataplane, managing requests is the primary purpose for implementing an API Security or Management system. The features that are being developed or implemented depend on the platform, it's key to not lose sight of this.

We developed the EPIC platform as an ideal execution environment for Proxy engines implementing API Security and Management. It provides the configuration and automation required by network and security engineers to enable developers to dynamically create and manage their applications. The list of required functionality is extensive and covered in this post.

There are two key components in the platform: the Proxy engine and the Proxy execution environment that hosts the Proxy engine.

Proxy Engine

The Security and Management components configure, program and manage Proxy engines therefore there is a tight coupling between the API product and the Proxy engine. In many solutions this results in the Proxy engine being bundled with the API Security or Management tooling.

Understanding the specifics of the Proxy engines used to implement API Security & Management is critical to understanding the solution capabilities.

There is a significant amount of variability in how Proxy engines are delivered with API Security & Management solutions.

• Bundled Proxy Engine. In these cases there is often little visibility to the Proxy Engine. It could be a custom built using libraries from a popular frameworks (node.js, rust, python, go) or a fork of a popular Open Source Proxy. There are so many Open Source Proxy engines that it doesn't make sense to build a new one.
• Supported Proxy Engines. Some solutions support commonly available Proxy Engines, these are usually one of the more popular Open Source engines such as Envoy, HAProxy or Nginx.

There are so many Proxy engines that creating a comprehensive list would be almost impossible and many Proxy engines are use case specific, such as Varnish for Content Distribution, and are not suited to the API Security and Management use case.

• Development Framework Proxies. Almost every modern development language has libraries that can be used to create a Proxy engine. The functionality available in those libraries defines the base functionality and additional features can be developed. However no two proxies developed using a single language/framework are the same. With the exception of very specific scale or use cases, I don’t see the need to develop another Proxy engine. This type of solution is often used to shortcut initial development at the expense of longer term support and stability.
• Complete Proxy Engines. There are lots of these as well, both Open Source and proprietary. From a functionality perspective there are differences but in general from a Request processing perspective their functionality is largely the same. Performance varies often based upon when development began and development language used but it’s usually not that important.

The important differences between Proxy Engines are how they are programmatically configured and how telemetry data is returned. In the case of Development Framework Proxy, any mechanism can be used as long as it can be developed, however complete Proxy engines have predefined configuration and telemetry interfaces. Two examples:

• Envoy Proxy. The programmatic configuration interface Envoy uses is called xDS and supports REST or gRPC. Most tooling that configures Envoy uses the gRPC interface as it provides more functionality than REST (including Acnodal EPIC). Go has libraries for gRPC and Envoy is widely used in Kubernetes. Considering its popularity in Kubernetes, it's not surprising that statistics are scraped from a REST endpoint.
• Nginx. In its Open Source form, Nginx is configured using a configuration file and that configuration reloaded from a CLI. (There is a commercial version with a REST interface, as with many of these Open Source/Commercial products it is hard to figure out what you have to pay for!). The security and management functionality are loaded as modules and where these modules are provided as part of the solution, in practise additional configuration mechanisms can be added by the module provider. By default telemetry is provided by logging however there are other modules such as OTEL that can be used.

Each Proxy engine has its own unique management interfaces and supporting each of them is a unique development for the API Security and Management solution, supporting multiple Proxy engines creates a feature matrix requiring additional attention.

Proxy Execution Environment

Proxy engines are executables that terminate and establish sessions, acting as a “middle-man” between the client and the server applications. In most cases with API Security and Management solutions, how and where the Proxy engine runs is left to the individual deploying the solution.

Proxy Engines need to run somewhere, the selection and configuration of the execution environment is critical to the security, scale and reliability of the solution.

The execution environment for the Proxy engine or engines is critical to security, scale and operation. The Proxy engine depends on the underlying infrastructure for operation and security. In the Security context, if the Proxy engine is executed in an insecure execution environment, the API Security platform is insecure. Put simply the execution environment is a key component of API security and Management.

API Security and Management provides access from untrusted clients to controlled resources. Therefore one side of the API Security and Management system is exposed to the untrusted Internet. This exposure creates an attack surface. Therefore the configuration of the execution environment is necessarily an extension of the API Security and Management system. For example, one or more public exposures (IP Address/DNS) are required for the API system and a limited set of open protocol ports.

Procedurally, and in a statically configured system, creating an execution environment in a Linux host consists of exposing at least one interface to the public internet, configuring security rules in the form of IP tables and matching the Proxy Engine listener port configuration. However such a solution is not practical for production use. Only one IP address/port tuple is available limiting the configuration of multiple upstream endpoints to DNS configuration.

Being limited to a small number of shared Proxy engines is in itself an operational and security problem. A shared Proxy engine results in a more complex Proxy engine configuration and in some cases necessitates Proxy and execution environment configuration specific to one of the shared Proxy use case potentially creating additional attack surface for others.

Scaling and operation for production is also limited with a simplistic static execution environment. Adding Proxy engines from new applications requires operational intervention. Scaling to meet demands that cannot be served by a execution environment creates even more complexity as the system will likely need to be redesigned to integrate with routings ECMP to provide distribution among execution environments.

The problem with Kubernetes Ingresses

If your environment has Kubernetes, it’s temping to believe that this problem can be solved with in-cluster Kubernetes specific solutions. There is however a major flaw with this assumption. This solution puts the untrusted access surface, the downstream side of the Proxy engine, inside the Kubernetes cluster. Compromising the Proxy engine POD will expose the cluster to lateral movement into application PODs.

Creating a platform for API Security and Management

The implementation of an API Security and Management platform is a complex task taking significant effort. Consider the following attributes that we have implemented in our platform and their need in a reliable, secure execution environment.

• uService Proxy Engines. Containerizing the Proxy Engines allows the platform to create as many individual Proxy engines as needed for each use case. These individual Proxy engines can each be configured to provide maximum security for that use case which avoid the need for merging use case configurations. Many Proxy engines have plugins for specific protocols or functions, using a uService platform further enhances security by limiting the plugin use to specific instances. Testing or using newer versions of the Proxy engine can be limited to specific use cases. uService Proxy engines when managed by the platform can also scale easily with each Gateway consisting of multiple instances of the Proxy engine as demanded by workload.
• Combine proxy, network & execution environment management. When a new Proxy is created or its configuration changed, that event should trigger the necessary configuration of the execution environment. While it may seem simple; by automating the allocation of IP addresses, advertising and exposing those addresses, generation of DNS records and opening only the ports necessary for the proxy listeners enables Secure APIs to be exposed on-demand in a manner consistent with predefined policies. If any of these steps is manual, adhoc or distributed among multiple organizations or functions they because a dangerous source of configuration error.
• Separate from Application resources. It's tempting to place the API functionality next to the applications in Kubernetes clusters. While many will argue that a Cluster can be made sufficiently secure by configuration, correct configuration over Clusters with the amount of complexity and possible variance of tooling makes this a difficult objective to achieve every-time at scale. Therefore the simpler solution of providing a high level of isolation between the execution environment and application resources simplifies this risk completely avoiding any possibility of lateral movement.
• Access Endpoints behind Firewalls. Isolating application resources also results in putting them behind additional security, in particular firewalls. In the case of Kubernetes, almost all POD resources are on networks where PODs are not externally addressable.
• Multicluster, multiple Endpoint. The API Security and Management environment needs to be able to communicate with all of the upstream application endpoints irrespective of location and operating environment. Tight integration with kubernetes workflow is required to simplify application operators. Not all applications are located in Kubernetes clusters, the ability to add application endpoints running on dedicated hosts, VMs or other environments is also required with a similar workflow.
• Scales across execution environment nodes. A proxy consisting of multipleproxy execution instances should scale across multiple compute instances. This required the implementation and integration with networking infrastructure to leverage network hardware capabilites to ensure performance. The addition and removal of Proxy engines should dynamically trigger network distribution of packets.
• Limit Proxy hops. Often the solution to scale Proxies is the addition of more Proxies in the request flow. Each proxy terminating either a TCP or HTTP session adds latency and configuration complexity. The fewer Proxies, ideally one, in the path the better the performance and simpler the configuration. In practise this takes attention to detail.
• The right Proxy Engine. The Proxy Engine used in API Security and Management defines the solution capabilities. How the Proxy engine manages configuration directly impacts scale and operations. For example, updating configuration files vs. a dynamic programmable API dramatically increases the functionality and security of the overall solution. Implementing security functionality often involves additional programming of the Proxy, a choice of programing interface entry points and languages makes this effort significantly easier. For example, the Envoy proxy engine enables dynamically loadable programs as part of the configuration written using WebAssembly or LUA to interact with the request flow. Where more low level functionality is required, it provides the ability to develop plugin modules to be added to the execution binary.

Conclusion

I could have written this as a list of things that the platform we developed provides, however I think it’s more valuable to provide insight into the requirements and why they are necessary in implementing API Security in particular and also API Management when Public APIs are exposed to the Internet. A detailed understanding of API technology is required to implement an API Security platform, I hope you agree that in addition a detailed understanding of how to create the Execution environment is equally important.

If you would like to leverage the EPIC platform in your API Security deployment or integrate it with your existing solution, please let me know.

You can find the documentation and source code at https://www.epic-gateway.org/