Exploring Kubernetes 1.28 Sidecar Containers

Kubernetes v1.28 comes with multiple new enhancements this year and we’ve already covered an overview of those in our previous blog, Do check this out before diving into sidecar containers.

We’re going to completely focus on the new sidecar feature for this post, which enables restartable init containers and is available in alpha in Kubernetes 1.28. The launch of sidecar containers in Kubernetes 1.28 overcomes the set of challenges faced in previous versions by implementing several key improvements. These advancements can be used to increase productivity by deploying and managing sidecars within a Kubernetes cluster.

The Sidecar Origin Story 

Sidecar term is not new for Kubernetes users and specialists, The concept of a “sidecar” has been part of Kubernetes since 2015 and was described in a blog post about ambassador container connecting proxy to the world, Adapter containers converting heterogeneous data from different systems into a single unified representation, composite containers as additional containers that “extend and enhance the ‘main’ container”. Sidecar containers are a common Kubernetes deployment pattern for better networking strategies, secure deployments, and simplified integration of observability tools. But, All of this was without the native support of Kubernetes with service meshes, gathering metrics, and more. The lack of support has caused some usage friction and implementation of additional workloads, which means implementing it hasn’t been inconsequential, or hassle free. Before Kubernetes v1.28, there was no way to differentiate a sidecar container from a main container. This provoked issues such as sidecars terminating before the main container or restarting to run after completing the job.

According to the above blog post by Brendan, There are different Use Cases and Patterns by using sidecar containers. Sidecar containers find extensive applications in the realm of service mesh. For instance, a sidecar proxy can be injected into the data plane of a pod, facilitating advanced features such as load balancing and service discovery. Additionally, sidecar containers excel in scenarios that require shared resources like a shared file system. By syncing the file system with a repository or bucket, a sidecar container can ensure seamless data access and synchronization between the main container and external storage.

One particularly useful sidecar pattern is the Ambassador container pattern, where a main container communicates with external services through a proxy container. This allows for added flexibility in connecting to different types of databases or enabling high availability setups.

Another powerful pattern is the adapter container, which acts as a reverse proxy, collecting metrics from the main container and pushing them to a centralized monitoring system. With sidecar containers, you can easily monitor the metrics of your applications without modifying the main container’s codebase.

How to use sidecar containers in Kubernetes 1.28?

Kubernetes 1.28 adds a new restartPolicy field to init containers that are available when the SidecarContainers feature gate is enabled that means you can set a new restartPolicy field to Always on an init container, and Kubernetes will handle it significantly:

• The kubelet won’t wait until the container has finished, instead, it will only wait until the startup has completed.
• Every time it will be restarted until the sidecar fails during startup and the Pod restartPolicy is Never.
• The sidecar container is dependent on the service container hence it will keep running for as long as the main container runs.
• The sidecar container will terminate after the main and another container shut down, allowing jobs to be completed always.

kind: Pod

…

spec:

containers:

- name: myapp

- image: alpine:latest

- command: ['sh', 'echo 'hello-lumigo' > /user/rohit/hello.txt']

  initContainers:

- name: secret-fetch

    image: secret-fetch:1.2.0

  - name: istio-proxy

    image: istio/proxyv2:1.16.0

    args: ["proxy", "sidecar"]

    restartPolicy: Always

  volumeMounts:

…

Why is Kubernetes 1.28 built differently and What is so special about sidecar containers? 

Previously, Kubernetes failed to differentiate between the lifecycle of the primary application or service container and the sidecar container which posed some difficulties in managing the startup and shutdowns of the pods. Manually Starting and shutting pods is really a pain for some use cases, With the introduction of container lifecycle hooks in K8s v1.28, Your primary application pods are decoupled from the sidecar container which eliminates this issue. There is better management and survival of pods now. Furthermore, Some of the things that are possible now are log forwarding, matrix collection, and capturing errors were complex tasks that required workarounds.

Let’s say you want to integrate some database like Redis or MySQL inside your application and connect it to the main service pod you just have to tell your sidecar container to do so. Same with proxy sidecar containers, Whatever traffic going through your services from one application to another can now go efficiently, sidecar containers will handle the traffic, security issues, collect log metrics, and many more. One of the common examples is a service mesh. Whenever you want to allocate any resource to your service in the application, Developers can add or manage resources to sidecar containers without worrying about the main container inside the application.

However, It’s amazing now to see in Kubernetes 1.28, sidecar containers are now prioritized citizens in the Kubernetes ecosystem. Kubernetes is going to provide dedicated APIs and resources for their deployment and management, which will greatly simplify the process. I’m excited because, with native support, Kubernetes will be a lot easier to integrate and manage sidecar containers within a Kubernetes cluster. Again, If you’re interested in contributing to the Kubernetes 1.28 sidecar containers version then don’t wait and hop on to Kubernetes issues, GitHub repository, and working groups to submit the bugs and help them with enhancements.

But, Why now after so many years? 

In previous releases of Kubernetes, the introduction of sidecar containers would have increased the overall complexity of your pods in the cluster. Kubernetes 1.28 starts to address all of the issues like pod raised operation, business logic concerns, and orchestration complexities. Node has different pods and a single pod owns N number of containers, Hence the number of containers within a pod is no longer a concern, and the orchestration and operational complexities are effectively managed.

Lastly, Kubernetes 1.28 includes significant updates to the existing tooling and kubelet, the primary node agent, to enable efficient management of sidecar containers. Implementing the necessary changes to support sidecar containers required updates to existing tooling, and these improvements ensure that the necessary tooling is available to effectively implement and manage sidecar containers in a Kubernetes environment. Kubernetes team recommends using the sidecar containers currently only on the short-lived testing clusters at the alpha stage. If you have any existing sidecar that is configured with the main container it can be moved to the initContainers section of the pod spec as we have shown above at the start.

Some Issues you should be aware of, Before starting to use the new version that will be solved before the feature moves to beta:

• The CPU, memory, device, and topology manager are unaware of the sidecar container lifetime and additional resource usage and will operate as if the Pod had lower resource requests than it actually does.
• The output of the kubectl describe node is incorrect when sidecars are in use. The output shows resource usage that’s lower than the actual usage because it doesn’t use the new resource usage calculation for sidecar containers.

Explore Kubernetes Sidecar Containers

The introduction of sidecar containers in Kubernetes 1.28 solves the challenges faced by previous versions in a comprehensive manner. With the introduction of sidecar containers as an alpha feature in Kubernetes 1.28, a new world of possibilities opens up for developers and operators. The ability to seamlessly integrate additional functionalities, share resources, and ensure smooth integrations makes sidecar containers an invaluable tool in the Kubernetes ecosystem. As Kubernetes continues to evolve and with your feedback from the community, sidecar containers are destined to become an even more integral part of container orchestration. With enhanced support and documentation, developers can leverage the power of sidecar patterns to build resilient, efficient, and scalable applications. The improvements include container lifecycle hooks for better management of startup and shutdown behaviors, native support for sidecar containers with dedicated APIs and resources, streamlined complexity in managing pod clusters, and improved tooling for efficient management. These enhancements provide a more robust and streamlined solution for deploying and managing sidecars within a Kubernetes cluster.

As the Kubernetes ecosystem continues to expand, opportunities for participation and innovation beckon. Whether you’re a seasoned developer or just beginning your journey, your contributions can help sculpt the future of the wider ecosystem. What better way to familiarize oneself than hands-on experience? Register for a free Lumigo account and deploy Kubernetes 1.28 in a test environment alongside our Kubernetes operator (which can auto-trace an entire namespace in a single bound), and experience the prowess of the v1.28 release firsthand.