When Kubernetes first got here onto the scene, it was a serious turning level, a revision of the infrastructure and operations house that reworked the way in which builders and ops personnel construct, deploy, and preserve purposes within the cloud. It has since change into the clear customary for the way fashionable purposes are constructed and operated. Because the CNCF famous in its newest Annual Cloud Native Survey report, “Amongst container customers, 82% are utilizing Kubernetes in manufacturing in 2025, up from 66% in 2023. This represents near-universal adoption throughout the container ecosystem.”
Over the previous couple of years, one other revision within the house has occurred with Kubernetes’s evolution from a container orchestrator to an AI infrastructure platform. Based on the CNCF survey, “The rise of Kubernetes because the de facto AI platform represents a basic shift in how organizations method machine studying operations. . .[with Kubernetes] offering a unified orchestration layer that handles each conventional software workloads and compute-intensive AI duties.” The emergence of seismic applied sciences like generative AI and agentic AI has solely accelerated this transformation.
The intersection of AI with Kubernetes is undoubtedly some of the impactful developments within the operations house. As Jonathan Johnson, software program architect at Dijure, observes, “AI on K8s may be very, crucial, and there may be not sufficient [resources] on the market.” Raju Gandhi, senior technical architect at Edward Jones, echoes this evaluation, noting that “operationalizing AI/ML on K8s is an enormous concern, [and it’s only] getting larger. It is a subject that wants consideration.” However what are among the issues that you need to learn about this development to maintain abreast and keep forward within the sport?
Generative AI
Anybody with entry to a pc or a smartphone has probably used some iteration of generative AI, a surprising truth when you think about that GenAI was on the outer edges of mainstream discourse and consumption a scant 5 years in the past. However on the finish of 2022, the debut of ChatGPT marked the start of a technological revolution, one that will influence and reshape practically each side of our working and private lives. Unsurprisingly, there are actually 1000’s of generative AI fashions, a proliferation that naturally has its personal set of complexities. Choosing a mannequin is easy, however if you happen to’re an software developer or MLOps engineer, how do you go about working that mannequin in a manufacturing system? Not solely do it’s important to be cognizant of things like resilience, scalability, safety, and operational prices, however there’s the truth that bringing a mannequin from experimentation into manufacturing may be arduous if not carried out correctly. That’s the place Kubernetes comes into play.
As Roland Huß and Daniele Zonca, distinguished engineers at Crimson Hat, word, “GenAI/LLM fashions are useful resource intensive, requiring substantial computational energy and huge datasets. Given its scalability and extensibility, Kubernetes is uniquely suited to operate as an environment friendly platform for AI and LLM mannequin pretraining, fine-tuning, deployment, and immediate engineering.” They additional elaborate that “this integration with Kubernetes not solely simplifies the adoption of cutting-edge AI applied sciences but in addition ensures a seamless and environment friendly operational circulate. Kubernetes, with its sturdy scalability and administration capabilities, stands as a really perfect platform for generative AI tasks, aligning DevOps and MLOps practices in a cohesive ecosystem.”
This sentiment is already shared by a large swath of the trade. Based on the CNCF survey above, as of 2025, 66% of organizations run generative AI workloads on Kubernetes. These organizations embody OpenAI, which makes use of Kubernetes for its AI/LLM software experimenting and testing; Tesla, which makes use of KServe to handle production-grade LLM inference; and Adobe, which makes use of Kubernetes to energy its suite of generative artistic fashions. Different firms taking this method embody Uber, Intuit, and Google. With extra firms adopting this apply for his or her generative AI and LLMs operations, it’d be prudent for any group to leverage Kubernetes for their very own GenAI and LLM workflows.
Agentic AI
Almost coinciding with the rise of GenAI has been the regular progress of agentic AI. In contrast to GenAI, agentic AI goes past answering easy prompts and producing textual content in its means to function autonomously to carry out complicated, multistep actions, make the most of instruments, and make impartial selections. With its means to help each conventional ML processes and GenAI and LLM operations, it ought to come as no shock that Kubernetes has a job within the agentic AI ecosystem as effectively.
Based on Ronald Petty, principal marketing consultant at RX-M, “Kubernetes has been leveraged to host machine studying pipelines, together with AI mannequin coaching and inference. As inference choices have change into plentiful and inexpensive, on and off-premise, we’ve got seen the rise of brokers. Coupling cloud native applied sciences and standard protocols, we now see brokers transferring from advert hoc demos to complicated fleets of brokers on techniques like Kubernetes.” So what are some examples of the combination between these two applied sciences?
One notable providing is Kagent, an OS programming framework that runs AI brokers in Kubernetes and “helps engineers construct highly effective inside platforms by tackling cloud native duties reminiscent of configuration, troubleshooting, complicated deployment situations, observability pipelines and dashboards, and safely enabling community safety.” Working alongside related strains is K8sGPT, an AI-powered instrument that leverages clever insights and automatic troubleshooting to research Kubernetes clusters for configuration issues and safety points, in addition to generates options to issues found in evaluation.
A newer entry within the subject is Sympozium, a Kubernetes-native coordination layer for multi-agent AI techniques that “solves the identical drawback Kubernetes solved for containers, however for brokers that must share context, hand off duties, and preserve shared situational consciousness.” One other newer providing is Agent Sandbox, which lets you run AI brokers as remoted, stateful workloads with a local API on Kubernetes.
The basics
Whereas it’s essential to pay attention to the most recent developments and traits affecting your area, that shouldn’t come on the expense of foundational information and abilities. As basketball nice Michael Jordan as soon as stated, “Get the basics down and the extent of every thing you do will rise.” Probably the most basic abilities for working with Kubernetes is networking, and frustratingly sufficient, it’s one of many tougher ones to grasp. As Cisco senior workers engineer Nico Vibert observes, “Platform engineers are typically snug with Linux networking however much less so with protocols like BGP and IPv6; community directors know these protocols effectively however discover Kubernetes abstractions unfamiliar. Each personas battle to navigate the handfuls of networking instruments seemingly required to satisfy connectivity and safety necessities.” But as organizations transfer mission-critical workloads, AI coaching pipelines, and controlled monetary providers onto Kubernetes, the engineers who can design, safe, and troubleshoot the community layer have change into among the most sought-after professionals within the trade.
In recognition of each the significance and troublesome nature of the Kubernetes networking ability, the CNCF not too long ago introduced a brand new certification centered on the Kubernetes community engineer function. The certification is designed to validate hands-on networking experience throughout the entire aforementioned layers, filling a spot that the Kubernetes neighborhood has lengthy acknowledged.
For organizations that use Kubernetes to develop and ship purposes, leaders and decision-makers have to be conscious that using Kubernetes together with the most recent AI instruments is now not a luxurious however a needed apply that may permit their firms to thrive. An analogous onus needs to be positioned on the fundamentals. When hiring your subsequent DevOps, community, or website reliability engineer, be certain that their means to design, safe, and troubleshoot the Kubernetes community layer is second to none.
If you wish to dive deeper, take a look at Roland Huß and Daniele Zonca’s Generative AI on Kubernetes, Jonathan Johnson’s GPU Kubernetes Homelab stay course, Alex Corvin, Taneem Ibrahim, and Kyle Stratis’s Scalable Kubernetes Infrastructure for AI Platforms, Ashok Srirama and Sukirti Gupta’s Kubernetes for Generative AI Options, and Yogesh Raheja’s K8sGPT Necessities on-demand course. They’re all on O’Reilly. When you’re not a member, you may get began with a free trial.
