KubeCon + CloudNativeCon Europe 2023

Resource management is a fundamental area in Kubernetes that focuses on how to properly reserve, allocate, and isolate finite resources on nodes such as CPU, memory, disk, network, accelerators, etc. Resource Management is a hot topic, with multiple proposals raised recently on how to improve things both in Kubernetes and container runtimes: Dynamic Resource Allocation, QoS class resources, improvements to CPU Management, to container lifecycle management and statistics, support in CRI-enabled container runtimes for advanced low-level runtimes such as Kata containers, Firecracker, gVisor, and Confidential Containers and many more. In this presentation, speakers will present the “big picture” for these proposals, how they are interconnected, how they are different, which problems they are targeting to solve, and what they mean for Kubernetes users. This presentation will be helpful for cluster administrators and users to understand the future direction in their resource management area and give a framework for them to provide feedback that can help shape these future efforts. We will also describe opportunities for folks who are more interested to get involved with the open source SIG-Node and runtime communities to drive these efforts forward.

Dynamic Resource Allocation (DRA) is a new Kubernetes feature that puts resource scheduling in the hands of 3rd-party developers. From an end-users perspective, it moves away from the limited "countable" interface for requesting access to resources (e.g. "nvidia.com/gpu: 2"), providing an API more akin to that of persistent volumes. Using GPUs as an example, DRA unlocks a host of new features without the need for awkward solutions shoehorned on top of the existing device plugin API. These features include: * Controlled GPU Sharing (both within a pod and across pods) * Multiple GPU models per node (e.g. T4 and A100) * Specifying arbitrary constraints for a GPU (min/max memory, device model, etc.) * Dynamic allocation of MIG devices * Dynamic repurposing of a GPU from full to MIG mode * Dynamic repurposing of a GPU for use as Passthrough vs. vGPU * ... the list goes on ... In this talk, you will learn how to build your own resource driver for DRA. This includes details of how to use Kubernetes's in-tree helper libraries for DRA, where to find an example driver to get you started, as well as best-practices for architecting the driver itself. Throughout this talk, we will use our existing NVIDIA and Intel GPU drivers as a guide, concluding with a demo of these drivers in action.

When we write load testing scripts against our applications, we write them sequentially: A, then B, then C. But this doesn't accurately reflect the organized chaos of a system in production, nor does it prepare the system for the unexpected. Emergence is a phenomenon where parts of a whole independently develop properties not originally present in the whole. Emergence is what helps ant workers develop roles without leadership, prompts animals to evolve adaptive traits without forethought, and facilitates non-toxic communities without moderators. The growing field of emergent software applies this swarm logic to the programs that we write. What would it take to write emergent load testing scripts? It turns out that there are a few ingredients for emergence: a large population size, opportunities to interact, feedback, and an element of control. In this talk, Nicole van der Hoeven discusses how to bring these elements to load testing by writing a script in Grafana k6 that can independently decide what requests to make next, modify Kubernetes app pods, and disrupt services based on a continual feed of results during runtime-- all without manual intervention. She shows how to wield this new breed of load testing to improve confidence in the complex systems we build.

GPUs and accelerators are changing traditional High Energy Physics (HEP) deployments while also being the key to enable efficient machine learning. GPU scheduling in Kubernetes has been limited until now. Not being able to easily share access to single GPUs by multiple workloads leads to inefficiencies when those are light or spiky. At the same time these resources are scarce, expensive and in high demand. In this talk we explore the different possibilities to improve overall usage of GPU resources. We explore the multiple options for GPU scheduling, time sharing and the recently introduced Nvidia Multi-Instance-GPU (MIG) for physical partitioning. We cover the features and limitations of each option and present extensive benchmark results that helped us assign each workload to the most appropriate layout. Finally we describe how we manage GPUs in a centralized way, ensuring optimal resource utilization for services like continuous integration, machine learning and batch.

Can containers and Kubernetes run anywhere? Yes, nearly. We have seen in the past fighter jets, fully isolated environments, security critical infrastructure and more with Kubernetes. So it is no wonder that Boston Dynamics Spot, the most advanced mobile quadrupled robot, is running on containers too. But this wasn’t enough for us. How and why we tweaked the (real world) bot a little and what are our lessons learned is part of this talk. We will show you the easy steps to migrate to K8s, the experimental integrations with Wasm and ideas on how to manage Spot like any other Kubernetes. Our targets are to provide a highly reliable, self-healing software infrastructure for industrial great robots that are secure, fast and autonomous.

Setting up clusters that need thousands of nodes can be challenging especially when it comes to etcd architecture and configuration. It’s especially common in use cases like large processing farms for AI/ML/HPC workloads,or in case of internet scale serving applications. In this session you’ll be able to learn best practices around etcd deployments architecture and configuration from tech leads from DataDog and Google Cloud. DataDog has been running their own Kubernetes clusters with thousands of nodes for many years already. Google Cloud has been offering managed clusters up to 15000 nodes since 2020. You’ll be able to hear from practitioners in the space how to squeeze performance, reliability and scale from etcd instances in your clusters. You'll be able to hear about topics like handling disk io or network throughput bottlenecks or how to handle api server restarts and their impact on etcd.

Cluster federation sounds easy: you take several k8s clusters and treat them as one. Right? Easy! What if the network is unstable? What if clusters are resource-diverse and on the move? What if clusters join, unjoin, and re-join spontaneously? What if a cluster running a stateful application leaves the federation? Cluster federation does not sound so easy anymore. Join us on an adventure of bringing cloud federation to the edge and uncovering just how far a mesh of elastic mobile clouds can stretch. See the use Liqo for distributed federation; a combination of TAS (Telemetry Aware Scheduler) and OLSR (Optimized Link State Routing) for network aware scheduling; and Chaos Mesh to simulate network effects. We will show you a tactical cloud concept developed together by TNO and the Dutch Ministry of Defence, where manned or unmanned vehicles join spontaneously in ad-hoc cloud constellations to deliver a resilient, distributed, and collaborative computation.

Containers allow applications to run anywhere and on anything (with a container runtime on Linux). This ability to run anywhere is perfect for Edge or IoT devices where device management may be complex. How do we bring container orchestration out to the Edge, IoT devices, and other distant, difficult, or constrained environments – K3s. K3s is a lightweight Kubernetes distribution that can run on as little as 1 CPU and 512MB of RAM and does not need etcd. K3s is a CNCF sandbox project that is perfect to deliver Kubernetes out to the Edge, IoT devices, and constrained environments. This talk isn't a K3s overview and goes beyond deploying Kubernetes at the Edge or on IoT devices. This talk is about "Cloud Nativifying" a few sysadmin duties for Edge and IoT devices where access is difficult. This talk covers how to control OS updates of Edge and IoT devices via Kubernetes by making the OS a Kubernetes resource. First we make a bootable Linux derivative OCI container image and store it in a container registry. Updates are delivered via container registry, controlled by Kubernetes, and accomplished by switching active/passive images. Not only do you manage an application through Kubernetes but also the underlying OS of the machine.