Cloud & Platform Engineering · Infrastructure · Open Source
100% open source stack · 6 integrated layers · 0 manual cluster setup
A client selling modular, plug-and-play compute deployment units needed a software layer to turn raw hardware into a fully-managed cloud experience — standard VMs, persistent storage, networking, and managed Kubernetes clusters — built entirely on open source primitives, from scratch.
Challenge
Full IaaS stack from bare metal with no managed cloud to lean on. Open source only. Kubernetes inside Kubernetes to offer managed clusters as a product. GitOps from day one.
Approach
Vanilla Kubernetes as the single control plane. Rook + Ceph for unified block and object storage from one operator. CAPI + KubeVirt + K0smotron for managed Kubernetes clusters as declarative workloads inside the host cluster.
Outcome
Production-grade IaaS platform with zero proprietary dependencies. All six layers — compute, storage, networking, managed clusters, and observability — managed through a single Kubernetes-native API.
The background
The client builds modular deployment units — self-contained, high-density compute designed to be deployed rapidly inside existing real estate. To make that product viable, they needed a software layer that turned raw hardware into a cloud experience. The entire stack was greenfield. Every architectural decision had to be made from scratch.
Open source was a non-negotiable requirement — for cost, auditability, and long-term independence. The stack had to cover the full IaaS surface: network, block storage, object storage, VM management, and managed Kubernetes, with no managed cloud provider to lean on.
The challenge
What was built
Rather than building bespoke orchestration, we put Vanilla Kubernetes at the centre of the architecture — the single control plane for all resources. This gave us a proven reconciliation engine and a declarative API surface without introducing custom abstractions that accumulate complexity.
We evaluated Longhorn — simpler setup, but limited scalability and no object storage — and OpenEBS — flexible, but less mature for production block storage. We chose Rook + Ceph for product maturity, proven scalability, and the ability to support both block and object storage patterns from a single unified operator. One operator handles persistent volumes, S3-compatible object storage, and file storage — eliminating an entire category of architectural decisions.
Customer Kubernetes control planes run as workloads inside the host cluster via K0smotron — fully isolated and GitOps-managed. KubeVirt runs VMs as native Kubernetes workloads. CAPI provides the declarative, GitOps-compatible interface for provisioning and lifecycle management of all clusters. Every customer cluster is a Kubernetes custom resource — provisioned by applying a manifest, upgraded the same way, with a complete audit trail of every state change.
Standard VM lifecycle — create, resize, migrate, delete — managed entirely through the Kubernetes control plane via KubeVirt. No separate hypervisor layer. Every VM is a Kubernetes custom resource managed through the same API surface as clusters and storage volumes. The operational model for the entire platform is consistent: kubectl, GitOps, declarative manifests — regardless of whether you're managing a VM or a storage bucket.
ClickHouse for analytics and time-series data. Grafana for dashboards. NATS for high-throughput internal messaging. PostgreSQL for persistent state. All wired into the same declarative control plane — no separate observability stack to operate. The same GitOps workflow that manages compute and storage also manages the monitoring infrastructure.
Architecture
All resources — VMs, storage volumes, network policies, customer clusters — are Kubernetes custom resources managed through a single API surface. Managed customer clusters run as workloads inside the host cluster, provisioned on demand with full isolation and a complete GitOps audit trail.
All layers are Kubernetes custom resources managed through one control plane API surface. Every cluster provisioned via CAPI as a CRD, GitOps-driven from day one.
Results