KubeCon EU 2024: Argo
Discover the Latest Innovations and Insights from Argo Project’s Collaborative Gathering
Argo Project is an open-source toolkit for deploying and running applications and workloads on Kubernetes. It extends the Kubernetes API and unlocks new powerful capabilities in application deployment, container orchestration, event automation, and progressive delivery. It comprises four main projects: Argo CD, Argo Workflows, Argo Rollouts, and Argo Events. On March 19, 2024, ArgoCon, a co-located event of KubeCon + CloudNativeCon Europe 2024, took place in Paris, France, aimed at fostering collaboration, discussion, and knowledge sharing around the Argo Project. This article shares highlights from ArgoCon Europe 2024.
ArgoCon Highlights
First up, Carlos Santana from AWS delivered the opening remarks for ArgoCon, welcoming new contributors and introducing updates to the Argo Project.
Meanwhile, Jesse Suen from Akuity sparked a discussion with “Argo CD is not enough” introducing a new open-source project, Karg. He highlighted the significant advantage of GitOps in rapid, seamless application delivery, especially for a single environment. However, existing GitOps platforms face challenges in supporting advanced pipelines spanning multiple stages, regions, and cloud services. These platforms also undergo time-consuming validation processes. The lack of these features often leads GitOps users to rely on CI tools and custom automation, resulting in complex and brittle setups. Karg, an open-source project, aims to overcome these obstacles while adhering to the core principles of GitOps, solving these issues in an understandable manner, and supporting popular tools known to GitOps users such as Kustomize, Helm, Argo CD, and Flux.
In the following talk, J.P. Zivalich and Elliot Gunton posed some questions: “Who writes Argo workflows spanning hundreds of lines of YAML in a single file? How many have undergone unit testing? Integration testing? Just system testing? How navigable are YAML files as workflows expand into behemoths?” They argued that as workflows iterate and grow into monstrosities, they become less secure and slow overall, continuing to use YAML at this stage may hinder rather than help. Hera addresses this by introducing native Python function templates for workflows, making the process more intuitive. Moreover, Hera conforms to the YAML specification of Argo’s context manager, making it user-friendly. Additionally, Hera boasts natural advantages in Python modularity and packaging compared to YAML, aiding in both unit testing of individual functions and end-to-end testing of workflows. They demonstrated how Hera makes it easier to create DAGs, loops, and more using Python functions.
Michael Crenshaw and Zach Aller from Intuit addressed pain points encountered when scaling GitOps: users can’t see exact changes before deployment, environment upgrades are tricky, and there’s no straightforward git-centric method for automatic rollback. They proposed three new rules for GitOps to address these issues: developers can review PR-level delta changes during the review process, and these change manifests are reviewable. Environment promotion and rollback can be handled as automated pull requests. Pull requests provide developers with a place to review, discuss changes, and implement progression events, fitting into ecosystems developers are already familiar with. They also shared new Argo CD feature proposals to enable next-gen GitOps practices to be effortlessly intuitive in any organization.
Based on practical Argo usage scenarios in a session by Yao Lin and Harris Khan from Bloomberg, they discussed challenges, examined their potential impacts, and shared observations and experiences, hoping to inspire prevention and solution approaches applicable to other types of batch workloads. They introduced a centralized, multi-tenant, workflow orchestration platform maintained for internal engineering teams. The system coped with rapid business growth and tackled unique challenges such as peak submission rates and unpredictable resource requests within a short timeframe. Furthermore, the platform accommodates diverse user characteristics, catering to various use case requirements within the cluster.
Carlos Santana and Raj Saha from AWS discussed exploring the future of Kubernetes cluster scaling using Karpenter. Karpenter is the latest, fastest Kubernetes native cluster autoscaler, now part of the CNCF ecosystem. While Cluster Autoscaler remains widely used among Kubernetes enthusiasts, they introduced this innovative approach, facilitating seamless and efficient transition of worker nodes and Pods to Karpenter. They emphasized the powerful synergy between Argo Workflows and Karpenter, such as seamless migration of Kubernetes resources, leveraging the flexibility of Argo Workflows, and its unique ability to execute CI pipelines within clusters. Karpenter enhances the security posture of Kubernetes clusters and helps developers adeptly handle challenges like massive node failures and time-intensive tasks.
Similarly, Nicholas Morey from the Akuity team addressed how GitOps aims to solve visibility and collaboration challenges in Kubernetes, emphasizing declarative desired states and continuous coordination. While this brings valuable solutions, issues arise regarding Git repository structure, configuration management tools, and branching strategies. Configuration tools like Helm and Kustomize simplify manifest management but also introduce some challenges. In GitOps, the true desired state for Kubernetes clusters is not Helm charts or Kustomizations but the manifests these tools present. Many organizations abstract storage in Git, which can have adverse effects. In this talk, he detailed how to address these issues through the Rendered Manifests Pattern, involving rendering manifests for the main branch of GitOps repositories through CI workflows, ensuring immutable and clear presentation of the desired state in Git.
References
If you’re interested in ArgoCD, you can learn more through the links below:
1. ArgoCon
2. Welcome + Project Updates
3. Navigating Multi-stage Deployment Pipelines via the GitOps Approach
4. Orchestrating Python Functions Natively in Argo Using Hera
5. Space-Age GitOps: The Rise of the Humble Pull Request
6. Growing Pain: Scaling to 10K Workflows per Week:
7. Harnessing Karpenter: Transforming Kubernetes Clusters with Argo Workflows
8. The Rendered Manifests Pattern: Reveal Your True Desired State
