What if the safest way to change a production system was to open a pull request? That single question drives GitOps architecture, a model that turns a version-controlled repository into the source of truth for everything running in your infrastructure. Instead of engineers running commands against live clusters, software agents watch the repository and quietly make reality match what the files describe. The idea sounds almost too tidy to work at scale, yet it has become one of the dominant operating patterns in cloud-native engineering. This article explains how the architecture works, where it works well, and where teams commonly run into trouble.
What GitOps Architecture Really Means
GitOps is an operating model in which the desired state of a system is stored declaratively in Git, and automated agents continuously reconcile the live system to match it. The term was coined by Weaveworks in 2017 to describe running Kubernetes with Git as the single source of truth, and it spread quickly through the cloud-native community.
The architecture rests on four principles published by the OpenGitOps working group under the Cloud Native Computing Foundation. The system is declarative, meaning you describe what you want rather than the steps to get there. The desired state is versioned and immutable, so every change has a full history and nothing is silently overwritten. Agents pull that state automatically rather than waiting for someone to push it. And the system is continuously reconciled, with agents constantly comparing actual state against desired state and acting to close the gap. The full set of principles and the accompanying glossary are maintained by the CNCF OpenGitOps project, which is the best starting point for anyone who wants the canonical wording.
What separates GitOps from ordinary “we keep our YAML in Git” practice is the reconciliation loop. Storing manifests in a repository is just configuration management. GitOps adds an always-on agent that refuses to let the cluster drift away from what the repository says.
The Core Components of a GitOps System
A working GitOps setup usually involves four moving parts, and understanding each one clarifies where problems tend to appear.
The Git repository holds the declarative manifests: Kubernetes objects, Helm charts, Kustomize overlays, or whatever format the chosen tool renders. It is often split into an application repository and an environment or config repository, though monorepo designs are common too.
The reconciliation agent runs inside the target cluster. It watches the repository, pulls changes, renders the manifests, and applies them. It also reports status back, so you can see whether the live system matches the desired state.
The CI pipeline builds and tests artifacts, then updates the repository with a new image tag or manifest change. Crucially, in the canonical model, CI does not touch the production cluster directly.
The observability layer surfaces sync status, drift, and health. This is where operators spend much of their time, because a dashboard full of green checkmarks is the whole point of the exercise.
For teams thinking about how this fits into a wider platform, it helps to see GitOps as one layer in a broader set of cloud infrastructure and architecture decisions rather than a standalone tool choice.
Pull-Based Versus Push-Based Deployment
The most consequential architectural decision in GitOps is whether the agent pulls from inside the cluster or an external system pushes changes in.
| Dimension | Pull-based (canonical) | Push-based |
|---|---|---|
| Who initiates the change | In-cluster agent watches Git | CI system applies manifests |
| Cluster credentials | Stay inside the cluster | Held by the CI system |
| Drift detection | Continuous, built in | Usually absent or bolted on |
| Blast radius if CI is compromised | Limited, no cluster access | Large, full deploy rights |
| Typical tools | Argo CD, Flux | Scripts, generic CI steps |
The pull model is the one most practitioners mean when they say GitOps. Because the agent lives inside the cluster and reaches out to Git, the CI system never needs production credentials. A compromised pipeline can at worst propose a bad change to the repository, where review and policy can catch it. The Argo CD documentation walks through how its in-cluster agent tracks Git and reports sync status, which is a useful concrete example of the pattern.
Push-based variants still exist and are not automatically wrong. They are simpler to bootstrap and fit teams with existing deployment scripts. But they give up the security and drift-detection advantages that make GitOps attractive in the first place.
The Reconciliation Loop, Explained Plainly
Reconciliation is the engine under the hood, and it is worth understanding because it explains most GitOps behavior, good and bad.
The loop works like this:
- The agent reads the desired state from the repository at a specific commit.
- It renders that state into concrete resource definitions.
- It compares the rendered result with what is actually running in the cluster.
- Where they differ, it applies the desired version.
- It records the outcome and repeats on an interval or when a change is detected.
That fourth step is where the magic and the friction live. If someone edits a deployment by hand, the agent notices the difference, labels it drift, and reverts it. Argo CD surfaces this as an OutOfSync status; Flux handles it through its controllers and explicit reconcile commands. The underlying pattern is the same one Kubernetes uses everywhere, and the Kubernetes controller documentation explains the reconciliation concept in general terms if you want the foundation beneath the tooling. (Kubernetes Documentation: Overview)
In practice, this behavior surprises newcomers in two ways. First, manual hotfixes vanish, sometimes within seconds. Second, if the repository itself is broken, the agent will faithfully try to apply a broken state. Reconciliation is obedient, not intelligent.
The Two Reference Implementations: Argo CD and Flux
Almost every GitOps conversation comes down to two tools, both under the CNCF umbrella.
Argo CD is part of the Argo project and presents itself as a declarative GitOps continuous delivery tool for Kubernetes. It ships with a web UI that many teams love for its visual diff and sync status, and it supports multi-cluster management out of the box. Its application model maps neatly onto how platform teams think about environments.
Flux is a set of composable controllers, each responsible for a slice of the reconciliation job. It leans toward a CLI-driven, Git-centric workflow with less emphasis on a dashboard. The Flux documentation describes its controller architecture and how the pieces fit together, which is worth reading even if you choose Argo CD, because it clarifies the underlying model.
Neither is objectively better. Teams that want a visual control plane and multi-cluster views tend toward Argo CD. Teams that prefer small composable components and a Git-first workflow tend toward Flux. Both implement the same principles, and switching later is painful but not impossible.
Where GitOps Architecture Fits in a Wider Platform
GitOps does not exist in a vacuum. It is usually one layer of a larger platform story that includes runtime infrastructure, model serving, and increasingly AI workloads. Teams building intelligent systems often find that the same reconciliation ideas apply to model deployment and configuration, which is why AI inference architecture: a practical enterprise guide is useful background when planning how deployment pipelines should behave.
The pattern also connects naturally to broader enterprise AI systems and architecture thinking, where reproducibility and auditability matter as much as speed. A Git history that records exactly which model version, configuration, and resource limits went live is a compliance asset rather than only a convenience.
As platforms grow more automated, the line between infrastructure management and application delivery blurs. Articles on AI cloud architecture: when infrastructure becomes intelligent explore that convergence, and GitOps is one of the mechanisms that makes it manageable.
Common Mistakes and What Actually Works in Practice
Most GitOps failures are architectural, not tool-related. A few patterns show up again and again.
Putting secrets in Git. Plain-text credentials in a repository is the single most common mistake. Teams solve it with external secret stores, sealed secrets, or encryption tooling that keeps the repository safe to share. Decide this before you migrate, not after.
Letting CI push to production. Many “GitOps” setups quietly keep a CI job that applies manifests directly. That reintroduces the credential exposure the model was meant to remove. If you adopt pull-based reconciliation, commit to it.
Ignoring drift instead of reverting it. Some teams disable auto-sync to avoid surprises, then wonder why environments diverge. Auto-sync with good review is usually safer than manual sync with good intentions.
Treating database migrations as ordinary manifests. Imperative, stateful operations do not fit the declarative model cleanly. Most mature teams run migrations as a separate, carefully gated step rather than pretending they are just another resource.
Letting the monorepo grow unchecked. Rendering thousands of manifests on every reconcile gets slow. Splitting repositories by team or environment, or using targeted reconciliation, keeps performance sane.
What actually works is boring: small, reviewable changes; strict branch protection; automated policy checks before merge; and a clear rule that the repository is the only path to production. Teams that follow those habits get the promised benefits. Teams that treat GitOps as a tool install rather than an operating discipline usually end up with a complicated way to deploy the same way they always did.
Security and Auditability Benefits
The security argument for GitOps is straightforward and worth stating plainly. Because the agent pulls from inside the cluster, the CI system never holds production credentials, which shrinks the blast radius of a compromised pipeline. Every change is a reviewed commit with an author, a timestamp, and a diff. Rollback becomes a revert rather than a frantic scramble.
Auditability follows from the same property. When an auditor asks who changed a production setting and when, the answer is a commit hash, not a support ticket. That alone justifies the model for many regulated organizations.
Limitations Worth Knowing Before You Commit
GitOps is not a universal answer. Secrets management requires extra tooling. Stateful operations like database migrations sit awkwardly outside the declarative model. Very large repositories strain rendering performance. And the model assumes a Kubernetes-like system with a strong declarative API; it works elsewhere in principle, but most tooling and practice assumes Kubernetes.
It is also worth being precise about standards. The OpenGitOps principles are a community specification, not a formal ISO or IEEE standard. Anyone claiming otherwise is overselling.
Conclusion
GitOps architecture is less a product than a discipline: describe what you want, store it in Git, and let an agent make it true, continuously. The pull-based model, the reconciliation loop, and the two reference implementations form a coherent system that improves security, auditability, and rollback speed. It demands real commitment, especially around secrets and pipeline design, and it does not erase the need for judgment about stateful operations. But for teams running declarative infrastructure at scale, it remains one of the most practical operating models available. Start small, protect your main branch, keep secrets out of the repository, and let the loop do its quiet work.
Frequently Asked Questions
What is the difference between GitOps and DevOps?
DevOps is a broad cultural and organizational movement about collaboration between development and operations. GitOps is a specific technical operating model within that movement, where Git holds the desired state and automated agents reconcile the live system to match it. You can practice DevOps without GitOps, and GitOps is one way to implement DevOps principles. (GitOps)
Is GitOps only for Kubernetes?
In practice, yes, most GitOps tooling assumes Kubernetes because its declarative API and controllers make reconciliation natural. In principle the model could apply to any system with a declarative interface, but the mature tools, community knowledge, and reference implementations are overwhelmingly Kubernetes-focused.
Do I need Argo CD or Flux to do GitOps?
No, but they are the two most widely used reference implementations, and both are CNCF projects. You could build a custom reconciler, and some large organizations do, but you would be rebuilding solved problems around rendering, drift detection, and status reporting. For most teams, adopting an existing tool is the faster and safer path.
How does GitOps handle secrets?
Not natively, which is why secrets management is the most common early obstacle. Teams typically use external secret stores, sealed secret tooling, or encryption at rest so the repository stays safe to share. The key rule is simple: never commit plain-text credentials, and decide your secrets strategy before migrating workloads.
What happens if someone changes the cluster manually?
The agent detects the difference between live state and desired state, labels it drift, and typically reverts it automatically if auto-sync is enabled. This is a feature, not a bug, but it surprises people who are used to hotfixing directly. If you need a manual change, make it through the repository.
Does GitOps work for database migrations?
Poorly, in its pure form. Database migrations are imperative and stateful, which sits awkwardly against a declarative reconciliation model. Most mature teams run migrations as a separate, gated step in the pipeline rather than treating them as ordinary declarative resources, and they plan rollback carefully.
