Argo CD · Software Engineering Wiki

Cheatsheet #

Task	Command
Application state	`argocd app get myapp`
Live vs desired	`argocd app diff myapp`
Sync now	`argocd app sync myapp`
Sync one resource	`argocd app sync myapp --resource apps:Deployment:api`
Force re-read of Git	`argocd app get myapp --hard-refresh`
Watch until healthy	`argocd app wait myapp --health --timeout 600`
Roll back	`argocd app rollback myapp <id>`
History	`argocd app history myapp`
Controller’s view of a resource	`argocd app manifests myapp --source live`
Prune removed objects	`argocd app sync myapp --prune`
Application YAML	`kubectl get app myapp -n argocd -o yaml`
Controller logs	`kubectl logs -n argocd deploy/argocd-application-controller`

An Application that will not sync #

Argo CD reconciles three states: what Git says (desired), what the cluster has (live), and what it last applied. OutOfSync compares desired with live; Degraded is a health assessment of live objects only.

argocd app get myapp
argocd app diff myapp                     # exact fields that differ
argocd app get myapp --hard-refresh       # bypass the manifest cache
kubectl get app myapp -n argocd -o jsonpath='{.status.conditions}' | jq
kubectl logs -n argocd deploy/argocd-repo-server --tail 100

Symptom	Cause
`Unknown` health, no resources	Repo server cannot render: bad path, missing values file, private repo credentials
`OutOfSync` immediately after sync	A controller or webhook mutates the object; needs `ignoreDifferences`
Sync succeeds, app still `Progressing`	Health check waiting on readiness, or a custom health script
`ComparisonError`	Rendering failed — read the condition message, it quotes the tool’s stderr
Resources reappear after deletion	Deleted by hand, restored by reconciliation; delete from Git instead
Nothing happens on push	Webhook not configured; polling interval defaults to 3 minutes

Application spec #

apiVersion: argoproj.io/v1alpha1
kind: Application
metadata:
  name: myapp
  namespace: argocd
  finalizers:
    - resources-finalizer.argocd.argoproj.io      # cascade delete children on app delete
spec:
  project: platform
  source:
    repoURL: https://github.com/example/infra.git
    targetRevision: main                           # branch, tag or commit SHA
    path: clusters/prod/myapp
    helm:
      valueFiles: [values.yaml, values.prod.yaml]
      parameters:
        - { name: image.tag, value: "1.4.2" }
  destination:
    server: https://kubernetes.default.svc
    namespace: myns
  syncPolicy:
    automated:
      prune: true                                  # delete objects removed from Git
      selfHeal: true                               # revert manual changes to the cluster
      allowEmpty: false
    syncOptions:
      - CreateNamespace=true
      - ServerSideApply=true
      - PruneLast=true
    retry:
      limit: 5
      backoff: { duration: 15s, factor: 2, maxDuration: 5m }
  revisionHistoryLimit: 10

Pin targetRevision to a tag or SHA for production. main means the next merge deploys itself, which is either the point of GitOps or an incident, depending on the repository.

selfHeal: true reverts manual kubectl edit within seconds — useful to know before spending an afternoon debugging why a change keeps vanishing.

Diffing and drift #

Argo CD diffs the rendered manifests against live objects after normalisation. Fields written by other controllers show as permanent drift unless excluded.

spec:
  ignoreDifferences:
    - group: apps
      kind: Deployment
      jsonPointers: ["/spec/replicas"]             # HPA owns this
    - group: ""
      kind: Secret
      name: db
      jqPathExpressions: ['.data["ca.crt"]']

ServerSideApply=true hands conflict resolution to the API server’s field ownership, which is the cleaner fix when several controllers legitimately write to one object.

argocd app diff myapp --local ./clusters/prod/myapp   # compare a working copy against live
argocd app manifests myapp --source git               # what Argo rendered
argocd app manifests myapp --source live              # what exists now

App of apps and ApplicationSets #

An Application whose source directory contains more Applications bootstraps a whole cluster from one object. An ApplicationSet generates Applications from a template plus a generator, which avoids writing one file per cluster or per team.

apiVersion: argoproj.io/v1alpha1
kind: ApplicationSet
metadata: { name: tenants, namespace: argocd }
spec:
  goTemplate: true
  generators:
    - git:
        repoURL: https://github.com/example/infra.git
        revision: main
        directories: [{ path: tenants/* }]
  template:
    metadata: { name: '{{.path.basename}}' }
    spec:
      project: tenants
      source: { repoURL: https://github.com/example/infra.git, targetRevision: main, path: '{{.path.path}}' }
      destination: { server: https://kubernetes.default.svc, namespace: '{{.path.basename}}' }
      syncPolicy: { automated: { prune: true, selfHeal: true } }

Generators include git (directories or files), cluster, list, matrix and pullRequest. Set applicationsSync: create-update on the ApplicationSet if deletions should not cascade while the pattern is being trialled.

Sync waves and hooks #

Within a sync, Argo CD orders resources by wave (ascending), then by kind, then by name. Each wave completes and reports healthy before the next begins.

metadata:
  annotations:
    argocd.argoproj.io/sync-wave: "-1"             # CRDs and namespaces first

metadata:
  annotations:
    argocd.argoproj.io/hook: PreSync               # PreSync, Sync, PostSync, SyncFail
    argocd.argoproj.io/hook-delete-policy: HookSucceeded

A PreSync Job that never completes blocks the sync indefinitely; give hook Jobs backoffLimit and activeDeadlineSeconds.

Projects and access #

An AppProject restricts which repositories, destinations and resource kinds its Applications may use. It is the only boundary that stops a team’s Application from creating a ClusterRoleBinding.

apiVersion: argoproj.io/v1alpha1
kind: AppProject
metadata: { name: tenants, namespace: argocd }
spec:
  sourceRepos: ["https://github.com/example/infra.git"]
  destinations:
    - { server: https://kubernetes.default.svc, namespace: "tenant-*" }
  clusterResourceWhitelist: []                      # no cluster-scoped objects
  namespaceResourceBlacklist:
    - { group: "", kind: ResourceQuota }

Health and readiness #

Built-in health checks cover Deployments, StatefulSets, Services, Ingresses and more. Custom resources are Healthy by default unless a Lua health script exists for them, so a broken CR can leave an Application green.

# argocd-cm ConfigMap
resource.customizations.health.example.com_Database: |
  hs = {}
  if obj.status ~= nil and obj.status.phase == "Ready" then
    hs.status = "Healthy"
  else
    hs.status = "Progressing"
  end
  return hs

Oneliners #

# Every application not synced or not healthy
argocd app list -o json | jq -r '.[] | select(.status.sync.status!="Synced" or .status.health.status!="Healthy") | [.metadata.name, .status.sync.status, .status.health.status] | @tsv'

# The same without the CLI
kubectl get app -n argocd -o custom-columns='NAME:.metadata.name,SYNC:.status.sync.status,HEALTH:.status.health.status'

# Which commit each app is running
kubectl get app -n argocd -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{.status.sync.revision}{"\n"}{end}'

# Sync everything in a project
argocd app list -p platform -o name | xargs -n1 -P4 argocd app sync

# Applications whose auto-sync is off
kubectl get app -n argocd -o json | jq -r '.items[] | select(.spec.syncPolicy.automated==null) | .metadata.name'

# Force refresh every app after a repo credential change
kubectl get app -n argocd -o name | xargs -n1 -I{} kubectl patch {} -n argocd --type merge -p '{"metadata":{"annotations":{"argocd.argoproj.io/refresh":"hard"}}}'

# Watch what the controller is doing
kubectl logs -n argocd deploy/argocd-application-controller -f | grep -E 'myapp|error'

# Remove a stuck finalizer on a deleted application
kubectl patch app myapp -n argocd --type json -p '[{"op":"remove","path":"/metadata/finalizers"}]'

# Initial admin password
kubectl get secret argocd-initial-admin-secret -n argocd -o jsonpath='{.data.password}' | base64 -d