App of Apps of Infra

Declarative infrastructure (IaC) by any means is necessary for the modern enterprise. A single source of truth in regards to not only applications but infrastructure and configuration is a must. Housing it all in git and adding necessary barriers further allows access control, multi-tenancy, and configuration boundaries via CI/CD and Gitops. Below, we will use Argo's App of Apps pattern to further describe the state of our Digital Ocean cluster using Crossplane. For the sake of brevity, I'll get straight to the needed components.

First lets stand up our bootstrap cluster using Kind

kind create cluster --name bootstrap

Next go ahead and fork, copy, whatever you wish in order to get the necessary files I've shared in this repository https://gitlab.com/jaymiracola/app-of-infra. After you've done so we will install ArgoCD followed by kicking off our app-of-apps pattern by applying the first application which is Argo itself. From there forward it will be managed via git.

helm repo add argo https://argoproj.github.io/argo-helm
helm repo update
helm install argo argo/argo-cd -n argocd --create-namespace --set version=5.51.4

kubectl apply -f applications.yaml

Now ArgoCD and Crossplane have been installed and are being defined entirely from our git repository. It's time to define the infrastructure.

kubectl get applications -A

NAMESPACE   NAME           SYNC STATUS   HEALTH STATUS
argocd      applications   Synced        Healthy
argocd      argocd         Synced        Healthy
argocd      crossplane     Synced        Healthy
argocd      infra          Synced        Healthy

Next, we will define some infrastructure in Digital Ocean. Why Digital Ocean vs AWS, GCP, or Azure? Simple, I use them all the time for low cost services I use personally! Of course, from here you could define whatever you'd like with Crossplane so the cloud and infrastructure is up to you.

Now we need to start defining Providers and Provider Configurations to Crossplane to configure which Cloud Providers we want and the keys to allow Crossplane to configure them on our behalf. I've already added the Digital Ocean Provider so now we need to configure access.

I'll add the following manifest to my /app-of-infra/applications/infra/manifests/ folder as I've already defined and declared it as an application that ArgoCD is tracking. You can get a Digital Ocean token here.

apiVersion: v1
kind: Secret
metadata:
  namespace: crossplane
  name: provider-do-secret
type: Opaque
data:
  token: BASE64ENCODED_PROVIDER_CREDS
---
apiVersion: do.crossplane.io/v1alpha1
kind: ProviderConfig
metadata:
  name: do-config
  namespace: crossplane
spec:
  credentials:
    source: Secret
    secretRef:
      namespace: crossplane
      name: provider-do-secret
      key: token

Important! Do NOT have this in a public repo. There is a reason this part of the instruction is omitted from my demonstration repository.

Last, it's time to declare infrastructure! As an example I will add the following configuration in /app-of-infra/applications/infra/manifests/ to create a Kubernetes cluster in my Digital Ocean account. More examples of different Digital Ocean infrastructure can be found here.

apiVersion: kubernetes.do.crossplane.io/v1alpha1
kind: DOKubernetesCluster
metadata:
  name: do-cluster
  namespace: infra
spec:
  providerConfigRef:
    name: do-config
  forProvider:
    region: nyc3
    version: 1.29.0-do.0
    nodePools:
      - size: s-1vcpu-2gb #lowest tier
        count: 1 #cost cutting for demo
        name: worker-pool
    maintenancePolicy:
      startTime: "03:00"
      day: sunday
    autoUpgrade: true
    surgeUpgrade: false
    highlyAvailable: false

Now you have successfully declared the entirety of your bootstrap cluster all the way into platform infrastructure in a not only declarative but also idempotent pattern! From here you can create more infrastructure as needed. You may also be interested in extending the control plane using Crossplane's Composite Resource Definitions allowing SRE teams to abstract infra with apps (and more) to easily consumable APIs.