Aller au contenu principal
TechLedger Registre technique
Sommaire de la formation

Workload cluster avec talos control-plane et proxmox

coolibra565 vues

4. Workload cluster avec talos control-plane et proxmox

  • Définition des CRDs (Cluster, MachineDeployment, etc.)
  • Création d’un cluster de workload
  • Vérification et gestion des ressources

Définition des CRDs Cluster, MachineDeployment

  • Définir le cluster et le control-plane
    • crd ProxmoxCluster
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: ProxmoxCluster
metadata:
  name: k8s-talos-dev-proxmox
  namespace: default
spec:
  schedulerHints:
    memoryAdjustment: 0
  allowedNodes:
  - proxmox-node-01
  controlPlaneEndpoint:
    host: 10.10.10.41
    port: 6443
  dnsServers:
  - 8.8.8.8
  - 8.8.4.4
  ipv4Config:
    addresses:
    - 10.10.10.50-10.10.10.53
    gateway: 10.10.10.1
    prefix: 24

 

    • crd ProxmoxMachineTemplate
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: ProxmoxMachineTemplate
metadata:
  name: k8s-talos-dev-cp
  namespace: default
spec:
  template:
    spec:
      disks:
        bootVolume:
          disk: scsi0
          sizeGb: 6
      format: qcow2
      full: true
      memoryMiB: 2048
      network:
        default:
          bridge: vmbr0
          model: virtio
      numCores: 2
      numSockets: 1
      sourceNode: proxmox-node-01
      templateID: 102
      checks:
        skipCloudInitStatus: true
        skipQemuGuestAgent: false

 

    • crd TalosControlPlane et le lier à ProxmoxMachineTemplate
apiVersion: controlplane.cluster.x-k8s.io/v1alpha3
kind: TalosControlPlane
metadata:
  name: k8s-talos-dev-cp
spec:
  version: v1.30.0
  replicas: 1
  infrastructureTemplate:
    kind: ProxmoxMachineTemplate
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
    name: k8s-talos-dev-cp
    namespace: default
  controlPlaneConfig:
    controlplane:
      generateType: controlplane
      strategicPatches:
        - |

          - op: replace
            path: /machine/install
            value:
              disk: /dev/sda
              extensions:
                - image: ghcr.io/siderolabs/qemu-guest-agent:9.2.0
          - op: add
            path: /machine/install/extraKernelArgs
            value:
              - net.ifnames=0
          - op: add
            path: /machine/network/interfaces
            value:
              - interface: eth0
                dhcp: false
                vip:
                  ip: 10.10.10.41

 

    • crd Cluster et le lier au crd TalosControlPlane et ProxmoxCluster
apiVersion: cluster.x-k8s.io/v1beta1
kind: Cluster
metadata:
  name: k8s-talos-dev-cluster
  namespace: default
spec:
  controlPlaneRef:
    apiVersion: controlplane.cluster.x-k8s.io/v1alpha3
    kind: TalosControlPlane
    name: k8s-talos-dev-cp
  infrastructureRef:
    apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
    kind: ProxmoxCluster
    name: k8s-talos-dev-proxmox

 

  • Définir le nodepool worker et le lier au cluster
    • crd ProxmoxMachineTemplate
apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: ProxmoxMachineTemplate
metadata:
  name: k8s-talos-dev-worker-pmt
  namespace: default
spec:
  template:
    spec:
      disks:
        bootVolume:
          disk: scsi0
          sizeGb: 6
      format: qcow2
      full: true
      memoryMiB: 2048
      network:
        default:
          bridge: vmbr0
          model: virtio
      numCores: 2
      numSockets: 1
      sourceNode: proxmox-node-01
      templateID: 102
      checks:
        skipCloudInitStatus: true

 

    • crd TalosConfigTemplate
---
apiVersion: bootstrap.cluster.x-k8s.io/v1alpha3
kind: TalosConfigTemplate
metadata:
  name: k8s-talos-dev-workers-tct
spec:
  template:
    spec:
      generateType: worker
      talosVersion: v1.9
      configPatches:
          - op: replace
            path: /machine/install
            value:
              disk: /dev/sda

 

    • crd MachineDeployment et le lier aux crd  TalosConfigTemplate et ProxmoxMachineTemplate
apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
  name: k8s-talos-dev-workers-pool
  namespace: default
spec:
  clusterName: k8s-talos-dev-cluster
  replicas: 1
  selector:
    matchLabels: null
  template:
    spec:
      bootstrap:
        configRef:
          apiVersion: bootstrap.cluster.x-k8s.io/v1alpha3
          kind: TalosConfigTemplate
          name: k8s-talos-dev-workers-tct
      clusterName: k8s-talos-dev-cluster
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
        kind: ProxmoxMachineTemplate
        name: k8s-talos-dev-worker-pmt
      version: v1.30.0

 

  • Définir le nodepool monito et le lier au cluster
############## Nodepool monito ############
apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
  name: k8s-talos-dev-monito-pool
  namespace: default
spec:
  clusterName: k8s-talos-dev-cluster
  replicas: 1
  selector:
    matchLabels: null
  template:
    spec:
      bootstrap:
        configRef:
          apiVersion: bootstrap.cluster.x-k8s.io/v1alpha3
          kind: TalosConfigTemplate
          name: k8s-talos-dev-monito-tct
      clusterName: k8s-talos-dev-cluster
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
        kind: ProxmoxMachineTemplate
        name: k8s-talos-dev-monito-pmt
      version: v1.30.0
---

apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
kind: ProxmoxMachineTemplate
metadata:
  name: k8s-talos-dev-monito-pmt
  namespace: default
spec:
  template:
    spec:
      disks:
        bootVolume:
          disk: scsi0
          sizeGb: 6
      format: qcow2
      full: true
      memoryMiB: 2048
      network:
        default:
          bridge: vmbr0
          model: virtio
      numCores: 2
      numSockets: 1
      sourceNode: proxmox-node-01
      templateID: 102
      checks:
        skipCloudInitStatus: true
---
apiVersion: bootstrap.cluster.x-k8s.io/v1alpha3
kind: TalosConfigTemplate
metadata:
  name: k8s-talos-dev-monito-tct
spec:
  template:
    spec:
      generateType: worker
      talosVersion: v1.9
      configPatches:
          - op: replace
            path: /machine/install
            value:
              disk: /dev/sda

 

On peut regrouper tous ces crds dans le fichier k8s-talos-dev-cluster.yaml et l'appliquer sur le management cluster

kubectl --kubeconfig ~/.kube/management-kubeconfig  apply -f 8s-talos-dev-cluster.yaml

 

 

Vérification et gestion des ressources

    • Une fois le cluster est up, on recupère le kube config

Le kubeconfig du workload cluster est énéré dans un secret dans le management cluster. Le secret est nommé nom-cluster-kubeconfig dans notre cas tools-cluster-dev-kubeconfig

kubectl --kubeconfig ~/.kube/management-kubeconfig get secret k8s-talos-dev-cluster-kubeconfig -o jsonpath='{.data.value}' | base64 --decode > k8s-talos-dev-cluster

 

kubectl --kubeconfig k8s-talos-dev-cluster  get no -o wide

 

 

    • Scale up le worker nodepool en 2 replicas

Pour scale up/down un nodepool, on passe tout simplement le replicas du crd MachineDeployment  au nombre souhaité. Ensuite on replique avec kubectl apply.

apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
  name: k8s-talos-dev-workers-pool
  namespace: default
spec:
  clusterName: k8s-talos-dev-cluster
  replicas: 2
  selector:
    matchLabels: null
  template:
    spec:
      bootstrap:
        configRef:
          apiVersion: bootstrap.cluster.x-k8s.io/v1alpha3
          kind: TalosConfigTemplate
          name: k8s-talos-dev-workers-tct
      clusterName: k8s-talos-dev-cluster
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
        kind: ProxmoxMachineTemplate
        name: k8s-talos-dev-worker-pmt
      version: v1.30.0

 

    • Appliquer la nouvelle configuration
kubectl --kubeconfig ~/.kube/management-kubeconfig  apply -f k8s-talos-dev-cluster.yaml

 

Le deuxième worker a été ajouté

 

  • Recuperer le kubeconfig
kubectl --kubeconfig ~/.kube/management-kubeconfig \
get secret tools-flactar-cluster-dev-kubeconfig \
-o jsonpath='{.data.value}' | base64 --decode > tools-flactar-cluster-dev-kubeconfig

 

k9s --kubeconfig .~/.kube/management-kubeconfig

 

 

Pour faire un upgrade, il faut juster changer la version de k8s dans  (version: v1.31.6) dans tous les MachineDeployment et faire un apply.

 

Ajouter des labels

############## Nodepool Workers ############
apiVersion: cluster.x-k8s.io/v1beta1
kind: MachineDeployment
metadata:
  name: k8s-talos-dev-workers-pool
  namespace: default
spec:
  clusterName: k8s-talos-dev-cluster
  replicas: 1
  selector:
    matchLabels: null
  template:
    metadata:
      labels:
        node-role.kubernetes.io/worker: "worker"
        node-role.kubernetes.io/ai: "ai"
    spec:
      bootstrap:
        configRef:
          apiVersion: bootstrap.cluster.x-k8s.io/v1alpha3
          kind: TalosConfigTemplate
          name: k8s-talos-dev-workers-tct
      clusterName: k8s-talos-dev-cluster
      infrastructureRef:
        apiVersion: infrastructure.cluster.x-k8s.io/v1alpha1
        kind: ProxmoxMachineTemplate
        name: k8s-talos-dev-worker-pmt
      version: v1.32.0

Commentaires

Connectez-vous pour rejoindre la discussion.

Aucun commentaire pour le moment.