↓Skip to main content

The New Adventures in Kubernetes - Beyond `HELLO WORLD` - 2020-12-09

·4 mins
code

The New Adventures in Kubernetes #

Beyond HELLO WORLD #

2020-12-09 #

Notes: Today, I plan to fix the issues I was having with finalizing cert-manager installation. A quick aside, to scale the deployments that are already running use a command like: kubectl -n tests scale deployments nginx-hello-world-test --replicas 10

  • install cert-manager to namespace kube-tools
  • secure current ingress endpoints behind TLS
  • install nextcloud to namespace devops

Fix Cert-Manager installation #

  • Change the Issuer to a Cluster-Issuer without a namespace
apiVersion: cert-manager.io/v1
kind: ClusterIssuer
metadata:
  name: mkcert-ca-issuer
spec:
  ca:
    secretName: mkcert-ca-key-pair
  • Apply the changes
kubectl apply -f issuer-mkcert.yaml
  • Check to be sure the issuer is working correctly
kubectl get clusterissuers mkcert-ca-issuer -o wide
  • Delete the unused issuer
kubectl delete issuers mkcert-ca-issuer -n kube-tools
  • Add a test certificate to ingress for devops-landing-page. In devops-landing-page-ingress.yaml:
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: devops-landing-page-ingress
  namespace: devops
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    cert-manager.io/cluster-issuer: mkcert-ca-issuer # change to cluster-issuer
spec:
  rules:
    - host: minikube.localhost
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: devops-landing-np
                port:
                  number: 80
  tls:
  - hosts:
    - minikube.localhost
    secretName: devops-landing-page-cert

Notes: The certificate is now being correctly issued by mkcert.

  • Update Grafana Ingress grafana-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: grafana-ingress
  namespace: kube-tools
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    cert-manager.io/cluster-issuer: mkcert-ca-issuer
spec:
  rules:
    - host: grafana.minikube.localhost
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: grafana-np
                port:
                  number: 3000
  tls:
  - hosts:
    - grafana.minikube.localhost
    secretName: grafana-cert
  • Apply the changes
kubectl apply -f grafana-ingress.yaml
  • Update Hello World Ingress nginx-hello-world-test-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-hello-world-test-ingress
  namespace: tests
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$1
    cert-manager.io/cluster-issuer: mkcert-ca-issuer
spec:
  rules:
    - host: hello-world.minikube.localhost
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: nginx-hello-world-test
                port:
                  number: 80
  tls:
  - hosts:
    - hello-world.minikube.localhost
    secretName: hello-world-cert
  • Apply the changes
kubectl apply -f nginx-hello-world-test-ingress.yaml
  • Update Hello World 2 Ingress nginx-hello-world2-test-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: nginx-hello-world2-test-ingress
  namespace: tests
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /$1
    cert-manager.io/cluster-issuer: mkcert-ca-issuer
spec:
  rules:
    - host: hello-world2.minikube.localhost
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: nginx-hello-world-test2
                port:
                  number: 80
  tls:
  - hosts:
    - hello-world2.minikube.localhost
    secretName: hello-world2-cert
  • Apply the changes
kubectl apply -f nginx-hello-world-test2-ingress.yaml
  • Update Portainer Ingress portainer-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: portainer-ingress
  namespace: devops
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    cert-manager.io/cluster-issuer: mkcert-ca-issuer
spec:
  rules:
    - host: portainer.minikube.localhost
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: portainer
                port:
                  number: 9000
  tls:
  - hosts:
    - portainer.minikube.localhost
    secretName: portainer-cert
  • Apply the changes
kubectl apply -f portainer-ingress.yaml
  • Update Prometheus Ingress prometheus-ingress.yaml
apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: prometheus-ingress
  namespace: kube-tools
  annotations:
    nginx.ingress.kubernetes.io/rewrite-target: /
    cert-manager.io/cluster-issuer: mkcert-ca-issuer
spec:
  rules:
    - host: prometheus.minikube.localhost
      http:
        paths:
          - path: /
            pathType: Prefix
            backend:
              service:
                name: prometheus-server-np
                port:
                  number: 9090
  tls:
  - hosts:
    - prometheus.minikube.localhost
    secretName: prometheus-cert
  • Apply the changes
kubectl apply -f prometheus-ingress.yaml

Notes: The last ingress I need to secure behind https is Kubeview, which was installed a little differently, using Helm, so the ingress file had to be copied from Lens.

  • 108

  • Update Kubeview Ingress kubeview-ingress.yaml

kind: Ingress
apiVersion: networking.k8s.io/v1
metadata:
  name: kubeview-ingress
  namespace: kube-tools
  selfLink: /apis/networking.k8s.io/v1/namespaces/kube-tools/ingresses/kubeview-ingress
  uid: 93b28ac4-ec4c-454b-acc5-d69b2c31bee8
  resourceVersion: '64021'
  generation: 1
  creationTimestamp: '2020-12-04T19:26:06Z'
  labels:
    app.kubernetes.io/instance: kubeview
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/name: kubeview
    app.kubernetes.io/version: 0.1.17
    helm.sh/chart: kubeview-0.1.17
  annotations:
    kubectl.kubernetes.io/last-applied-configuration: >
            {"apiVersion":"extensions/v1beta1","kind":"Ingress","metadata":{"annotations":{"kubernetes.io/ingress.class":"nginx","meta.helm.sh/release-name":"kubeview","meta.helm.sh/release-namespace":"kube-tools"},"creationTimestamp":"2020-12-04T17:19:37Z","generation":1,"labels":{"app.kubernetes.io/instance":"kubeview","app.kubernetes.io/managed-by":"Helm","app.kubernetes.io/name":"kubeview","app.kubernetes.io/version":"0.1.17","helm.sh/chart":"kubeview-0.1.17"},"managedFields":[{"apiVersion":"extensions/v1beta1","fieldsType":"FieldsV1","fieldsV1":{"f:metadata":{"f:annotations":{".":{},"f:kubernetes.io/ingress.class":{},"f:meta.helm.sh/release-name":{},"f:meta.helm.sh/release-namespace":{}},"f:labels":{".":{},"f:app.kubernetes.io/instance":{},"f:app.kubernetes.io/managed-by":{},"f:app.kubernetes.io/name":{},"f:app.kubernetes.io/version":{},"f:helm.sh/chart":{}}},"f:spec":{"f:rules":{}}},"manager":"Go-http-client","operation":"Update","time":"2020-12-04T17:19:37Z"},{"apiVersion":"networking.k8s.io/v1beta1","fieldsType":"FieldsV1","fieldsV1":{"f:status":{"f:loadBalancer":{"f:ingress":{}}}},"manager":"nginx-ingress-controller","operation":"Update","time":"2020-12-04T17:19:46Z"}],"name":"kubeview-ingress","namespace":"kube-tools","selfLink":"/apis/extensions/v1beta1/namespaces/kube-tools/ingresses/kubeview","uid":"2bb1a3ae-3122-448c-9cc7-25abf55e1870"},"spec":{"rules":[{"host":"kubeview.minikube.localhost","http":{"paths":[{"backend":{"serviceName":"kubeview","servicePort":80},"path":"/","pathType":"ImplementationSpecific"}]}}]}}
    kubernetes.io/ingress.class: nginx
    cert-manager.io/cluster-issuer: mkcert-ca-issuer
    meta.helm.sh/release-name: kubeview
    meta.helm.sh/release-namespace: kube-tools
  managedFields:
    - manager: kubectl-client-side-apply
      operation: Update
      apiVersion: extensions/v1beta1
      time: '2020-12-04T19:26:06Z'
      fieldsType: FieldsV1
      fieldsV1:
        'f:metadata':
          'f:annotations':
            .: {}
            'f:kubectl.kubernetes.io/last-applied-configuration': {}
            'f:kubernetes.io/ingress.class': {}
            'f:meta.helm.sh/release-name': {}
            'f:meta.helm.sh/release-namespace': {}
          'f:labels':
            .: {}
            'f:app.kubernetes.io/instance': {}
            'f:app.kubernetes.io/managed-by': {}
            'f:app.kubernetes.io/name': {}
            'f:app.kubernetes.io/version': {}
            'f:helm.sh/chart': {}
        'f:spec':
          'f:rules': {}
    - manager: nginx-ingress-controller
      operation: Update
      apiVersion: networking.k8s.io/v1beta1
      time: '2020-12-08T03:28:14Z'
      fieldsType: FieldsV1
      fieldsV1:
        'f:status':
          'f:loadBalancer':
            'f:ingress': {}
spec:
  rules:
    - host: kubeview.minikube.localhost
      http:
        paths:
          - path: /
            pathType: ImplementationSpecific
            backend:
              service:
                name: kubeview
                port:
                  number: 80
  tls:
  - hosts:
    - kubeview.minikube.localhost
    secretName: kubeview-cert
status:
  loadBalancer:
    ingress:
      - ip: 192.168.99.100
  • Apply the changes
kubectl apply -f kubeview-ingress.yaml

Install NextCloud to devops namespace #

  • Add the Helm repository
helm repo add nextcloud https://nextcloud.github.io/helm/ && helm repo update
  • Create a values.yaml for installation configuration
ingress:
  enabled: true
  annotations: 
   nginx.ingress.kubernetes.io/proxy-body-size: 4G
   cert-manager.io/cluster-issuer: mkcert-ca-issuer
   nginx.ingress.kubernetes.io/server-snippet: |-
     server_tokens off;
     proxy_hide_header X-Powered-By;

     rewrite ^/.well-known/webfinger /public.php?service=webfinger last;
     rewrite ^/.well-known/host-meta /public.php?service=host-meta last;
     rewrite ^/.well-known/host-meta.json /public.php?service=host-meta-json;
     location = /.well-known/carddav {
       return 301 $scheme://$host/remote.php/dav;
     }
     location = /.well-known/caldav {
       return 301 $scheme://$host/remote.php/dav;
     }
     location = /robots.txt {
       allow all;
       log_not_found off;
       access_log off;
     }
     location ~ ^/(?:build|tests|config|lib|3rdparty|templates|data)/ {
       deny all;
     }
     location ~ ^/(?:autotest|occ|issue|indie|db_|console) {
       deny all;
     }     
  tls:
  - hosts:
    - nextcloud.minikube.localhost
    secretName: nextcloud-cert
  
nextcloud:
  host: nextcloud.minikube.localhost
  username: admin
  password: PRoject2501

metrics:
  enabled: true

  replicaCount: 1
  # The metrics exporter needs to know how you serve Nextcloud either http or https
  https: true
  timeout: 5s

Notes: I had a hell of a time trying to get the values.yaml to parse correctly when it came to the ingress.annotations field. I had to run helm --debug install nextcloud --namespace devops -f values.yaml nextcloud/nextcloud and helm --debug delete nextcloud --namespace devops over and over again, commenting and un-commenting lines at a time. I finally got it working with what is recorded above.

  • Install NextCloud
helm install \
  nextcloud \
  --namespace devops \
  -f values.yaml \
  nextcloud/nextcloud
  • Add an entry to /etc/hosts
192.168.99.100 nextcloud.minikube.localhost
↑