Maaz Zindani

Loading the index…

↑↓ move↵ openesc close4072 pages
Servers, nginx and Docker

ingress-nginx controller stops working after upgrading Kubernetes to 1.22 or later

ErrorIngress manifests using extensions/v1beta1 are rejected after a Kubernetes upgrade

Short answer

Kubernetes 1.22 stopped serving the extensions/v1beta1 and networking.k8s.io/v1beta1 Ingress APIs. An old controller release or old manifests that still use them break on upgrade. Move manifests to networking.k8s.io/v1, then upgrade the controller to a release that supports your cluster version.

  • Applies to Kubernetes 1.22+, ingress-nginx Helm chart, Azure AKS
  • Updated
  • By Maaz Zindani, Senior Software Engineer

Fix it fast

  1. 1Find out which cause you have

    Check the controller's logs and the image version, then list which Ingress API versions your manifests use.

    kubectl -n ingress-nginx get pods
    kubectl -n ingress-nginx logs deploy/ingress-nginx-controller --tail=50
    grep -rn "apiVersion: .*v1beta1" .
  2. 2Match what you see, then jump to its fix

    1. 1kubectl apply or helm fails on kind Ingress with extensions/v1beta1 or networking.k8s.io/v1beta1The manifests still use the removed Ingress API versionsFix
    2. 2The controller pod crashes or does not become ready right after the control plane upgradeThe controller release is older than the cluster supportsFix

What does the error look like?

  • Ingress manifests using extensions/v1beta1 are rejected after a Kubernetes upgrade
  • ingress-nginx controller does not start after an AKS upgrade to 1.22

Cause 1 of 2

The manifests still use the removed Ingress API versions

1Check

The Kubernetes deprecation guide lists both versions as no longer served from 1.22. The replacement networking.k8s.io/v1 has been available since 1.19.

grep -rn "extensions/v1beta1\|networking.k8s.io/v1beta1" .

2Fix

Migrate each Ingress: pathType is now required, backend becomes defaultBackend, serviceName and servicePort become service.name and service.port.number, and ingressClassName replaces the old class annotation.

apiVersion: networking.k8s.io/v1
kind: Ingress
metadata:
  name: example
spec:
  ingressClassName: nginx
  rules:
  - host: foo.example.com
    http:
      paths:
      - path: /foo
        pathType: Prefix
        backend:
          service:
            name: foo
            port:
              number: 80

3Verify

Apply the manifest and read it back from the cluster.

kubectl apply -f ingress.yaml
kubectl get ingress

Confirmed by kubernetes.io

Did this fix it?

Cause 2 of 2

The controller release is older than the cluster supports

1Check

The ingress-nginx project publishes a table of which controller versions support which Kubernetes versions, with the matching Helm chart. Compare it with what you run.

kubectl -n ingress-nginx get deploy ingress-nginx-controller -o jsonpath='{.spec.template.spec.containers[0].image}'
kubectl version
helm list -n ingress-nginx

2Fix

Upgrade the Helm release to a chart whose controller version supports your Kubernetes version. Note that the project's README says it is being retired, so plan a move to a maintained ingress controller too.

helm repo update
helm upgrade ingress-nginx ingress-nginx/ingress-nginx -n ingress-nginx --reuse-values

3Verify

The controller pod is Running and ready.

kubectl -n ingress-nginx get pods

Confirmed by github.com

Did this fix it?

How do I stop it coming back?

  • Read the Kubernetes deprecation guide before every minor upgrade and scan manifests for removed API versions.
  • Upgrade the ingress controller before the control plane when its table says the new version needs it.
  • Keep Ingress manifests in Git so they can be searched and migrated in one change.

Sources

How was this page made?

I drafted this page with AI assistance from the official documents listed under Sources. Every cause above names the document that confirms it. I have not reproduced this fix on a machine of my own yet, so read it as documented guidance, not a test report. Found a mistake? Tell me.

Which stack is behind this page?

Projects of mine built on the same technologies:

Related fixes

Next step

Want this handled for good?

I look after servers, apps and releases end to end. Tell me what keeps breaking.