Question
I am deploying the nginx based ingress controller on Kubernetes cluster managed by RKE. ( I have also tried the same directly without RKE ).
In both the cases , it tries to use/bind to Ports 80
, and 443
on the
host, and it fails because in the pod security policy
for all service
accounts I am not allowing host ports.
In fact I don't need to access the ingress directly on the hosts, but I want
to access the ingress controller
as a Service
on the NodePort
from
external LoadBalancer
.
Is there way to deploy Nginx ingress controller
not to use any hostPort.
Answer
Done by disabling hostNetwork , and remove unnecessary privileges and capabilities:
C02W84XMHTD5:Downloads iahmad$ kubectl get deployments -n ingress-nginx -o yaml
apiVersion: v1
items:
- apiVersion: extensions/v1beta1
kind: Deployment
metadata:
annotations:
deployment.kubernetes.io/revision: "1"
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
name: nginx-ingress-controller
namespace: ingress-nginx
resourceVersion: "68427"
selfLink: /apis/extensions/v1beta1/namespaces/ingress-nginx/deployments/nginx-ingress-controller
uid: 0b92b556-12fa-11ea-9d82-08002762a3c5
spec:
progressDeadlineSeconds: 600
replicas: 1
revisionHistoryLimit: 10
selector:
matchLabels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
strategy:
rollingUpdate:
maxSurge: 25%
maxUnavailable: 25%
type: RollingUpdate
template:
metadata:
annotations:
prometheus.io/port: "10254"
prometheus.io/scrape: "true"
creationTimestamp: null
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
spec:
containers:
- args:
- /nginx-ingress-controller
- --configmap=$(POD_NAMESPACE)/nginx-configuration
- --tcp-services-configmap=$(POD_NAMESPACE)/tcp-services
- --udp-services-configmap=$(POD_NAMESPACE)/udp-services
- --publish-service=$(POD_NAMESPACE)/ingress-nginx
- --annotations-prefix=nginx.ingress.kubernetes.io
env:
- name: POD_NAME
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.name
- name: POD_NAMESPACE
valueFrom:
fieldRef:
apiVersion: v1
fieldPath: metadata.namespace
image: quay.io/kubernetes-ingress-controller/nginx-ingress-controller:0.26.1
imagePullPolicy: IfNotPresent
lifecycle:
preStop:
exec:
command:
- /wait-shutdown
livenessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
initialDelaySeconds: 10
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
name: nginx-ingress-controller
ports:
- containerPort: 80
name: http
protocol: TCP
- containerPort: 443
name: https
protocol: TCP
readinessProbe:
failureThreshold: 3
httpGet:
path: /healthz
port: 10254
scheme: HTTP
periodSeconds: 10
successThreshold: 1
timeoutSeconds: 10
resources: {}
securityContext:
runAsUser: 33
terminationMessagePath: /dev/termination-log
terminationMessagePolicy: File
dnsPolicy: ClusterFirst
restartPolicy: Always
schedulerName: default-scheduler
securityContext: {}
serviceAccount: nginx-ingress-serviceaccount
serviceAccountName: nginx-ingress-serviceaccount
terminationGracePeriodSeconds: 300
status:
availableReplicas: 1
conditions:
- lastTransitionTime: 2019-11-29T22:46:59Z
lastUpdateTime: 2019-11-29T22:46:59Z
message: Deployment has minimum availability.
reason: MinimumReplicasAvailable
status: "True"
type: Available
- lastTransitionTime: 2019-11-29T22:46:13Z
lastUpdateTime: 2019-11-29T22:46:59Z
message: ReplicaSet "nginx-ingress-controller-84758fb96c" has successfully progressed.
reason: NewReplicaSetAvailable
status: "True"
type: Progressing
observedGeneration: 1
readyReplicas: 1
replicas: 1
updatedReplicas: 1
kind: List
metadata:
resourceVersion: ""
selfLink: ""
and then creating a nodeport service pointing to the ingress controller ports:
C02W84XMHTD5:Downloads iahmad$ kubectl get svc -n ingress-nginx -o yaml
apiVersion: v1
items:
- apiVersion: v1
kind: Service
metadata:
labels:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
name: ingress-nginx
namespace: ingress-nginx
resourceVersion: "68063"
selfLink: /api/v1/namespaces/ingress-nginx/services/ingress-nginx
uid: 7aa425a4-12f9-11ea-9d82-08002762a3c5
spec:
clusterIP: 10.97.110.93
externalTrafficPolicy: Cluster
ports:
- name: http
nodePort: 30864
port: 80
protocol: TCP
targetPort: 80
- name: https
nodePort: 30716
port: 443
protocol: TCP
targetPort: 443
selector:
app.kubernetes.io/name: ingress-nginx
app.kubernetes.io/part-of: ingress-nginx
sessionAffinity: None
type: NodePort
status:
loadBalancer: {}
kind: List
metadata:
resourceVersion: ""
selfLink: ""
C02W84XMHTD5:Downloads iahmad$