2

Currently able to resolve all services to IP addresses and telnet and ping them. Unable to resolve pods to IP addresses. Though can lookup pod IP addresses with kubectl and telnet and ping them. How do I need DNS configured to resolve pods in Kubernetes 1.9.6, dns-controller:1.9.1.

3
3

@mk_sta 's answer a bit easier to work with you can run this one liner to test your DNS:

kubectl run busybox --image=busybox --rm --attach --command -- sh -c "cat /etc/resolv.conf; nslookup $POD.$NAMESPACE.pod.cluster.local"

Example output:

If you don't see a command prompt, try pressing enter.

Server:         100.64.0.10
Address:        100.64.0.10:53


*** Can't find $POD.$NAMESPACE.pod.cluster.local: No answer

deployment.apps "busybox" deleted

Bonus is that it deletes the deployment/pods after running.


For me the problem was that my pods are in a statefulset and therefore Pod DNS resolution is a bit different. You have to use (for example):

web-{0..N-1}.nginx.default.svc.cluster.local

pod-N.$GOVERNING_STATEFULSET.$NAMESPACE.svc.cluster.local

https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/

That's one thing that can go wrong. You may want to provide more info about your particular case.

0

You can create test Pod with busybox in order to check DNS resolving function and debug the issue:

apiVersion: v1
kind: Pod
metadata:
  name: busybox
  namespace: default
spec:
  containers:
  - image: busybox
    command:
      - sleep
      - "3600"
    imagePullPolicy: IfNotPresent
    name: busybox
  restartPolicy: Always

Check /etc/resolv.conf file on the busybox Pod:

kubectl exec busybox cat /etc/resolv.conf

The content should be like this:

search default.svc.cluster.local svc.cluster.local cluster.local .....
nameserver 10.0.0.10
options ndots:5

Make a lookup request to the target Pod in your cluster:

kubectl exec -ti busybox -- nslookup XXX-XX-XX-XX.default.pod.cluster.local

In case you have not been resolved Pods DNS name, you can check DNS components logs withkubectl logs command:

kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c kubedns
kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c dnsmasq
kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c healthz
3
  • The bottom 3 commands exit with an error: kubectl logs --namespace=kube-system $(kubectl get pods --namespace=kube-system -l k8s-app=kube-dns -o name) -c kubedns error: only one of -c or an inline [CONTAINER] arg is allowed See 'kubectl logs -h' for help and examples. – the_frank Sep 6 '18 at 14:46
  • Have you used kube-dns as DNS resolving service in your cluster? – Nick_Kh Sep 14 '18 at 9:40
  • kube-dns is the current DNS resolver in the cluster – the_frank Sep 20 '18 at 15:39
0

The best way that I have seen to resolve this is by creating a Jump Pod.

Step 1. Create the pod with a spec file.

cat << ENDL >> jumpod.yml
apiVersion: v1
kind: Pod
metadata:
  name: jumpod
  namespace: default
spec:
  containers:
  - name: busybox
    image: busybox:1.28
    command:
      - sleep
      - "3600"
    imagePullPolicy: IfNotPresent
  restartPolicy: Always
ENDL

Step 2. Ether apply or create the pod into the cluster and on to a node. Note that it will be located at the default namespace.

kubectl apply -f jumpod.yml

Now you can use this pod to either get the DNS of a Pod or Service.

For Example:

Pod nslookup Step 3. Get the ip address of the pod you are looking for.

kubectl get pods -o wide

Step 4. Use the Jump Pod to do an nslookup

kubectl exec -it jumpod ping 10.244.0.149

Service nslookup Step 5. Lookup the services ip address that you may want to use.

kubectl get services --all-namespaces

Step 6. Now use the jump pod to do an nslookup on the service ip address

kubectl exec -it jumpod nslookup 10.245.150.103

Below is an example on how the main cluster using nslookup will not work and how the busybox (aka jump pod) will work.

$ kubectl exec -it jumpod nslookup 10.245.150.103
Server:    10.245.0.10
Address 1: 10.245.0.10 kube-dns.kube-system.svc.cluster.local
Name:      10.245.150.103
Address 1: 10.245.150.103 hello.develop.svc.cluster.local
$ nslookup 10.245.150.103
Server:     8.8.8.8
Address:    8.8.8.8#53
** server can't find 103.150.245.10.in-addr.arpa: NXDOMAIN

Hope this helps!

P.S If by chance you need to export the DNS file you created within the pod you can use this.

kubectl cp default/POD_NAME:bin/FILE_NAME /Users/username/FILE_NAME

or just use > to export it to a file.

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.