The tenants of immutable infrastructure strongly discourage accessing containers. For day-to-day operation and deployment activities, that's a sage bit of wisdom. When things start breaking, however, you'll need to get in and take a look around. Kubernetes gives us two ways of doing this:
When you run
kubectl exec, you are creating a brand new process inside of the existing container. This is often used to get an interactive shell (assuming the container image has one to execute):
$ kubectl exec -it pod/foo /bin/bash root@foo:/# root@foo:/# ps -ef UID PID PPID C STIME TTY TIME CMD root 1 0 0 15:25 ? 00:00:01 /bin/bash /opt/run root 5472 0 0 19:58 pts/0 00:00:00 /bin/bash root 5488 1 0 19:59 ? 00:00:00 sleep 6 root 5489 5472 0 19:59 pts/0 00:00:00 ps -ef
This is extra handy for OS-based images (where you can run a shell), but if your image contains any troubleshooting utilities, you can execute them.
Attaching to a pod is a different matter entirely. Rather than spin up a brand new process,
kubectl attach hooks up standard input (file descriptor 0), output (fd 1), and error (fd 2) to that of your terminal, letting you view the output of the running process. If it's an interactive container, you can also send input keystrokes.
This file defines a pod with an interactive container that asks for a name, and then prints a custom message in response. With that running, we can attach to it and interact:
$ kubectl attach -i attach-to-me Defaulting container name to main. Use 'kubectl describe pod/attach-to-me -n trick11' to see all of the containers in this pod. If you don't see a command prompt, try pressing enter. Hello, welcome to the Pod! What is your name? James Pleased to meet you, James ^C
Note: if you have multiple processes running in a single container, you're only going to be able to attach to the init process, PID #1. All the more reason to break up multi-process containers.