Kubernetes does not start after restart system (Ubuntu)

I had a bad exported variable KUBECONFIG which is needed by kubelet (history details are in a comment under question).

To ~/.zprofile I saved KUBECONFIG=$HOME/admin.conf which solved my problem.

After reloading ENV variables is kubelet working:

# kubectl get nodes
NAME      STATUS     AGE       VERSION
master    Ready      5d        v1.6.1
node01    NotReady   5d        v1.6.1

As the comment there, you really need check whether apiserver is started, because kubectl will talk to apiserver. While from you description and version of kubeadm, I believe this is a duplicate question I just answered, so I just copy the answer to here.

---

In current version of kubeadm(v1.6.1), insecure port of ApiServer is abandoned by default, you can verify this by checking api-server yaml file in /etc/kubernetes/manifests/kube-apiserver.yaml, there is kube-apiserver parameter --insecure-port=0.

You can

• Correct this in a running cluster:

  $ mv kube-apiserver.yaml ../kube-apiserver.yaml
  // edit ../kube-apiserver.yaml to remove --insecure-port=0 
  // or change it to --insecure-port=<WHATERER_YOUR_LIKE>
  $ mv ../kube-apiserver.yaml kube-apiserver.yaml
  

• Do it right at startup. You need a kubeadm config file to do this. A simple one would like:

  apiVersion: kubeadm.k8s.io/v1alpha1
  kind: MasterConfiguration
  apiServerExtraArgs:
    insecure-port: 8080 //or whatever you like
  
  // Then you can start a master node use `kubeadm init --config=<this-configure-file-path>`
  

I had the same problem with kubernetes 1.12.3 and ubuntu 16.04.05. I then looked at the kubernetes log by running the command

$ journalctl -u kubelet

and then in the log i saw that k8s was complaining (exiting with status 255) about swap being on.

So i then turned swap off by running

$ swapoff -a

Then i edited fstab and commented out the entry for swap

$ vi /etc/fstab
#comment out line with swap

and then rebooting the system. After the system came back up, i verified that swap was disabled by running

$ free -m

and checking whether the row for swap has 0. It did.

Then i verified that kubeapi service had successfully started by executing

$ systemctl status kubelet

It has successfully started. I verified by also re-checking journalctl logs. Did not see swap error this time.

I verified k8s node status by running

$ kubectl get nodes

which was now working and showing expected output.

NOTE: I had KUBECONFIG set in my .bash_profile file as well, previously.

root@k8s-master:~# cat .bash_profile
export KUBECONFIG="/etc/kubernetes/admin.conf"