126 lines
5.4 KiB
XML
126 lines
5.4 KiB
XML
<chapter xmlns="http://docbook.org/ns/docbook" xmlns:xlink="http://www.w3.org/1999/xlink" xml:id="sec-kubernetes">
|
|
<title>Kubernetes</title>
|
|
<para>
|
|
The NixOS Kubernetes module is a collective term for a handful of
|
|
individual submodules implementing the Kubernetes cluster
|
|
components.
|
|
</para>
|
|
<para>
|
|
There are generally two ways of enabling Kubernetes on NixOS. One
|
|
way is to enable and configure cluster components appropriately by
|
|
hand:
|
|
</para>
|
|
<programlisting language="bash">
|
|
services.kubernetes = {
|
|
apiserver.enable = true;
|
|
controllerManager.enable = true;
|
|
scheduler.enable = true;
|
|
addonManager.enable = true;
|
|
proxy.enable = true;
|
|
flannel.enable = true;
|
|
};
|
|
</programlisting>
|
|
<para>
|
|
Another way is to assign cluster roles ("master" and/or
|
|
"node") to the host. This enables apiserver,
|
|
controllerManager, scheduler, addonManager, kube-proxy and etcd:
|
|
</para>
|
|
<programlisting language="bash">
|
|
services.kubernetes.roles = [ "master" ];
|
|
</programlisting>
|
|
<para>
|
|
While this will enable the kubelet and kube-proxy only:
|
|
</para>
|
|
<programlisting language="bash">
|
|
services.kubernetes.roles = [ "node" ];
|
|
</programlisting>
|
|
<para>
|
|
Assigning both the master and node roles is usable if you want a
|
|
single node Kubernetes cluster for dev or testing purposes:
|
|
</para>
|
|
<programlisting language="bash">
|
|
services.kubernetes.roles = [ "master" "node" ];
|
|
</programlisting>
|
|
<para>
|
|
Note: Assigning either role will also default both
|
|
<xref linkend="opt-services.kubernetes.flannel.enable" /> and
|
|
<xref linkend="opt-services.kubernetes.easyCerts" /> to true. This
|
|
sets up flannel as CNI and activates automatic PKI bootstrapping.
|
|
</para>
|
|
<para>
|
|
As of kubernetes 1.10.X it has been deprecated to open
|
|
non-tls-enabled ports on kubernetes components. Thus, from NixOS
|
|
19.03 all plain HTTP ports have been disabled by default. While
|
|
opening insecure ports is still possible, it is recommended not to
|
|
bind these to other interfaces than loopback. To re-enable the
|
|
insecure port on the apiserver, see options:
|
|
<xref linkend="opt-services.kubernetes.apiserver.insecurePort" />
|
|
and
|
|
<xref linkend="opt-services.kubernetes.apiserver.insecureBindAddress" />
|
|
</para>
|
|
<note>
|
|
<para>
|
|
As of NixOS 19.03, it is mandatory to configure:
|
|
<xref linkend="opt-services.kubernetes.masterAddress" />. The
|
|
masterAddress must be resolveable and routeable by all cluster
|
|
nodes. In single node clusters, this can be set to
|
|
<literal>localhost</literal>.
|
|
</para>
|
|
</note>
|
|
<para>
|
|
Role-based access control (RBAC) authorization mode is enabled by
|
|
default. This means that anonymous requests to the apiserver secure
|
|
port will expectedly cause a permission denied error. All cluster
|
|
components must therefore be configured with x509 certificates for
|
|
two-way tls communication. The x509 certificate subject section
|
|
determines the roles and permissions granted by the apiserver to
|
|
perform clusterwide or namespaced operations. See also:
|
|
<link xlink:href="https://kubernetes.io/docs/reference/access-authn-authz/rbac/">
|
|
Using RBAC Authorization</link>.
|
|
</para>
|
|
<para>
|
|
The NixOS kubernetes module provides an option for automatic
|
|
certificate bootstrapping and configuration,
|
|
<xref linkend="opt-services.kubernetes.easyCerts" />. The PKI
|
|
bootstrapping process involves setting up a certificate authority
|
|
(CA) daemon (cfssl) on the kubernetes master node. cfssl generates a
|
|
CA-cert for the cluster, and uses the CA-cert for signing
|
|
subordinate certs issued to each of the cluster components.
|
|
Subsequently, the certmgr daemon monitors active certificates and
|
|
renews them when needed. For single node Kubernetes clusters,
|
|
setting <xref linkend="opt-services.kubernetes.easyCerts" /> = true
|
|
is sufficient and no further action is required. For joining extra
|
|
node machines to an existing cluster on the other hand, establishing
|
|
initial trust is mandatory.
|
|
</para>
|
|
<para>
|
|
To add new nodes to the cluster: On any (non-master) cluster node
|
|
where <xref linkend="opt-services.kubernetes.easyCerts" /> is
|
|
enabled, the helper script
|
|
<literal>nixos-kubernetes-node-join</literal> is available on PATH.
|
|
Given a token on stdin, it will copy the token to the kubernetes
|
|
secrets directory and restart the certmgr service. As requested
|
|
certificates are issued, the script will restart kubernetes cluster
|
|
components as needed for them to pick up new keypairs.
|
|
</para>
|
|
<note>
|
|
<para>
|
|
Multi-master (HA) clusters are not supported by the easyCerts
|
|
module.
|
|
</para>
|
|
</note>
|
|
<para>
|
|
In order to interact with an RBAC-enabled cluster as an
|
|
administrator, one needs to have cluster-admin privileges. By
|
|
default, when easyCerts is enabled, a cluster-admin kubeconfig file
|
|
is generated and linked into
|
|
<literal>/etc/kubernetes/cluster-admin.kubeconfig</literal> as
|
|
determined by
|
|
<xref linkend="opt-services.kubernetes.pki.etcClusterAdminKubeconfig" />.
|
|
<literal>export KUBECONFIG=/etc/kubernetes/cluster-admin.kubeconfig</literal>
|
|
will make kubectl use this kubeconfig to access and authenticate the
|
|
cluster. The cluster-admin kubeconfig references an auto-generated
|
|
keypair owned by root. Thus, only root on the kubernetes master may
|
|
obtain cluster-admin rights by means of this file.
|
|
</para>
|
|
</chapter>
|