Chapter 5.  Manage Storage for Application Configuration and Data

When an application is run in Kubernetes with a pre-existing image, the application uses the default configuration. This action is valid for testing purposes. However, for production environments, you might need to customize your applications before deploying them.

With Kubernetes, you can use manifests in JSON and YAML formats to specify the intended configuration for each application. You can define the name of the application, labels, the image source, storage, environment variables, and more.

The following snippet shows an example of a YAML manifest file of a deployment:

apiVersion: apps/v1 
kind: Deployment 
metadata: 
  name: hello-deployment
spec: 
  replicas: 1
  selector:
    matchLabels:
      app: hello-deployment
  template:
    metadata:
      labels:
        app: hello-deployment
    spec: 
      containers:
      - env: 
        - name: ENV_VARIABLE_1
          valueFrom:
            secretKeyRef:
              key: hello
              name: world
        image: quay.io/hello-image:latest

Sometimes your application requires configuring a combination of files. For example, at the time of creation, a database deployment must have preloaded databases and data. You most commonly configure applications by using environment variables, external files, or command-line arguments. This process of configuration externalization ensures that the application is portable across environments when the container image, external files, and environment variables are available in the environment where the application runs.

Kubernetes provides a mechanism to externalize the configuration of your applications by using configuration maps and secrets.

You can use configuration maps to inject containers with configuration data. The ConfigMap (configuration map) namespaced objects provide ways to inject configuration data into containers, which helps to maintain platform independence of the containers. These objects can store fine-grained information, such as individual properties, or coarse-grained information, such as entire configuration files or JSON blobs (JSON sections). The information in configuration maps does not require protection.

The following listing shows an example of a configuration map:

apiVersion: v1
kind: ConfigMap 
metadata:
  name: example-configmap
  namespace: my-app
data: 
  example.property.1: hello
  example.property.2: world
  example.property.file: |-
    property.1=value-1
    property.2=value-2
    property.3=value-3
binaryData: 
  bar: L3Jvb3QvMTAw

Applications often require access to sensitive information. For example, a back-end web application requires access to database credentials to query a database. Kubernetes and OpenShift use secrets to hold sensitive information. For example, you can use secrets to store the following types of sensitive information:

The following listing shows an example of a secret:

apiVersion: v1
kind: Secret
metadata:
  name: example-secret
  namespace: my-app
type: Opaque 
data: 
  username: bXl1c2VyCg==
  password: bXlQQDU1Cg==
stringData: 
  hostname: myapp.mydomain.com
  secret.properties: |
    property1=valueA
    property2=valueB

A secret is a namespaced object and it can store any type of data. Data in a secret is Base64-encoded, and is not stored in plain text. Secret data is not encrypted; you can decode the secret from Base64 format to access the original data. The following example shows the decoded values for the username and password objects from the example-secret secret:

[user@host] echo bXl1c2VyCg== | base64 --decode
myuser
[user@host] echo bXlQQDU1Cg== | base64 --decode
myP@55

Kubernetes and OpenShift support the following types of secrets:

When you store information in a specific secret resource type, Kubernetes validates that the data conforms to the type of secret.

For more information, refer to the Encrypting Etcd Data chapter in the Red Hat OpenShift Container Platform 4.14 Security and Compliance documentation at https://docs.redhat.com/en/documentation/openshift_container_platform/4.14/html-single/security_and_compliance/index#about-etcd_encrypting-etcd

If a pod requires access to sensitive information, then create a secret for the information before you deploy the pod. Both the oc and kubectl command-line tools provide the create secret command. Use one of the following commands to create a secret:

To create an opaque secret from the web console, click the Workloads → Secrets menu. Click Create and select Key/value secret. Complete the form with the key name, and specify the value by writing it in the following section, or by extracting it from a file.

To create a secret from the web console that stores the credentials for accessing a container image registry, click the Workloads → Secrets menu. Click Create and select Image pull secret. Complete the form or upload a configuration file with the secret name, select the authentication type, and add the registry server address, the username, password, and email credentials.

The syntax for creating a configuration map and for creating a secret closely match. You can enter key-value pairs on the command line, or use the content of a file as the value of a specified key. You can use either the oc or kubectl command-line tools to create a configuration map. The following command shows how to create a configuration map:

[user@host ~]$ kubectl create configmap my-config \
--from-literal key1=config1 --from-literal key2=config2

You can also use the cm shortname to create a configuration map.

[user@host ~]$ oc create cm my-config \
--from-literal key1=config1 --from-literal key2=config2

To create a configuration map from the web console, click the Workloads → ConfigMaps menu. Click Create ConfigMap and complete the configuration map by using the form view or the YAML view.

You can use files on each key that you add by clicking Browse beside the Value field. The Key field must be the name of the added file in the Value field.

You can use configuration maps to populate individual environment variables that configure your application. Unlike secrets, the information in configuration maps does not require protection. The following listing shows an initialization example of environment variables:

apiVersion: v1
kind: ConfigMap
metadata:
  name: config-map-example
  namespace: example-app 
data:
  database.name: sakila 
  database.user: redhat 

You can then use the configuration map to populate environment variables for your application. The following example shows a pod resource that populates specific environment variables by using a configuration map.

apiVersion: v1
kind: Pod
metadata:
  name: config-map-example-pod
  namespace: example-app
spec:
  containers:
    - name: example-container
      image: registry.example.com/mysql-80:1-237
      command: [ "/bin/sh", "-c", "env" ]
      env: 
        - name: MYSQL_DATABASE 
          valueFrom:
            configMapKeyRef:
              name: config-map-example 
              key: database.name 
        - name: MYSQL_USER
          valueFrom:
            configMapKeyRef:
              name: config-map-example
              key: database.user 
              optional: true 

The following example shows a pod resource that injects all environment variables from a configuration map:

apiVersion: v1
kind: Pod
metadata:
  name: config-map-example-pod2
  namespace: example-app
spec:
  containers:
    - name: example-container
      image: registry.example.com/mysql-80:1-237
      command: [ "/bin/sh", "-c", "env" ]
  envFrom: 
    - configMapRef:
        name: config-map-example 
  restartPolicy: Never

You can use secrets with other Kubernetes resources such as pods, deployments, builds, and more. You can specify secret keys or volumes with a mount path to store your secrets. The following snippet shows an example of a pod that populates environment variables with data from the test-secret Kubernetes secret:

apiVersion: v1
kind: Pod
metadata:
  name: secret-example-pod
spec:
  containers:
    - name: secret-test-container
      image: busybox
      command: [ "/bin/sh", "-c", "export" ]
      env: 
        - name: TEST_SECRET_USERNAME_ENV_VAR
          valueFrom: 
            secretKeyRef: 
              name: test-secret 
              key: username 

In contrast with configuration maps, the values in secrets are always encoded (not encrypted), and their access is restricted to fewer authorized users.

To expose a secret to a pod, you must first create the secret in the same namespace, or project, as the pod. In the secret, assign each piece of sensitive data to a key. After creation, the secret contains key-value pairs.

The following command creates a generic secret that contains key-value pairs from literal values that are typed on the command line: user with the demo-user value, and root_password with the zT1kTgk value.

[user@host ~]$ oc create secret generic demo-secret \
--from-literal user=demo-user \
--from-literal root_password=zT1KTgk

You can also create a generic secret by specifying key names on the command line and values from files:

[user@host ~]$ oc create secret generic demo-secret \
--from-file user=/tmp/demo/user \
--from-file root_password=/tmp/demo/root_password

You can mount a secret to a directory within a pod. Kubernetes creates a file for each key in the secret that uses the name of the key. The content of each file is the decoded value of the secret. The following command shows how to mount secrets in a pod:

[user@host ~]$ oc set volume deployment/demo \ 
--add --type secret \ 
--secret-name demo-secret \ 
--mount-path /app-secrets 

To assign a secret as a volume to a deployment from the web console, list the available secrets from the Workloads → Secrets menu.

Select a secret and click Add Secret to workload.

Select the workload, choose the Volume option, and define the mount path for the secret.

Similar to secrets, you must first create a configuration map before a pod can consume it. The configuration map must exist in the same namespace, or project, as the pod. The following command shows how to create a configuration map from an external configuration file:

[user@host ~]$ oc create configmap demo-map \
--from-file=config-files/httpd.conf

You can similarly add a configuration map as a volume by using the following command:

[user@host ~]$ oc set volume deployment/demo \
--add --type configmap \
--configmap-name demo-map \
--mount-path /app-secrets

To confirm that the volume is attached to the deployment, use the following command:

[user@host ~]$ oc set volume deployment/demo
 demo
  configMap/demo-map as volume-du9in
    mounted at /app-secrets

You can also use the oc set env command to set application environment variables from either secrets or configuration maps. In some cases, you can modify the names of the keys to match the names of environment variables by using the --prefix option. In the following example, the user key from the demo-secret secret sets the MYSQL_USER environment variable, and the root_password key from the demo-secret secret sets the MYSQL_ROOT_PASSWORD environment variable. If the key name from the secret is lowercase, then the corresponding environment variable is converted to uppercase to match the pattern that the --prefix option defines.

[user@host ~]$ oc set env deployment/demo \
--from secret/demo-secret --prefix MYSQL_

Secrets and configuration maps occasionally require updates. OpenShift provides the oc extract command to ensure that you have the latest data. You can save the data to a specific directory by using the --to option. Each key in the secret or configuration map creates a file with the same name as the key. The content of each file is the value of the associated key. If you run the oc extract command more than one time, then you must use the --confirm option to overwrite the existing files. You can also use the --confirm option to create the target directory for the extracted content.

[user@host ~]$ oc extract secret/demo-secrets -n demo \
--to /tmp/demo --confirm
[user@host ~]$ ls /tmp/demo/
user  root_password
[user@host ~]$ cat /tmp/demo/root_password
zT1KTgk
[user@host ~]$ echo k8qhcw3m0 > /tmp/demo/root_password

After updating the locally saved files, use the oc set data command to update the secret or configuration map. For each key that requires an update, specify the name of a key and the associated value. If a file contains the value, then use the --from-file option.

[user@host ~]$ oc set data secret/demo-secrets -n demo \
--from-file /tmp/demo/root_password

You must restart pods that use environment variables for the pods to read the updated secret or configuration map. Pods that use a volume mount to reference secrets or configuration maps receive the updates without a restart by using an eventually consistent approach. By default, the kubelet agent watches for changes to the keys and values that are used in volumes for pods on the node. The kubelet agent detects changes and propagates the changes to the pods to keep volume data consistent. Despite the automatic updates that Kubernetes provides, a restart of the pod is still required if the software reads configuration data only at startup time.

Similar to other Kubernetes resources, you can use the delete command to delete secrets and configuration maps that are no longer needed or in use.

[user@host ~]$ kubectl delete secret/demo-secrets -n demo

[user@host ~]$ oc delete configmap/demo-map -n demo