Linting
Linting is the process of running validations on descriptors.
Bundlebee is able to read common "pre-rendered" descriptors (from Helm for example) as long as a manifest.json is properly formatted so you can run the linter on any descriptor even if it is working out of the box for Bundlebee alveoli (where you just run bundlebee lint … command without pre-processing).
Configure
The main configuration of the linter are the inclusions or exclusions (see lint command details).
You can also exclude rules in ignoredLintingRules of the manifest.json (per manifest not global) or in the descriptors directly using virtual $bundlebeeIgnoredLintingRules attribute which is equivalent.
|
Tip
|
while in the manifest the list is a list of object with a name attribute, manifest and descriptor lists are list of rule names to keep it simple.
|
Indeed the lint command enables to exclude alveoli and descriptors but it is generally more convenient to create a my-project:lint alveolus which will disable (using placeholders or includeIf conditions) undesired external dependencies and descriptors during the linting process.
This makes it quite easy and efficient to control what you deploy and avoid to trigger rules you don’t want.
Rules
Here is the available rules (built-in).
Linting Checks
Here is the list of available checks by defaults.
access-to-create-pods
Name: access-to-create-pods.
Indicates when a subject (Group/User/ServiceAccount) has create access to Pods. CIS Benchmark 5.1.4: The ability to create pods in a cluster opens up possibilities for privilege escalation and should be restricted, where possible.
Remediation
Where possible, remove create access to pod objects in the cluster.
access-to-secrets
Name: access-to-secrets.
Indicates when a subject (Group/User/ServiceAccount) has access to Secrets. CIS Benchmark 5.1.2: Access to secrets should be restricted to the smallest possible group of users to reduce the risk of privilege escalation.
Remediation
Where possible, remove get, list and watch access to secret objects in the cluster.
cpu-limits
Name: cpu-limits.
Ensures resources.limits.cpu is set.
Remediation
Set resources.limits.cpu.
cpu-requests
Name: cpu-requests.
Ensures resources.requests.cpu is set.
Remediation
Set resources.requests.cpu.
default-service-account
Name: default-service-account.
Indicates when pods use the default service account.
Remediation
Create a dedicated service account for your pod. Refer to https://kubernetes.io/docs/tasks/configure-pod-container/configure-service-account/ for details.
deprecated-service-account-field
Name: deprecated-service-account-field.
Indicates when deployments use the deprecated serviceAccount field.
Remediation
Use the serviceAccountName field instead. If you must specify serviceAccount, ensure values for serviceAccount and serviceAccountName match.
dnsconfig-options
Name: dnsconfig-options.
Alert on deployments that have no specified dnsConfig options
Remediation
Specify dnsconfig options in your Pod specification to ensure the expected DNS setting on the Pod. Refer to https://kubernetes.io/docs/concepts/services-networking/dns-pod-service/#pod-dns-config for details.
docker-sock
Name: docker-sock.
Alert on deployments with docker.sock mounted in containers.
Remediation
Ensure the Docker socket is not mounted inside any containers by removing the associated Volume and VolumeMount in deployment yaml specification. If the Docker socket is mounted inside a container it could allow processes running within the container to execute Docker commands which would effectively allow for full control of the host.
drop-net-raw-capability
Name: drop-net-raw-capability.
Indicates when containers do not drop NET_RAW capability
Remediation
NET_RAW makes it so that an application within the container is able to craft raw packets, use raw sockets, and bind to any address. Remove this capability in the containers under containers security contexts.
duplicate-env-var
Name: duplicate-env-var.
Check that duplicate named env vars aren’t passed to a deployment like.
Remediation
Confirm that your DeploymentLike doesn’t have duplicate env vars names.
host-ipc
Name: host-ipc.
Alert on pods/deployment-likes with sharing host’s IPC namespace
Remediation
Ensure the host’s IPC namespace is not shared.
host-network
Name: host-network.
Alert on pods/deployment-likes with sharing host’s network namespace
Remediation
Ensure the host’s network namespace is not shared.
host-pid
Name: host-pid.
Alert on pods/deployment-likes with sharing host’s process namespace
Remediation
Ensure the host’s process namespace is not shared.
hpa-minimum-three-replicas
Name: hpa-minimum-three-replicas.
Indicates when a HorizontalPodAutoscaler specifies less than three minReplicas
Remediation
Increase the number of replicas in the HorizontalPodAutoscaler to at least three to increase fault tolerance.
memory-limits
Name: memory-limits.
Ensures resources.limits.memory is set.
Remediation
Set resources.limits.memory.
memory-requests
Name: memory-requests.
Ensures resources.requests.memory is set.
Remediation
Set resources.requests.memory.
minimum-three-replicas
Name: minimum-three-replicas.
Indicates when a deployment uses less than three replicas
Remediation
Increase the number of replicas in the deployment to at least three to increase the fault tolerance of the deployment.
mismatching-selector
Name: mismatching-selector.
Indicates when deployment selectors fail to match the pod template labels.
Remediation
Confirm that your deployment selector correctly matches the labels in its pod template.
missing-anti-affinity
Name: missing-anti-affinity.
When replicas > 1 setting an anti-affinity enables to distribute the load accross machines setting 'topologyKey: "kubernetes.io/hostname"', see https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/#inter-pod-affinity-and-anti-affinity.
Remediation
Add podAntiAffinity in your deployment descriptor.
no-extensions-v1beta
Name: no-extensions-v1beta.
Indicates when objects use deprecated API versions under extensions/v1beta.
Remediation
Migrate using the apps/v1 API versions for the objects. Refer to https://kubernetes.io/blog/2019/07/18/api-deprecations-in-1-16/ for details.
no-latest
Name: no-latest.
Ensures no latest image is used.
Remediation
Ensure to tag any image you use and use an immutable tag if possible or at least versionned flavor.
no-liveness-probe
Name: no-liveness-probe.
Ensures a liveness probe is defined.
Remediation
Any container (from containers array) should have a liveness probe.
no-read-only-root-fs
Name: no-read-only-root-fs.
Indicates when containers are running without a read-only root filesystem.
Remediation
Set readOnlyRootFilesystem to true in the container securityContext.
no-readiness-probe
Name: no-readiness-probe.
Ensures a readinessProbe probe is defined.
Remediation
Any container (from containers array) should have a readinessProbe probe.
privilege-escalation-container
Name: privilege-escalation-container.
Alert on containers of allowing privilege escalation that could gain more privileges than its parent process.
Remediation
Ensure containers do not allow privilege escalation by setting allowPrivilegeEscalation=false, privileged=false and removing CAP_SYS_ADMIN capability. See https://kubernetes.io/docs/tasks/configure-pod-container/security-context/ for more details.
privileged-container
Name: privileged-container.
Indicates when deployments have containers running in privileged mode.
Remediation
Do not run your container as privileged unless it is required.
privileged-ports
Name: privileged-ports.
Alert on deployments with privileged ports mapped in containers
Remediation
Ensure privileged ports [0, 1024] are not mapped within containers.
read-secret-from-env-var
Name: read-secret-from-env-var.
Indicates when a deployment reads secret from environment variables. CIS Benchmark 5.4.1: "Prefer using secrets as files over secrets as environment variables. "
Remediation
If possible, rewrite application code to read secrets from mounted secret files, rather than from environment variables. Refer to https://kubernetes.io/docs/concepts/configuration/secret/#using-secrets for details.
run-as-non-root
Name: run-as-non-root.
Indicates when containers are not set to runAsNonRoot.
Remediation
Set runAsUser to a non-zero number and runAsNonRoot to true in your pod or container securityContext. Refer to https://kubernetes.io/docs/tasks/configure-pod-container/security-context/ for details.
sensitive-host-mounts
Name: sensitive-host-mounts.
Alert on deployments with sensitive host system directories mounted in containers.
Remediation
Ensure sensitive host system directories are not mounted in containers by removing those Volumes and VolumeMounts.
unsafe-proc-mount
Name: unsafe-proc-mount.
Alert on deployments with unsafe /proc mount (procMount=Unmasked) that will bypass the default masking behavior of the container runtime
Remediation
Ensure container does not unsafely exposes parts of /proc by setting procMount=Default. Unmasked ProcMount bypasses the default masking behavior of the container runtime. See https://kubernetes.io/docs/concepts/security/pod-security-standards/ for more details.
unsafe-sysctls
Name: unsafe-sysctls.
Alert on deployments specifying unsafe sysctls that may lead to severe problems like wrong behavior of containers
Remediation
Ensure container does not allow unsafe allocation of system resources by removing unsafe sysctls configurations. For more details see https://kubernetes.io/docs/tasks/administer-cluster/sysctl-cluster/ https://docs.docker.com/engine/reference/commandline/run/#configure-namespaced-kernel-parameters-sysctls-at-runtime.
use-namespace
Name: use-namespace.
Indicates when a resource is deployed to the default namespace. CIS Benchmark 5.7.1: Create administrative boundaries between resources using namespaces. CIS Benchmark 5.7.4: The default namespace should not be used.
Remediation
Create namespaces for objects in your deployment.
wildcard-in-rules
Name: wildcard-in-rules.
Indicate when a wildcard is used in Role or ClusterRole rules. CIS Benchmark 5.1.3 Use of wildcards is not optimal from a security perspective as it may allow for inadvertent access to be granted when new resources are added to the Kubernetes API either as CRDs or in later versions of the product.
Remediation
Where possible replace any use of wildcards in clusterroles and roles with specific objects or actions.
writable-host-mount
Name: writable-host-mount.
Indicates when containers mount a host path as writable.
Remediation
Set containers to mount host paths as readOnly, if you need to access files on the host.
Write your own rule
To write your own rule you have to implement and add to bundlebee (core) classpath a CDI bean implementing LintingCheck - or SynchronousLintingCheck which can be easier to implement.
Synchronous or not
The generic LintingCheck uses CompletionStage because it is possible to use validation rules which connect to a cluster (the bundlebee configured one) and using KubeClient - or any other logic - you can do checks against a particular cluster (does role exists for example).
CompletionStage enable to run validations in parallel easily.
When not needed, just go with the SynchronousLintingCheck base interface or one of its children.
Anatomy of a check
A check is composed of:
-
A
namewhich enables to disable/force the check to run, -
A
remediationstring which explains how to fix the violation when it happens, -
A
decriptionexplaining the validation, -
A
validate(or sync flavor) method doing validation when a descriptor is visited, -
Sometimes an
afterAllmethod doing validations after all descriptors were seen (for joins accross desriptors for example).
JSON, JSON, JSON
Yaml descriptors are converted to JsonObject and descriptors containing a list are flattened.
What does it means is that you always validate a single descriptor which is an object in check code which makes it easier to work with.
Going further
It is not yet possible to define custom rules, it is not very complicated to do it technically but the language (must be JSON based) is challenging. Here are the current evaluated options:
-
Custom DSL: likely the most powerful and consistent option since we could define it in
manifest.jsonbut requires to learn yet another language, -
JSON-Logic: a very good option but requires a lot of operators and resulting DSL is not that friendly/simple,
-
Template based: idea there is to define a few options ("check this value at this JSON pointer", "check this chain of json pointer", …) but this becomes quite hard to have a relevant dictionary set without making it overcomplex,
-
Scripting based: we can run jruby/jython/js engine rules but they all required to embed a script manager runtime and they tend to be fatty for this need and not all are graalvm friendly to convert bundlebee to native mode.
So as of today, since we didn’t find a compromise making us happy we didn’t enable to define JSON DSL based rules. Since it is easy to add custom rules in Java (and using the maven plugin), this is not blocking but we’d like to tackle it if we can come up with a better idea. Don’t hesistate to propose something on bundlebee github project if you have an idea or a common need.