Bundlebee K8s Operator

Warning

this module is new and can change.

Bundlebee operator provides a Kubernetes operator which creates a custom resource of type Alveolus. This way you can deploy directly an alveolus and the operator will take care to deploy it. Similarly, if you delete the alveolus, the operator will delete it all.

Important

in current version there is no catch up logic so if the deletion fails you should re-run it or finish to clean up the data manually (this is on the roadmap).

Installation

Warning

as of today, the image is not yet available on docker hub (>= 1.0.14) so you will need to build it yourself. This can be done 1. cloning the project, 2. building it with mvn install -DskipTests -Pdocker, 3. push the yupiik/bundlebee/operator:<version> on your kubernetes image registry.

You can install the Bundlebee operator using bundlebee client on your machine (or the maven plugin/fat jar if you prefer): bundlebee --from io.yupiik:bundlebee-operator:<version> --alveolis io.yupiik:bundlebee-operator:<version.

The bundlebee alveolus installation supports the following configuration:

bundlebee.operator.crd.scope: scope of the operator. Default value: Namespaced.
bundlebee.operator.namespace: namespace of the operator. Default value: default.
bundlebee.operator.deployment.dryRun: is dry run mode enabled (for testing purposes). Default value: false.
bundlebee.operator.deployment.http.threads: how many threads are allocated to the http client. Default value: 8.
bundlebee.operator.deployment.downloads.enabled: can the operator download alveoli or should it use the local provisionned maven repository only (kind of explicit enablement of dependencies). Default value: false.
bundlebee.operator.deployment.verbose: does the operator log or not the HTTP requests/responses it does (for debug purposes). Default value: false.

Alternatively you can use the following "all in file" JSON file you can use with kubectl apply -f <bundlebee.operator.install.json>. See all in one descriptor.

Tip

if you don’t install it in default namespace, don’t hesitate to customize KUBECONFIG value in the all in one descriptor or use placeholders for the bundlebee installation. You can also desire to customize the roles affected to the ServiceAccount mounted in the operator Deployment to give it more or less permissions on what it can deploy/delete.

Configuration

The operator configuration is globally the same than standard Bundlebee with a small difference: the global configuration is configured once - in the Deployment of the installation part - and only command configuration can be set in the Alveolus descriptors.

Global configuration

BUNDLEBEE_OPERATOR_STORAGE: Where to store the internal operator state to not restart from scratch when restarting. Default value: /opt/yupiik/state/bundlebee-operator.
BUNDLEBEE_AWAITER_RETRYINTERVAL: How often to retry for a descriptor condition. Increasing it will reduce the pressure on the Kubernetes REST API (rate limiting for example). Default value: 500
BUNDLEBEE_HTTPCLIENT_CONNECTTIMEOUT: The HTTP client connect timeout (in java Duration format), none can be used to ignore this setting. Default value: none
BUNDLEBEE_HTTPCLIENT_FOLLOWREDIRECTS: The HTTP client redirect policy. Default to NORMAL, can be set to ALWAYS or NEVER. Default value: NORMAL
BUNDLEBEE_HTTPCLIENT_FORCEDHTTPVERSION: The HTTP client version, none mean the JVM default (v2), HTTP_1_1 v1.1 and HTTP_2 v2.0. Default value: none
BUNDLEBEE_HTTPCLIENT_THREADS: How many threads are allocated to async HTTP client, negative or zero value means to use common pool. Default value: -1
BUNDLEBEE_KUBE_API: When kubeconfig is not set the base API endpoint. Default value: http://localhost:8080
BUNDLEBEE_KUBE_CONTEXT: When kubeconfig (explicit or not) is used, the context to use. If not set it is taken from the kubeconfig itself.
BUNDLEBEE_KUBE_CUSTOMMETADATAINJECTIONPOINT: When using custom metadata (bundlebee ones or timestamp to force a rollout), where to inject them. Default uses labels since it enables to query them later on but you can switch it to annotations. Default value: labels
BUNDLEBEE_KUBE_DEFAULTPROPAGATIONPOLICY: Default value for deletions of propagationPolicy. Values can be Orphan, Foreground and Background. Default value: Foreground
BUNDLEBEE_KUBE_DRYRUN: If true http requests/responses are skipped. Note that dry run implies verbose=true for the http client. Note that as of today, all responses are mocked by a HTTP 200 and an empty JSON payload. Default value: false
BUNDLEBEE_KUBE_FIELDVALIDATION: fieldValidation - server side validation - value when applying a descriptor, values can be Strict, Warn pr Ignore. Note that using skip will ignore the query parameter. Default value: Strict
BUNDLEBEE_KUBE_FILTERS_STATEFULESET_SPEC_ALLOWED: The attributes to keep from StatefulSet (spec children) descriptor on updates. Default value: replicas,template,updateStrategy,persistentVolumeClaimRetentionPolicy,minReadySeconds,serviceName,selector
BUNDLEBEE_KUBE_FORCE: By default a descriptor update is done using PATCH with strategic merge patch logic, if set to true it will use a plain PUT. Note that io.yupiik.bundlebee/putOnUpdate annotations can be set to true to force that in the descriptor itself and for cases it is not enough, you can set force to true to delete the descriptor before applying it again (move from clusterip to nodeport or the opposite in a serice for ex). Note that you can set it to true in a descriptor annotation io.yupiik.bundlebee/force too to not be global. Default value: false
BUNDLEBEE_KUBE_HTTP_TIMEOUT: HTTP timeout in ms, ignored if ⇐ 0. Default value: 60000
BUNDLEBEE_KUBE_IMPLICITLYDROPPEDATTRIBUTES: Enables to tolerate custom attributes in the descriptors. Typically used to drop /$schema attribute which enables a nice completion in editors. Values are | delimited and are either a JSON-Pointer (wrapped in a remove JSON-Patch) or directly a JSON-Patch. Using none ignores this processing. Default value: /$schema|/$bundlebeeIgnoredLintingRules
BUNDLEBEE_KUBE_LOGDESCRIPTORONPARSINGERROR: Should YAML/JSON be logged when it can’t be parsed. Default value: true
BUNDLEBEE_KUBE_NAMESPACE: When kubeconfig is not set the namespace to use. Default value: default
BUNDLEBEE_KUBE_PATCHCONTENTTYPE: Default header value for PATCH content-type requests header. It uses strategic merge patch algorithm but in some cases you just want to use application/json or (better) application/merge-patch+json. Annotation io.yupiik.bundlebee/patchContentType is also supported. Default value: application/strategic-merge-patch+json
BUNDLEBEE_KUBE_PROXY_HOST: If a proxy is needed to contact the target cluster API, its host, ignore if not set.
BUNDLEBEE_KUBE_PROXY_PASSWORD: If a proxy is needed to contact the target cluster API, its password if it needs an authentication (take care the JVM can nee -Djdk.http.auth.tunneling.disabledSchemes= options).
BUNDLEBEE_KUBE_PROXY_PORT: If a proxy is needed to contact the target cluster API, its port. Default value: 3128
BUNDLEBEE_KUBE_PROXY_SETPROXYSYSTEMPROPERTIES: If a proxy is configured to use authentication, automatically set -Djdk.http.auth.tunneling.disabledSchemes=, note that setting it on the JVM is still more reliable depending how you run bundlebee (in particular with maven or embed). Important: the system property is "leaked", ie it is not cleaned up to limit side effect in concurrent mode. Default value: true
BUNDLEBEE_KUBE_PROXY_USERNAME: If a proxy is needed to contact the target cluster API, its username if it needs an authentication (take care the JVM can nee -Djdk.http.auth.tunneling.disabledSchemes= options).
BUNDLEBEE_KUBE_PUTONUPDATE: By default a descriptor update is done using PATCH with strategic merge patch logic, if set to true it will use a plain PUT. Note that io.yupiik.bundlebee/putOnUpdate annotations can be set to true to force that in the descriptor itself. Default value: false
BUNDLEBEE_KUBE_RATELIMITER_ENABLED: Should HTTP client requests be limited and HTTP 427 responses be handled. Default value: false
BUNDLEBEE_KUBE_RATELIMITER_PERMITS: How many calls can be done if rate limiting is enabled. Note that setting it to Integer.MAX_VALUE will disable the client rate limiting and only enable server one. Default value: 100
BUNDLEBEE_KUBE_RATELIMITER_WINDOW: Rate limiting window duration in milliseconds (default being 1 second). Default value: 1000
BUNDLEBEE_KUBE_RESOURCEMAPPING: Enables to define resource mapping, syntax uses propeties one: <lowercased resource kind>s = /apis/….. Default value: ``
BUNDLEBEE_KUBE_SKIPDRYRUNFORGET: If true GET http requests are not skipped when dryRun is true. Default value: false
BUNDLEBEE_KUBE_SKIPUPDATEFORKINDS: List of kind of descriptors updates can be skipped, it is often useful for PersistentVolumeClaim. Default value: PersistentVolumeClaim
BUNDLEBEE_KUBE_TOKEN: When kubeconfig is set to explicit, the bearer token to use (if set).
BUNDLEBEE_KUBE_VALIDATESSL: Should SSL connector be validated or not. Default value: true
BUNDLEBEE_KUBE_VERBOSE: If true http requests/responses to Kubernetes will be logged. Default value: false
BUNDLEBEE_MAVEN_CACHE: Where to cache maven dependencies. If set to auto, tries to read the system property maven.repo.local then the settings.xml localRepository and finally it would fallback on $HOME/.m2/repository. Default value: auto
BUNDLEBEE_MAVEN_FORCECUSTOMSETTINGSXML: If true we only use cache value and never fallback on default maven settings.xml location. Default value: false
BUNDLEBEE_MAVEN_HTTP_CONNECTTIMEOUT: When fetching a dependency using HTTP, the connection timeout for this dependency. Default value: 30000
BUNDLEBEE_MAVEN_PREFERCUSTOMSETTINGSXML: If false we first try to read settings.xml file(s) in cache location before the default one. Default value: true
BUNDLEBEE_MAVEN_REPOSITORIES_DOWNLOADS_ENABLED: Enable the download, i.e. ensure it runs only with local maven repository. Default value: false
BUNDLEBEE_MAVEN_REPOSITORIES_HTTPHEADERS: Properties to define the headers to set per repository, syntax is host1=headerName headerValue and it supports as much lines as used repositories. Note that you can use maven ~/.m2/settings.xml servers (potentially ciphered) username/password pairs. In this last case the server id must be bundlebee.<server host>. Still in settings.xml case, if the username is null the password value is used as raw Authorization header else username/password is encoded as a basic header.
BUNDLEBEE_MAVEN_REPOSITORIES_RELEASE: Default release repository. Default value: https://repo.maven.apache.org/maven2/
BUNDLEBEE_MAVEN_REPOSITORIES_SNAPSHOT: Default snapshot repository, not set by default.
KUBECONFIG: Kubeconfig location. If set to auto it will try to guess from your $HOME/.kube/config file until you set it so explicit where it will use other bundlebee.kube properties to create the client. The content can also be set inline!. Default value: auto

Delete/Apply an alveolus

To apply a custmo Alveolus you can write a descriptor like:

apiVersion: bundlebee.yupiik.io/v1 (1)
kind: Alveolus (2)
metadata: (3)
  name: test-alveolus
spec:
  args: (4)
    - "--from"
    - "io.yupiik.alveoli:elasticsearch-local:1.0.13"
    - "--alveolus"
    - "io.yupiik.alveoli:elasticsearch-local:1.0.13"

Use apiVersion bundlebee.yupiik.io/v1,
Use kind Alveolus,
metadata object is a standard one so you can set a name, labels, etc…,
args are the options of the apply command for this deployment (without the global configuration indeed).

Tip	you can check `apply` documentation to see all the options available.

Appendix, all in one installation

bundlebee.operator.install.json

{
  "apiVersion": "apiextensions.k8s.io/v1",
  "kind": "CustomResourceDefinition",
  "metadata": {
    "name": "alveoli.bundlebee.yupiik.io"
  },
  "spec": {
    "group": "bundlebee.yupiik.io",
    "scope": "Namespaced",
    "names": {
      "plural": "alveoli",
      "singular": "alveolus",
      "kind": "Alveolus",
      "shortNames": [
        "av"
      ]
    },
    "versions": [
      {
        "name": "v1",
        "served": true,
        "storage": true,
        "schema": {
          "openAPIV3Schema": {
            "type": "object",
            "properties": {
              "spec": {
                "type": "object",
                "properties": {
                  "args": {
                    "type": "array",
                    "items": {
                      "type": "string"
                    }
                  }
                }
              }
            }
          }
        }
      }
    ]
  }
}
{
  "apiVersion": "rbac.authorization.k8s.io/v1",
  "kind": "Role",
  "metadata": {
    "name": "bundlebee-operator",
    "labels": {
      "app.kubernetes.io/component": "controller",
      "app.kubernetes.io/name": "bundlebee-operator",
      "app.kubernetes.io/version": "1.0.28-SNAPSHOT"
    }
  },
  "rules": [
    {
      "apiGroups": [
        "bundlebee.yupiik.io"
      ],
      "resources": [
        "alveoli"
      ],
      "verbs": [
        "*"
      ]
    },
    {
      "apiGroups": [
        "apps"
      ],
      "resources": [
        "daemonsets",
        "statefulsets",
        "deployments"
      ],
      "verbs": [
        "get",
        "create",
        "update",
        "delete"
      ]
    },
    {
      "apiGroups": [
        ""
      ],
      "resources": [
        "configmaps",
        "secrets",
        "pods",
        "persistentvolumes",
        "persistentvolumeclaims",
        "services"
      ],
      "verbs": [
        "*"
      ]
    }
  ]
}
{
  "apiVersion": "rbac.authorization.k8s.io/v1",
  "kind": "RoleBinding",
  "metadata": {
    "name": "bundlebee-operator",
    "labels": {
      "app.kubernetes.io/component": "controller",
      "app.kubernetes.io/name": "bundlebee-operator",
      "app.kubernetes.io/version": "1.0.28-SNAPSHOT"
    }
  },
  "roleRef": {
    "apiGroup": "rbac.authorization.k8s.io",
    "kind": "Role",
    "name": "bundlebee-operator"
  },
  "subjects": [
    {
      "kind": "ServiceAccount",
      "name": "bundlebee-operator",
      "namespace": "default"
    }
  ]
}
{
  "apiVersion": "v1",
  "kind": "ServiceAccount",
  "metadata": {
    "name": "bundlebee-operator",
    "namespace": "default",
    "labels": {
      "app.kubernetes.io/component": "controller",
      "app.kubernetes.io/name": "bundlebee-operator",
      "app.kubernetes.io/version": "1.0.28-SNAPSHOT"
    }
  }
}
{
  "apiVersion": "v1",
  "kind": "Secret",
  "metadata": {
    "name": "bundlebee-operator-token",
    "namespace": "default",
    "annotations": {
      "kubernetes.io/service-account.name": "bundlebee-operator"
    }
  },
  "type": "kubernetes.io/service-account-token"
}
{
  "apiVersion": "apps/v1",
  "kind": "Deployment",
  "metadata": {
    "name": "bundlebee-operator",
    "labels": {
      "app.kubernetes.io/component": "controller",
      "app.kubernetes.io/name": "bundlebee-operator",
      "app.kubernetes.io/version": "1.0.28-SNAPSHOT"
    }
  },
  "spec": {
    "replicas": 1,
    "selector": {
      "matchLabels": {
        "app.kubernetes.io/component": "controller",
        "app.kubernetes.io/name": "bundlebee-operator"
      }
    },
    "template": {
      "metadata": {
        "annotations": {
          "kubectl.kubernetes.io/default-container": "bundlebee-operator"
        },
        "labels": {
          "app.kubernetes.io/component": "controller",
          "app.kubernetes.io/name": "bundlebee-operator",
          "app.kubernetes.io/version": "1.0.28-SNAPSHOT"
        }
      },
      "spec": {
        "serviceAccountName": "bundlebee-operator",
        "containers": [
          {
            "name": "bundlebee-operator",
            "image": "yupiik/bundlebee/operator:1.0.28-SNAPSHOT",
            "env": [
              {
                "name": "BUNDLEBEE_KUBE_DRYRUN",
                "value": "false"
              },
              {
                "name": "BUNDLEBEE_KUBE_SKIPDRYRUNFORGET",
                "value": "false"
              },
              {
                "name": "BUNDLEBEE_KUBE_VERBOSE",
                "value": "false"
              },
              {
                "name": "BUNDLEBEE_MAVEN_REPOSITORIES_DOWNLOADS_ENABLED",
                "value": "false"
              },
              {
                "name": "BUNDLEBEE_HTTPCLIENT_THREADS",
                "value": "8"
              },
              {
                "name": "KUBECONFIG",
                "value": "{\n  \"apiVersion\": \"v1\",\n  \"kind\": \"Config\",\n  \"current-context\": \"ctx\",\n  \"clusters\": [{\"name\": \"clt\",\"cluster\": {\n    \"certificate-authority\": \"/var/run/secrets/kubernetes.io/serviceaccount/ca.crt\",\n    \"server\": \"https://kubernetes.default.svc\"}}],\n  \"contexts\": [{\"name\": \"ctx\",\"context\": {\"cluster\": \"clt\", \"user\": \"sa\",\n    \"namespace\": \"default\"}}],\n  \"users\": [{\"name\": \"sa\",\"user\": {\n    \"tokenFile\": \"/var/run/secrets/kubernetes.io/serviceaccount/token\"}}]\n}"
              }
            ]
          }
        ]
      }
    }
  }
}

Menu