CanSupportMe.Operator

Overview

What is this?

This package will allow you to create a simple Kubernetes Operator in .NET.
It can pick up changes to CRDs, ConfigMaps or Secrets and say if it was added, modified or deleted. It can check if a CRD exists as well as retrieve, create, replace and delete configmaps and secrets. It can also update the status of any manifest that supports it.

What isn't this?

It is not a full blown Kubernetes Operator SDK. These exist and, if desired, this package can be used in conjunction with them. It has been developed to do what the project required and nothing more however this will likely fit the requirements of many similar projects too.

Getting Started

Prerequisites

• .NET 8 or later
• A Kubernetes cluster (either local or remote)
• Sufficient access to the cluster to read and watch secrets (for the demo)
• The ability to create a service account and cluster role binding with access to the resources you will be watching or creating
• A suitable pair of test secrets, created using the following bash script: kubectl apply -f - <<EOF apiVersion: v1 kind: Secret metadata: name: test-secret-with-label namespace: default labels: app.kubernetes.io/managed-by: DemoOperator stringData: notImportant: SomeValue type: Opaque --- apiVersion: v1 kind: Secret metadata: name: test-secret-without-label namespace: default stringData: notImportant: SomeValue type: Opaque EOF

Create a new .NET console application

Create a new .NET 8 console application and call it CanSupportMe.Operator.Demo.

Add the following packages from NuGet:

• Microsoft.Extensions.Configuration.Abstractions
• Microsoft.Extensions.Hosting
• Microsoft.Extensions.Hosting.Abstractions
• Serilog.AspNetCore

Note that the Serilog.AspNetCore package is not required but is used in this demo to show how to produce nicer logs.

Install the package

dotnet add package CanSupportMe.Operator

Create a simple operator

Replace the contents of your Program.cs with the following:

using CanSupportMe.Operator.Extensions; using CanSupportMe.Operator.Options; using Microsoft.Extensions.Hosting; using Serilog; using Serilog.Events; // const string FAILOVER_TOKEN = "<YOUR_TOKEN_GOES_HERE>"; Log.Logger = new LoggerConfiguration() .MinimumLevel.Debug() .MinimumLevel.Override("Microsoft", LogEventLevel.Warning) .MinimumLevel.Override("System", LogEventLevel.Warning) .WriteTo.Console() .CreateLogger(); try { Log.Information("Application starting up"); IHost host = Host.CreateDefaultBuilder(args) .ConfigureServices((context, services) => { services.AddOperator(options => { options.Group = ""; options.Kind = "Secret"; options.Version = "v1"; options.Plural = "secrets"; options.Scope = ResourceScope.Namespaced; options.LabelFilters.Add("app.kubernetes.io/managed-by", "SeqApiKeyOperator"); options.OnAdded = (kind, name, @namespace, item, dataObjectManager) => { Log.Information("On {Kind} Add: {Name} to {Namespace} which is of type {Type} with {ItemCount} item(s)", kind, name, @namespace, item.Type, item.Data?.Count); }; // options.FailoverToken = FAILOVER_TOKEN; }); }) .UseSerilog() .Build(); host.Run(); } catch (Exception ex) { Log.Fatal(ex, "Application start failed"); } finally { Log.CloseAndFlush(); }

Running the project will show it connecting to the cluster and watching for changes to secrets. It will then show test-secret-with-label as being added and not mention test-secret-without-label.

Retrieve service token

If a options.FailoverToken is required, one can be generated using the following bash script:

export SERVICEACCOUNT="cansupportme-operator" kubectl apply -f - <<EOF apiVersion: v1 kind: Secret metadata: name: ${SERVICEACCOUNT}-token namespace: default annotations: kubernetes.io/service-account.name: $SERVICEACCOUNT type: kubernetes.io/service-account-token EOF export TOKEN=$(kubectl -n default get secret ${SERVICEACCOUNT}-token -o jsonpath='{.data.token}' | base64 --decode) echo $TOKEN

Uncomment the two commented out lines of code in the above snippet and paste in the key generated above into the FAILOVER_TOKEN constant.

Listening for Custom Resource Definition (CRD) Changes

The more likely scenario for an operator is to listen out for CRD changes. The following example shows the simple file you need to create and how to configure the operator. To set up a CRD, apply the following CRD manifest to your cluster:

apiVersion: apiextensions.k8s.io/v1 kind: CustomResourceDefinition metadata: # name must match the spec fields below, and be in the form: <plural>.<group> name: apikeys.demo.cansupport.me spec: # group name to use for REST API: /apis/<group>/<version> group: demo.cansupport.me # list of versions supported by this CustomResourceDefinition versions: - name: v1alpha1 # Each version can be enabled/disabled by Served flag. served: true # One and only one version must be marked as the storage version. storage: true schema: openAPIV3Schema: type: object properties: spec: type: object properties: secretName: type: string x-kubernetes-validations: - rule: self == oldSelf message: "The secret name cannot be changed once set" required: - secretName status: type: object properties: status: type: string default: "New" enum: - "Ready" - "Reconciling" - "Failed" - "New" - "Deleting" additionalPrinterColumns: - name: Secret Name type: string description: The name of the secret that contains the API Key jsonPath: .spec.secretName - name: Status type: string description: The status of the Seq API key being reconciled jsonPath: .status.status # either Namespaced or Cluster scope: Namespaced names: # plural name to be used in the URL: /apis/<group>/<version>/<plural> plural: apikeys # singular name to be used as an alias on the CLI and for display singular: apikey # kind is normally the CamelCased singular type. Your resource manifests use this. kind: ApiKey # shortNames allow shorter string to match your resource on the CLI shortNames: - ak

Create a new file called MyCrd.cs in the root of the project and add the following:

using CanSupportMe.Operator.Interfaces; using CanSupportMe.Operator.Models; using System.Text.Json.Serialization; namespace CanSupportMe.Operator.Demo; public class MyCrd : KubernetesResource<MySpec, MyStatus> { } public class MySpec : IKubernetesSpec { [JsonPropertyName("secretName")] public required string SecretName { get; set; } } public class MyStatus : IKubernetesStatus { [JsonPropertyName("status")] public required string Status { get; set; } }

Then, in Program.cs add the following:

services.AddOperator<MySpec, MyStatus>(options => { options.OperatorName = "SeqApiKeyOperator"; options.Group = "demo.cansupport.me"; options.Kind = "ApiKey"; options.Version = "v1alpha1"; options.Plural = "apikeys"; options.Scope = ResourceScope.Namespaced; options.OnReconcile = async (kind, name, @namespace, item, dataObjectManager) => { Log.Information("On {kind} reconcile: {Name} to {Namespace} to create secret called {SecretName}", kind, name, @namespace, item.Spec.SecretName); var existingSecret = await dataObjectManager.Get("Secret", item.Spec.SecretName, @namespace); if (existingSecret.IsSuccess) { Log.Debug("Secret {SecretName} already exists in {Namespace} and has a value of {ExistingValue}", existingSecret.Value.Metadata.Name, existingSecret.Value.Metadata.Namespace, existingSecret.Value.Data["myApiKey"]); } else { Log.Debug("Secret {SecretName} does not exist in {Namespace}", item.Spec.SecretName, @namespace); await dataObjectManager.Create("Secret", item.Spec.SecretName, @namespace, new Dictionary<string, string> { { "myApiKey", $"Some random string at {DateTime.UtcNow}" } }, labels: new() { { "cansupport.me/owned-by", name } }); } }; options.FailoverToken = FAILOVER_TOKEN; });