Skip to content

Customize Helm charts

In the Getting Started guide, we designed a SaaS service that deploys Redis Clusters using a Helm chart. However, in that example, we had all the pods deploy to any available VM in the Kubernetes cluster. In addition, these pods were colocated on the same VM, not an ideal scenario for High Availability.

In this guide, we will show you how to customize the infrastructure for the Helm chart to deploy the Redis Master and Replica pods on separate VMs. We will also introduce a custom instance type for the Redis Master and Replica pods that your customers can specify through the SaaS portal we generated.

Add the REST API Parameters

First, we need to update the Helm chart to support the new features. We will add two new parameters to the values.yaml file to allow customers to choose the instance type for their workload and the number of replicas they want to deploy.

...
    apiParameters:
      - key: replicas
        description: Number of Replicas
        name: Replica Count
        type: Float64
        modifiable: true
        required: false
        export: true
        defaultValue: "1"
      - key: instanceType
        description: Instance Type
        name: Instance Type
        type: String
        modifiable: true
        required: false
        export: true
        defaultValue: "t4g.small"
...

The apiParameters section in the specification defines the REST API parameters that you want your customers to provide as part of the provisioning APIs for your SaaS. For more information see: API Parameters.

Define affinity rules to configure High Availability

We will also update the deployment.yaml file to deploy the Redis Master and Replica pods to separate VMs. We will override the affinity rules to make this happen.

First, we add a pod-label for setting the anti-affinity between the master and replica pods.

...
        master:
          podLabels:
            omnistrate.com/schedule-mode: exclusive

        replica:
          podLabels:
            omnistrate.com/schedule-mode: exclusive
...

Then, we set the affinity rules appropriately for the Redis Master and Replica pods.

...
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                    - key: omnistrate.com/managed-by
                      operator: In
                      values:
                      - omnistrate
                    - key: topology.kubernetes.io/region
                      operator: In
                      values:
                      - $sys.deploymentCell.region
                    - key: node.kubernetes.io/instance-type
                      operator: In
                      values:
                      - $sys.compute.node.instanceType
                    - key: omnistrate.com/resource
                      operator: In
                      values:
                      - $sys.deployment.resourceID
            podAntiAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
              - labelSelector:
                  matchExpressions:
                  - key: omnistrate.com/schedule-mode
                    operator: In
                    values:
                    - exclusive
                namespaceSelector: {}
                topologyKey: kubernetes.io/hostname
...

You might have noticed the use of system variables ($sys.deploymentCell.region, $sys.compute.node.instanceType, and $sys.deployment.resourceID) in the affinity rules. These variables are used to dynamically set the affinity rules based on the customer's deployment configuration. For more information, see System Variables.

Reference REST API parameters

Finally, we will use the new REST API parameters to set the number of replicas and the instance type for the Redis Master and Replica pods in the values.yaml file.

...
    compute:
      instanceTypes:
        - apiParam: instanceType
          cloudProvider: aws
        - apiParam: instanceType
          cloudProvider: gcp

    replica:
      ...
      replicaCount: $var.replicas
...

Final specification

name: Redis Server # Service Plan Name
deployment:
  hostedDeployment:
    AwsAccountId: "<AWS_ID>"
    AwsBootstrapRoleAccountArn: arn:aws:iam::<AWS_ID>:role/omnistrate-bootstrap-role

services:
  - name: Redis Cluster
    compute:
      instanceTypes:
        - apiParam: instanceType
          cloudProvider: aws
        - apiParam: instanceType
          cloudProvider: gcp
    network:
      ports:
        - 6379
    helmChartConfiguration:
      chartName: redis
      chartVersion: 19.6.2
      chartRepoName: bitnami
      chartRepoURL: https://charts.bitnami.com/bitnami
      chartValues:
        master:
          podLabels:
            omnistrate.com/schedule-mode: exclusive
          persistence:
            enabled: false
          service:
            type: LoadBalancer
            annotations:
              external-dns.alpha.kubernetes.io/hostname: $sys.network.externalClusterEndpoint

          resources:
            requests:
              cpu: 100m
              memory: 128Mi
            limits:
              cpu: 150m
              memory: 256Mi
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                    - key: omnistrate.com/managed-by
                      operator: In
                      values:
                      - omnistrate
                    - key: topology.kubernetes.io/region
                      operator: In
                      values:
                      - $sys.deploymentCell.region
                    - key: node.kubernetes.io/instance-type
                      operator: In
                      values:
                      - $sys.compute.node.instanceType
                    - key: omnistrate.com/resource
                      operator: In
                      values:
                      - $sys.deployment.resourceID
            podAntiAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
              - labelSelector:
                  matchExpressions:
                  - key: omnistrate.com/schedule-mode
                    operator: In
                    values:
                    - exclusive
                namespaceSelector: {}
                topologyKey: kubernetes.io/hostname
        replica:
          podLabels:
            omnistrate.com/schedule-mode: exclusive
          persistence:
            enabled: false
          replicaCount: $var.replicas
          resources:
            requests:
              cpu: 100m
              memory: 128Mi
            limits:
              cpu: 150m
              memory: 256Mi
          affinity:
            nodeAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
                nodeSelectorTerms:
                  - matchExpressions:
                    - key: omnistrate.com/managed-by
                      operator: In
                      values:
                      - omnistrate
                    - key: topology.kubernetes.io/region
                      operator: In
                      values:
                      - $sys.deploymentCell.region
                    - key: node.kubernetes.io/instance-type
                      operator: In
                      values:
                      - $sys.compute.node.instanceType
                    - key: omnistrate.com/resource
                      operator: In
                      values:
                      - $sys.deployment.resourceID
            podAntiAffinity:
              requiredDuringSchedulingIgnoredDuringExecution:
              - labelSelector:
                  matchExpressions:
                  - key: omnistrate.com/schedule-mode
                    operator: In
                    values:
                    - exclusive
                namespaceSelector: {}
                topologyKey: kubernetes.io/hostname
    apiParameters:
      - key: replicas
        description: Number of Replicas
        name: Replica Count
        type: Float64
        modifiable: true
        required: false
        export: true
        defaultValue: "1"
      - key: instanceType
        description: Instance Type
        name: Instance Type
        type: String
        modifiable: true
        required: false
        export: true
        defaultValue: "t4g.small"

Apply changes to the service

For this we will run the same command that was used to setup the service the first time.

omnistrate-ctl build -f spec.yaml --name 'RedisHelm' --release-as-preferred --spec-type ServicePlanSpec

# Example output shown below
 Successfully built service
Check the service plan result at: https://omnistrate.cloud/product-tier?serviceId=s-dEhutaDa2X&environmentId=se-92smpU2YAm
Access your SaaS offer at: https://saasportal.instance-w6vidhd14.hc-pelsk80ph.us-east-2.aws.f2e0a955bb84.cloud/service-plans?serviceId=s-dEhutaDa2X&environmentId=se-92smpU2YAm

Deploying a Redis Cluster through your dedicated Customer Portal

Now, your customers can deploy Redis Clusters with the desired instance type and number of replicas through the dedicated customer portal. The portal will use the REST API parameters to customize the deployment based on the customer's requirements. Cluster Deploy

The final cluster status Cluster status

And the workload status in the Kubernetes Dashboard confirming the placement of pods on independent VMs Kubernetes Dashboard