Serverless¶

Overview¶

Omnistrate enables your Resources to have advanced auto-scaling capabilities and scale down to zero with just a few clicks. This allows you to save on infrastructure costs when your customers are not actively using the underlying infrastructure.

Additionally, Omnistrate allows you to extend your Resources to behave completely serverless, including preserving any session state. This ensures that your client connections remain unaware when you automatically pause and resume your infrastructure.

How serverless works¶

Omnistrate's serverless implementation uses an intelligent proxy layer to enable seamless scale-to-zero functionality for your Resources. Here's how it works:

Architecture overview¶

When you enable serverless for a Resource, Omnistrate automatically deploys a lightweight proxy in front of your application. This proxy serves as the stable endpoint for client connections while your backend Resources can scale down to zero during periods of inactivity.

Client → Proxy (Always Running) → Backend Resource (Scales 0-N)

Scale down to zero process¶

1. Monitoring: The proxy continuously monitors activity on the target port(s) of your Resource
2. Idle detection: When no active connections are detected for the configured period, the proxy signals Omnistrate to scale down the backend
3. Graceful shutdown: Your Resource instances are gracefully stopped, reducing infrastructure costs to zero
4. Proxy standby: The proxy remains active, listening for new connection attempts

Wake up process¶

1. Connection attempt: A client attempts to connect to your Resource through the proxy endpoint
2. Automatic provisioning: The proxy detects the incoming connection and signals Omnistrate to provision backend Resources
3. Warm pool (optional): If configured, a pre-warmed instance from the pool is immediately allocated, reducing startup time from minutes to seconds
4. Connection establishment: Once the backend is ready, the proxy establishes the connection and forwards traffic
5. Transparent to clients: The client experiences a brief connection delay but remains unaware of the underlying scale-to-zero operation

Benefits of the proxy architecture¶

• Stable endpoints: Your clients always connect to the same proxy endpoint, regardless of backend state
• Seamless scaling: Backend Resources can scale up and down without disrupting client connectivity
• Cost optimization: Only pay for compute resources when actively in use
• Session preservation (Advanced mode): The proxy can maintain session state during scale-down operations

Scale down to zero¶

Autoscaling requires you to have a minimum of one machine, but if you want to scale down to zero and automatically restart machines during load, you can enable the serverless capability for the corresponding Resource instances.

Enabling serverlessConfiguration will enable auto-stop and auto-wakeup capabilities for your Resource with just one click. There are two serverless modes: Basic and Advanced.

Configuration¶

You can enable serverless in basic mode for one or more Resources by adding x-omnistrate-capabilities. Here is an example using compose spec:

services:
  app: 
    x-omnistrate-capabilities:
      serverlessConfiguration:
        enableAutoStop: true
        targetPort: 3306

The default serverless mode only works for TCP-based services and uses the number of active connections to a target port as a metric to scale down to zero. For custom metrics, see Advanced serverless for more details.

Testing serverless functionality¶

Create an instance of the serverless Resource. Once it's up and running, check the connectivity.

Wait for the configured period of inactivity for the infrastructure to be automatically stopped. You can confirm this by checking the state of the Resource instance.

As soon as new activity resumes, the underlying Resource instance should start automatically within several seconds.

Faster wakeup using warm pool¶

To speed up the wakeup time, Omnistrate offers a warm pool capability that you can enable for faster resume times.

    minimumNodesInPool: 5

• minimumNodesInPool: Configure a static number of machines in the pool. You can change this value at any time programmatically through APIs or manually through the UI.

Demo Example¶

Here is a demo example using PostgreSQL:

Achieving full elasticity¶

Scale down to zero enables your underlying software to go from 0 to 1 when in use and back to 0 when not in use. However, in practice, you may want to scale from 0 to N as the load increases and back to 0 when not in use.

To achieve this, enable auto-scaling in addition to scale down to zero. Here is an example configuration using compose spec:

services:
  app: 
    x-omnistrate-capabilities:
      autoscaling:
        maxReplicas: 5
        minReplicas: 1
        idleMinutesBeforeScalingDown: 2
        idleThreshold: 20
        overUtilizedMinutesBeforeScalingUp: 3
        overUtilizedThreshold: 80
        scalingMetric:
          metricEndpoint: "http://localhost:9187/metrics"
          metricLabelName: "application_name"
          metricLabelValue: "custom_metric_label"
          metricName: "custom_metric_name"
      serverlessConfiguration:
        targetPort: 3306
        enableAutoStop: true
        minimumNodesInPool: 5

Advanced serverless¶

You may need advanced serverless if you require any of the following capabilities:

• Resume client state after scaling to zero: For example, if you want to scale down the infrastructure for your database application on idle connections but have prepared statements or temporary tables as part of the session state, the default behavior will lose all connection state and client connections will fail. With advanced serverless, you can store the connection state so that on resume, clients continue to behave the same way, making them completely unaware of the scale down to zero.
• Custom metrics: Use custom metrics to determine when to scale down to zero or resume.
• Proxy configuration control: Take control of the proxy configuration, including its image, infrastructure, capabilities, and managing the lifecycle of the proxy Resource.

To enable advanced serverless using compose spec:

services:
  app: 
    serverlessConfiguration:
      referenceProxyKey: "proxyResourceKey"
      portsMappingProxyConfig: 
        numberOfPortsPerCluster: 4
        maxNumberOfClustersPerProxyInstance: 1
      proxyStorageConfig:
        aws:
          storageType: "s3"

• referenceProxyKey: Reference to the proxy Resource that will be used to listen to client traffic
• proxyStorageConfig: State store to keep the connection state(s)
• portsMappingProxyConfig: Configurations to map proxy Resource ports to the database application

In summary, if you require any customization, you will likely need advanced mode. For simple use cases, the basic serverless mode should suffice.