Integrations¶

Overview¶

Integrations are 1st/3rd party SaaS applications that you want to integrate with for your SaaS.

We have classified SaaS integrations into the following categories:

Observability: Examples - Datadog, NewRelic, Loggly, Signoz, Axiom

Billing: Examples - Metronome, Amberflo

Continuous Integration: Examples - GitHub Actions, CircleCI
Marketplace: Examples - Clazar, Suger, Feenix, Labra, Tackle
Cloud Insurance: Example - Archera
Operational Status: Examples - Atlassian, BetterStack

If you don't see a category for your SaaS integration and would like us to support, please reach out to us support@omnistrate.com

Supported Integrations¶

Observability¶

Omnistrate supports 2 different 'scopes' for observability features:

Internal - provides visibility for service provider across fleet
Customer - provides visibility for your customer into their instance(s)

Different observability providers are available based on intended scope.

Internal observability¶

Omnistrate native¶

Omnistrate internal observability creates dashboards accessible by service provider on fleet page.

You can enable logging dashboard integration from your compose file using:

x-internal-integrations:
  logs:

Enabling this integration will enable real-time logging for your support team to manage the customer fleet.

You can enable metrics dashboard integration from your compose file using:

x-internal-integrations:
  metrics:

Enabling this integration will enable real-time infra metrics dashboard.

Note

We launch a service deployment in same kubernetes cluster where instances are running, this deployment is responsible to collect, store, aggregate, and stream metrics and logs across all nodes running in the same kubernentes cluster.

To enable logs and metrics respectively through API, call EnableServicePlanFeature with the respective params.

Custom metrics¶

Omnistrate metrics allow you to export additional application metrics. You can add custom metrics for each service component that hosts prometheus exporter by specifying endpoint and list of metrics as follows:

x-internal-integrations:
  metrics:
    additionalMetrics:
      postgres: # service component
        prometheusEndpoint: "http://localhost:9187/metrics"
        metrics: # list of metric names to export
          pg_settings_wal_receiver_status_interval_seconds:
          pg_settings_wal_buffers_bytes:
          pg_settings_tcp_keepalives_interval_seconds:

Warning

additionalMetrics are only supported when using Omnistrate as observability provider. For 3rd party OTEL integrations, see serviceComponentsConfiguration that will allow you to configure additional prometheus exporters.

This is the most basic configuration that will export listed metrics from configured prometheus exporter. In some cases, such basic configuration is not enough. Single metric can have multiple series with different labels. Additional configuration for each metric can be passed to define custom exported series, each with optional aggregation functions and/or labels filtering.

Say you want to export metric "pg_scrape_collector_success", but out of multiple values, you only want to export one where label "collector" has value "database" or "replication":

x-internal-integrations:
  metrics:
    additionalMetrics:
      postgres: # service component
        prometheusEndpoint: "http://localhost:9187/metrics"
        metrics:
          pg_scrape_collector_success: # metric name
            database_scrape_success: # name for 1st custom serie
              labelFilters: # filter defining how to produce custom serie "database_scrape_success"
                collector: database 
            replication_scrape_success: # name for 2nd custom serie
              labelFilters:
                collector: replication

This configuration will export 2 series for same metric, each with corresponding name (i.e: database_scrape_success and replication_scrape_success)

If instead of multiple series, single metric value should be produced, you can specify aggregation function. Aggregation function will produce single value out of all discovered series. 4 aggregations are supported: - Minimum (min) - Maximum (max) - Average (avg) - Sum (sum)

Say you want to export sum of all "pg_stat_activity_count" across all dimensions. You can configure such metric as:

x-internal-integrations:
  metrics:
    additionalMetrics:
      postgres: # service component
        prometheusEndpoint: "http://localhost:9187/metrics"
        metrics:
          pg_stat_activity_count: # metric name
            sum_of_all_activities: # name defined for this custom metric serie
              aggregationFunction: sum

Filters and aggregations can be combined. Filters specify which metric values should be taken based on their labels, while aggregation defines how to produce single output value if there are multiple matches found.

API and UI require custom metric configuration to be passed in JSON form. Here is an example of how to define 2 custom series:

{
  "average_active_activity": {
    "aggregationFunction": "avg",
    "labelFilters": {
      "state": "active"
    }
  },
  "max_postgres_activity": {
    "aggregationFunction": "max",
    "labelFilters": {
      "datname": "postgres"
    }
  }
}

OpenTelemetry providers¶

Supported open-telemetry providers: NewRelic, Signoz, Datadog

Omnistrate allows you to ship metrics to your open telemetry provider account. This will enable you to:

Provide visibility to your customers.
Stage/debug any form of production issues that your customers may be facing while provisioning/operating your SaaS.

Metrics¶

You can enable metrics integration by following the steps below:

Generate an API key from your open telemetry providers account and store them in the cloud native secret manager (for AWS that would be "Secrets Manager").
You can then enable metrics integration by adding the following to your compose specification:

x-internal-integrations:
  metrics:
    provider: <signoz | newRelic | datadog>
    endpoint: <provider endpoint>
    secretLocators:
      gcp: projects/123456789012/secrets/mySecret123456-abc123/versions/latest
      aws: arn:aws:secretsmanager:us-west-2:123456789012:secret:mySecret123456-abc123

The specification includes the following parameters:

Provider name - specifies which open-telemetry provider to use. Supported values:
- signoz
- newRelic
- datadog
Provider endpoint - specify endpoint for open telemetry provider of choice. Examples:
- ingest.us.signoz.cloud:443 - for Signoz. For list of available cloud endpoints, see Signoz docs. You can also export OTEL telemetry to self-hosted Signoz instance.
- https://otlp.nr-data.net - for NewRelic. For list of available endpoints, see NewRelic docs.
- us5.datadoghq.com - for Datadog. This endpoint needs to be one of the datadog ingest endpoints. For more information, see one of the options named "Site Parameter" in Datadog docs.
Providing secret store locator for API key - specify secret identifier from within secret store of used cloud provider.
- For AWS, we expect ARN of secret from within "AWS Secrets Manager"
- For GCP, we expect secret ID (including version) of secret stored in "Secret Manager".

Note

Most otel providers require single API key / token as authentication mechanism. This "key" can be directly stored as secret value and would be used as needed based on the otel provider.

In some scenarios, such as self-hosting or hosting Signoz using Omnistrate, authentication is performed using combination of username and password. In that case, secret value needs to be in form of following json object that provides both: {"username":"replace_with_username","password":"replace_with_password"}

We will automatically add infrastructure metrics. If you want to add your own custom metrics in addition, you can define your own prometheus endpoint (per service component) for us to scrape and add them. You can enable custom metrics by adding the following to your service component:

  metrics:
    serviceComponentsConfiguration:
      postgres: # service component 
        prometheusEndpoint: "http://localhost:9187/metrics" # Add the local host endpoint to supply promotheus metrics.

Omnistrate will automatically scrape the metrics of the services and ship them to your open telemetry provider account.

Logging¶

Logging can be enabled using same configuration parameters, but specifying 'logs' as feature:

x-internal-integrations:
  logs:
    provider: <signoz | newRelic>
    endpoint: <provider endpoint>
    secretLocators:
      gcp: projects/123456789012/secrets/mySecret123456-abc123/versions/latest
      aws: arn:aws:secretsmanager:us-west-2:123456789012:secret:mySecret123456-abc123

Logging integration will upload service logs to open telemetry provider of your choice.

Both metrics/logs can be activated using API method with following format:

{
   "feature": "METRICS",
   "scope": "INTERNAL",
   "configuration": {
      "provider": "newRelic",
      "endpoint": "https://otlp.nr-data.net",
      "secretLocators": {
         "aws": "arn:aws:secretsmanager:us-west-2:123456789012:secret:mySecret123456-abc123",
         "gcp": "projects/my-project/secrets/my-secret/versions/latest"
      },
      "serviceComponentsConfiguration": {
         "postgres": {
            "prometheusEndpoint": ":9090" 
         }
      }
   }
}

Note

Logs use "LOGS" as feature name and will not expect "serviceComponentsConfiguration" parameter.
For metrics, "serviceComponentsConfiguration" parameter with additional prometheus endpoint configuration is optional.

Info

Please note that this integration is only available in the 'Provider hosted' hosting model

Custom OpenTelemetry¶

In case open-telemetry provider of your choice is not supported out of the box, Omnistrate allows you to inject your own configuration for open-telemetry collector. For list of available exporters, see open-telemetry docs.

Custom open-telemetry provider requires entire configuration, including access credentials to be provided as a part of configuration. To configure custom open-telemetry provider, pass yaml configuration as part of the compose file:

x-internal-integrations:
  logs:
    provider: custom 
    template:
      exporters:
        otlp:
          endpoint: custom.endpoint:80 
          tls:
            insecure: false
          headers:
            custom-auth-header: abc11231394

Calling EnableServicePlanFeature API with the following parameters will achieve the same result:

{
  "feature": "LOG",
  "scope": "INTERNAL",
  "configuration": {
    "provider": "custom",
    "template": {
      "exporters": {
        "otlp": {
          "endpoint": "custom.endpoint:80",
          "tls": {
            "insecure": false
          },
          "headers": {
            "custom-auth-header": "abc11231394"
          }
        }
      }
    }
  }
}

Info

Please note that this integration is only available in the 'Provider hosted' hosting model

Cloud native¶

Each cloud provider has its own built-in observability platform (CloudWatch for AWS and Operations Suite for GCP). Omnistrate will automatically configure permissions necessary in order to easily integrate with observability platform of cloud provider you are using. To enable native integration from compose specification, add following configuration for logs:

x-internal-integrations:
  logs:
    provider: native

For metrics:

x-internal-integrations:
  metrics:
    provider: native

To enable logs through API, call EnableServicePlanFeature with the following params:

{
   "feature": "LOG",
   "scope": "INTERNAL",
   "configuration": {
      "provider": "native"
   }
}

Customer observability¶

Omnistrate¶

Omnistrate customer observability creates dashboards accessible by customer on instance detail page.

You can enable logging dashboard integration from your compose file using:

x-customer-integrations:
  logs:

Enabling this integration will enable real-time logging for your customers.

You can enable metrics dashboard integration from your compose file using:

x-customer-integrations:
  metrics:

Enabling this integration will enable real-time infra metrics for your customers. Additional metrics can be configured and added to the customer dashboard (please see internal omnistrate metrics documentation for details).

To enable logs and metrics respectively through API, call EnableServicePlanFeature with the respective params.

Cloud native¶

Your customers' observability is also supported when using the BYOA model. It can be activated in similar fashion as internal native observability using compose spec:

x-customer-integrations:
  logs:
    provider: native 
  metrics:
    provider: native

or via API EnableServicePlanFeature

Omnistrate Metering¶

You can enable Omnistrate Metering integration directly from GUI.

You can also enable metering using:

x-internal-integrations:
  billing:
    S3BucketARN: arn:aws:s3:::billing-bucket-name

Enabling this integration will enable metering from your infrastructure to capture the usage per customer, aggregate and store them at a defined location in your account.

Omnistrate Billing¶

If you want to charge your customers directly, you can enable billing integration. To enable, please see here

Continuous Integration¶

Omnistrate allows you to integrate with GitHub Actions. To integrate, you will need to:

Create GitHub Actions file if you haven't done so
Configure it to build your dockerfile
(Optional) if you want to run any test, configure them accordingly
Finally, configure it to release to Omnistrate using the appropriate service_id and image_id

- name: Publish to Omnistrate
  action: omnistrate/publish-image@v1
  with:
    service-id: <filled in automatically>
    image-config-id: <filled in automatically>
    image-tag: # To be filled in by the CI step prior

Warning

We don't recommend to configure images in Omnistrate with a fixed tag as brings unpredictability.

Let's say, you create an image with a tag "latest" and its underlying version is 1.0. Let's say, you change the underlying version to 2.0 but the tag stays the same. Because, Omnistrate has no knowledge of image versioning, if one of your customers' clusters adds a new node, we will pull the latest image with the tag latest and run that node with 2.0. On the other hand, if the process restart, we may continue to use the cached image and continue to run with version 1.0 on that node.

Our recommendation is to use versioned tags and update the image tags (say with commit id as a suffix) and update your image object every time there is a new change. With this, Omnistrate platform will continue to operate with the older version for the existing resource instances and give you the fine-grained control on when to upgrade those existing resource instances your customers.

If you need help with building docker images quickly, you can checkout depot here and configure it to build your docker image. For other CI toolings, see this

Alerting¶

Omnistrate platform allows you to enable real-time alerting with PagerDuty. Omnistrate will notify you when there is an issue with your SaaS infrastructure and enabling your operators to take action. For example, we can send an alert when your service account is hitting quota limits for creating new resources on the cloud.

To enable PagerDuty alerting, you need to:

Generate a PagerDuty integration key from your PagerDuty account. For further guidance, please refer to the PagerDuty Integration Guidance.
You can enable PagerDuty integration directly from GUI

You will soon be able to enable alerting by adding the following to your compose specification:

x-omnistrate-integrations:
  - integration-type: "Alerts"
    integration-provider: "PagerDuty"
    integration-key: <your-integration-key>

Warning

PagerDuty integration has been deprecated. It was moved to a global notification setting instead. Please see this page on how to configure it.

Marketplace¶

If you want to list your offerings to cloud marketplace, you can enable marketplace integration.

Info

Please note that this integration is only available in the enterprise plan for now.

Cloud Insurance¶

If you want to purchase cloud with unprecedented flexibility, you can enable cloud insurance integration.

Info

Please note that this integration is only available in the enterprise plan for now.

Operational Status¶

If you want to want to keep your customers informed about service status and streamline incident reporting, you can enable this integration by reaching out to us at [email protected].

Info

Please note that this integration is only available in the enterprise plan for now.

Future Integrations¶

We are constantly adding native support for new integrations. If you would like us to prioritize any particular integration, please reach out us at support@omnistrate.com

Note

Please note that you can also directly integrate your favorite tooling yourself as you have full control on the underlying infrastructure

If you looking to partner and integrate your solution with us, please reach out to us at partnerships@omnistrate.com