Operational insights
Debug Events¶
Omnistrate’s Debug Events feature provides real-time, detailed insights into your deployments, allowing you to quickly pinpoint and resolve issues by offering a transparent view of each operation’s progress and status.
How It Works¶
For each instance operation—such as create, modify, upgrade, start, stop, or delete—Omnistrate launches a dedicated workflow to carry out the task. This workflow moves through several defined stages, including bootstrap, storage, network, compute, deployment, and monitoring. Within each stage, actions are tracked as individual debug events in chronological order, giving a precise view of each action's execution status. This structure enables you to monitor the workflow’s progression step-by-step and quickly locate the root cause of any issues.
Example Scenarios¶
When a workflow encounters an issue, such as an invalid instance type parameter, debug events identify the specific error within the workflow. This detailed view enables you to diagnose and resolve issues swiftly, ensuring minimal disruption.
By utilizing Omnistrate’s Debug Events, you can streamline issue resolution, improve operational efficiency, and ensure a smoother service experience.
Debug Mode¶
Omnistrate's Debug Mode allows you to interact with ongoing deployments in real-time. This powerful feature enables you to quickly troubleshoot and fix issues without restarting or upgrading your entire deployment from scratch. Debug Mode is ideal for testing complex deployment scripts or implementing urgent fixes while maintaining deployment continuity, saving you valuable time and resources.
How It Works¶
First, set up the proper Kubernetes context to your dataplane cluster, you can go to instance details page and see commands in K8s Config tab.
After context is properly setup, then run to access operational bastion:
kubectl -n dataplane-agent exec -it $(kubectl -n dataplane-agent get pods | grep dp-agent | awk '{print $1}' | head -1) -- /bin/bash
To enable debug mode for instance deployment:
Once debug mode is enabled, retrieve the current deployment artifacts:
omnistrate-ctl instance get-deployment <instance-id> --resource-name <service component name> --output-path <path to workspace to save deployment artifacts>
Apply your modifications to the deployment:
omnistrate-ctl instance patch-deployment <instance-id> --resource-name <service component name> --deployment-action apply --patch-files <path to workspace to save deployment artifacts>
When satisfied with your changes, resume the deployment workflow:
omnistrate-ctl instance continue-deployment <instance-id> --resource-name <service component name> --deployment-action apply
The system will then continue with the deployment process, progressing through any remaining deployment steps.
Warning
Debug mode restricts instance operations to only delete and upgrade functions. After resuming deployment, you'll need to properly correct the service plan specification and upgrade your instance, then manually disable debug mode to restore all instance operations.
Next, fix the service plan spec and publish a new version. Then upgrade your instance to this updated version. After verifying that everything is functioning properly, disable debug mode to resume normal operations.
Example Scenarios¶
Taking terraform as an example, you may encounter a typo in your terraform deployment scripts causing the deployment to stall. By following the steps above, you can quickly fix these scripts without needing to restart the entire deployment process.
Once you've fixed your terraform artifacts locally and patched your changes, the Omnistrate platform applies the patch immediately without unnecessary retries. You can monitor terraform execution logs at /tmp/tf-<service component id>-<instance-id>-log-output/<service component version>
You can continue to iterate and refine your terraform scripts in debug mode while maintaining the ongoing deployment. After resolving all issues and confirming your changes work as expected, simply continue to complete the deployment.
Warning
Don't forget to fix your main service plan spec and release a new version of it once you have proper artifacts been fixed. You will need to upgrade your instance to the correct version before you safely disable debug mode.