High-Level Architecture

Root Orchestrator#

The root orchestrator is the centralized control plane that coordinates the participating clusters.

The image above illustrates the components of the root orchestrator. Each component operates as an independent service, integrated and managed using the Docker Compose plugin.

• The System Manager serves as the primary interface for users to access the system as an application deployment platform. It provides two sets of APIs:

  1. To receive deployment commands from users via CLI, Dashboard, or directly via REST API.
  2. To handle child Oakestra Clusters.

• The Scheduler determines the most suitable cluster for deploying a given application.

• Mongo acts as the interface for database access. The root manager stores aggregated information about its child clusters. Oakestra categorizes this data into:

  1. Static metadata—such as IP addresses, port numbers, cluster names, and locations.
  2. Dynamic data—such as the number of worker nodes per cluster, total CPU cores and memory, disk space, GPU capabilities, etc.

• The Resource Abstractor standardizes resource management by abstracting generic resources into a unified interface. Whether managing clusters or workers, this abstraction ensures interoperability of scheduling algorithms between root and child clusters. Additionally, it provides an interface for managing the service lifecycle.

• Grafana offers a dashboard with global system alerts, logs, and performance statistics.

• The Root Network Component manages Semantic IP and Instance IP addresses for each service and the cluster’s subnetworks. Refer to the Networking concepts and manuals for more details.

Cluster Orchestrator#

The cluster orchestrator functions as a logical twin of the root orchestrator but differs in the following ways:

• Worker Management: Unlike the root orchestrator, the cluster orchestrator manages worker nodes instead of clusters.

• Resource Aggregation: The cluster orchestrator aggregates resources from its worker nodes and abstracts the cluster’s internal composition from the root orchestrator. At the root level, a cluster appears as a generic resource with a total capacity equal to the sum of its worker node resources.

• Intra-Cluster Communication: MQTT is used as the communication protocol for the intra-cluster control plane.

Worker Node#

The worker node is the component responsible for executing workloads requested by developers.

A worker node is any Linux machine running two essential services:

• NodeEngine: Manages the deployment of applications based on the installed runtimes.
• NetManager: Provides the networking components required for seamless inter-application communication.

Resilience and Failure Recovery#

Root Orchestrator Failure#

A centralized control plane introduces a single point of failure. Oakestra mitigates this risk by enabling clusters to autonomously satisfy SLAs for deployed applications. Consequently, a Root Orchestrator failure impacts only the following:

• Deployment of new applications: System Manager APIs become unavailable, making it impossible to deploy new workloads.
• Inter-cluster root discovery: Pre-existing P2P communication between worker nodes remains unaffected, but new inter-cluster routes cannot be established.
• Onboarding new clusters: New clusters cannot join the infrastructure.

Cluster Orchestrator Failure#

By design, a cluster orchestrator failure does not impact (i) other clusters or (ii) workloads already running on worker nodes. However, the following limitations arise:

• The cluster cannot accept new workloads.
• The cluster cannot onboard new worker nodes.
• The cluster cannot reschedule failed workloads.
• Inter-cluster route discovery is disabled, though pre-existing connections are preserved.

Worker Node Failure#

Worker node failures are common and expected in edge environments. The cluster handles such failures as follows:

• Workloads affected by the failure are automatically re-deployed on other worker nodes.
• Failures are detected using a heartbeat mechanism. If a worker node is unresponsive for more than 5 seconds, the cluster scales up and reallocates the workload.