Load Balancing

The load balancing in Oakestra consists of a multistep address conversion protocol using semantic addresses as part of an overlay network.

Semantic Addressing

The semantic addressing happens on the service layer. Similar to a cluster IP in Kubernetes, these addresses reference all the instances (replicas) of a microservice with a single address. This address does not change when scaling up the instances or when migrating them. Unlike the Kubernetes cluster IP, when deploying a service in Oakestra, the platform provides as many Service IP addresses as the number of balancing policies supported (and active) within the platform.

Example

Service A
Instance 1
Service B
Instance 1
Service B
Instance 2
Service B
Instance 3
Service B Service IP
- Closest:  fdff:1000::1
10.30.0.1
 - RoundRobin: fdff:2000::1
10.30.0.2
Namespace IP
10.18.3.1
fc00::18:3:1
Namespace IP
10.21.3.2
fc00::21:3:2
fdff:1000::1
fc00::18:3:1
fdff:2000::1
fdff:2000::1
fc00::21:3:2
Oakestra Network Component
Proxy
Conversion Table
Instance IP
10.30.0.3
fdff::3
Instance IP
10.30.0.4
fdff::4
Instance IP
10.30.0.5
fdff::5
Instance IP
10.30.1.5
fdff::105
Namespace IP
10.101.3.23
fc00::101:3:23
fdff::5
10.30.0.1
fc00::41:3:7
10.18.3.1
Service A
Instance 1
Service B
Instance 1
Service B
Instance 2
Service B
Instance 3
Service B Service IP
- Closest:  fdff:1000::1
10.30.0.1
 - RoundRobin: fdff:2000::1
10.30.0.2
Namespace IP
10.18.3.1
fc00::18:3:1
Namespace IP
10.21.3.2
fc00::21:3:2
Namespace IP
10.41.3.7
fc00::41:3:7
fdff:1000::1
fc00::18:3:1
fdff:2000::1
fdff:2000::1
fc00::21:3:2
Oakestra Network Component
Proxy
Conversion Table
Instance IP
10.30.0.3
fdff::3
Instance IP
10.30.0.4
fdff::4
Instance IP
10.30.0.5
fdff::5
Instance IP
10.30.1.5
fdff::105
Namespace IP
10.101.3.23
fc00::101:3:23
fdff::5
10.30.0.1
fc00::41:3:7
10.18.3.1

A developer can communicate with any instance of Service B either with Round Robin balancing policy or the Closest balancing policy. The former balances the traffic evenly between all the instances, and the latter finds the geographically closer instance.

Click on the following tabs to see how every request is handled.

Service A performs the first request using the ServiceIP fdff:1000::1 representing the closest instance balancing policy. The network components’ proxy converts the address to the Namespace IP of Service B Instance 1, which looks like it is the geographically closer service. The message will be, therefore transparently delivered to the closest instance of Service B. Note that the Namespace IP is the real address of the instance, the one provisioned at deployment time, unique and dynamic. An application developer never sees or uses this address, as it depends on the subnetwork of the specific machine where the service is deployed.
Service A performs a second request using the ServiceIP fdff:2000::1 representing the Round Robin balancing policy. The network components’ proxy converts the address to the Namespace IP of Service B Instance 3, which is randomly chosen among all the available instances.
When Service A performs a third request using again the ServiceIP fdff:2000::1 representing the Round Robin balancing policy, this time, the network components’ proxy randomly chooses Service B Instance 2.

The fourth request from Service A to Service B uses the Instance IP representing instance 3.

The proxy component then automatically chooses Service B Instance 3.

The last request demonstrates how the requests using the IPv4 Round Robin semantic address works exactly the same way as the IPv6 Round Robin semantic address used in the first request.

Reminder

Currently the IPv6 Closest balancing strategy IP addresses (reserved, but not implemented) are in the fdff:1000::/21 subnet, and IPv6 Round Robin in fdff:2000::/21. In IPv4, both balancing strategies take their IPs from the 10.30.0.0/16 subnet. For further information, take a look at the IPv4 or IPv6 address space documentation pages.

Mixing IP protocols

6to4 or 4to6 service translation is not supported (yet).

Why do we need Instance IPs?

Instance IPs represent a service’s instance uniquely within the platform. Even when the instance migrates toward new devices, the Instance IP always represents the instance and not the machine where the instance is deployed. The Instance IP is the foundation that enables an overlay network that abstracts services from machines.

When forwarding the packet, the proxy uses the sender’s Instance IP in the from header field of the packet. This way, any response or connection-oriented protocol can transparently work, and we guarantee the original sender receives the response.

Why Service IPs? Why do we need multiple balancing policies?

At the Edge, Oakestra’s net component enables flexibility in the way developers can balance the traffic without the requirement of adapting the code. Just by using a Service IP instead of a regular IP, a developer can achieve balancing by using any protocol based on UDP or TCP and can also customize the balancing behavior of each request accordingly to their need. Edge computing brings resources closer to the users, so one might need to forward some traffic with very low latency using Closest balancing policy, or one might just want to evenly balance another endpoint with Round Robin policy.

Prev
IPv6 Addressing
Next
Proxy Conversion