How does e2routerd work?

5.4, 5.5, 6.0, 6.1
How does e2routerd do load balancing?  Is it strictly a round robin, or does it use some other method to distribute traffic to the various render engines?

According to the Vault engineering team,

> Can you quickly sum up how e2router works?
> > Basically a customer is asking does it Round Robin?
e2routerd uses a (somewhat obscure) algorithm to distribute requests.
It is based on 4 weights, in order of priority:
(1) failed requests since last heartbeat (~30 sec)
(2) hold ticks (holds come from previous heartbeats with failures) 
(3) running commands
(4) completed commands
(1) and (2) manage connection failures. Connections that fail during the current heartbeat are avoided. Connections with failures during the last heartbeat get increased hold ticks which act as a delay until the connection is retried.
(3) is the main rule, causing commands to be distributed to rendering engines running the fewest commands. It doesn't know the costs of each command, just the number of commands that were sent and have not completed.
(4) just acts to distribute load across connections under light load. This helps keep the live-ness information about connections fresh. This actually produces around a robin effect but only under light load.
> > In the new configuration do you envisage that the load balancer on top
> > of the render clients will be controlled by consideration of the
> > network traffic, or will it be round robin?  Will we need a separate
> > box / virtual server for the load balancer?
Failure modes aside, e2routerd distributes requests based on the number of requests each rendering engine is currently working on.
This is not round robin. Neither is it balancing based on network traffic (perhaps something to consider as another weight in the future).
UPDATED:  September 18, 2017