Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add support for multiple directors / HA federations #2023

Open
wants to merge 9 commits into
base: main
Choose a base branch
from

Conversation

bbockelm
Copy link
Collaborator

Add preliminary support for having multiple directors in a single federation.

This PR allows the services in a federation to calculate all the available directors in a federation and advertise to all available ones. If the director knows of more directors than the service, it will also forward received ads to the director.

This way, all available directors - whether configured statically or discovered through advertisement - will eventually receive all ads. It's left as an exercise for the operations to figure out how to allow fallback of the default URLs to a different director if one goes down.

@bbockelm bbockelm linked an issue Feb 16, 2025 that may be closed by this pull request
@bbockelm bbockelm added enhancement New feature or request cache Issue relating to the cache component origin Issue relating to the origin component director Issue relating to the director component labels Feb 17, 2025
When `Range` was called on an empty ttlcache object, the older version
of ttlcache would trigger a process panic.
We will need version information initialized in the `server_structs`
module; having `server_structs` include the `config` module for the
version information causes a circular dependency.

This factors out the version-related calls (keeping backward API
compatibility) into a new module, breaking the dependency.
With this, an empty configuration string indicates to use the default.
The common base includes the name -- but also the version number,
a process/instance identifier, and a generation number.

The instance ID and generation number provide a total ordering for
all ads generated by the name/type of ad.  This will allow a process
to compare any two ads and determine which one is the "newer" of the
two.  This will allow the director to keep the right ad in memory
even if it receives them out-of-order.
The director ad needs to be able to determine its own name, just like
the cache and origin.  Pull these into a single location.

There's a need for future improvements -- the service should persist
its currently registered name -- but this simple refactor should suffice
for this line of work.
This provides a helper function to ensure we consistently calculate
the audience value for our token configuration.
Provide a framework for any service to detect the presence of multiple
director services in a federation, whether from static configuration,
from metadata discovery, or from querying known directors.  All the
sources are merged together to create one set of known directors.

Once the directors are known, and advertisements from the service
are sent to all directors directly from the service.

Additionally, the directors will forward the ads they receive from
a service to all other known directors.  This prevents a director
from missing updates just because the service hadn't discovered it
yet.

The directors will compare the generation and instance IDs of the
incoming ads to remove duplicates and ensure they only keep the
latest copy of a given service description.

A simple unit test is included.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
cache Issue relating to the cache component director Issue relating to the director component enhancement New feature or request origin Issue relating to the origin component
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add support for multiple directors for high availability
1 participant