8000 Use Validator List (VL) cache files in more scenarios by ximinez · Pull Request #5323 · XRPLF/rippled · GitHub
[go: up one dir, main page]
More Web Proxy on the site http://driver.im/
Skip to content

Use Validator List (VL) cache files in more scenarios #5323

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 30 commits into
base: develop
Choose a base branch
from
Open
Changes from all commits
Commits
Show all changes
30 commits
Select commit Hold shift + click to select a range
e1f8395
Use Validator List (VL) cache files in more scenarios
ximinez Feb 26, 2025
d78aacb
Merge remote-tracking branch 'upstream/develop' into ximinez/fix/vali…
ximinez Feb 28, 2025
f156874
Merge remote-tracking branch 'upstream/develop' into ximinez/fix/vali…
ximinez Mar 11, 2025
c7f6cd7
Merge remote-tracking branch 'upstream/develop' into ximinez/fix/vali…
ximinez Mar 11, 2025
c1b1b05
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Mar 12, 2025
f6f6620
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Mar 17, 2025
d6821b9
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Mar 19, 2025
bb37a5b
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Mar 20, 2025
93e1c9a
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Mar 20, 2025
039a07e
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Mar 24, 2025
14cf0df
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Mar 25, 2025
0bf4cd7
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Mar 27, 2025
965bd77
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Mar 31, 2025
0626bff
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Apr 1, 2025
8737023
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Apr 4, 2025
6f71852
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Apr 4, 2025
6bb6706
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Apr 8, 2025
da2fd89
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Apr 9, 2025
0d77314
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Apr 9, 2025
70ce609
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Apr 10, 2025
7e7a7b1
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Apr 10, 2025
2173aef
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Apr 11, 2025
896bc73
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Apr 24, 2025
366c4fc
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Apr 28, 2025
ed9c606
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Apr 29, 2025
489d234
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez May 1, 2025
103b78b
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez May 4, 2025
c00bc94
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez May 14, 2025
9c315ba
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Jul 2, 2025
9e92413
Merge branch 'develop' into ximinez/fix/validator-cache
ximinez Jul 3, 2025
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 21 additions & 2 deletions src/xrpld/app/misc/detail/ValidatorSite.cpp
Original file line number Diff line number Diff line change
Expand Up @@ -148,7 +148,12 @@ ValidatorSite::load(
{
try
{
sites_.emplace_back(uri);
// This is not super efficient, but it doesn't happen often.
bool found = std::ranges::any_of(sites_, [&uri](auto const& site) {
return site.loadedResource->uri == uri;
});
if (!found)
sites_.emplace_back(uri);
Copy link
Collaborator
@a1q123456 a1q123456 Jun 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we want to use a set or an unordered_set instead?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Possibly. The main reason I didn't is that there are several places that access individual items in sites_ using the vector index (i.e. operator[]). See, for example, ValidatorSite::makeRequest.

It could be rewritten to use a set-based class and some other lookup method, but as I mentioned in the PR description, I decided it wasn't worth it because:

ValidatorSite::load checks for duplicate URIs. Since it's a vector, it's not very efficient, but this function is only called at startup and via missingSites, so it won't be called often, and not in any critical path.

This saved some effort (e.g. I am lazy), and kept the PR small and easier to reason about.

}
catch (std::exception const& e)
{
Expand Down Expand Up @@ -210,6 +215,17 @@ ValidatorSite::setTimer(
std::lock_guard<std::mutex> const& site_lock,
std::lock_guard<std::mutex> const& state_lock)
{
if (!sites_.empty() && //
Copy link
Collaborator
@a1q123456 a1q123456 Jun 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better if we call onError when we can't verify the VL or when there's something wrong with the json so that we could catch more cases and potentially log something?

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Would it be better if we call onError when we can't verify the VL or when there's something wrong with the json so that we could catch more cases and potentially log something?

onError is a lambda defined in ValidatorSite::onSiteFetch, and it is called if there's something wrong with the VL, because ValidatorSite::parseJsonResponse throws exceptions, which are handled by calling onError.

The point of the new check here in ValidatorSite::setTimer in particular is to handle the case where there were no problems with the VL downloads from [validator_list_sites], but we still don't have VLs for all the keys defined in [validator_list_keys].

That can be possible because [...sites] and [...keys] are completely independent.

I could configure the sites list to request from multiple URLs that mirror the same VL. Someone could set up an aggregator that returns multiple VLs from different publishers, and I could use that. I could define a key that doesn't actually have a site to publish their VL, and instead relies on propagation through the P2P network. I simulated that example above where I commented out one of the URLs in the config file.

A760
std::ranges::all_of(sites_, [](auto const& site) {
return site.lastRefreshStatus.has_value();
}))
{
// If all of the sites have been handled at least once (including
// errors and timeouts), call missingSite, which will load the cache
// files for any lists that are still unavailable.
missingSite(site_lock);
}

auto next = std::min_element(
sites_.begin(), sites_.end(), [](Site const& a, Site const& b) {
return a.nextRefresh < b.nextRefresh;
Expand Down Expand Up @@ -322,13 +338,16 @@ ValidatorSite::onRequestTimeout(std::size_t siteIdx, error_code const& ec)
// processes a network error. Usually, this function runs first,
// but on extremely rare occasions, the response handler can run
// first, which will leave activeResource empty.
auto const& site = sites_[siteIdx];
auto& site = sites_[siteIdx];
if (site.activeResource)
JLOG(j_.warn()) << "Request for " << site.activeResource->uri
<< " took too long";
else
JLOG(j_.error()) << "Request took too long, but a response has "
"already been processed";
if (!site.lastRefreshStatus)
site.lastRefreshStatus.emplace(Site::Status{
clock_type::now(), ListDisposition::invalid, "timeout"});
}

std::lock_guard lock_state{state_mutex_};
Expand Down
Loading
0