-
Notifications
You must be signed in to change notification settings - Fork 1.5k
Use Validator List (VL) cache files in more scenarios #5323
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: develop
Are you sure you want to change the base?
Changes from all commits
e1f8395
d78aacb
f156874
c7f6cd7
c1b1b05
f6f6620
d6821b9
bb37a5b
93e1c9a
039a07e
14cf0df
0bf4cd7
965bd77
0626bff
8737023
6f71852
6bb6706
da2fd89
0d77314
70ce609
7e7a7b1
2173aef
896bc73
366c4fc
ed9c606
489d234
103b78b
c00bc94
9c315ba
9e92413
File filter
Filter by extension
Conversations
Jump to
Diff view
Diff view
There are no files selected for viewing
Original file line number | Diff line number | Diff line change |
---|---|---|
|
@@ -148,7 +148,12 @@ ValidatorSite::load( | |
{ | ||
try | ||
{ | ||
sites_.emplace_back(uri); | ||
// This is not super efficient, but it doesn't happen often. | ||
bool found = std::ranges::any_of(sites_, [&uri](auto const& site) { | ||
return site.loadedResource->uri == uri; | ||
}); | ||
if (!found) | ||
sites_.emplace_back(uri); | ||
} | ||
catch (std::exception const& e) | ||
{ | ||
|
@@ -210,6 +215,17 @@ ValidatorSite::setTimer( | |
std::lock_guard<std::mutex> const& site_lock, | ||
std::lock_guard<std::mutex> const& state_lock) | ||
{ | ||
if (!sites_.empty() && // | ||
There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more. Would it be better if we call There was a problem hiding this comment. Choose a reason for hiding this commentThe reason will be displayed to describe this comment to others. Learn more.
The point of the new check here in That can be possible because I could configure the sites list to request from multiple URLs that mirror the same VL. Someone could set up an aggregator that returns multiple VLs from different publishers, and I could use that. I could define a key that doesn't actually have a site to publish their VL, and instead relies on propagation through the P2P network. I simulated that example above where I commented out one of the URLs in the config file. |
||
std::ranges::all_of(sites_, [](auto const& site) { | ||
return site.lastRefreshStatus.has_value(); | ||
})) | ||
{ | ||
// If all of the sites have been handled at least once (including | ||
// errors and timeouts), call missingSite, which will load the cache | ||
// files for any lists that are still unavailable. | ||
missingSite(site_lock); | ||
} | ||
|
||
auto next = std::min_element( | ||
sites_.begin(), sites_.end(), [](Site const& a, Site const& b) { | ||
return a.nextRefresh < b.nextRefresh; | ||
|
@@ -322,13 +338,16 @@ ValidatorSite::onRequestTimeout(std::size_t siteIdx, error_code const& ec) | |
// processes a network error. Usually, this function runs first, | ||
// but on extremely rare occasions, the response handler can run | ||
// first, which will leave activeResource empty. | ||
auto const& site = sites_[siteIdx]; | ||
auto& site = sites_[siteIdx]; | ||
if (site.activeResource) | ||
JLOG(j_.warn()) << "Request for " << site.activeResource->uri | ||
<< " took too long"; | ||
else | ||
JLOG(j_.error()) << "Request took too long, but a response has " | ||
"already been processed"; | ||
if (!site.lastRefreshStatus) | ||
site.lastRefreshStatus.emplace(Site::Status{ | ||
clock_type::now(), ListDisposition::invalid, "timeout"}); | ||
} | ||
|
||
std::lock_guard lock_state{state_mutex_}; | ||
|
Uh oh!
There was an error while loading. Please reload this page.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Do we want to use a
set
or anunordered_set
instead?There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Possibly. The main reason I didn't is that there are several places that access individual items in
sites_
using the vector index (i.e.operator[]
). See, for example,ValidatorSite::makeRequest
.It could be rewritten to use a
set
-based class and some other lookup method, but as I mentioned in the PR description, I decided it wasn't worth it because:This saved some effort (e.g. I am lazy), and kept the PR small and easier to reason about.