Add statistic correction option #5061
Conversation
f60b9bb to
922a9af
Compare
|
How does this deal with different event types? Is that already in the stat parsing machinery, and if so can you remind me where? |
|
Different event types are handled in the config file. SO one specifies something like: (old naming for this argument, but same idea). |
|
Right, that is what I suspected & seems the easiest way of doing this (modulo the case where we would want a different number for the same event type in different coinc times, but that seems not to be needed quite yet). |
Co-authored-by: Thomas Dent <thomas.dent@usc.es>
Co-authored-by: Thomas Dent <thomas.dent@usc.es>
Co-authored-by: Thomas Dent <thomas.dent@usc.es>
Co-authored-by: Thomas Dent <thomas.dent@usc.es>
|
So to me the only remaining concern is around |
I am happy with the redundancy - all the benchmark_lograte means is that the numbers are a bit more natural this way |
Co-authored-by: Thomas Dent <thomas.dent@usc.es>
|
If both the benchmark lograte and this term are used in exactly the same ranking statistics, we ought to be able to get rid of one of them by defining a constant, eg This leaves the output identical but takes out one redundant option which could confuse any users who tried to understand all the optional kwargs. |
|
I guess alternatively, in the spirit of keeping the production branch changes as simple as possible, we could cherry pick this minimal change to the 2.3 branch and then clean up the benchmark redundancy on master with a separate PR. |
* Merge part of 4997 to avoid failure * Bump to v2310 * Compressed waveforms bank workflow (#4969) * Add compressed waveforms to bank workflow * Allolw plotting script to use any bank conversion parameter * Some fixes to allow the joined bank to be plotted * Use inference's parameter labels: they are available and mostly good * Add mismatch to plotting, make some tweaks * some tidying * thinko * Try to make the CI workflow run * Fix do-not-compress default * Use different examples in compress bank workflow * Proper name for the github workflow * Thinko * python shebang in compression workflow script * minor edits * move to readily-available waveform * TRy IMRPhenomD instead * revert change to workflow.core * Warn for KeyError in get_decompressed_waveform * Fix issue with if get_decompressed_waveform raised a ValueError * Combined plotifar (#5034) * added plot script * cleanups * remove now unused bits * Generalize fit plotting * rename script * Added page_farstat in summary (#5052) * Fix release naming * Stat correction patch (#5061) * Limit number of stage output jobs * Reorganize FAR/stat plots on summary page (#5061) * Get the dq files into a nice layout (#5064) * Sphinx version CI fix (#5060) * Update pycbc_page_fars_vs_stat (#5067) --------- Co-authored-by: Gareth S Cabourn Davies <gareth.cabourndavies@ligo.org> Co-authored-by: Thomas Dent <thomas.dent@usc.es> Co-authored-by: Rahul Dhurkunde <rahul.dhurkunde@ldas-pcdev6.ligo.caltech.edu>
* Add statistic correction option * Update pycbc/events/stat.py Co-authored-by: Thomas Dent <thomas.dent@usc.es> * Update pycbc/events/stat.py Co-authored-by: Thomas Dent <thomas.dent@usc.es> * Update pycbc/events/stat.py Co-authored-by: Thomas Dent <thomas.dent@usc.es> * Update pycbc/events/stat.py Co-authored-by: Thomas Dent <thomas.dent@usc.es> * Update pycbc/events/stat.py Co-authored-by: Thomas Dent <thomas.dent@usc.es> --------- Co-authored-by: Thomas Dent <thomas.dent@usc.es>
* Add statistic correction option * Update pycbc/events/stat.py Co-authored-by: Thomas Dent <thomas.dent@usc.es> * Update pycbc/events/stat.py Co-authored-by: Thomas Dent <thomas.dent@usc.es> * Update pycbc/events/stat.py Co-authored-by: Thomas Dent <thomas.dent@usc.es> * Update pycbc/events/stat.py Co-authored-by: Thomas Dent <thomas.dent@usc.es> * Update pycbc/events/stat.py Co-authored-by: Thomas Dent <thomas.dent@usc.es> --------- Co-authored-by: Thomas Dent <thomas.dent@usc.es>
This patch adds a
statistic_correction"statistic keyword" instat.py. This option will add a constant factor to all returned statistic values.The immediate problem this patch sets out to fix is this plot (from https://inspirehep.net/literature/2053424)
This shows the desired behaviour using the state of the art statistic as of a few years ago. In particular HL doubles lie above other things.
When adding the new KDE term we see this relative behaviour change and H/L singles lie above the doubles line, causing singles to be weighted more than HL doubles when combining. Using this option we can downrank singles relative to doubles (and add appropriate factors when considering 3-ifo behaviour).
However, the patch is broader than that, it could be used (for e.g.) to try to make
0correspond to a 50/50 chance of being real ... Or any other thing you could dream up where we just want to scale the statistic.I've tested that this produces the desired effect.