JP-3635: Account for pixel area ratio in outlier detection blotting #8553
Conversation
|
Regression tests started here. edit: a second regtest run started because the first one had some 100 unrelated failures |
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #8553 +/- ##
==========================================
+ Coverage 58.02% 58.54% +0.52%
==========================================
Files 388 388
Lines 38977 38963 -14
==========================================
+ Hits 22617 22812 +195
+ Misses 16360 16151 -209 ☔ View full report in Codecov by Sentry. |
|
@mairanteodoro Does romancal need a similar fix? |
|
Looks good to me; as a test I tried deliberately messing up the header PIXAR keywords for some MIRI data by 20% to really throw off the outlier detection routine. In the image attached the left panel is the original drizzled i2d image (with good PIXAR values), the middle panel is the existing jwst code working on the data with bad PIXAR values, and the right panel is this PR working on the data with bad PIXAR values. As expected the current code does really bad things because the scaling isn't being applied when blotting. This PR looks like it cleans that up really well, and the result looks identical to the original (except, of course, for the fluxcal error that I introduced by hacking the PIXAR keywords). |
|
Most of the 80 or so regtest failures stem from The |
Currently, spectral data does not have iscale computed/applied in resampling, so it shouldn't be applied in blotting either. You'll need to check for |
|
started another regtest run with the suggested change |
|
@emolter Your latest regtest run is still corrupted with unrelated changes due to 1) new NIRCam photom ref files and 2) changes to the jump step in stcal. The truth files for the NRC photom related tests have been updated now, and a new version of stcal (1.7.2) has been released that contains the jump updates (and truth files updated). So try running again, being sure you're using stcal 1.7.2 (or stcal/main). |
edit: There are now 8 failures left, and I believe that they are all expected: I checked the value of The fraction of pixels that differ is <~0.01% for all the i2d files (nircam_image, miri_image, nircam_mtimage, except for test_image3_closedfile which shows a 1% fraction. |
Resolves JP-3635
Closes #8509
This PR addresses a small bug in outlier detection: resample accounted for the difference between the nominal pixel area in steradians used for the photometric calibration (img.meta.photometry.pixelarea_steradians) and the as-computed pixel area from the WCS when resampling cal files. That wasn't performed in reverse when blotting median images back to match those individual cal files. This PR applies that reverse ratio so that the total flux is conserved between input models and blotted models.
Checklist for PR authors (skip items if you don't have permissions or they are not applicable)
CHANGES.rstwithin the relevant release sectionHow to run regression tests on a PR