v0.4.1
What's Changed
- add timestamp to workflow by @rsriya in #192
- Transfer data to local after processing by @dachengx in #194
- Download log of jobs by @dachengx in #196
- Untar outpus if transferred to local by @dachengx in #197
- Check if there is no crucial user installed package by @dachengx in #198
- Upload miscellaneous information to DB by @dachengx in #199
- Ignore processed data while processing by @dachengx in #200
- Download
raw_records*from onlyraw_records_rseby @dachengx in #201 - Use dependency tree for instruction of processing by @dachengx in #202
- [pre-commit.ci] pre-commit autoupdate by @pre-commit-ci in #204
- Limit the number CPU usage of JAX and TensorFlow by @dachengx in #209
- Optimize the order of data_types in processing by @dachengx in #210
- Clearly list the
data_typeto process when submitting job by @dachengx in #211 - If self-installed package, do not upload to rucio by @dachengx in #212
- Set environmental variable of CPUs usage for all jobs by @dachengx in #213
- Still need local sites even if do not transfer to local by @dachengx in #214
- Set defualt
dagman_maxjobsto be 5000 by @dachengx in #215 - Fix $lte for single run by @shenyangshi in #217
- Ask reporter to include the
Outsourcesection ofXENON_CONFIGby @dachengx in #222 - Forbid using development container by @dachengx in #221
- No per-chunk processing of MV by @dachengx in #223
- Force renew
.dbtokenbefore workflow submission by @dachengx in #224 - Limit the filename in 255 characters by @dachengx in #226
- Allow staging-out lower
data_typesand keeping downloadedraw_records*by @dachengx in #228 - Decide whether the dependency exists based on specified RSE by @dachengx in #229
- Add script to test the resources usage: memory, time, storage by @dachengx in #230
- Automatically assign resources by @dachengx in #231
- Fix bug when multiple json files exists by @dachengx in #232
- Keep the folder being tarred by @dachengx in #233
- Stage out lower, combine or upper level outputs by @dachengx in #234
- List a directory in DAV by @dachengx in #235
- FIx bug that non-LED
data_typesis not filtered out in LED mode by @dachengx in #236 - Assign the resources usage of LED calibration by @dachengx in #237
- Allow multiple
raw_records*RSEs by @dachengx in #238 - Set same requirements for lower level and combine jobs by @dachengx in #239
- Only extract input and output folders by @dachengx in #240
- Prioritize the first assign RSE by @dachengx in #241
- Allow multiple per-chunk processing by @dachengx in #242
- Small updates and fixes by @cfuselli in #244
- Change logic for EU sites by @cfuselli in #245
- Print locations of
raw_records*by @dachengx in #243 - Lower memory usage of lower level processing by @dachengx in #246
- Revert part of #244 and #245 by @dachengx in #247
- Set special memory usage for LED calibration runs by @dachengx in #249
- Add constraint for
SURFSARA2_USERDISKby @dachengx in #250 - Set lowest disk requirement as 200MB by @dachengx in #251
- Update memory usage of upper level jobs by @dachengx in #252
- Save lineage of data_types in
summary.jsonby @dachengx in #253 - Copy
.xenon_configto avoid its change after the job submission by @dachengx in #254 - Remove "download" jobs because they are not used by @dachengx in #255
- Add option to first stage the rucio files then download by @dachengx in #256
- Piecewise function for NV disk usage by @dachengx in #257
- Add suffix of combine and untar jobs by @dachengx in #258
- Piecewise function for NV disk usage again by @dachengx in #259
- Update memory usage of NV data by @dachengx in #260
- Copy local files to workflow folder to avoid overwritten by @dachengx in #261
- First
dagman_static_retryretries will use the same resources by @dachengx in #262 - Bump version: 0.4.0 → 0.4.1 by @dachengx in #263
New Contributors
- @pre-commit-ci made their first contribution in #204
- @shenyangshi made their first contribution in #217
- @cfuselli made their first contribution in #244
Full Changelog: v0.4.0...v0.4.1
Contributors
rsriya, dachengx, and 3 other contributors