Issue during the closing of xarray open with dask backend #11960

LouisPauchet · 2025-05-21T11:30:39Z

Hello,

I didn't manage to create an example to reproduce the bug in few lines, but the complete is example is available on GitHub : https://github.com/MET-OM/metocean-api and https://github.com/MET-OM/metocean-api/actions/runs/15142723801.

When having the test processing, they randomly pass and randomly fail at the closing of dataset open in a with statement.

import xarray as xr
from dask.diagnostics import ProgressBar

tempfiles = [...] # list of NetCDF4 files

with xr.open_mfdataset(tempfiles, parallel=True, engine="netcdf4") as ds, ProgressBar():
            ds.load()
            if save_nc:
                
                # Save the unaltered structure
                ds = ds.sel({"time": slice(ts.start_time, ts.end_time)})
                save_to_netcdf(ds, ts.datafile.replace(".csv", ".nc"))

            df = self.create_dataframe(
                ds=ds,
                lon_near=lon_near,
                lat_near=lat_near,
                outfile=ts.datafile,
                start_time=ts.start_time,
                end_time=ts.end_time,
                save_csv=save_csv,
                **flatten_dims,
            )

def save_to_netcdf(ds, outfile):
    remove_if_datafile_exists(outfile)
    ds.to_netcdf(outfile)
    print(f"NetCDF file created at {outfile}")

Here are the logs when running the same test command 'pytest -v --full-trace >>log4.txt'.

log4.txt

I absolutly don't know what is happening and it's maybe not really a bug but a not complete understanding of what is happening.

Environment:

Dask version: 2025.5.1
xarray version: 2025.4.0
Python version: 3.13.3
Operating System: Ubuntu 24.04.2 LTS / Windows (both same behaviour)
Install method (conda, pip, source): conda forge

Thank you very much for your help

github-actions bot added the needs triage Needs a response from a contributor label May 21, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Issue during the closing of xarray open with dask backend #11960

Issue during the closing of xarray open with dask backend #11960

Issue during the closing of xarray open with dask backend #11960

Issue during the closing of xarray open with dask backend #11960

Comments