I'm new to zarr, and trying to use Xarray to output files to zarr.storage.DBMStore. Since, the dataset is quite large, I calculate and output the results to one zarr.storage.DBMStore multiple times (append dim='time'). However, everytime when the program failed because of timeout (I request 4hours from the queue every time), I seems to unable to access the files that have already been written to this zarr store during previous times. I wonder how to dealing with issues like this?
Below is my code to write and read files to and from zarr store, and also the error message after a timeout attempt:
import zarr
import xarray
store= zarr.DBMStore('/scratch/brown/kong97/WBGT/cmip6/GFDL_CM4_WBGT.bdb', open=bsddb3.btopen)
#doing some calculations and get the output: WBGT
WBGT.to_zarr(store,append_dim='time')
store.close()
After a error of timeout:
store= zarr.DBMStore('/scratch/brown/kong97/WBGT/cmip6/GFDL_CM4_WBGT.bdb', open=bsddb3.btopen)
test=xr.open_zarr(store)
raise error: DBPageNotFoundError: (-30985, 'BDB0075 DB_PAGE_NOTFOUND: Requested page not found')
Appreciate any help!
