Utilities for reading and writing

Functions

xr_fresh.io.WriteStackedArray(src: DataArray, file_path='/tmp/test.parquet')[source]

Writes stacked ie. flattened by (y,x,time) to parquet in chunks.

Parameters:

  • src (xr.DataArray) – [description]

  • file_path ([type], optional) – [description], defaults to “/tmp/test.parquet”:path

xr_fresh.io.parquet_append(file_list: list, out_path: str, filters: list)[source]

Read, filter and append large set of parquet files to a single file. Note: resulting file must be read with pd.read_parquet(engine=’pyarrow’)

See read_table docs

Parameters:

  • file_list (list) – list of file paths to .parquet files

  • out_path (str) – path and name of output parquet file

  • filters (list) – list of