3

I know that Apache Arrow Parquet can read spec compliant Delta encoded files, but can not write them out. I am wondering if there is any commonly used open source C++/Python library that can write out Parquet spec compliant delta encoding.

cogle
  • 997
  • 1
  • 12
  • 25

1 Answers1

1

There's a Rust library with Python bindings called delta-rs that has a file writer that can take an apache arrow Table or RecordBatch and write to Delta format. Note that it doesn't support transactions or checkpoints yet.

Seems like a pretty active project though, with recent contributions around Delta optimizations so that's cool.

Note: the Delta writer feature of delta-rs is labeled Experimental, so it might not be completely stable.

zyd
  • 833
  • 7
  • 16