0

As seen from the documentation here it is possible to use the deltalake::open_table method to open the delta lake table located on the file system. This I have been able to do.

But I have not been able to figure out how to open delta lake table on s3.

The documentation for deltalake::open_table which can be found here says

Creates and loads a DeltaTable from the given path with current metadata. Infers the storage back-end to use from the scheme in the given table path.

I tried to pass in a url to s3 bucket since the documentation says the storage back-end will be inferred. But doing this open_table("s3://127.0.0.1:9000/deltalake/delta-table") leads to this panic

thread 'main' panicked at 'called `Result::unwrap()` on an `Err` value: LoadCheckpoint { source: Storage { source: Generic { store: "S3", source: Error { retries: 0, message: "request error", source: Some(reqwest::Error { kind: Request, url: Url { scheme: "http", cannot_be_a_base: false, username: "", password: None, host: Some(Ipv4(169.254.169.254)), port: None, path: "/latest/api/token", query: None, fragment: None }, source: hyper::Error(Connect, ConnectError("tcp connect error", Os { code: 60, kind: TimedOut, message: "Operation timed out" })) }) } } } }', src/main.rs:71:10
stack backtrace:
   0: rust_begin_unwind

I had a feeling this should be trivial but unfortunately it has not been :(

So anyone knows how to get open_table to open a table located on an s3 bucket?

Finlay Weber
  • 2,989
  • 3
  • 17
  • 37
  • Is your table really called `detla-table`, not `delta-table`? And is it really on `127.0.0.1`? – Jmb Feb 20 '23 at 07:47
  • Thanks for spotting that, But corrected the typo and still errors – Finlay Weber Feb 20 '23 at 07:59
  • The full `stack backtrace` output is likely to be enlightening, you should post it. Also, have you enabled the `s3` feature on your deltalake dependency? – Caesar Feb 20 '23 at 08:12

0 Answers0