0

For an existing library, what is the best (or at least good) way to structure the project when creating Python bindings via pyo3 / maturin ?

So far I figure I can either just annotate all the structs and impls with #[pyclass] and #[pymethods] directly in the original library along with a #[pymodule] fn, but this changes the return types and litters the original code with the additional attribute annotations.

The other option I can see is to basically copy + paste the library into a new lib with the appropriate Cargo.toml and pyproject.toml, and annotate all the desired structs/impls etc there instead. The benefit of this is you retain the original library as it was, but the downside is that for every implementation in the original, it requires doing the same thing in the secondary pyo3 project which is a tedious task.

The crates polars and tokenizers both seem to do the latter, which I would prefer since I want to keep the original library without the additional attributes everywhere.

Is there a smart way to accomplish this, such as automate it ?

FISR
  • 13
  • 1
  • 5
  • 2
    You don't need to copy the library, you just need to create wrapper structs and methods. – Chayim Friedman Jun 06 '23 at 10:51
  • Thanks for the advice, do you mind giving an example ? – FISR Jun 06 '23 at 13:27
  • 1
    `polars` is an example. It doesn't copy behavior, only wraps the types. – Chayim Friedman Jun 06 '23 at 13:28
  • I see, I need to have a better look at their method it seems. Nevertheless, does that not essentially mean doing the latter method I described? i.e. re-implementing the whole library, just adding the attributes above each object (wrapper structs, etc) ? – FISR Jun 06 '23 at 13:52
  • No, that's the point: you only need to reimplement the **interface**, not the **behavior**. – Chayim Friedman Jun 06 '23 at 13:53
  • Okay thank you very much for your help. This is the sort of thing you mean right? `#[pyclass] pub struct PySQLContext { pub context: SQLContext, }` Where SQLContext is imported from polars via: `use polars::sql::SQLContext;` – FISR Jun 06 '23 at 14:05
  • Yes, this is what I meant. Wrap the types and the methods. – Chayim Friedman Jun 06 '23 at 14:07
  • One last question, is there any benefit to keeping the pyo3 project separate from the original library (like polars and tokenizers) ? Why not just create an additional file in each module in the original library to contain the wrappers, etc ? – FISR Jun 06 '23 at 15:05
  • It may be more clean this way, and it avoids from building pyo3 when all you want is to use the Rust library. – Chayim Friedman Jun 06 '23 at 15:07
  • Okay great, thanks a lot for the help, much appreciated. – FISR Jun 06 '23 at 15:19

0 Answers0