-2

Recently I had to convert a module written in pure python to pyx in in order to compile with cython. The procedure for converting from py to pyx was very straight forward since the all the variables and functions were type hinted. So it was a just a matter of looking up the cython static type for each types in python.

What are the current options available for automating the conversion of a module written in pure Python to Cython, specifically for converting .py files to .pyx files, taking into account the use of Python type hints in the original code? Are there any existing modules or tools that can facilitate this process? if NO, is it theoretically possible to develop a module that can automatically convert Python type hints to Cython static types, and if so, what challenges may arise in the development of such a module?

Samm Flynn
  • 314
  • 2
  • 13
  • 1
    Please, check [ask] and [help/on-topic]. This is off-topic and also needs more focus. – buran Jan 29 '23 at 11:00
  • @buran can you explain further the reasons this is off-topic and which points to focus on. I would like to improve the question. – Samm Flynn Jan 29 '23 at 11:22
  • 1
    Did you check the links I shared - Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow as they tend to attract opinionated answers and spam. Instead, describe the problem and what has been done so far to solve it. Also you ask like several questions in one. And finally - what challenges may arise is more or less opinion-based... – buran Jan 29 '23 at 11:36
  • @buran, I have yet to see a recommendation question that attracts all those opinionated answers. Especially for something as specialized as this. But an expert who can really help, can still comment, and probably will have enough reputation to reopen this. But the OP doesn't seem to appreciate the answer he already got. – hpaulj Jan 29 '23 at 15:28

1 Answers1

1

Cython already uses Python type annotations so the chances are you didn't need to do anything.

There's a few small notes:

  • the most recent Cython 3 alpha versions have better support for this than the 0.29.x branch (so upgrade if you can),
  • int is assumed to to be a Python object rather than a C int to avoid overflow for large ints and keep the semantics the same as Python. Use cython.int as the annotation for a C int.

"A bit more clarification:" Cython is generally pretty conservative with type annotations. It assumes the annotations are correct but aren't for its benefit. Therefore it doesn't do things where the annotation might change the answer. int is the main example - Python ints can be infinitely large while C ints have a fixed maximum value. Therefore, Cython doesn't change an int annotation to a C int because the result could change. Obviously if you're happy with the potentially different behaviour you can make this change yourself.

If you want to automate a fully conversion then the builtin ast module is probably the easiest way to do it. I'd still recommend sticking with "pure Python mode" for simplicity, just because the ast module can generate Python code. But essentially what you'd do is create a visitor that looks for annotations (i.e. ast.AnnAssign, ast.Arg) and replaces it will an appropriate Cython-friendly annotation. You then use ast.dump to rewrite the modified code.

DavidW
  • 29,336
  • 6
  • 55
  • 86
  • I did that at first and decided to convert to cython types anyway and the cython typed version ran significantly faster. – Samm Flynn Jan 29 '23 at 11:29
  • 1
    It's obviously a bit hard to tell what the difference is without being able to compare some representative code (looking at the html output from `cython -a` is likely to pick up the main differences). But `int` is probably the main one. The bits where Cython differs are the ones where just using type hints *might* do something different, which makes it quite hard to automate. – DavidW Jan 29 '23 at 11:34
  • Many thanks for the clarifications, wrote a simple script s, using your methods, you were right, getting some compilation warning regarding int – Samm Flynn Jan 29 '23 at 19:48