1

I am using Inline::Python 0.56, trying to call a Python function with named parameters from Perl. I have replaced the original code fragments with an example that can be run, as requested:

use strict;
use warnings;
use 5.16.3;

use Inline Python => <<'END_OF_PYTHON_CODE';
# Interface
def get_sbom_of_package (
    package_name = None,
    target_path  = None,
):
    assert package_name is not None
    assert target_path is not None

    print ( package_name, target_path )
    return package_name

END_OF_PYTHON_CODE

my $package = 'thePackage';
my $target_path  = $ENV{HOME};

my $sbp = get_sbom_of_package(
    package_name => $package ,
    target_path  => $target_path
);
say $sbp;

and my error is:

TypeError: get_sbom_of_package() takes from 0 to 2 positional arguments but 8 were given at (eval 3) line 3.

Is there something else I need to do the inline Python to understand I am feeding it named parameters? I have replaced the call in Perl with

my $sbp = get_sbom_of_package(
    $package ,
    $target_path
);

and it works just fine. I am thinking it's either

  • bad Python on my part or
  • an Inline::Python configuration issue
  • an Inline::Python limitation

sorted from most likely to least. :-)

Jens
  • 69,818
  • 15
  • 125
  • 179
mpersico
  • 766
  • 7
  • 19
  • It would be better if you could publish an isolated test case we can run. – Evan Carroll Sep 14 '22 at 19:10
  • Also when you say "Inline::Perl 0.56" you mean "Inline::Python"? – Evan Carroll Sep 14 '22 at 19:11
  • Thank you for correcting the question. I have replaced the fragments with a runnable. – mpersico Sep 14 '22 at 20:08
  • I don't see any examples of keyword arguments for Python functions in any of the documentation. From the error message, it sounds like the named arguments in the Perl function call are just translated into a sequence of positional arguments, alternating keys and values. – chepner Sep 14 '22 at 20:40
  • Thanks for updating. Please correct me if I'm goofing off here, but: your Perl function call receives 4 (four) arguments, your Python function is written to take 2 (each with a default value) ... it disagrees, no? – zdim Sep 14 '22 at 20:58
  • @zdim The Python equivalent call would be `get_sbom_of_package( package_name=package, target_path=target_path)`. See https://treyhunner.com/2018/04/keyword-arguments-in-python/ Though I'm not sure how passing 2 pairs turns into 8 arguments, you'd think it would be 4. – Schwern Sep 14 '22 at 21:20
  • @Schwern Yeah, I'm not sure exactly what they're after but I posted what it seems to be to me. (I think that 8 is from a previous version in the question, they had four pairs in Perl call...?) – zdim Sep 14 '22 at 21:36
  • @Schwern Ah, I see what you mean -- that's a totally different thing though; that's about how a function is _called_. I don't think they'll have that capability here, since an actual (Python) function call is tranaslated from Perl, by the module; so all we get is to make the _Perl call_ and that won't work. But they don't need that, they can define Python's function to take a dictionary, pass it from Perl (well, a reference to it) and voila. (That's what I posted anyway...) – zdim Sep 14 '22 at 21:54

3 Answers3

2

Turns out it is not supported: https://rt.cpan.org/Public/Bug/Display.html?id=91360

mpersico
  • 766
  • 7
  • 19
  • 1
    That is not "unsupported" -- in Perl, a call `func(a => 1)` _passes two arguments to the subroutine `func`_. It's just how the language is. The expression in parens is evaluated to a _list of scalars_ and then that's passed to the sub. (Just a comment on the "Bug," which I don't think is a bug -- not a criticism of your post :) – zdim Sep 14 '22 at 22:03
  • So, I don't see how one would implement Python's call with named arguments (`func(a=7)`) on the Perl side ...? Anything that comes to mind (for me) is an existing Perl syntax meaning something else. Perhaps one could say that that feature isn't implemented, as it would require new syntax on Perl side? – zdim Sep 14 '22 at 22:05
  • @zdim did you even read the rt issue? the author answers your question `py_call_function_named("package", "function", \@positional, \%named)` function, but I'm not sure it's worth the effort."_ Ie., named arguments could provided in a separate named hr, and then simply passed to the python api. Which is how python itself actually works, it's just sugar. – Evan Carroll Sep 14 '22 at 22:45
  • @EvanCarroll But that's not exactly what they're asking here I thought? But firstly my comment is about raising a bug report over this! (The developer's response contains some language which I didn't have time to parse in detail right now so I _may_ have missed some points.) Then, what you explain is pretty much what I answered (if I understand correctly what you mean)...? – zdim Sep 14 '22 at 22:57
  • @EvanCarroll Then, I'm not sure I get the question fully, and I asked and nobody is countering that ... 1) of course the number of passed arguments (four) mismatches the ones read (two) 2) They want to pass a hash, for named arguments ... OK (a reference), and write the function to take a dictionary. How does that _not_ cut it? It's precisely what they want ...? (As an example, variable named arguments in Pythong _are_ implemented with a dictionary I think) – zdim Sep 14 '22 at 23:00
  • Well, because you're assuming you have control over the Python code you're providing to `Inline::Python`.. The library doesn't presuppose that. You can execute arbitrary python you load from a python module or file. BTW, I'm not saying `pyfunc(name => val)` is part of the solution. That's a total distraction. The bug here is that the module says it "allows you to put Python source code directly "inline" in a Perl script or module." But the Python code is not callable in Perl (at all) if it has named parameters because there is 0-module-api to call it (regardless of CORE). – Evan Carroll Sep 14 '22 at 23:04
  • @EvanCarroll Finally, as for "_the author answers your question_" -- yeah, in affirmative. And I didn't ask a question, I stated (reaffirmed) that it's not possible.I brought that up because when you glance at this answer it appears that there is a bug of sorts, something unsupported (which thus presumably should be). Who's going to go and read the thread with a detailed discussion from the implementer ... (kudos to them for responding) – zdim Sep 14 '22 at 23:04
  • It is possible. I'm not sure why you're talking past everyone. The syntax is simple. You could call a python function with named parameters like this `py_call_function_named("package", "function", \@positional, \%named)`. There is nothing impossible about it. That the API isn't implemented is a bug in the implementation. Literally. – Evan Carroll Sep 14 '22 at 23:05
  • @EvanCarroll "_But the Python code is not callable in Perl (at all) if it has named parameters_" But it doesn't "have them" - "_named parameters_" is about how it's called, not how the function is written. And I don't hold it against a library that it cannot reproduce API, of course. (perhaps I'm missing something?) But OK, maybe I pushed it too far, maybe it is a "bug" ... – zdim Sep 14 '22 at 23:07
  • @EvanCarroll Perhaps that's the problem, I don't understand what that means -- is this `py_call_function_named("package", "function", \@positional, \%named)` meant to be how to call it out of Perl? What is "package" and "function"? What I'm trying to say is that you cannot call in Perl `perl_func(a=>1)` and have it mean in Python `py_func(a=1)`. Not sensibly. Perhaps I _am_ misunderstanding something... (But you _can_ call `perl_func({a=>1})` and have it mean `py_func({"a":1})` and I don't see what's wrong with that for this question) – zdim Sep 14 '22 at 23:09
  • _"But it doesn't "have them" - "named parameters"_ is about how it's called, not how the function is written." That's not true in python, thanks to `kwargs` That code snipped is the suggestion of how to supply kwargs explicitly, as python does (it just has 0 sugar). – Evan Carroll Sep 14 '22 at 23:11
  • "_`kwargs`_" is what I was referring to, that works by dictionary (when used out of caller). Again, I absolutely accept I may be missing something... – zdim Sep 14 '22 at 23:14
  • You're missing a few things here, this bug isn't about calling Perl functions with named parameters. It's about calling python functions that take named parameters, from perl. The request isn't about syntax (in my read), it's an abstract question like "how do I call any python function with named parameters in Perl". In order to do that you have to have some method to supply kwargs in Python, and the API simply doesn't support it yet. – Evan Carroll Sep 14 '22 at 23:14
  • @EvanCarroll "_you have to have some method to supply kwargs_" ... hm, ok. I didn't realize they wanted that. If I send in a hashref from Perl and take a dictionary in Python then it may have variables number of key-value pairs. OR, can I not define it as `def py_func(**kwargs):`? – zdim Sep 14 '22 at 23:16
  • Put it another way, let's say you have this function in Python to keep it simple, `def myfunc(*, foo=42): print(foo)`. The question here is how do you call that function from Perl. It only accepts a named parameter. The proposed and unimplemented syntax by the `Inline::Python` author is `py_call_function_named(__PACKAGE__, "myfunc", [], {foo => 7})` which would be the Perl equiv of Python's `myfunc(foo = 7)` – Evan Carroll Sep 14 '22 at 23:18
  • @EvanCarroll I think I should back off and reconsider who is asking what :) Some things just aren't clear. As for what I see in the question, my answer takes care of it (I'm not rooting for answer-points, just saying how I see the problem). So I'd suggest we leave it, I'll reread this exhcange and think about it. I 'll leave it for awhile (in case I wrote something outrages :)), then remove I suggest... – zdim Sep 14 '22 at 23:18
  • 1
    @EvanCarroll "_Put it another way,..._" OK, I see what you mean ... that's the opposite direction from what I was thinking. I take it that I have a wanted API in Perl and am writting code in Python for it; a "normal" way to develop libraries I'd say. But, of course, one could say "I want to port this great Python code to Perl"... then the "bug" makes some sense (even though one can't really expect to implement language features word by word) – zdim Sep 14 '22 at 23:19
  • I have an eerie recollection of the author/OP telling me this was how he was calling Python code he wasn't writing at $work. ;) I did have beers with him a few months ago. But sure, you can do a lot with `Inline::Python` but calling code you're not writing is certainly how a lot of people use it (make use of Python libraries and such). – Evan Carroll Sep 14 '22 at 23:24
  • 1
    hah :) (you mean he had some Python code he couldn't change and was calling it from Perl the way he shows in that rt?) ok, that makes sense then... (it's not at all what this question asks though but that's ok) – zdim Sep 14 '22 at 23:26
  • You got it, that's my assumption and reading anyway. – Evan Carroll Sep 14 '22 at 23:28
1

If I am reading the XS correctly and remembering my Python extension module work, this is not directly possible.

Inline::Python appears to invoke py callables using PyObject_CallObject which passes a tuple of arguments — without any named/keyword args.

So, you'd have to do it yourself, perhaps by putting a wrapper around the py callable, which wrapper understands what you mean by parameter => value, ... and constructs the right tuple to pass along. You'd need to redundantly know default values, too, of course. You might rev Inline::Python to support interrogating the callable's signature to look for keyword args ... but then you'd still have to adapt those semantics that to ordinary Perl subroutine calls yourself.

pilcrow
  • 56,591
  • 13
  • 94
  • 135
  • Interesting. I'll go look at the code you referenced. I wonder, though; that seems to imply that if the function I was calling in Python *required* named params, then I couldn't call it directly. From my research, I think that Python functions can be called with named or positional commands, so there is nothing that can be interrogated. – mpersico Sep 14 '22 at 20:48
  • It's little more complicated. "Regular" parameters can be set using positional or keyword arguments. Python also has positional-only and keyword-only parameters, which (as their names imply) can only be set using positional and keyword arguments, respectively. Such a function could be called using this syntax: `foo(**{name: value, name: value})`, so perhaps in Perl you could pass a hash ref, like `foo({name => value, name => value})`? – chepner Sep 14 '22 at 21:21
  • (Though that looks like `Inline::Python` would have to use `PyObject_Call`, not `PyObject_CallObject`. I wonder why they didn't.) – chepner Sep 14 '22 at 21:24
  • I tried that hash ref idea. Didn't work, probably for the reason you suggested. – mpersico Sep 14 '22 at 21:26
  • Looks like the package predates keyword-only parameters, and the author assumed keyword arguments would never be *needed*. – chepner Sep 14 '22 at 21:26
1

Note   This is meant to sketch the idea that "named arguments" be implemented as a hash(ref) in Perl --> dictionary in Python, and then do whatever need be done on the Python side, with named arguments in hand. By all means add other needed arguments to the list (or to the dictionary), like function names, arrayrefs, or whatnot.

The ultimate purpose isn't explained but if that is clarified I can add more specific code. (A guess at how this might be used, plus another fairly generic way, are given in a footnote.)


The attempted code passes four (4) arguments to Perl's function call, while Python's function is written to take two (each with a default value).

If you want to pass named parameters to a Python function, one way would be to pass a dictionary. Then this incidentally mimics your Perl use, as well

def adict(d):
    for key in d:
        print("key:", key, "value:", d[key])

This way one maps named arguments from the caller in Perl (hash) to Python (dictionary), leaving it to normal python->python calls to be built in this wrapper._

Then your code could go as

use warnings;
use strict;
use feature 'say';

use Inline::Python;

my $package = 'thePackage';
my $target_path  = $ENV{HOME};

call_py( {  # note -- pass a hashref
    package_name => $package, target_path  => $target_path
} );

use Inline Python => <<'END_PY'
def call_py(d):
    for key in d:
        print(key + " => " + d[key])
    # ...
    # prepare and make other (python-to-python) calls as needed
END_PY

If specific further calls from the Python function need be made (by named parameters?) please show how they'd look and I can add to this bare-bones example.

See below on my guesses of what could be intended with this question. Please clarify?

This prints (with my path redacted)

target_path => /...
package_name => thePackage

Or one could devise a system for passing Perl's four arguments and taking them in Python and interpreting them as name => value pairs, but why.

Once arguments in the Python function are unpacked from the dictionary (and whatever else may have been added to the argument list along with the dictionary for named arguments sent from Perl via a hash(ref)), one can make further python -> python calls as needed.


A function in Perl takes a list of scalars. Period. So this

func(package_name => $package, target_path  => $target_path)

or even this

my %ph = (package_name => $package, target_path  => $target_path);

func( %ph );

is really this

func('package_name', $package, 'target_path', $target_path)

That "fat comma" (=>) is a comma, whereby its left argument also gets quoted.


Imagine that the purpose of this is to be able to call from Perl a great py_lib in Python, somehow via "named parameters" (please clarify). Here are a couple of guesses

use warnings;
use strict;
use feature 'say';

use Inline::Python;

my $package = 'thePackage';
my $target_path  = $ENV{HOME};

call_py({ package_name => $package, target_path => $target_path });

use Inline Python => <<'END_PY'
def call_py(d):
    print(d)
    # prepare and make other (python-to-python) calls as needed

    print("\nArgs with these names passed from Perl:")
    py_lib_kw( pack=d['package_name'], path=d['target_path'] )

    print("\nVariable number of named args passed from Perl:")
    py_lib_kwargs( **d )

def py_lib_kw(path=None, pack=None):
   print ("package is " + pack + " and path is " + path)

def py_lib_kwargs(**data):
    for val in data:
        print(val + " => " + data[val])

END_PY
zdim
  • 64,580
  • 5
  • 52
  • 81
  • You showed how to call a func that accepts a dictionary. That wasn't the question. But it might part of the answer. Is it possible to write a Python function that accepts a function name and a dictionary of arguments, and call the appropriate Python function with the appropriate args? – ikegami Sep 14 '22 at 23:35
  • 1
    @ikegami "_That wasn't the question._" -- how was that not the question? They want a function that works with named parameters; that's a dictionary -- I'm suggesting to them to implement it as a dictionary (on Python side). I don't see that their wording of "named parameters" means what we think, judged by their example. (Has nothing to do with Python's named parameters, just a function with two arguments.) I am trying to read the actual question and it wants a function with parameters. – zdim Sep 15 '22 at 01:23
  • @ikegami I'm totally lost to see any of what people ar talking about in this question. Example: "_///write a Python function that accepts a function name and a dictionary of arguments,..._" -- A function name? Why? Who asked for that? It's asking to send in `arg_name => arg_value` and for the function to be able to operate by arguments' names (supposedly, by invoking words "named arguments"). That's all I see as actually posted in this question. And a dictionary is an answer, I think... – zdim Sep 15 '22 at 01:34
  • [contd] Now, if they want variable number of named parameters ala `**kwargs` then that is different but I see no hint of that in the question. The wording "named parameters" is much too generic to invoke all that given all that is actually shown in the question. My take on it, after rereading it for the fifth time – zdim Sep 15 '22 at 01:35
  • Your code can't be used to call `def get_sbom_of_package( package_name = None, target_path = None ):` using names, which is what the OP wants. Simple as that. – ikegami Sep 15 '22 at 03:50
  • The wrapper doesn't have to take the function name. Makes it reusable, though. It would be silly to have a copy the same wrapper with a different hardcoded name for each function. – ikegami Sep 15 '22 at 03:51
  • 1
    @ikegami "_call `def get_sbom_of_package( package_name = None, target_path = None ):` using names, which is what the OP wants. Simple as that._" -- not simple: 1) I don't see that they ask that. I see that they want a Perl function to pass named arguments and a Python function for it, that uses those named args, and they wrote something they thought might work. I don't see that the shown Python function is _the_ requirement? 2) But even if they want what you say, I am clearly suggesting they write it differently, with a dictionary, and then they can call using names. never mind... – zdim Sep 15 '22 at 06:27
  • At best the question is unclear in what exactly it wants. But we can just leave it, I clearly don't see in this question what others do – zdim Sep 15 '22 at 06:29
  • 1
    @ikegami "_The wrapper doesn't have to take the function name. Makes it reusable, though. It would be silly to have a copy the same wrapper with a different hardcoded name for each function_" --- wait, so you are saying they want to write a wrapper to use some yet other Python function (a great library or some such), so that's the function name to pass? But the question doesn't even hint at anything like it!? Yes, it's a perhaps the most frequent use, but it _doesn't_ ask that! And the rest of it ... you say the dictionary, just what I'm proposing here! whatever. i'm outta here – zdim Sep 15 '22 at 06:40
  • Re "*I don't see that they ask that*", You don't have to look further than the title! – ikegami Sep 15 '22 at 14:17
  • Re "*so you are saying they want to write a wrapper to use some yet other Python function*", They'd rather call it directly, but it's been established that this is impossible. I was just trying to help you salvage you answer. – ikegami Sep 15 '22 at 14:20
  • 1
    @ikegami "_I was just trying to help you salvage you answer_" -- well thank you, I thought so at first, but I don't see how. you mention a function name and a dictionary? I did pass a hashref -> dictionary? so they can add a function name and their python function can call that target library with parameters unpacked from the dictionary. Called out of Perl via named args (hash). Anything else? This is even more of an answer if they are writing merely a wrapper (Add `"func_name": name` to that dictionary for an even simpler interface. Heck, add _all_ needed to that one hashref->dictionary) – zdim Sep 15 '22 at 16:23
  • 1
    @ikegami I propose to pass a hash(ref) into a dictionary. you say 'no that's not what they are asking', instead 'write a Python function that accepts a function name and a dictionary' (paraphrasing) ...? I don't get it. Evan-Carrol below talks about the author's comment, in a linked rt issue, about how this can be implemented using ... a dictionary, guess what! (And then some, sure, but "named" goes by dict) While I get crap (and a downvote for good measure I guess). I don't understand. But clearly something isn't matching here so I I'll just leave it then. Sorry for the bother... – zdim Sep 15 '22 at 16:35
  • Re "*I propose to pass a hash(ref) into a dictionary*", Yes, and that's wrong. 1) The function has two parameters (not one), 2) The parameters expect strings (not dictionaries), and 3) The OP wants to specify the arguments by name (not by position). – ikegami Sep 15 '22 at 17:11
  • Re "*about how this can be implemented using ... a dictionary*", Indeed, it might be possible to devise a solution using a dictionary (e.g. by passing it to a wrapper). But the solution isn't to use a dictionary. That's just part of the solution. Where's the rest of it? Specifically, you didn't provide the wrapper! (Since I just finished repeating my first comment, this is my last comment.) – ikegami Sep 15 '22 at 17:18
  • 1
    "_The function has two parameters_" -- and I suggested to pass these two parameters in via a dictionary instead, so they come by names. (It's their bloody function they can write it any way they like, then in it they can prepare calls to yet other functions as needed.) /// "_Where's the rest of it? Specifically, you didn't provide the wrapper!_" -- but they don't ask for any of that! What I posted _would be_ the wrapper, and inside they can do what they need (what was all left unspoken). But fine, clearly I don't get something here. (I added a tweak. thanks for the discussion) – zdim Sep 15 '22 at 18:14
  • Re "*I suggested to pass these two parameters in via a dictionary*", Doesn't work. First arg gets the dictionary instead of the correct value, and second arg always gets default value – ikegami Sep 15 '22 at 18:53
  • 1
    "_First arg gets the dictionary instead of the correct value, and second arg always gets default value_" --- no, I suggested that the Python function be written to take a dictionary. Then inside of that function they can unpack arguments, by name, as they need them. So not to use the Python function they show in the question (which doesn't reveal what they need), but instead to use one which takes a dictionary. So pass "named arguments" from Perl (as hash(ref)), and do with them what they need from the dictionary -- I don't know what they really need; they don't show. I edited a little – zdim Sep 15 '22 at 19:07