Call `+=` on a PyObject in C++

Question

I'm writing a Python module in C++.

At a certain point, I need to add a PyObject of an arbitrary type to another one. In other words, do the same as a += b would do in Python. But I haven't been able to find a function in the API that does that.

I tried to do the following,

PyObject* increment(PyObject* a, PyObject* b) {
  const auto tp = Py_TYPE(a);
  [&]{
    if (const auto nb = tp->tp_as_number)
      if (auto iadd = nb->nb_inplace_add)
        return iadd;
    if (const auto sq = tp->tp_as_sequence)
      if (auto iadd = sq->sq_inplace_concat)
        return iadd;
    throw error(PyExc_TypeError,
      "type "s+(tp->tp_name)+" does not provide __iadd__");
  }()(a,b);
  return a;
}

but found out that neither Python float, nor int, nor str implement these methods.

Is there a function in the API that applies a generic +=? If not, how would I write one?

"found out that neither Python float, nor int, nor str implement these methods": because those types are immutable. You cannot increment/change them and the "+=" operator is a "fake". `a += 1` is the same thing as `a = a + 1` (well, disassembly shows `INPLACE_ADD` but that's not really inplace and `__iadd__` doesn't exist) — Jean-François Fabre, Jun 21 '20 at 21:22
in the case the object doesn't provide `__iadd__` you could fall back to addition then return the result — Jean-François Fabre, Jun 21 '20 at 21:28
The `operator` module has an `__iadd__` function that should do the right thing. Just call it. — tdelaney, Jun 21 '20 at 21:32
I see. Would there be any detriment in mutating, say, a `PyFloatObject` in C++, e.g. `a->ob_fval += 4.2;`? — SU3, Jun 21 '20 at 21:34

user2357112 · Accepted Answer · 2020-06-21T23:03:35.410

At a certain point, I need to add a PyObject of an arbitrary type to another one. In other words, do the same as a += b would do in Python. But I haven't been able to find a function in the API that does that.

The function you weren't able to find is PyNumber_InPlaceAdd.

x += y

at Python level is equivalent to

sum = PyNumber_InPlaceAdd(x, y);
Py_DECREF(x);
x = sum;

at C level. (You can also keep both sum and x if you want, which can be useful for exception handling.)

PyNumber_InPlaceAdd handles the full dispatch logic of +=, including __iadd__, both sides' __add__ methods (in the correct order), the NotImplemented sentinel, and the nb_inplace_add, nb_add, sq_inplace_concat, and sq_concat C-level hooks.

score 0 · Answer 2 · answered Jun 21 '20 at 22:19

0

I ended up doing something like this:

static PyObject* increment(PyObject*& a, PyObject* b) {
  const auto tp = Py_TYPE(a);
  if (const auto nb = tp->tp_as_number) {
    if (auto iadd = nb->nb_inplace_add)
      return iadd(a,args), a;
    if (auto add = nb->nb_add)
      return a = add(a,b);
  }
  if (const auto sq = tp->tp_as_sequence) {
    if (auto iadd = sq->sq_inplace_concat)
      return iadd(a,b), a;
    if (auto add = sq->sq_concat)
      return a = add(a,b);
  }
  throw error(PyExc_TypeError,
    "cannot increment "s+(tp->tp_name)+" by "+(Py_TYPE(b)->tp_name));
}

That is falling back on __add__ if __iadd__ is not available, as Jean-François Fabre suggested.

answered Jun 21 '20 at 22:19

SU3

5,064
3
35
66

You still need to assign back to `a` in the `iadd` cases - it's possible, though unusual, for the "in-place" methods to return a different object. – user2357112 Jun 21 '20 at 22:24
I did the opposite of your suggestion exactly for the same reason as yours. For my application, if `iadd` returned a different object, I want my original, supposedly incremented, object rather than the returned one. – SU3 Jun 21 '20 at 22:30
1

You're also missing a dispatch to the right-hand-side's method, as well as the logic for which side's `nb_add` to try first. – user2357112 Jun 21 '20 at 22:32
"if `iadd` returned a different object, I want my original, supposedly incremented, object rather than the returned one" - that's kind of weird and not how Python's `+=` works. – user2357112 Jun 21 '20 at 22:34
"that's kind of weird": This is a part of a histogramming code. If `iadd` returned a sentinel signaling whether the addition was successful, which could fail if the two types were not compatible, I would lose my bin object. – SU3 Jun 21 '20 at 22:40
You are right about the sides. I overlooked the fact that the types might not be commutative with respect to availability of addition. – SU3 Jun 21 '20 at 22:42
"If `iadd` returned a sentinel signaling whether the addition was successful, which could fail if the two types were not compatible" - are you talking about `NotImplemented`? You need to avoid assigning `NotImplemented` over the original object, but that only applies to `NotImplemented`, and it applies in the `add` cases too. – user2357112 Jun 21 '20 at 22:47

Call `+=` on a PyObject in C++

2 Answers2