Why are static variables in an xsub not thread safe?

Question

According to perldoc threads:

Since Perl 5.8, thread programming has been available using a model called interpreter threads which provides a new Perl interpreter for each thread, and, by default, results in no data or state information being shared between threads.

What type of data or state information is referred to in the above quote? According to perldoc perlxs:

Starting with Perl 5.8, a macro framework has been defined to allow static data to be safely stored in XS modules that will be accessed from a multi-threaded Perl.

So it seems to me static variables are shared between threads? But Perl variables are not shared? (I am trying to figure out exactly what kind of data is thread safe, and how to create a thread safe module)

Answer 1

Each thread has its own interpreter. This struct^[1] stores everything that makes up perl, including parser state, regex engine state, symbol table, and all "SV" (which includes scalars, arrays, hashes, code, etc). Creating a new thread from within Perl copies makes a copy of the current interpreter.

XS code may safely use the Perl API because each function has a parameter that specifies the interpreter to use. This is often invisible to the code thanks to macros, but you might have noticed references to "THX" or "Perl context". Just don't pass an SV that belongs to one interpreter to another. (You might have heard of the "Free to wrong pool" error message that can result from this.)

But Perl can't offer any protection to things outside of its knowledge or control, such as the static storage of the external libraries it loads. No copies of those are made. Two threads could call the same C function at the same time, so precautions need to be taken as if you were writing a multi-threaded C program.

That macro framework to which your quote alludes gives access to per-interpreter storage. It also allows the library to specify a function to call on creation of new Perl threads to clone the variables into to the new interpreter.

---

1. If Perl is built without -Dusemultiplicity, the Perl interpreter consists of a bajillion global (static) variables instead. MULTIPLICITY moves them into a struct and adds a context parameter to Perl API calls. This has a performance penalty, but it allows a process to have multiple Perl interpreters. Because threaded builds of Perl requires this, building a threaded perl (-Dusethreads) presumes -Dusemultiplicity.