3

I am a little bit confused, when I create a dirty NIF (for example, by setting the appropriate flags value for the dirty NIF in its ErlNifFunc entry), this creates a dirty scheduler that runs on a dirty thread.

I understand that I can have only N cpu-bond dirty threads as the number of N cpu cores. But, there is also the enif_thread_create function.

What is the difference between them? Is there a limit of threads that I can create using enif_thread_create? Will they be a dirty threads also? I would appreciate a simple code example of using dirty threads through enif_thread_create.

ziv
  • 53
  • 3

1 Answers1

5

When you define a NIF as dirty, you're telling the VM to execute it only via a dirty scheduler. You're not creating a dirty scheduler; only the VM does that.

By default, the VM gives you N dirty CPU schedulers, where N is the number of normal schedulers. The number of normal schedulers defaults to the number of logical processors configured on the system. As explained in the erl man page, the numbers of normal and dirty schedulers can be controlled via various command line options.

The enif_thread_create function provides access to the underlying operating system's thread creation functionality. This function existed prior to dirty NIFs and schedulers, and essentially existed prior to normal NIFs as well, since it's just a wrapper around the erl_drv_thread_create function, which has been part of the driver API for quite some time. These threads are independent of scheduler threads and so are unrelated to NIF scheduling. Rather, they're more like threads a regular C or C++ program might create and use. In other words, the Erlang runtime uses scheduler threads to run Erlang jobs, including dirty jobs via dirty schedulers, whereas your internal NIF or driver code can use threads it creates via enif_thread_create or erl_drv_thread_create for jobs running (mostly) independently of the Erlang runtime. The maximum number of threads you can create through these functions is limited by the underlying operating system.

Steve Vinoski
  • 19,847
  • 3
  • 31
  • 46
  • What is the benefit of using this envelope function enif_thread_create / erl_drv_thread_create over using the std::thread for create a new thread for processing something beside Erlang VM? – ddleon Dec 02 '21 at 14:30
  • 1
    For non-Erlang tasks, I think the only benefit of using the wrappers is the portability they provide -- they're guaranteed to work wherever the Erlang VM works. But they don't provide any extra portability benefits over those provided by `std::thread` or other portable threading APIs. – Steve Vinoski Dec 02 '21 at 18:53