6

I asked this question over at the rakulang sub reddit and was suggested to post here:

I keep falling back to Perl 5 for a lot of my work so I can "get it done" simply because I am so much more familiar with Perl 5.

However, I have the need to build something that will subscribe to multiple MQTT topics (similar conceptually to a websocket subscription) and process data, keeping a lot of this data as internal state. A concurrency project. So I see this as a great opportunity to immerse myself in some Raku :)

So far I understand I need to create a supply / given / when setup, but I'm not totally sure how I will deal with each stream of data's state received over each topic. And a reply from my reddit post suggested Cro, which I think will fit the bill very nicely. But there are still some implementation details I am unclear on.

For example, a message payload arrives on topic foo, I want to add data from that payload to an existing array (my internal state).

But this subscription to topics will be happening for an 'undetermined' number topics, and will adjust at runtime. So it will not be possible to have a hard coded array to store and manage this data in like @foo

In a non concurrent world, I could create a hash (associative array) with a key that matches my topic name, %data<foo> for example, and store the array there.

However in the world of concurrency, I would need an answer to the mutex problem. If each member of the hash is having it's data modified concurrently by different threads, then I would think the entire hash would require a lock.

This has potential to result in a deadlock or poor performance at the very least (I am expecting some several hundred messages per second, across multiple topic subscriptions).

Perhaps I can create a variable 'dynamically' (or better yet, object) based the topic name, so there is a separate memory address for each array of data. However, I'm not sure how to do that, or indeed if that is the 'best' approach in this scenario.

In summary, Question 1: Is creating an object or variable dynamically for this purpose a sound pattern? Question 2: Is there a design approach I am simply not aware of that would be more suitable?

So, any specific advice would be greatly appreciated. I feel like this is a case of "I don't know what I don't know" type of problem!

Thanks!

camstuart
  • 623
  • 3
  • 13
  • 1
    *Can anyone advise books/blogs/tutorials specifically designed for Raku concurrency?* - unfortunately *Questions asking us to recommend or find a book, tool, software library, tutorial or other off-site resource are off-topic for Stack Overflow* as explained in the [help center](https://stackoverflow.com/help/on-topic). – dbc Feb 11 '21 at 17:23
  • 2
    You don't need mutexes. The whole point of react/whenever is to avoid things like that. Whatever variable, state or lexical in the scope of the whenever block is guaranteed to be thread safe. – Holli Feb 12 '21 at 01:24
  • 1
    @dbc, I recognize that, per the normal rules of SO you are 100% correct. However, the `raku` tag is _typically_ a bit more informal about that sort of thing: Raku is a small language, and we're a small community, and that lets us avoid some of the flooding problems that would come up in a larger group. So, moderators who are more used to the local norms would likely have left this question open. That doesn't (of course!) mean that you were wrong to close it, but I thought you might appreciate the context. Thanks for helping to keep SO useful :) – codesections Feb 12 '21 at 01:34
  • 1
    I haven't done all that much with concurrency in Raku (it's on my list!) and am not 100% sure I understand your use case. That said, if I understand what you're trying to do correctly, this sounds like a great use case for a [Channel](https://docs.raku.org/language/concurrency#Channels). As the example in the docs shows, you can use a channel to send data from multiple locations to one central location, which would let you use your Hash without the need for a mutex/other locking mechanism. Would that fit what you have in mind? – codesections Feb 12 '21 at 01:42
  • That's it. I'm done with StackOverflow. – Elizabeth Mattijsen Feb 12 '21 at 11:11
  • @ElizabethMattijsen - 3 people voted to close this question. If you disagree you can always [vote to reopen](https://stackoverflow.com/help/reopen-questions), and if two others do as well, it will get reopened. – dbc Feb 12 '21 at 20:44
  • 1
    3 people *NOT* of the Raku community, voted to close this question. This is at least the second time I know of, that this happened. So clearly, SO is not about its users, it's about its moderators. Well, good luck with it then: I will recommend people to ask their Raku questions on places such as Reddit instead. And yes, I know I could vote to reopen, but I won't be playing such games anymore: there are too many of you versus to few of us. Simple. – Elizabeth Mattijsen Feb 12 '21 at 23:47
  • @ElizabethMattijsen I agree with your frustration. And I completely disagree that any recommendations for learning materials is cause to close – camstuart Feb 13 '21 at 05:32
  • @codesections thanks for your advice! I had not encountered channels in Raku so far. And I think they might offer a possible solution. If I understand it's usage, In my case I could create 1 channel per topic subscription, but I would still need to name the channels dynamically. But if I understand your suggestion, the channels themselves could live In the hash. Is that correct? – camstuart Feb 13 '21 at 05:38
  • @Holli thanks for clarifying that, it sounds like a neat, well thought out feature of Raku. – camstuart Feb 13 '21 at 05:41
  • Thanks @raiph I have update the question to fit SO guidelines (I hope!) – camstuart Feb 13 '21 at 05:47

0 Answers0