Design choice of Haskell data types in multithreaded programs

Question

In a multiple threaded server application, I use the type Client to represent a client. The nature of Client is quite mutable: clients send UDP heartbeat messages to keep registered with the server, the message may also contain some realtime data (think of a sensor). I need to keep track of many things such as the timestamp and source address of the last heartbeat, the realtime data, etc. The result is a pretty big structure with many states. Each client has a client ID, and I use a HashMap wrapped in an MVar to store the clients, so lookup is easy and fast.

type ID        = ByteString
type ClientMap = MVar (HashMap ID Client)

There's a "global" value of ClientMap which is made available to each thread. It's stored in a ReaderT transformer along with many other global values.

The Client by itself is a big immutable structure, using strict fields to prevent from space leaks:

data Client  = Client
  {
    _c_id        :: !ID
  , _c_timestamp :: !POSIXTime
  , _c_addr      :: !SockAddr
  , _c_load      :: !Int
    ...
  }
makeLenses ''Client

Using immutable data structures in a mutable wrapper in a common design pattern in Concurrent Haskell, according to Parallel and Concurrent Programming in Haskell. When a heartbeat message is received, the thread that processes the message would construct a new Client, lock the MVar of the HashMap, insert the Client into the HashMap, and put the new HashMap in the MVar. The code is basically:

modifyMVar hashmap_mvar (\hm ->
  let c = Client id ...
  in return $! M.insert id c hm)

This approach works fine, but as the number of clients grows (we now have tens of thousands of clients), several problems emerge:

The client sends heartbeat messages pretty frequently (around every 30 seconds), resulting in access contention of the ClientMap.
Memory consumption of the program seems to be quite high. My understanding is that, updating large immutable structures wrapped in MVar frequently will make the garbage collector very busy.

Now, to reduce the contention of the global hashmap_mvar, I tried to wrap the mutable fields of Client in an MVar for each client, such as:

data ClientState  = ClientState
  {
    _c_timestamp :: !POSIXTime
  , _c_addr      :: !SockAddr
  , _c_load      :: !Int
    ...
  }
makeLenses ''ClientState

data Client = Client
  {
    c_id    :: !ID
  , c_state :: MVar CameraState
  }

This seems to reduce the level of contention (because now I only need to update the MVar in each Client, the grain is finer), but the memory footprint of the program is still high. I've also tried to UNPACK some of the fields, but that didn't help.

Any suggestions? Will STM solve the contention problem? Should I resort to mutable data structures other than immutable ones wrapped in MVar?

See also Updating a Big State Fast in Haskell.

Edit:

As Nikita Volkov pointed out, a shared map smells like bad design in a typical TCP-based server-client application. However, in my case, the system is UDP based, meaning there's no such thing as a "connection". The server uses a single thread to receive UDP messages from all the clients, parses them and performs actions accordingly, e.g., updating the client data. Another thread reads the map periodically, checks the timestamp of heartbeats, and deletes those who have not sent heartbeats in the last 5 minutes, say. Seems like a shared map is inevitable? Anyway I understand that using UDP was a poor design choice in the first place, but I would still like to know how can I improve my situation with UDP.

Are you really constructing a new Client for every heartbeat, or only for the first heartbeat from a particular source (IP, clientId, or some other mechanism)? If it's a new client for every heartbeat, what prunes the old ones? I suspect part of the problem is that you're doing too much work in the critical section, in which case stm-containers will only partially mitigate the problems. A read-then-update might allow for less contention; this pattern is better with STM because STM TVar reads can be performed without taking a lock at all. — John L, Jul 05 '14 at 17:25
@JohnL Unfortunately in the first immutable Client version I did construct a new Client for every heartbeat, in order to update the c_timestamp, c_addr (may change for a Client), and other fields (realtime data). Apart from the contention problem I would also like to reduce the memory footprint, which I find particularly difficult if I stick with this “immutable structure wrapped in MVar" approach. — Aufheben, Jul 06 '14 at 13:48
how do you get rid of old Client data then? I think you'll need to periodically evict data from your working set. — John L, Jul 06 '14 at 15:48
@JohnL If I insert a new Client into ClientMap, wouldn't the old value be GC'ed? — Aufheben, Jul 06 '14 at 16:12
if it's inserted at the same key, yes. But what about when a client disconnects? Regardless, WRT memory usage you probably will need to do some heap profiling to see what's being retained. — John L, Jul 07 '14 at 01:40
@JohnL I have a thread that checks all the Clients periodically and remove those who have not sent heartbeats in 5 minutes. Anyway, profiling seems like the way to go when dealing with memory issues. Thanks for the comments! — Aufheben, Jul 07 '14 at 02:02

score 2 · Accepted Answer · answered Jul 05 '14 at 07:08

2

First of all, why exactly do you need that shared map at all? Do you really need to share the private states of clients with anything? If not (which is the typical case with a client-server application), then you can simply get around without any shared map.

In fact, there is a "remotion" library, which encompasses all the client-server communication, allowing you to create services simply by extending it with your custom protocol. You should take a look.

Secondly, using multiple MVars over fields of some entity is always potentially a race condition bug. You should use STM, when you need to update multiple things atomically. I'm not sure if that's the case in your app, nonetheless you should be aware of that.

Thirdly,

The client sends heartbeat messages pretty frequently (around every 30 seconds), resulting in access contention of the ClientMap

Seems like just a job for Map of the recently released "stm-containers" library. See this blog post for introduction to the library. You'll be able to get back to the immutable Client model with this.

answered Jul 05 '14 at 07:08

Nikita Volkov

42,792
11
94
169

1

Indeed, in a typical TCP-based client-server application, the server uses a thread to serve each client connection, in which case there's usually no need to share anything. But in this case the client does not connect to the server, but uses the connectionless UDP messages to keep registered with the server (which is a poor design choice that is too late to change now). The "stm-containers" library does look interesting, I'll look into it later. Thanks! – Aufheben Jul 05 '14 at 08:28
@Aufheben Well, then the "stm-containers" seems to be what you need. – Nikita Volkov Jul 05 '14 at 08:52
remotion is deprecated. Is there alternatives? – Daniil Iaitskov Apr 25 '20 at 01:08
I don't know of any unfortunately. – Nikita Volkov Apr 26 '20 at 08:32

Design choice of Haskell data types in multithreaded programs

1 Answers1