5

I'm writing a bittorrent tracker in erlang. Given the nature of the service, I won't need absolute consistency (ie. a client can be perfectly happy with a slightly outdated list of peers or torrent status).

My strategy so far has been to create mnesia tables in RAM with disc_copies enabled, so to have mnesia automatically dump the memory to disk when the log size exceeds a certain size.

If the server crashes, some information will be lost. Not a big deal.

A different approach would be to instance two tables (one ram only and one disk only) and have a process copy from ram to disk every minute or so. This is more naive, but would allow to dump just a subset of what's in memory, reducing the overall disk overhead and possibly avoid the usage of a log altogether (I'm actually not sure about this last statement).

I'm sure there are many other ways to do this. What's yours?

-teo

Matteo Caprari
  • 2,869
  • 4
  • 27
  • 35

3 Answers3

2

You may want to checkout redis & erldis. Redis takes the second approach: everything is stored in memory then periodically dumped to disk.

Jacob
  • 4,204
  • 1
  • 25
  • 25
1

Here is a project that might interest you http://github.com/jlouis/etorrent/tree/master.

Christian
  • 9,417
  • 1
  • 39
  • 48
1

This is certainly off-topic in respect to your original question, but if you're really writing a pure tracker, then it might be best to forego persistence completely and keep the data solely in memory.

For a minimal tracker, an announce weighs in at only a few bytes: 16 bytes for the SHA1-hash, 6 bytes for the peer's IP and port, and some more bytes as you'll need to keep a timestamp as well. But even with a bit of overhead, you'll be able to literally keep millions of records in memory.

earl
  • 40,327
  • 6
  • 58
  • 59
  • 1
    Currently announce and torrent data are stored in memory-only mnesia tables (I still keep the disc copied disabled), which is of course extremely fast. Persisting to disk would anyway make the service more resilient to a system failure (even accounting for bittorrent implicit resilience) and to keep minimal torrent related information over time (numer of completed downloads). (http://mcaprari.github.com/peasy-torrent-tracker/) – Matteo Caprari Aug 06 '09 at 08:33