1

We are working on an event-driven accounting engine and so far we are doing everything in a batch/sequential manner.

Problem is there are thousands of events created per day and processing everything sequentially makes it slow.

Is there a safe way we can implement a multi-thread event processing accounting engine without worrying about financial data integrity and consistency?

Or is it just better to play it safe and allows follow a batch/sequential approach?

code-ninja
  • 1,586
  • 1
  • 15
  • 23

3 Answers3

2

Perhaps you might want to look into software transactional memory model. The concept has been discussed in this paper

Neera
  • 1,577
  • 8
  • 10
1

You already have managed building an accounting engine satisfying your financial data integrity and consistency requirements. With this, it should be feasible to parallelize processing.

You would need

  • a new dispatcher component which would be responsible for feeding multiple threads with to-be-processed event data while maintaining consistency (i.e. bill each event exactly once) and
  • a new aggregator component for muxing the rated events on the accounts.

Typically the aggregator component will become the bottleneck in your parallel architecture.

Since you explicitly intend to build a multi-thread application: Depending on requirements and environment it might be feasible to build a multi-process design, i.e. have several independent processes running in parallel. You would gain a simpler concurrency model versus the evemtual need for interprocess communication.

Virtually all telecom billing systems scale by multi-threading or multi-processing so this is definitelly a sane way forward.

Bernd
  • 3,390
  • 2
  • 23
  • 31
0

You can use multiple concurrent transactions if you follow the rules

check out
en.wikipedia.org/wiki/Atomicity_(database_systems)
en.wikipedia.org/wiki/ACID
en.wikipedia.org/wiki/Record_locking

or read all relevant articles in this section
http://en.wikipedia.org/wiki/Category:Transaction_processing

with the correct level of locking for what you are doing

ColinDNZ
  • 1
  • 1