1

Architectural question here. I have a Web App (HTML5/Javascript/AngularJS) which logs user activity into a Backend DB via Web API. This function runs quite frequently on events such as Menu and button Clicks throughout the app. Making an API call for each such event seems expansive, and I was wondering what are best practices for these kind of updates, Or, How is that being implemented with a service such as Google Analytics on their client scripts?

kob490
  • 3,177
  • 4
  • 25
  • 40
  • The key question here is, what do you mean by 'log user activity'? I'd be tempted to use GA where you can, and your own database logging where the information is confidential, or too complex for GA to handle, or (if you are using a free GA account) where GA is snapshotting and you don't want that. Re your final question, GA snapshots (i.e. doesn't handle absolutely every single event) where it wants to, unless you pay lots for a full licence. btw you might like to look at GA custom dimensions. – MandyShaw Nov 03 '18 at 20:40
  • thank you for the answer. the reason for wanting to log user activity locally is because of the need of the client to retain confidentiality and full control over the logged information. – kob490 Nov 05 '18 at 18:49
  • All depends how many of your events/users you really need to track, then .... and whether your database can take the extra traffic, obviously. You might perhaps like to consider some sort of queuing mechanism that has a lightweight Javascript client, with the database writes happening asynchronously. (IBM MQ apparently has one: https://www.ibm.com/support/knowledgecenter/en/SSFKSJ_7.5.0/com.ibm.mm.tc.doc/tc60302_.htm) – MandyShaw Nov 05 '18 at 20:30

1 Answers1

3

I've been looking at something very similar where I work. First you really want the data sent to your API as small as possible so that sending all of these events doesn't use to much bandwidth. The next thing we realised was to use a queue of some sort, RabbitMQ, Azure Service Bus, MSMQ or Amazon SQS or something similar.

The API endpoint should just take the data and put it on the queue. Then have another program or service that then reads the messages on the queue at it's own pace and does any inserts into databases.

At peak times you shouldn't then see any dip in performance from the API, the only thing that will happen when your system is being hammered is that the data may take a few seconds to a few minutes to be inserted into the DB.

You're essentially batching up on the queue and don't have to worry about batching client side

matt_lethargic
  • 2,706
  • 1
  • 18
  • 33