Meteor, mongodb - canteen optimization

Question

TL;DR: I'm making an app for a canteen. I have a collection with the persons and a collection where I "log" every meat took. I need to know those who DIDN'T take the meal.

Long version:

I'm making an application for my local Red Cross.

I'm trying to optimize this situation:

• there is a canteen at wich the helped people can take food at breakfast, lunch and supper. We need to know how many took the meal (and this is easy).

• if they are present they HAVE TO take the meal and eat, so we need to know how many (and who) HAVEN'T eat (this is the part that I need to optimize).

• When they take the meal the "cashier" insert their barcode, the program log the "transaction" in the log collection.

Actually, on creation of the template "canteen" I create a local collection "meals" and populate it with the data of all the people in the DB, (so ID, name, fasting/satiated), then I use this collection for my counters and to display who took the meal and who didn't. (the variable "mealKind" is = "breakfast" OR "lunch" OR "dinner" depending on the actual serving.)

Template.canteen.created = function(){
  Meals=new Mongo.Collection(null);
  var today= new Date();today.setHours(0,0,1);
  var pers=Persons.find({"status":"present"},{fields:{"Name":1,"Surname":1,"barcode":1}}).fetch();
  pers.forEach(function(uno){
    var vediamo=Log.findOne({"dest":uno.codice,"what":mealKind, "when":{"$gte": today}});
    if(typeof vediamo=="object"){
      uno['eat']="satiated";
    }else{
      uno['eat']="fasting";
    }
    Meals.insert(uno);
  });
};

Template.canteen.destroyed = function(){
   meals.remove({});
};

From the meal collection I estrapolate the two colums of people satiated (with name, surname and barcode) and fasting, and I also use two helpers:

  fasting:function(){
    return Meals.find({"eat":"fasting"});
  }
  "countFasting":function(){
    return Meals.find({"eat":"fasting"}).count();
  }
//same for satiated

This was ok, but now the number of people is really increasing (we are arount 1000 and counting) and the creation of the page is very very slow, and usually it stops with errors so I can read that "100 fasting, 400 satiated" but I have around 1000 persons in the DB.

I can't figure out how to optimize the workflow, every other method that I tried involved (in a manner or another) more queries to the DB; I think that I missed the point and now I cannot see it. I'm not sure about aggregation at this level and inside meteor, because of minimongo.

Although making this server side and not client side is clever, the problem here is HOW discriminate "fasting" vs "satiated" without cycling all the person collection.

+1 if the solution is compatibile with aleed:tabular

Answer 1

EDIT

I am still not sure about what is causing your performance issue (too many things in client memory / minimongo, too many calls to it?), but you could at least try different approaches, more traditionally based on your server.

By the way, you did not mention either how you display your data or how you get the incorrect reading for your number of already served / missing Persons?

If you are building a classic HTML table, please note that browsers struggle rendering more than a few hundred rows. If you are in that case, you could implement a client-side table pagination / infinite scrolling. Look for example at jQuery DataTables plugin (on which is based aldeed:tabular). Skip the step of building an actual HTML table, and fill it directly using $table.rows.add(myArrayOfData).draw() to avoid the browser limitation.

---

Original answer

I do not exactly understand why you need to duplicate your Persons collection into a client-side Meals local collection?

This requires that you have first all documents of Persons sent from server to client (this may not be problematic if your server is well connected / local. You may also still have autopublish package on, so you would have already seen that penalty), and then cloning all documents (checking for your Logs collection to retrieve any previous passages), effectively doubling your memory need.

Is your server and/or remote DB that slow to justify your need to do everything locally (client side)?

Could be much more problematic, should you have more than one "cashier" / client browser open, their Meals local collections will not be synchronized.

If your server-client connection is good, there is no reason to do everything client side. Meteor will automatically cache just what is needed, and provide optimistic DB modification to keep your user experience fast (should you structure your code correctly).

• With aldeed:tabular package, you can easily display your Persons big table by "pages".
• You can also link it with your Logs collection using the dburles:collection-helpers (IIRC there is an example en the aldeed:tabular home page).