Sails.js Redis How to stream data?

Question

I have been looking for info to know how to connect Redis to Sails.js throw model - controller - view and I think I've got it.

My problem is that I think I don't understand the philosophy of Redis, its keys, etc.

I guess my Redis has not any keys, I mean it's not something like "name : Victor, surname : Garcia" but " Victor:García", wihtout a key, so I don't know how to set an attribute in the model of Redis.

So I've tried either no setting attributes or set just an attribute for the whole key (e.g. "Victor:Garcia:33:Seville:Spain") but without a result.

My model looks like this:

module.exports = {
    schema: false,
    connection: 'redis',
    autoPK: false,
    autoCreatedAt: false,
    autoUpdatedAt: false,
    attributes: {
        ta : 'string',
        // Override toJSON instance method 
        toJSON: function() {
          var obj = this.toObject();
          return obj;
        }
    }
};

I tried this:

Redis.find()
.where({ ta: 'MB:400V:TRAF004:EP:MvMoment' })
.exec(function(err, users) {
     // Do stuff here 
     console.log("User: " + users + " - " + err);
});

With no result, in fact, I get in console:

User: - null

No matter what I set in .where.

I also Tried to do something like this:

Model

module.exports = {
    schema: false,
    connection: 'redis',
    autoPK: false,
    autoCreatedAt: false,
    autoUpdatedAt: false
};

Without attributes, and I tried in the controller

Redis.find()
    .where({ ta: 'MB:400V:TRAF004:EP:MvMoment' })
    .exec(function(err, users) {
         // Do stuff here 
         console.log("User: " + users + " - " + err);
    });

Taking a look in google I've found this

http://sailsjs.org/#!/documentation/reference/waterline/models/stream.html

And I've tried to do the sample in my application.

Model (Redis.js):

module.exports = {
        schema: false,
        connection: 'redis',
        autoPK: false,
        autoCreatedAt: false,
        autoUpdatedAt: false
    };

Controller:

estStream: function(req,res){

    if (req.param('startStream') && req.isSocket){

        var getSocket = req.socket;

        // Start the stream.  Pipe it to sockets.
        Redis.stream().pipe(getSocket.emit);

    } else {

      res.view();

    }

View:

<script type="text/javascript">
window.onload = function startListening(){
    socket.on('gotUser',function(data){
      console.log(data+' has joined the party');
    });
};

</script>
<div class="addButton" onClick="socket.get('/monweb/testStream/',{startStream:true})">Stream all the Users !</div>

Bur when I click the button the result by console is as below:

error: Sending 500 ("Server Error") response: 
 TypeError: Object function (ev) {
  if (ev == 'newListener') {
    return this.$emit.apply(this, arguments);
  }

  var args = util.toArray(arguments).slice(1)
    , lastArg = args[args.length - 1]
    , packet = {
          type: 'event'
        , name: ev
      };

  if ('function' == typeof lastArg) {
    packet.id = ++this.ackPackets;
    packet.ack = lastArg.length ? 'data' : true;
    this.acks[packet.id] = lastArg;
    args = args.slice(0, args.length - 1);
  }

  packet.args = args;

  return this.packet(packet);
} has no method 'on'
    at Stream.pipe (stream.js:65:8)
    at module.exports.testStream (/home/victor/gestamp-PRUEBAS/api/controllers/MonWebController.js:1364:24)
    at routeTargetFnWrapper (/home/victor/gestamp-PRUEBAS/node_modules/sails/lib/router/bind.js:178:5)
    at callbacks (/home/victor/gestamp-PRUEBAS/node_modules/sails/node_modules/express/lib/router/index.js:164:37)
    at param (/home/victor/gestamp-PRUEBAS/node_modules/sails/node_modules/express/lib/router/index.js:138:11)
    at param (/home/victor/gestamp-PRUEBAS/node_modules/sails/node_modules/express/lib/router/index.js:135:11)
    at pass (/home/victor/gestamp-PRUEBAS/node_modules/sails/node_modules/express/lib/router/index.js:145:5)
    at nextRoute (/home/victor/gestamp-PRUEBAS/node_modules/sails/node_modules/express/lib/router/index.js:100:7)
    at callbacks (/home/victor/gestamp-PRUEBAS/node_modules/sails/node_modules/express/lib/router/index.js:167:11)
    at /home/victor/gestamp-PRUEBAS/node_modules/sails/lib/router/bind.js:186:7
    at alwaysAllow (/home/victor/gestamp-PRUEBAS/node_modules/sails/lib/hooks/policies/index.js:209:11)
    at routeTargetFnWrapper (/home/victor/gestamp-PRUEBAS/node_modules/sails/lib/router/bind.js:178:5)
    at callbacks (/home/victor/gestamp-PRUEBAS/node_modules/sails/node_modules/express/lib/router/index.js:164:37)
    at param (/home/victor/gestamp-PRUEBAS/node_modules/sails/node_modules/express/lib/router/index.js:138:11)
    at param (/home/victor/gestamp-PRUEBAS/node_modules/sails/node_modules/express/lib/router/index.js:135:11)
    at pass (/home/victor/gestamp-PRUEBAS/node_modules/sails/node_modules/express/lib/router/index.js:145:5) [TypeError: Object function (ev) {
  if (ev == 'newListener') {
    return this.$emit.apply(this, arguments);
  }

  var args = util.toArray(arguments).slice(1)
    , lastArg = args[args.length - 1]
    , packet = {
          type: 'event'
        , name: ev
      };

  if ('function' == typeof lastArg) {
    packet.id = ++this.ackPackets;
    packet.ack = lastArg.length ? 'data' : true;
    this.acks[packet.id] = lastArg;
    args = args.slice(0, args.length - 1);
  }

  packet.args = args;

  return this.packet(packet);
} has no method 'on']

events.js:74
        throw TypeError('Uncaught, unspecified "error" event.');
              ^
TypeError: Uncaught, unspecified "error" event.
    at TypeError (<anonymous>)
    at emit (events.js:74:15)
    at ModelStream.end (/home/victor/gestamp-PRUEBAS/node_modules/sails/node_modules/waterline/lib/waterline/utils/stream.js:61:10)
    at module.exports.stream (/home/victor/gestamp-PRUEBAS/node_modules/sails/node_modules/waterline/lib/waterline/adapter/stream.js:25:66)
    at /home/victor/gestamp-PRUEBAS/node_modules/sails/node_modules/waterline/lib/waterline/query/stream.js:42:20
    at process._tickDomainCallback (node.js:492:13)

Sorry, I know I am new with this, I am sure that I am doing something (or a lots of things) wrong, could you help me? All what I want is stream data from Redis to the View. I would like to see in the view every change in Redis.

Thank you all.

1. Do you already have data in Redis DB? You need to get data from present Redis DB, right? 2. You can get data without _where_ : `Redis.find().exec(function(err, users) { console.log("User: " + users + " - " + err); });` So what if you request data like that? — Bulkin, Jun 01 '15 at 12:15

score 2 · Answer 1 · answered Jun 04 '15 at 10:35

There are several questions you are asking so let's try to answer them one by one:

Redis schema format

Redis has support for several data structures out of which sails-redis uses set for indexes and binary string for storing data. If a part of the data can be used as primary key it can increase your system performance if you use it, if not sails-redis auto generates a primary key based on a sequence.

According to your definitions looks like your schema should look like that:

module.exports = {
  connection: 'your redis connection name'
  attributes: {
    ta : { type: 'string' }
  }
}

to start working with it you can use the blueprints api by creating an empty controller.

Retrieving data

To get the data from redis you can use the blueprints api to get the data by the generated id or that you can create a new controller using the find method, this is an example (sorry for calling the model simply Data):

module.exports = {
 findByTa: function (req, res) {
   Data.find({ where: { ta: req.param('ta') }}).exec(function (err, result) {
     res.json(result);
   });
 }

}

you'll also need to add a route for the new controller function in conf/routes.js

'GET /data/ta/:ta': 'DataController.findByTa',

Streaming data

The stream method is not implemented for sails-redis for now, that's the reason for the error you got.

Notifications on data changes

To get notifications on data changes you need to register the socket for the model to receive new instances and for model instances (records) to receive the update and delete events, this is a sample controller that registers on the model itself and subscribers to all existing instances.

testEvents: function (req, res) {
  if (req.isSocket){
    Data.watch(req); //register on the model
    Data.find({}).exec(function (err, result) {
      Data.subscribe(req.socket, result); // subscribe to model instance
    });  
  }
}

A route should be defined for the action:

'GET /data/testEvents': 'DataController.testEvents'

On the client side you should add some code that will call the controller and listen for events:

window.onload = function startListening(){
  io.socket.on('data',function(msg){
    console.log(msg);
  });
  io.socket.get('/data/testEvents');
};

Sails.js Redis How to stream data?

1 Answers1

Redis schema format

Retrieving data

Streaming data

Notifications on data changes