How to best use socket.io and pub/sub to create connection to API call - Nodejs

Question

I have created my first application of Nodejs. I am using socket.io and express. I tried this tutorial: https://socket.io/get-started/chat/ and I find emit is what I might need not sure.

My application shows stock tickers, I am using API's, I have url like: https://quotes.example.com/v1/file.json/johndoe?&_token=tokenumber

All the users who open http://example.com/live page, I would like to push or emit stock tickers to this page. How do I best achieve this? I get the idea of live page needs to be a channel to which all user would subscribe and I am pushing data to it.

I have never worked on real-time apps before, all suggestions are welcomed.

Edited

Front-end code

<!doctype html>
<html>
  <head>
    <title>Live Stock Quotes App</title>
    <style>
      body { font: 26px Helvetica, Arial; font-weight:bold;}

      #livequotes { text-align: center;}
    </style>
  </head>
  <body>
    <p id="livequotes"></p>


    <script src="/socket.io/socket.io.js"></script>
<script src="https://code.jquery.com/jquery-1.11.1.js"></script>

<script>
  $(function () {
    var socket = io('/channel1');

     socket.on('live-quote', function(msg){
      $('#livequotes').text(msg);
    });
  });
</script>

  </body>
</html>

server-side code

var app = require('express')();
var http = require('http').Server(app);
var httpk = require('http');
var io = require('socket.io')(http);
var nsp = io.of('/channel1');

app.get('/', function(req, res){
  res.sendFile(__dirname + '/index.html');
});

nsp.on('connection', function(socket){

  nsp.emit('live-quote', 'Welcome User!');
  //Make a http call

  function test()
  {
    httpk.get("url-to-api", function(res) {
        var body = ''; // Will contain the final response

        res.on('data', function(data){
            body += data;
        });

        res.on('end', function() {
            var parsed = JSON.parse(body);
            console.log(parsed.johndoe.bid_price);
            return parsed.johndoe.bid_price;
        });
    });
  }

  setInterval(test,500);

  socket.on('disconnect', function(){
    console.log('1 user disconnected');
  });

});

http.listen(3000, function(){
  console.log('listening on *:3000');
});

Above code I have put as you suggested. Now, I wanted to do add something like:

setInterval(test,500);

function test (){
    http.get("url-for-api", function(res) {
        var body = ''; // Will contain the final response

            res.on('data', function(data){
            body += data;
        });

        // After the response is completed, parse it and log it to the console
        res.on('end', function() {
            var parsed = JSON.parse(body);
            console.log(parsed.johndoe.bid_price);
            data = parsed.johndoe.ask_price;

        });
    })
    // If any error has occured, log error to console
        .on('error', function(e) {
            console.log("Got error: " + e.message);
        });

}

Are you pushing the same data to all users? But basically yeah, user loads your page, they connect to your socket, you pump the data back out using emit. — Gary Batterbee, Jun 11 '17 at 20:15
@GaryBatterbee Yes, I would be using same data to all. The data will be changing every 500ms. I will be emitting every 500ms or whever there is new data on the API end. Can you show how to achieve this? I will accept the answer. — Murlidhar Fichadia, Jun 11 '17 at 20:38
Everything you need should be in the chat example - broadcasting section . Server side, just do io.emit('stock message', msg); client side just do socket.on('stock message', function(msg){ $('#messages').append($('
@GaryBatterbee if you could look at my code. It works fine. But I dont know how to send or push the data to the front-end. I am able to console.log the data returned from an API call but not to the client-side. I did nsp.emit('live-quote', data); where data = setInterval(test,500); — Murlidhar Fichadia, Jun 11 '17 at 22:00

jfriend00 · Accepted Answer · 2017-06-11T21:00:38.357

1

It's not entirely clear exactly what you're asking, but I'll take a stab at it.

If you want to be able to "push" new stock updates to a client using socket.io, then the usual way you would do that is a client would make a socket.io connection to your server and then your server can issue updates to all connected clients by doing an io.emit(msg, data) on the server. That will send that message and data to all connected clients.

If you want these updates to be sent to a web page, then you put Javascript in that web page that makes the socket.io connect to your server and then listens on that socket for the appropriate messages from the server. When it receives the messages from the server, it can then insert them into the page in whatever fashion you find appropriate.

This client code from the socket.io doc pages would be typical of the client-side:

<script src="/socket.io/socket.io.js"></script>
<script>
  var socket = io();
  socket.on('udpate', function (data) {
    console.log(data);
    // do something with the data here (like insert it into the web page)
  });
</script>

If you have to tell the server which stock updates you're interested in so only those updates are sent to the client page, then you will need to do more work on both client and server to allow the server to keep track of which connection is interested in which updates and only send to the right connections. You could use socket.io rooms to help manage that. A client connection requests to join a room for a given stock and then stock updates for a particular stock are sent only to the clients in that corresponding room.

edited Jun 11 '17 at 21:00

answered Jun 11 '17 at 20:55

jfriend00

683,504
96
985
979

I have added the code as you have suggested. Please have a look. I have added additional code after server-side code in my question. Is it good or correct strategy to setInterval of 500ms and make a call to Test function and then finally push the data to the front-end? – Murlidhar Fichadia Jun 11 '17 at 21:40
Just wanted to know, how many users could join a channel simultaneously? – Murlidhar Fichadia Jun 11 '17 at 22:07
1

@murlidhar - there is no particular limit. It would depend upon dozens of factors and could be thousands or hundreds of thousands. – jfriend00 Jun 11 '17 at 22:10
yes, I understand. one more question, let's say there is channelA which contains 15,000 concurrent connections and let's say it starts to act weird above 15,000 connections (due to some factors). If I were to have channelB with other 5000 connections. Do I have to consider the no. of connections limited as a whole regardless number of channels I have. Or do each channel have their own limit depending on what that channel does? I am just wondering if sum of connections in all the channels is what matters or what – Murlidhar Fichadia Jun 11 '17 at 22:24
@MurlidharFichadia - Since "channel" is not a socket.io term, I don't know exactly what you mean by that. A limitation on how many simultaneous connections can be influenced by all sorts of things from how much a single node.js process can get out of the CPU to how many sockets your server can handle, to total bandwidth available. There are lots of things that could affect it. A high scale farm would have specially configured servers to support large numbers of sockets, multiple node.js processes, multiple network cards per server, multiple server boxes running multiple processes, etc... – jfriend00 Jun 12 '17 at 01:47
@MurlidharFichadia - Only heavy load testing could reveal where the bottlenecks would be in your particular application and thus what the first roadblock is to work-around. – jfriend00 Jun 12 '17 at 01:48

How to best use socket.io and pub/sub to create connection to API call - Nodejs

Edited

1 Answers1