Variable not set in function nodejs

Question

I want to assign JSON data to a variable by parsing a warc file in a function. The variable is inaccessible outside a function and returns an empty array on the console.

var metadataObj = {
  metadata: []
};
fs
  .createReadStream('mywarc-file.warc')
  .pipe(new WARCStreamTransform())
  .on('data', record => {
    if (targetURL === record.warcHeader['WARC-Target-URI']){
      if(record.warcHeader['WARC-Type'] === 'response'){
        metadataObj.metadata.push({
          Url: record.warcHeader['WARC-Target-URI'],
          WarcID:record.warcHeader['WARC-Warcinfo-ID'],
          Timestamp:record.warcHeader['WARC-Date'],
          ContentType:record.warcHeader['Content-Type']
        })
      }else{
        metadataObj.metadata.push({
          Host: record.httpInfo.headers['Host'],
          userAgent: record.httpInfo.headers['User-Agent']
        })
      }
    }
  })
console.log(metadataObj.metadata)

Max Larionov · Answer 1 · 2019-01-09T10:23:02.727

Actually, the variable metadataObj is being accessible and, probably, set. The problem is, console.log statement is being executed before the new variable value is set.

I would recommend you to read more about async code handling and variable scopes in javascript.

Here is your code fixed:

var metadataObj = {
  metadata: []
};
fs
  .createReadStream('mywarc-file.warc')
  .pipe(new WARCStreamTransform())
  .on('data', record => {
    if (targetURL === record.warcHeader['WARC-Target-URI']){
      if(record.warcHeader['WARC-Type'] === 'response'){
        metadataObj.metadata.push({
          Url: record.warcHeader['WARC-Target-URI'],
          WarcID:record.warcHeader['WARC-Warcinfo-ID'],
          Timestamp:record.warcHeader['WARC-Date'],
          ContentType:record.warcHeader['Content-Type']
        })
      }else{
        metadataObj.metadata.push({
          Host: record.httpInfo.headers['Host'],
          userAgent: record.httpInfo.headers['User-Agent']
        })
      }
    }
  })
  .on('end', () => {
    console.log(metadataObj.metadata)
  })

Useful sources to read:

https://developer.mozilla.org/en-US/docs/Learn/JavaScript/Asynchronous/Introducing#Asynchronous_JavaScript

https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/Promise

https://medium.com/codebuddies/getting-to-know-asynchronous-javascript-callbacks-promises-and-async-await-17e0673281ee

https://medium.com/javascript-scene/master-the-javascript-interview-what-is-a-promise-27fc71e77261

Variable not set in function nodejs

1 Answers1