Read Excel file from Google Cloud Storage

Question

I am trying to read a spreadsheet uploaded to Google Storage from Nodejs App running on App Engine.

The function offered by @google-cloud/storage package in Nodejs is createReadStream(). I am pass this stream to the Exceljs object.

The code is as follows:

const blob = bucket.file(req.file.originalname);
const blobStream = blob.createWriteStream();
blobStream.on('finish', () => {

    let readStream = storage.bucket(bucket.name).file(blob.name).createReadStream();

    let returnString = '';

    const workbook = new Excel.stream.xlsx.WorkbookReader();
        var options = {
        entries: "emit",
        sharedStrings: "cache",
        worksheets: "emit"
    };

    workbook.read(readStream, options);

    workbook.on('worksheet', function (worksheet) {
        console.log("worksheet", worksheet.name);
        worksheet.on('row', function (row) {
            if(row.values.length > 0){
                console.log(" row.values", row.values[1]);
                returnString += row.values[1] + " ";
            }
        });

        worksheet.on('close', function () {
            console.log("worksheet close");
        });

        worksheet.on('finished', function () {
            console.log("worksheet finished");
            res.send(returnString);
        });
    });
});

blobStream.end(req.file.buffer);

Is workbook.read(readStream, options) getting the right readStream?

As I am getting the following error mentioning this:

UnhandledPromiseRejectionWarning: Error: Could not recognise input
2019-07-07 20:35:09 default[20190708t020308]      at module.exports._getStream (/srv/node_modules/exceljs/dist/es5/stream/xlsx/workbook-reader.js:58:11)
2019-07-07 20:35:09 default[20190708t020308]      at module.exports.read (/srv/node_modules/exceljs/dist/es5/stream/xlsx/workbook-reader.js:79:37)
2019-07-07 20:35:09 default[20190708t020308]      at Pumpify.blobStream.on (/srv/app.js:80:14)
2019-07-07 20:35:09 default[20190708t020308]      at Pumpify.emit (events.js:198:13)
2019-07-07 20:35:09 default[20190708t020308]      at finishMaybe (/srv/node_modules/readable-stream/lib/_stream_writable.js:630:14)
2019-07-07 20:35:09 default[20190708t020308]      at afterWrite (/srv/node_modules/readable-stream/lib/_stream_writable.js:492:3)
2019-07-07 20:35:09 default[20190708t020308]      at onwrite (/srv/node_modules/readable-stream/lib/_stream_writable.js:483:7)
2019-07-07 20:35:09 default[20190708t020308]      at Pumpify.WritableState.onwrite (/srv/node_modules/readable-stream/lib/_stream_writable.js:180:5)
2019-07-07 20:35:09 default[20190708t020308]      at Object.onceWrapper (events.js:286:20)
2019-07-07 20:35:09 default[20190708t020308]      at Pumpify.emit (events.js:198:13)
2019-07-07 20:35:09 default[20190708t020308]      at Pumpify.Duplexify.uncork (/srv/node_modules/duplexify/index.js:77:50)

Looking at the source of the exceljs code here ... https://github.com/exceljs/exceljs/blob/master/lib/stream/xlsx/workbook-reader.js it seems that your stream is not an instance of "Stream.Readable". This would be my first test ... make sure that your readStream feels good when it is to be used. — Kolban, Jul 07 '19 at 22:50
Earlier I was uploading the file to cloud storage and then reading it from cloud function, but then now I am reading it directly from buffer. — thewebjackal, Oct 12 '19 at 03:45

score 1 · Accepted Answer · answered Mar 28 '20 at 18:36

When you upload a file using multer and cloud storage, the "process" it's: You have a file in a buffer by multer, and then, with this buffer, you writes a file in cloud storage, so you can take advantage of this just reading the multer buffer with exceljs.

const Stream = require('stream'); //Stream is a node resourse
const blob = bucket.file(req.file.originalname);
const blobStream = blob.createWriteStream();
blobStream.on('finish', () => {
  // ToDo when the file has uploaded in storage
})
let workbook = new Excel.Workbook();
let stream = new Stream.Readable(); 
stream.push(req.file.buffer); // using multer buffer
stream.push(null);
workbook.xlsx.read(stream).then((workbook)=> {
  // ToDo with excel file readed
})

Read Excel file from Google Cloud Storage

1 Answers1