0

I've written a program that creates HTML files. I then attempt to upload the files to my S3 bucket at the end of the program. It seems that the problem is that my program terminates before allowing the function to complete or receiving a callback from the function.

Here is the gist of my code:

let aws = require('aws-sdk');

aws.config.update({
    //Censored keys for security
    accessKeyId: '*****',
    secretAccessKey: '*****',
    region: 'us-west-2'
});

let s3 = new aws.S3({
    apiVersion: "2006-03-01",
});

function upload(folder, platform, browser, title, data){
    s3.upload({
        Bucket: 'html',
        Key: folder + platform + '/' + browser + '/' + title + '.html',
        Body: data
    },  function (err, data) {
        if (err) {
            console.log("Error: ", err);
        }
        if (data) {
            console.log("Success: ", data.Location);
        }
    });
}

/*
*
* Here is where the program generates HTML files
*
*/

upload(folder, platform, browser, title, data);

If I call the upload() function (configured with test/dummy data) before the HTML generation section of my code, the upload succeeds. The test file successfully uploads to S3. However, when the function is called at the end of my code, I do not receive an error or success response. Rather, the program simply terminates and the file isn't uploaded to S3.

Is there a way to wait for the callback from my upload() function before continuing the program? How can I prevent the program from terminating before uploading my files to S3? Thank you!

Edit: After implementing Deiv's answer, I found that the program is still not uploading my files. I still am not receiving a success or error message of any kind. In fact, it seems like the program just skips over the upload() function. To test this, I added a console.log("test") after calling upload() to see if it would execute. Sure enough, the log prints successfully.

Here's some more information about the project: I'm utilizing WebdriverIO v4 to create HTML reports of various tests passing/failing. I gather the results of the tests via multiple event listeners (ex. this.on('test:start'), this.on('suite:end'), etc.). The final event is this.on('end'), which is called when all of the tests have completed execution. It is here were the test results are sorted based on which Operating System it was run on, Browser, etc.

I'm now noticing that my program won't to do anything S3 related in the this.on('end') event handler even if I put it at the very beginning of the handler, though I'm still convinced it's because it isn't given enough time to execute because the handler is able to process the results and create HTML files very quickly. I have this bit of code that lists all buckets in my S3:

s3.listBuckets(function (err, data) {
    if (err) {
        console.log("Error: ", err);
    } else {
        console.log("Success: ", data.Buckets);
    }
});

Even this doesn't return a result of any kind when run at the beginning of this.on('end'). Does anyone have any ideas? I'm really stumped here.

Edit: Here is my new code which implement's Naveen's suggestion:

this.on('end', async (end) => {

/*
* Program sorts results and creates variable 'data', the contents of the HTML file.
*/

    await s3.upload({
        Bucket: 'html',
        Key: key,
        Body: data
    }, function (err, data) {
        if (err) {
            console.log("Error: ", err);
        }
        if (data) {
            console.log("Success: ", data.Location);
        }
    }).on('httpUploadProgress', event => {
        console.log(`Uploaded ${event.loaded} out of ${event.total}`);
    });
}

The logic seems sound, but still I get no success or error message, and I do not see the upload progress. The HTML file does not get uploaded to S3.

Brian
  • 135
  • 4
  • 16
  • have you confirmed that the HTML was generated and didn't have a syntax error, etc? It might be useful to see that code. – chennighan Apr 09 '19 at 20:52
  • Yes, the HTML files are also saved locally and I am able to open, view, and use them correctly. Even if the HTML files did have syntax errors, shouldn't the file still upload to S3? And shouldn't I receive an `err` or `data` callback? – Brian Apr 09 '19 at 20:56

1 Answers1

3

You can use promises to wait for your upload function to finish. Here's what it will look like:

function upload(folder, platform, browser, title, data) {
    return new Promise((resolve, reject) => {
        s3.upload({
            Bucket: 'html',
            Key: folder + platform + '/' + browser + '/' + title + '.html',
            Body: data
        }, function(err, data) {
            if (err) {
                console.log("Error: ", err);
                return reject(err);
            }
            if (data) {
                console.log("Success: ", data.Location);
                return resolve();   //potentially return resolve(data) if you need the data
            }
        });
    });
}

/*
*
* Here is where the program generates HTML files
*
*/

upload(folder, platform, browser, title, data)
    .then(data => { //if you don't care for the data returned, you can also do .then(() => {
        //handle success, do whatever else you want, such as calling callback to end the function
    })
    .catch(error => {
        //handle error
    }
Deiv
  • 3,000
  • 2
  • 18
  • 30
  • Thanks for the response, it looked great and seemed like it was going to work, but for some reason, it doesn't :( I'm now starting to think that it's not being uploaded due to another issue. I'm going to update my question to include more information about my project that might shed some light on why the program refuses to upload my files. – Brian Apr 09 '19 at 22:26
  • If it's encountering an error it should show up in the logs...can you see where the logs stop? – Deiv Apr 09 '19 at 22:27
  • No need to use the Promise() wrapper any more (actually since March 2016). All SDK functions have a .promise() method: data = await s3.upload(params).promise() – jarmod Apr 09 '19 at 22:31
  • I've updated the question with more information. There is no error or success shown in the console - it simply shows the results of my tests. I can give you the console log if you'd really like to see it, but I think the takeaways are: `npm ERR! code ELIFECYCLE npm ERR! errno 1 npm ERR! proj@10.5.0 login: wdio test/webdriver/webdriverio/wdio.conf.js --suite login npm ERR! Exit status 1 npm ERR! npm ERR! Failed at the proj@10.5.0 login script.` – Brian Apr 09 '19 at 23:24
  • This is actually an error directly from npm, not related to the code itself. I'm not sure how to fix this but I suggest posting this error as a new question with the node.js tag and getting assistance that way. Make sure to include details like npm version, etc – Deiv Apr 10 '19 at 13:58
  • @Brian might be you need to check at which point in the WDIO code you are trying to call the `upload` function. Since you need this at the end of all test, you can make use of the `onComplete` hook. Make the `onComplete` function as `async` and call the `upload` function inside that with `await`. Like: onComplete: async function(....){ .... let status = await upload(arguments) .... } – Naveen Thiyagarajan Apr 13 '19 at 04:27
  • @NaveenThiyagarajan Thank you so much for your help Naveen. But could you clarify? I do not use `onComplete`, instead I use `this.on('end', (end) => {}` . I'll edit my question to show how I tried to implement your solution – Brian Apr 16 '19 at 18:28
  • Hi @Brian .. Please try by including Deiv's answer. i.e., 1. Edit your `function upload(..) ...` as mentioned by Deiv. So that `upload(..)` would return a `promise`. 2. Try this: `this.on('end', async (end) => { await upload(.. arguments ..) })` – Naveen Thiyagarajan Apr 16 '19 at 18:59