3

I have a D3 project that uses a very large CSV file to read the data. It works fine except sometimes user sees the spinning loading image indefinitely because of one the following issues.
I need to be able to catch these issues and display some meaningful message to user rather than the never-ending wait message:

  1. When the data file (MyData.csv) doesn’t exist
  2. When the MyData.csv exists but is empty
  3. When the MyData.csv exists, is not empty but is improperly formatted (change the name of a column header on first row for testing).

Ideally, I would like to be able to distinguish among them.

function loadData() {
    d3.csv('MyMap.csv', function(data) {
        data.forEach(function(d) {
            doSomething()
        });

        // load primary data set
        d3.csv('MyData.csv', function (data) {
            doSomethingWithData(data);
        }); // main data set
    });
};

Updated with error checking

var ErrorEnum = Object.freeze({NOTFOUND: 1, EMPTY: 2, INVALIDFORMAT: 3});
var mapFileName = 'MyMap_empty.csv';
var dataFileName = 'MyData_empty.csv';

function redirectOnFileError(fileName, url, errID) {
    var errTitle = errID == ErrorEnum.NOTFOUND ? "File Not Found" : (errID == ErrorEnum.EMPTY ? "Empty File" : "Invalid File Format");
    var errText = fileName + (errID == ErrorEnum.NOTFOUND ? " could not be found" : (errID == ErrorEnum.EMPTY ? " is empty" : " has invalid format"));
    console.log(errText);

    swal({
        title: errTitle,
        text: errText,
        type: "error",
        confirmButtonText: "OK"
        },
        function(isConfirm){
            // somehow we never get here!
            console.log("here ...");
            if (isConfirm) {
                    window.location.href = url;
            }
        }); 
        // hack ... callback function above is not executed, so using below jQuery to use "OK" button's class and 'click' it
        $('.swal2-confirm').click(function(){
            window.location.href = url;
    });
}

function loadData() {
    d3.csv(mapFileName, function (error, data) {
        if (error && error.status === 404) {
            redirectOnFileError(mapFileName, "fnf.html", ErrorEnum.NOTFOUND);
        }
        else if (data.length === 0) {
            redirectOnFileError(mapFileName, "ef.html", ErrorEnum.EMPTY);
        }
        else {
            data.forEach(function (d) {
                // DoSomethingHere();
            });

        // load primary data set
        d3.csv(dataFileName, function (error, data) {
            if (error && error.status === 404) {
                redirectOnFileError(dataFileName, "fnf.html", ErrorEnum.NOTFOUND);
            }
            else if (data.length === 0) {
                redirectOnFileError(dataFileName, "ef.html", ErrorEnum.EMPTY);              }
            else {
                // DomSomethingWithData();
            }); // main data set
    });
}
}
};
NoBullMan
  • 2,032
  • 5
  • 40
  • 93

1 Answers1

2

Both the first and the second scenarios can be answered here (for the third scenario, see the PS below).

Before we start, let's see a working, existing CSV file:

 d3.csv("https://gist.githubusercontent.com/GerardoFurtado/2e9f269f7a0f56cf7e23b480afcf280a/raw/5e361753a695534915e81786013976aa94685695/csvfile.csv", function(error, data) {
  console.log(data)
});
<script src="https://d3js.org/d3.v4.min.js"></script>

As you can see, everything works. This is the CSV I'll use in the next snippets.

First scenario

Your first scenario...

When the data file (MyData.csv) doesn’t exist

... can be handled using the first parameter in the d3.csv function, which is the error (if any):

d3.csv(url, function(error,data){...
//error here ----------^

In the following snippet we're checking for target.status in the error. It will give you a 404, which is printed in the console. I'm just changing the url of the previous CSV file:

d3.csv("https://gist.githubusercontent.com/GerardoFurtado/2e9f269f7a0f56cf7e23b480afcf280a/raw/fakeurl.csv", function(error, data) {
  if (error && error.target.status === 404) {
      console.log("File not found")
    }
});
<script src="https://d3js.org/d3.v4.min.js"></script>

Second scenario

Your second scenario...

When the MyData.csv exists but is empty

... can be fixed checking for the length of the data. Here is a demo, using a modified version of my first CSV (without the rows):

d3.csv("https://gist.githubusercontent.com/GerardoFurtado/2e9f269f7a0f56cf7e23b480afcf280a/raw/5e361753a695534915e81786013976aa94685695/csvfileempty.csv", function(error, data) {
  if (error && error.target.status === 404) {
      console.log("File not found")
    }
  if(data.length === 0){
  console.log("File empty")
  }
});
<script src="https://d3js.org/d3.v4.min.js"></script>

PS: Your third situation ("file is improperly formatted") can be solved only by you: you are the only one who knows the structure of the file.

Gerardo Furtado
  • 100,839
  • 9
  • 121
  • 171
  • Thank you very much. I will test this out and post back the result. – NoBullMan Mar 18 '17 at 04:58
  • I edited the question and the code, incorporating what you had suggested into it, but it seems I am doing something wrong; I get "d3 is undefined" in dc.js which I wasn't getting before. I guess it is something with the way I incorporated your code into mine. I am new to D3/CrossFilter/DC world, could be the way I added if...then...else. – NoBullMan Mar 18 '17 at 06:09
  • @NoBullMan Don't edit your answer to put my code in it: people will think that it was your attempt. Leave *your* code in the question. That way, people can propose solutions based on your code, not mine. Also, have in mind that, since I didn't see the DC tag in the question, my answer uses "pure" D3. Please, edit your question and add the DC tag. – Gerardo Furtado Mar 18 '17 at 11:21
  • I updated the post and left the original question intact and added the updated code, with you suggestion, under it. I didn't think DC was relevant because the code works if the files exist and are not empty. It was the file error checks I was missing. I still have the issue I mentioned after I added the updates; probably something with the way I added your suggested changes (the way "{" and "}: are added in the if...then...else statements. – NoBullMan Mar 18 '17 at 15:32
  • In my code i have other stuff and i added your code and i think i broke something; like wrong braces locations! – NoBullMan Mar 18 '17 at 16:58
  • I got this to work except when for testing I created an empty MyData.csv file, instead of displaying "File Empty" in console, it displays "File Not Found". – NoBullMan Mar 19 '17 at 07:30
  • After adding previous comment, I updated the post with latest code to show how I am using the file error scenarios. I am also trying to use SweetAlert2, with redirection, which works in FF and Chrome (except for redirection part, just shows the alert) and totally chokes in IE, but at least I can check it in Chrome (different issue). Not sure why empty csv file is taken as "file does not exist". – NoBullMan Mar 19 '17 at 07:47
  • It depends on *how* do you define empty. My empty file has the header row. That's why the `data.length` is zero. – Gerardo Furtado Mar 19 '17 at 07:49
  • OK, so is it possible to catch the error where file is completely empty (zero length)? – NoBullMan Mar 19 '17 at 16:17
  • Please, send the link of this zero bytes file. – Gerardo Furtado Mar 20 '17 at 01:17
  • I am sorry; what do mean by sending the link? This is for an intranet application. What i did was that I created an empty text file and named it MyData.csv, for testing zero byte file size situation. – NoBullMan Mar 20 '17 at 04:40
  • If you copy/paste the link of this empty csv file here in the comments I can put it in the snippet and test the result. – Gerardo Furtado Mar 20 '17 at 04:50