Android reading and graphing data from text file using GraphView

Question

In my android application, I am attempting to read from a text file containing data points which I wish to graph. The number of data points in each file is expected to be anywhere between 24,000 to 150,000.

Currently, I have implemented an asynchronous task that reads the data from a compressed text file and stores it in a local vector. On the completion of the task, the data points are added to a data set in order to be graphed.

The code implementation for the asynchronous task is as follows:

class ReadFileService extends AsyncTask<Void, String, Boolean> {

    @Override
    protected Boolean doInBackground(Void... args) {        
        Scanner strings = null;
        InputStream stream = null;
        ZipInputStream zipInput = null;
        try {
            System.out.println(externalStorageDirectory + Constants.APP_DIRECTORY + recordingName + Constants.ZIP_FILE_EXTENTION);
            File file = new File(externalStorageDirectory + Constants.APP_DIRECTORY, recordingName + Constants.ZIP_FILE_EXTENTION);
            ZipFile zipFile = new ZipFile(file);
            Enumeration<? extends ZipEntry> entries = zipFile.entries();

            while (entries.hasMoreElements()) {
                ZipEntry zipEntry = entries.nextElement();
                stream = zipFile.getInputStream(zipEntry);
                strings = new Scanner(stream);

                // Extract the value of sampling frequency from header
                System.out.println("Extracting value of sampling frequency.");
                String regexPattern = "\"ColumnLabels\"";
                strings.useDelimiter(regexPattern);
                String extracted = strings.next();
                Pattern pattern = Pattern.compile("\"SamplingFrequency\": \"(\\d+)\"");
                Matcher matcher = pattern.matcher(extracted);
                if (matcher.find()) {
                    samplingFrequency = Integer.parseInt(matcher.group(1));
                    System.out.println(samplingFrequency);
                }

                // Locate the end of the header and use tabs as the delimiter
                strings.findWithinHorizon(endOfHeader,0);           
                strings.useDelimiter("\t *");
                strings.next();
            }
        }
        catch (FileNotFoundException error) {
            System.out.println("@IOERROR: " + error);
            return false;
        }
        catch (IOException error) {
            System.out.println("@IOERROR: " + error);
            return false;
        }
        while (strings.hasNext())
        {
            // Get raw value of data point
            double dataPoint = Double.parseDouble(strings.next());
            // Convert data point into the proper scale
            dataSet.add(SensorDataConverter.scaleEMG(dataPoint));
            // Skip the index value if it exists and break from loop if it does not
            if (strings.hasNext())
                strings.next();
            else
                break;
        }
        System.out.println("Closing strings.");
        try {
            stream.close();
        } catch (IOException e) {
            // TODO Auto-generated catch block
            e.printStackTrace();
        }
        strings.close();            
        return true;
    }

    protected void onProgressUpdate(String...progress) {
        //called when the background task makes any progress
    }

    protected void onPreExecute() {
        //called before doInBackground() is started
        super.onPreExecute();
        // Show Progress Bar Dialog before calling doInBackground method
        prgDialog.setTitle("Opening File");
        prgDialog.setMessage("Opening " + recordingName + "\nPlease wait...");
        prgDialog.show();
    }

    //called after doInBackground() has finished
    protected void onPostExecute(Boolean readFileSuccess) { 
        // If unable to read from file, print error and generate a random set of sample data
        if(!readFileSuccess) {
            Random randomGenerator = new Random();          
            System.out.println("@IOERROR: Unable to read from file. Creating random dataset");
            for(int i=0; i<100; i++)
            {
                dataSet.add(randomGenerator.nextDouble());
            }
        }
        // Create a set of graph data to use with GraphView
        exampleSeries1 = new GraphViewSeries(new GraphViewData[] {
        });
        for (int i=0; i<dataSet.size(); i++) {
            double pointX = i;
            double pointY = dataSet.get(i);
            exampleSeries1.appendData(new GraphViewData(pointX, pointY), true, dataSet.size());
        }
        // Plot the data points
        graphData();
        prgDialog.dismiss();
        prgDialog = null;
    }
}

This appears to work when saving and loading approximately 10,000 data points and takes approximately 15 seconds to load. However, when the number of data points jumps up to 20,000, it takes upwards to 45 seconds to load.

My main question is: Is there another way of more efficiently handling all this data in order to minimize the length of time required to load data? For example, would it be possible to dynamically graph data points depending on the view port range?

Any help and suggestions would be much appreciated.

Answer 1

you have to resample the data to have smaller amount of data points. normally it doesn't make sense to plot more that 100 datapoints at a single viewport.

and use manual viewport and manual label width/height to improve performance