Getting JSON from System.out.println

Question

I am trying to do sentiment analysis and project the values on Google Visualization.

I am calling this python script using my java program

Code Snippet (for AlchemyAPI)

https://github.com/AlchemyAPI/alchemyapi-twitter-python

I wrote a java program to call the python script.

import java.io.*;
public class twitmain {
    public String twittersentiment(String[] args) throws IOException {
        // set up the command and parameter
        String pythonScriptPath = "/twitter/analyze.py"; // I'm calling AlchemyAPI                                          
        String[] cmd = new String[2 + args.length];
        cmd[0] = "C:\\Python27\\python.exe";
        cmd[1] = pythonScriptPath;
        for (int i = 0; i < args.length; i++) {
            cmd[i + 2] = args[i];
        }
        // create runtime to execute external command
        Runtime rt = Runtime.getRuntime();
        Process pr = rt.exec(cmd);

        // retrieve output from python script
        BufferedReader bfr = new BufferedReader(new InputStreamReader(
                pr.getInputStream()));
        String line = ""; int i=0;
        while ((line = bfr.readLine()) != null) {
            System.out.println(line);
        }
        return line;
    }
}

Output: I am getting tweets and final statistics as follows:

##########################################################
#    The Tweets                                          #
##########################################################

@uDiZnoGouD
Date: Mon Apr 07 05:07:19 +0000 2014
To enjoy in case you win!
To help you sulk in case you loose!
#IndiavsSriLanka #T20final http://t.co/hRAsIa19zD
Document Sentiment: positive (Score: 0.261738)


##########################################################
#    The Stats                                           #
##########################################################
Document-Level Sentiment:
Positive: 3 (60.00%)
Negative: 1 (20.00%)
Neutral: 1 (20.00%)
Total: 5 (100.00%)

Problem (Question):

How do I just scrape Positive, Negitive, Neutral and send it to Google Visualization? (I.e make a JSON?)

Any help would be really appreciated.

Answer 1

Shoot, I just realized you were asking the other way around. To write the parsing app in Java.

Anyway the idea would be the same, but the language would differ. But that would also mean that you have access to the sources of the python app, so you can maybe dig around there, and you could dump the result object into the console as a JSON object.

Original answer in python:

You should identify the types of lines and parse them and construct the JSON object yourself.

Like for every line:

import re

json_obj = {}
pattern = "^(\w+): (\d) \((\d{2}\.\d{2}%)\)$"
match = re.match(pattern, line)

if match:
    prop_obj = { "value": match[2], "percent": match[3] }
    json_obj[match[1]] = prop_obj

This would transform the line:

Positive: 3 (60.00%)

into

{
     Positive: {
         value: "3"
         percent: "60.00%"
     }
}

Taking this idea further, the parsing rules should be a dictionary of pattern-extractor_methods as key-values

var parse_rules = {
    "^(\w+): (\d) \((\d{2}\.\d{2}%)\)$": 
         def (matches): 
            return { match[1]: { "value": match[2], "percent": match[3] }}
    , ...
}

And for each line you would test against the parse rules and execute the methods if a match is found, and the result of the method is merged in the JSON result object

This is a lot of work (depending on the complexity of the java app, but I'd go this way if the Java app cannot be modified.

Regex explanation & example