Array.length in map reduce throwing Null pointer exception

Question

This is the code i am using in mapreduce and getting a null pointer exception..I am passing a variable via configuration, getting that as string and parsing and storing in an int array and processing that--

  int[] results;

        public void configure(JobConf job) {
            // targetPeriod = Integer.parseInt(job.get("Predict"));
            String[] pred = job.get("Predict").split(",");
            results = new int[pred.length];

            for (int i = 0; i < pred.length; i++) {
                try {
                    results[i] = Integer.parseInt(pred[i]);
                } catch (NumberFormatException nfe) {
                }
                ;
            }




        protected void reduce(final IntWritable key, final Iterable<Text> values,
                final Context context) throws IOException, InterruptedException {

            // int targetPeriod = Integer.parseInt(predict.trim());
            String predictfin = null;
            // int targetPeriod = 10;
        "EXCEPTION HERE"---->
            double[] projectedCumulativeScoreAtTargetPeriod = new double[results.length];
            // Participant participant = new Participant(values,targetPeriod);
            for (int i = 0; i < results.length; i++) {
                Participant participant = new Participant(values, results[i]);




14/07/25 15:23:12 INFO mapred.JobClient: Task Id : attempt_201407110824_0728_r_000000_0, Status : FAILED
java.lang.NullPointerException
        at ProjectionReducer.reduce(ProjectionReducer.java:38)
        at ProjectionReducer.reduce(ProjectionReducer.java:1)
        at org.apache.hadoop.mapreduce.Reducer.run(Reducer.java:177)
        at org.apache.hadoop.mapred.ReduceTask.runNewReducer(ReduceTask.java:649)
        at org.apache.hadoop.mapred.ReduceTask.run(ReduceTask.java:418)
        at org.apache.hadoop.mapred.Child$4.run(Child.java:255)
        at java.security.AccessController.doPrivileged(Native Method)
        at javax.security.auth.Subject.doAs(Subject.java:396)
        at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1190)
        at org.apache.hadoop.mapred.Child.main(Child.java:249)

score 2 · Answer 1 · answered Jul 25 '14 at 16:08

2

You're most likely not running the configure method before, so the int[] results is still null when you call results.length. To be able to access its methods and variables, you'll have to initialize it before, but because your errors are showing a null pointer exception, it's because the int[] results has not been initialized, so you need to make sure you're initializing it before you get its length attribute.

answered Jul 25 '14 at 16:08

StoneMan

423
3
18

Sorry i may be wrong, but i get the length to be initialized only inside the configure method(pred.length), so how can i initialize it before. The length it=s dynamic and varies.... – Muthu Palaniappan Jul 25 '14 at 18:13
@MuthuPalaniappan I can't say without seeing more code. What is the order in which the methods are called? – StoneMan Jul 25 '14 at 18:49

score 1 · Answer 2 · answered Jul 25 '14 at 16:01

The problem is that the results variable is null on the line that throws the NPE. Now you should verify this by testing the variables for null and/or their contents before the offending line, and then look back in your code to see why. Perhaps you're calling reduce before calling the configure method, but hard to tell based on what you've posted.

Most importantly, you will want to learn the general techniques of debugging NPE's -- that you check the variables on the offending line, find the one that is null and is throwing the exception, and then trace back into the code to see why. You will run into these buggers time and time again, trust me.

You'll also want to work on your search skills, as this sort of question gets asked more than once a day on this site. If I can find a good duplicate answer (the usual one that we use is not so good in my opinion), I'll delete this answer and lock this thread as a duplicate.

score 0 · Answer 3 · answered Jul 25 '14 at 17:26

0

I think your variable hasn't been initilized, that is why the NullPointerException is thrown.

answered Jul 25 '14 at 17:26

Aeon

87
1
11

Array.length in map reduce throwing Null pointer exception

3 Answers3