0

I am trying to load a .mmdb file in Pig to then pass it into a python script. However I get the error message: "Invalid scalar projection: db: A column needs to be projected from a relation for it to be used as a scalar". My code is:

REGISTER 'py_pigscript.py' USING jython AS myudf;
log = LOAD 'test.txt' USING PigStorage(',') AS (x:int);
db = LOAD 'data.mmdb';
result = FOREACH log GENERATE myudf.function(x,db);

Any help would be appreciated. Thank you!

-edit:

The goal of this script is to extract a value from each row in 'test.txt' and pass them to 'data.mmdb' to return additional data.

bjurstrs
  • 949
  • 2
  • 7
  • 17
  • 2
    You are passing a relation to your `udf` which makes no sense. In the absence of further information though, it will be difficult to help beyond that. – o-90 Jun 26 '15 at 01:44

1 Answers1

1

A similar issue is discussed here [1]. In the context of your question, the code would look like:

log = LOAD 'test.txt' USING PigStorage(',') AS (x:int);
db = LOAD 'data.mmdb' AS (entry:(field_1:chararray, field_2....));
result = FOREACH log GENERATE myudf.function(x, db.entry);

[1] Pig pass relation as argument to UDF

Community
  • 1
  • 1
Dan M
  • 1,175
  • 12
  • 23