0

There is a property in pig named

'pig.maxCombinedSplitSize' – Specifies the size, in bytes, of data to be processed by a single map. Smaller files are combined until this size is reached.

Is there a similar property in hive for specifying the size of data to be processed by a single map?

I am trying the below command but it doesn't work.

'SET hive.maxCombinedSplitSize=64mb';

Any suggestions?

leftjoin
  • 36,950
  • 8
  • 57
  • 116
dheee
  • 1,588
  • 3
  • 15
  • 25

1 Answers1

0

Try this:

set hive.input.format=org.apache.hadoop.hive.ql.io.CombineHiveInputFormat;
set mapred.min.split.size=67108864;
Joe K
  • 18,204
  • 2
  • 36
  • 58