3

To put in other words,

Is there a problem that can be solved by directly defining your map reduce jobs, but for which you cannot form a Hive QL query?

If yes, then it means that Hive QL is limited in it's expressive power and cannot express all possible map reduce jobs.

Practically, that means Hive QL is not a complete replacement for defining your own Map Reduce jobs.

user855
  • 19,048
  • 38
  • 98
  • 162

1 Answers1

3

Hive QL does not express everything that can be written with MapReduce. There will always be cases when you know something about the data that Hive cannot infer.

I don't think it would be fair to avoid using Hive for this reason, if that's what you're considering. SQL has the same problem with relation to writing an algorithm over flat files. Properly designed, you can always do at least as good or better than a query language by writing an algorithm from scratch.

Andrew McKnight
  • 634
  • 1
  • 6
  • 12