1

In order to emit 2 matrices as Key Value pairs:

Key  - Matrix A

Value - Matrix B.

Should I create a custom datatype or can directly go with TwoDArrayWritable? And what about the compareTo() method in custom class?

jamesmortensen
  • 33,636
  • 11
  • 99
  • 120
USB
  • 6,019
  • 15
  • 62
  • 93

1 Answers1

1

You'll definitely need to write a custom class for your Key as TwoDArrayWritable doesn't implement WritableComparable (even if you just extend the TwoDArrayWritable class to add the interface and compareTo method).

As for whether you should use TwoDArrayWritable over a custom class - depends on a couple of factors (in my opinion):

  • Are your arrays fixed in size / dimensions? If so i would say use a custom class so you can save performance and not re-create the arrays for each object deserialization call
  • Do you arrays hold primitive types (int, double, float etc) - If so i would again say use a custom class so you don't have to worry about wrapping / unwrapping the primitive values from their wrapper writables (IntWritable, DoubleWritable etc) when performing matrix operations downstream
Chris White
  • 29,949
  • 4
  • 71
  • 93
  • Thx Chris White for ur reply.My arrays are not with fixed size/dimension and but it will be double always. So it is better to create a Custom class for key and value right? – USB Nov 04 '13 at 04:31
  • yes , i created [MatrixWritable](http://stackoverflow.com/questions/19655071/custom-hadoop-key-and-value-get-values-in-a-two-dimensional-double-array) class.but how to write CompareTo() method inorder to emit the key(matrix) – USB Nov 04 '13 at 08:21
  • 1
    a compareTo method could be implemented in a number of ways, one method would be to initially compare the two dimension sizes for ordering, and if they match then drill down in to the matrix, one element at a time – Chris White Nov 04 '13 at 10:59