2

I am using logging for debugging, however it's extremely inefficient. What other approaches are there?

kgu87
  • 2,050
  • 14
  • 12

3 Answers3

1

Unfortunately, logging tends to be your best best. It is fairly difficult to debug with a traditional debugger, especially in a production environment because of the distributed nature of iterators. For prod level debugging, your best bet is to put together some scripts to fetch all of your logs into one place so that you can analyze them efficiently. In development mode, you could set up your Hadoop cluster in local/single/standalone mode which would make it easier to figure out where your iterators are running so that you could attach a debugger to the running process.

This question isn't for Accumulo per say, but might give you some ideas for doing logging debugging

Consolidate MapReduce logs

Community
  • 1
  • 1
FuriousGeorge
  • 4,561
  • 5
  • 29
  • 52
1

Accumulo has two debug versions you can run locally to help with development or use in unit testing.

MockAccumulo - a lightweight, in-memory version. I believe this might be removed from the code base relatively soon, in favour of Mini Accumulo Cluster.

Mini Accumulo Cluster - this is far more representative of Accumulo than MockAccumulo, it actually runs Zookeeper and runs the same code a normal Accumulo cluster would run.

I've previously used MockAccumulo a lot to help with iterator development and enable at least some kinds of unit tests and found it very helpful.

Binary Nerd
  • 13,872
  • 4
  • 42
  • 44
  • 1
    Thanks, that works. Also found this in between question and answer - http://affy.blogspot.com/2013/03/testing-your-grepiterator-without.html – kgu87 Aug 04 '16 at 18:48
1

The iterator-test-harness might be of help to you.

https://github.com/apache/accumulo/tree/1.8/iterator-test-harness

https://github.com/apache/accumulo/blob/master/docs/src/main/asciidoc/chapters/iterator_test_harness.txt

elserj
  • 1,236
  • 6
  • 6