0

I tried to put a break point and do the following in the watch window: check .getSize() which is supposed to return size in bytes. And .materialize() to see if I can look at the java objects.

The .getSize() does show a number >0 but I doubt if that should be an indicator of the PTable having elements. The .materialize() did not show anything to indicate the presence of elements.

Thanks in advance.

Gadam
  • 2,674
  • 8
  • 37
  • 56

2 Answers2

3

Instead of relying on PCollection.size() method to check whether your collection is empty or not, you should use PCollection.length(), which does exactly what you need.

Zmicier Zaleznicenka
  • 1,894
  • 1
  • 19
  • 22
  • I stumbled upon the same issue now. Does .length call .materialize in the background? Is it a resource sensitive method in any way? And importantly, it doesn't seem to work like it should. {code} LOGGER.info("size:{}", personRecordTable.length()); {code} gives me {code} size:PTables.values {code} – Gadam Oct 12 '15 at 21:19
1

I have come across this issue sometimes and the API methods like materialize don't really give a satisfactory result. I would suggest to create a simple DoFn that takes this PCollection as input and use loggers to see if it has elements or not. And PCollection getSize() method will be helpful to know how many elements it has.

shubh586
  • 23
  • 4