DynamoDB cross-regional table copy only copies partial data

Asked Nov 19 '15 at 08:56

Active Nov 19 '15 at 09:05

Viewed 205 times

I've tried default setup of Data Pipeline for cross-regional table copy. Copying one table to another in same region (eu-west-1).

On pipeline activation, EMR cluster is launched, runs for approx 20 minutes and then it's terminated with pipeline being in "success" state.

Problem is that only 389 entries are copied from my table :/ (number is the same on multiple runs). Total number of entries is close to 100000.

I've tried turning on logs (no errors there), walking through them, launching 4.1.0 cluster, increasing throughputs, etc, nothing solves the case.

Does cross-regional table copy work? What could be the problem? Why no error? How do I debug it?

edited Nov 19 '15 at 09:05

asked Nov 19 '15 at 08:56

Marius Grigaitis

After the cluster is launched, set 'termination protection' on for the cluster. In that way, the cluster wont terminate and may be you can get some useful logs out of it. – user1452132 Nov 20 '15 at 03:50
So I get all the logs of the cluster (in s3 bucket), but logs show that success, only 389 entries. No errors. – Marius Grigaitis Nov 20 '15 at 14:32
Sorry, cant think of anything else. That 389 is a fishy limit. – user1452132 Nov 20 '15 at 15:32

0 Answers0