1

There seems to be a cap on CDC files while using a s3 source endpoint for ongoing replication.

Is there a hard cap of 999999 possible CDC files?

I am able to load files that are increasing in sequence number ie:

CDC000001.csv CDC002001.csv CDC002009.csv

so not in increment of one each which is fine, but when I try to load a file numbered

CDC1000000.csv

It is not pick up by the DMS task. I would think since 1000000 > 2009 it would follow the same pattern and be picked up by the task. My concern is 999999 is the highest possible value available so if we need to load more than 999999 we cannot.

Is this the case?

Thank you in advance!

ejj
  • 172
  • 2
  • 3
  • 18

1 Answers1

0

The documentation here states

To replicate CDC files in the change data folder successfully upload them in a lexical (sequential) order.

and this is your mistake:

1000000 > 2009

That is incorrect. In reality 2009 > 1000000 in lexical order, i.e. the character '1' comes before the character '2' in ASCII/Unicode. For a lexical sort scheme to work on numeric values, you have to left zero pad the strings, e.g. 1000000 > 0002009 (because '1' > '0').

I stumbled upon this question because I was searching for confirmation that you can skip sequence numbers, which seems implied in the documentation.

Alex R
  • 11,364
  • 15
  • 100
  • 180