-1

What is the best way to import >5 million records of customer into commerce tools?

I only need it to load initial load but I don't need any sync services between external sources and commercetools multiple times

process of finding delta can be done outside.

I went through the import API documentation https://docs.commercetools.com/import-export/

#option 1: Impex UI, I believe its node backed application. Does it uses Import API?
#option 2: Project Sync. It uses Import API and abstract batching, http request etc
#option 3: Import API SDK. It abstract http request

With large dataset to import for initial load, it sounds like #2 Project sync would be better choice since it has batching. It looks like it calculates delta before syncing to commercetools which I dont need since I want to use it for initial load.

Which option should i use?

George
  • 41
  • 8

1 Answers1

1

The Import API is a good choice for customers and order imports as the import api will resolve customer references on the orders by key.

You can also consider using the API directly using the sdk of your choice.

Does that help?

Best Regards Brian

Brian Tomkins
  • 376
  • 1
  • 3
  • Thanks Brian. I tried to use the Project Sync library to push new customers into system but application fails. com.commercetools.sync.commons.exceptions.SyncException: Failed to fetch existing customers with keys: '[customer-key-john-doe-2]'. Summary: 1 customers were processed in total (0 created, 0 updated and 1 failed to sync). I am following the example given here https://github.com/commercetools/commercetools-sync-java/blob/master/src/integration-test/java/com/commercetools/sync/integration/externalsource/customers/CustomerSyncIT.java – George Jun 21 '22 at 01:20
  • Never mind. I found the issue. Looks like library is returning "Failed to fetch existing customers with key" as generic error message for all errors. It is slightly misleading. In my case, I did'nt set the region for client configuration so it is defaulting to wrong region. I am able to create customer records now – George Jun 21 '22 at 02:39