0

I’m currently doing some testing for an upcoming data migration project and came across Kingswaysoft which seemed like it would be ideal for this purpose. However I’m currently testing importing 225,000 contact records into a new sandbox Dynamics 365 instance and it is on course to take somewhere between 10 and 13 hours. Is this typical of the speeds I should expect or am I doing something silly? I am setting only some out of the box fields such as first name, last name, dob and address data. I have a staging contact SQL database holding the 225k records to be uploaded. I have the CRM Destination Component setup to use multi threading batch size of 250 with up to 16 threads. Have tested using both Create and Upsert and both very slow. Am I doing something wrong - I would have expected it to be much quicker.

user2425239
  • 19
  • 2

1 Answers1

1

When it comes to the data load to Dynamics 365 Online, the most important aspect that affects your data load performance is the network latency. You should try to put the data migration solution as close as possible to the Dynamics 365 online server. If you have the configuration right, you should be able to achieve something like 1m to 2m records per hour. The speed that you are getting is too slow. There must be something. There are many other things that could affect the data load performance, but start from network latency first. We have some other tips shared at https://www.kingswaysoft.com/products/ssis-integration-toolkit-for-microsoft-dynamics-365/help-manual/crm/advanced-topics#MaximizedPerformance, which you should check out.

Daniel Cai
  • 446
  • 2
  • 7
  • Hi Daniel, I am having similar issues, I am moving approximately 4M records at a time and sometimes it takes me 13 hours to completed a set. Could you tell me what you mean by put the Data Migration solution as close to the D365 server as possible? Currently they are both cloud servers. – acolene May 04 '20 at 21:56
  • 1
    What I meant is, you should have your VM or SSIS-IR instance sitting in the same data center as your Dynamics instance, which gives the closest proximity from the Dynamics 365 server. If you are running a VM, you can run CRM diagnostics tool to check the network latency - ideally you should have a network latency of about 10 milliseconds or lower. – Daniel Cai May 04 '20 at 22:06
  • My network latency is 46 ms. and i can transfer 1M records in 3hrs. My settings are Batch size -100 and thread count- 10. Is this rate of transfer typical for the network latency and application settings? – acolene May 05 '20 at 17:09
  • 1
    You could have a try of a smaller batch size (such as 10) and a higher thread number (say 20), and see if you are getting any improvement. For Dynamics 365 online, it is generally believed that a batch size of 10 works out better than a higher number. Your network latency is definitely not ideal by the way. You could relocate your VM or move your SSIS-IR instance (if that is what you are using). – Daniel Cai May 05 '20 at 22:46
  • Thank you for the advice thus far. With some assistance I was able to move the Integration server to the same data center as the CRM destination. However the transfer times are about the same. What would be the next thing we should check? Also what is the average number of records that can be transferred via kingswaysoft migration tool? Lastly does kingswaysoft offer support for these types of situations? And if so how do we start this support process? – acolene May 13 '20 at 17:04