1

I am currently testing inserting a large amount of mappings for existing data into a shardmap using Elastic Scale. Turns out the whole process is time consuming. It's inserting around ~10 mappings/second. Is there anyway to speed up the insertion e.g. by inserting batches of mappings or directly via stored procedures?

Marcel
  • 189
  • 7

2 Answers2

2

We know from our own testing that inserting mappings is time consuming. Here are a couple of options I'd suggest you try:

  • You can run multiple parallel threads inserting the mappings.
  • You can increase the service level objective for the shard map database for the time where you do the bulk load.

I understand why you would want to load mappings in bulk for test scenarios. However, I am not sure I understand the reason why you will need so many mappings that this becomes a problem. Could you explain a bit more?

Thanks, Torsten

Torsten Grabs
  • 481
  • 1
  • 3
  • 5
  • Hi Torsten, thank you for your answer! Good question, it made me rethink my requirements regarding the mappings. My initial thought was to use a ListShardMap in order to be able to move single shardlets. Is it possible when I use a RandeShardMap with range mappings, to still move a single shardlet within a range using the Split/Merge Service ? – Marcel Mar 10 '15 at 08:16
  • Hi Marcel, thanks for your reply. Yes, range shard maps support what we call shardlet moves. Those take a single sharding key value (that you specify as part of the request) out of the current range and move them off to a different shard. If you have setup the web role for split-merge, you can find shardlet move among the options for the Operation Type at the top of the web page. If you select that option, you just need to specify the key that you want to move and split-merge takes care of the rest (like managing the mappings in your shard map and moving the data). Hope that helps. – Torsten Grabs Mar 10 '15 at 16:46
0

Coming back to this question, since now we have published ShardManagement PowerShell module along with some sample scripts here: https://gallery.technet.microsoft.com/scriptcenter/Azure-SQL-DB-Elastic-731883db. This should help you in settings and querying the existing range/list mappings quickly.

Silvia Doomra
  • 947
  • 8
  • 16