Increasing shards in Citus

Question

How can we add shards to existing colocated tables in citus?

I found out that the default is 32, which is definitely lower than expected. Is there a easy way of resizing without downtime?

Answer 1

You can use alter_distributed_table and supply shard_count param to update the total number of shards of a distributed table. It is a nice idea to apply this change to all the distributed tables in the colocation group so that you do not see any degraded performance on distributed table joins.

-- change shard count of all tables in colocation group
SELECT alter_distributed_table('github_events', shard_count:=6, cascade_to_colocated:=true);

See Official Docs for Citus 10.2

Downtime

There will be a small time frame where Citus will need to acquire an ExclusiveLock to change the shard count, where concurrent operations will not be able to access the relations in the colocation group.

If you are running a high workload on your Citus cluster, alter_distributed_table can be blocked while waiting for acquiring the necessary locks on tables.