Citus: make existing database as worker node of citus

Question

I already have a database on one of my server used by a web app. Lets say D1

I want to make a new web app with new database D2 on some other server with some relations with D1.

I want to create citus data server and add both D1 and D2 as shard table and their worker node as their respective server.

What I found in citus documentations is that shard database tables are created on citus server and automatically distributed to available worker nodes.

How can I use existing database as worker node and shard table with a freshly new citus server.

Answer 1

Question is little vague, I did not quite get what is being asked.

We have 2 servers S1 and S2 containing 2 databases D1 and D2. We would like to create a distributed database containing both D1 and D2, we will have a 3rd server S0 that we wanted to use as a citus coordinator.

Using S1 (D1) and S2 (D2) as worker is not a problem. Citus creates main tables at the coordinator node. Respective shards are created at workers. Assuming there would be name collision, shard tables can co-exist with the old tables.

However, you can't blindly use existing data in D1 and D2 as it is and expect them to be shard tables. Data needs to be re-sharded.

Steps

1. read citus documentation on how to setup clusters, pick distribution keys, load data

2. install/setup citus extension on S1, S2, and S0 (coordinator)

3. add S1 and S2 as a worker nodes in S0

4. Decide on table distribution. Some tables need to be on all nodes (reference tables), some of them is sharded according to some key.

5. create tables on coordinator node, distributed them using create_distributed_table/create_reference_table function

6. load data, this could be a problem. Easiest way is to export (copy out) existing data to an external file, and import (copy in) from that file to a citus cluster. You will need to use coordinator node (S0) while importing data.

7. test your application using the new cluster pretty well before switching.