One master node. Several worker nodes.
The worker nodes hold sensitive data (running a database each, which is not accessible from outside) that should not be transmitted over any network. Nevertheless, we need to execute arbitrary* operations on that data and get only the results to the master node.
*by arbitrary, I mean that the operations will not be limited to a finite set of calculations, like sum, count, mean etc. but general complex calculations.
So we implement our own client/server application on top of that. The client side, runs on the master and transmits the source code for the given calculation. The server side, runs on each worker, has access to the database, executes the source code received by the master node and (somehow, irrelevant for the discussion) transmits the results back to the master node.
My question is, if a system like that makes sense security-wise. Secure in the sense that the source code to be executed on the worker nodes can only come from the master node. I am thinking of a setup where the code transmitted to the worker nodes is signed by the master node, in order to prevent arbitrary code execution. Could a system on these lines work securely* or will it (inevitably) be vulnerable to arbitrary code execution attacks? Any points to a similar system or any readings to this direction would also be helpful.
*and again by securely, I mean that it will only execute code that comes from the master node.
note: In this kind of setup, the master could obviously transmit code that could upload the raw data but this is not the concern here. Let us assume full trust to the master's intentions.