8

I'm working on a multi-tenant reactive application using Spring-Webflux + Spring-data-r2dbc with the r2dbc driver to connect to a Postgresql database. The multi-tenant part is schema-based: one schema per tenant. So depending of the context (the user logged in for instance) the requests will hit a certain schema of the database.

I'm struggling on how to accomplish that in r2dbc. Ideally it would be the way Hibernate does with MultiTenantConnectionProvider (see example 16.3).

What I found and what I've done so far:

  • Could use AbstractRoutingConnectionFactory as mentioned here. But I'm force to create a ConnectionFactory by tenant/schema. It seems to me that is far from efficient/scalable, I would rather use a connection pool like r2dbc-pool
  • I looked at PostgresqlConnectionFactory. Interesting thing here is that on prepareConnection there's a call on setSchema(connection):

    private Mono<Void> setSchema(PostgresqlConnection connection) {
        if (this.configuration.getSchema() == null) {
            return Mono.empty();
        }
    
        return connection.createStatement(String.format("SET SCHEMA '%s'", this.configuration.getSchema()))
            .execute()
            .then();
    }
    

May be I need to find a way to override this in order to get the schema dynamically from the context instead of the configuration ?

  • Otherwise I could try to specify the schema in the request as a table prefix:

        String s = "tenant-1";
        databaseClient.execute("SELECT * FROM \"" + s + "\".\"city\"")
                .as(City.class)
                .fetch()
                .all()
    

But I can't use SpringData anymore or I need to override every request to pass the tenant as a parameter.

Any hints/help appreciated :)

Arnaud42
  • 101
  • 1
  • 4

4 Answers4

1

I ran into this as well.

Here is what I am currently doing:

  • Publish the PostgresqlConnectionConfigurationBuilder and the PostgresqlConnectionFactory as a Bean:

    @Bean
    public PostgresqlConnectionConfiguration.Builder postgresqlConnectionConfiguration() {
        return PostgresqlConnectionConfiguration.builder()
                .host("localhost")
                .port(5432)
                .applicationName("team-toplist-service")
                .database("db")
                .username("user")
                .password("password");
    }
    
    @Bean
    @Override
    public PostgresqlConnectionFactory connectionFactory() {
        return new PostgresqlConnectionFactory(postgresqlConnectionConfiguration()
                .build());
    }
    

So that I can later (in my business method) create a new PostgresqlConnectionFactory using the injected PostgresqlConnectionConfigurationBuilder instance - but now with the "schema" setter called on the builder as well (after extracting tenant information from the incoming org.springframework.web.reactive.function.server.ServerRequest which i passed down from my route bean.

My db schemas follow the pattern appname_tenantId, so we have an "appName" statically configured to ie "app_name", so i end up with schema names like "app_name_foo_bar123"

next, we have a tenant identifier which in my case would come from a request header which is guaranteed to be set by an apache server sitting upstream (passing along a X-Tenant-Id header for incoming requests so to not rely on URLs to do tenant specific routing)

So my "logic" currently looks a bit like this:

public Flux<TopTeam> getTopTeams(ServerRequest request) {

    List<String> tenantHeader = request.headers().header("X-Tenant-Id");
    // resolve relevant schema name on the fly
    String schema = (appName+ "_" + tenantHeader.iterator().next()).replace("-", "_");
    System.out.println("Using schema: " + schema);
    // configure connfactory with schema set on the builder
    PostgresqlConnectionFactory cf = new PostgresqlConnectionFactory(postgresqlConnectionConfiguration.schema(schema).build());
    // init new DatabaseClient with tenant specific connection
    DatabaseClient cli = DatabaseClient.create(cf);


        return cli
                .execute("select * from top_teams ").fetch().all()
                .flatMap(map -> {

                    ...
                    });
                });
    }

This logic of course can be abstracted away, not sure however where to put it, maybe this can be moved to a MethodArgumentResolver so we can just inject an already configured DatabaseClient


ps: this only solves the multitenancy issue when using the DatabaseClient. I am not sure how one would make this work with R2dbcRepositories

salgmachine
  • 519
  • 1
  • 3
  • 14
  • At least, this is SUPER USERFUL for ddl, because Spring DAta R2dbc does not make DDL operations, such as Spring Data JPA. In other words, we can use this strategy for DDL, and R2dbcRepositories for DML. Awesome! – GtdDev Sep 25 '20 at 00:35
1

I have created a multi-tenancy example for r2dbc but using a per-databases strategy.

Check the complete example codes here.

In some databases, the schema and database concept is equivalent. If you are stick on using a per-schema strategy, add a SQL to select schema(please do research the database you are using, and determine the right clause to set a schema) when obtaining the connection.

Hantsy
  • 8,006
  • 7
  • 64
  • 109
1

Thanks for the answers. I finally ended up with this solution:

Build a ConnectionFactory by tenant/schema:

public class CloudSpringUtilsConnectionFactoryBuilder implements ConnectionFactoryBuilder {

@Override
public ConnectionFactory buildConnectionFactory(String schema) {
    PostgresqlConnectionConfiguration configuration = getPostgresqlConnectionConfigurationBuilder(schema)
            .build();
    return new PostgresqlConnectionFactory(configuration);
}

@Override
public ConnectionFactory buildSimpleConnectionFactory() {
    PostgresqlConnectionConfiguration configuration = getPostgresqlConnectionConfigurationBuilder(null)
            .build();
    return new PostgresqlConnectionFactory(configuration);
}

protected PostgresqlConnectionConfiguration.Builder getPostgresqlConnectionConfigurationBuilder(String schema) {
    return PostgresqlConnectionConfiguration
            .builder()
            .username(dbUser)
            .password(dbPassword)
            .host(dbHost)
            .port(dbPort)
            .database(dbName)
            .schema(schema);
}

Create a TenantRoutingConnectionFactory to get the right ConnectionFactory depending of the tenant. In our case tenant is extracted from the authentication Principal (a token convert to a UserProfile):

public class TenantRoutingConnectionFactory extends AbstractRoutingConnectionFactory {

private final DatabaseMigrationService databaseMigrationService;
private final ConnectionFactoryBuilder connectionFactoryBuilder;

private final Map<String, ConnectionFactory> targetConnectionFactories = new ConcurrentHashMap<>();

@PostConstruct
private void init() {
    setLenientFallback(false);
    setTargetConnectionFactories(new HashMap<>());
    setDefaultTargetConnectionFactory(connectionFactoryBuilder.buildConnectionFactory());
}

@Override
protected Mono<Object> determineCurrentLookupKey() {
    return ReactiveSecurityContextHolder.getContext()
            .map(this::getTenantFromContext)
            .flatMap(tenant -> databaseMigrationService.migrateTenantIfNeeded(tenant)
                    .thenReturn(tenant));
}

private String getTenantFromContext(SecurityContext securityContext) {
    String tenant = null;
    Object principal = securityContext.getAuthentication().getPrincipal();
    if (principal instanceof UserProfile) {
        UserProfile userProfile = (UserProfile) principal;
        tenant = userProfile.getTenant();
    }
    ...
    log.debug("Tenant resolved: " + tenant);
    return tenant;
}

@Override
protected Mono<ConnectionFactory> determineTargetConnectionFactory() {
    return determineCurrentLookupKey().map(k -> {
        String key = (String) k;
        if (!targetConnectionFactories.containsKey(key)) {
            targetConnectionFactories.put(key, connectionFactoryBuilder.buildConnectionFactory(key));
        }
        return targetConnectionFactories.get(key);
    });
}

Note that we use Flyway in DatabaseMigrationService to create and migrate schema for each tenant we get.

prosoitos
  • 6,679
  • 5
  • 27
  • 41
Arnaud42
  • 101
  • 1
  • 4
0

Thanks/Based on the @charlie carver answer, That's how I solved this problem:

Controller:

    @PostMapping(MAP + PATH_DDL_PROC_DB)  //PATH_DDL_PROC_DB = "/database/{db}/{schema}/{table}"
    public Flux<Object> createDbByDb(
            @PathVariable("db") String db,
            @PathVariable("schema") String schema,
            @PathVariable("table") String table) {
        return ddlProcService.createDbByDb(db,schema,table);

Service:

    public Flux<Object> createDbByDb(String db,String schema,String table) {
        return ddl.createDbByDb(db,schema,table);
    }

Repository:

    @Autowired
    PostgresqlConnectionConfiguration.Builder connConfig;

    public Flux<Object> createDbByDb(String db,String schema,String table) {
        return createDb(db).thenMany(
                Mono.from(connFactory(connConfig.database(db)).create())
                    .flatMapMany(
                            connection ->
                                    Flux.from(connection
                                                      .createBatch()
                                                      .add(sqlCreateSchema(db))
                                                      .add(sqlCreateTable(db,table))
                                                      .add(sqlPopulateTable(db,table))
                                                      .execute()
                                             )));
    }

    private Mono<Void> createDb(String db) {

        PostgresqlConnectionFactory
                connectionFactory = connFactory(connConfig);

        DatabaseClient ddl = DatabaseClient.create(connectionFactory);

        return ddl
                .execute(sqlCreateDb(db))
                .then();
    }

Connection Class:

@Slf4j
@Configuration
@EnableR2dbcRepositories
public class Connection extends AbstractR2dbcConfiguration {

    /*
     **********************************************
     * Spring Data jdbc:
     *      DDL: does support JPA.
     *
     * Spring Data R2DBC
     *      DDL:
     *          -does no support JPA
     *          -To achieve DDL, uses R2dbc.DataBaseClient
     *
     *      DML:
     *          -it uses R2dbcREpositories
     *          -R2dbcRepositories is different than
     *          R2dbc.DataBaseClient
     * ********************************************
     */
    @Bean
    public PostgresqlConnectionConfiguration.Builder connectionConfig() {
        return PostgresqlConnectionConfiguration
                .builder()
                .host("db-r2dbc")
                .port(5432)
                .username("root")
                .password("root");
    }

    @Bean
    public PostgresqlConnectionFactory connectionFactory() {
        return
                new PostgresqlConnectionFactory(
                        connectionConfig().build()
                );
    }
}

DDL Scripts:

@Getter
@NoArgsConstructor(access = AccessLevel.PRIVATE)
public final class DDLScripts {

    public static final String SQL_GET_TASK = "select * from tasks";

    public static String sqlCreateDb(String db) {
        String sql = "create database %1$s;";
        String[] sql1OrderedParams = quotify(new String[]{db});
        String finalSql = format(sql,(Object[]) sql1OrderedParams);
        return finalSql;
    }

    public static String sqlCreateSchema(String schema) {
        String sql = "create schema if not exists %1$s;";
        String[] sql1OrderedParams = quotify(new String[]{schema});
        return format(sql,(Object[])  sql1OrderedParams);
    }

    public static String sqlCreateTable(String schema,String table) {

        String sql1 = "create table %1$s.%2$s " +
                "(id serial not null constraint tasks_pk primary key, " +
                "lastname varchar not null); ";
        String[] sql1OrderedParams = quotify(new String[]{schema,table});
        String sql1Final = format(sql1,(Object[])  sql1OrderedParams);

        String sql2 = "alter table %1$s.%2$s owner to root; ";
        String[] sql2OrderedParams = quotify(new String[]{schema,table});
        String sql2Final = format(sql2,(Object[])  sql2OrderedParams);

        return sql1Final + sql2Final;
    }

    public static String sqlPopulateTable(String schema,String table) {

        String sql = "insert into %1$s.%2$s values (1, 'schema-table-%3$s');";
        String[] sql1OrderedParams = quotify(new String[]{schema,table,schema});
        return format(sql,(Object[]) sql1OrderedParams);
    }

    private static String[] quotify(String[] stringArray) {

        String[] returnArray = new String[stringArray.length];

        for (int i = 0; i < stringArray.length; i++) {
            returnArray[i] = "\"" + stringArray[i] + "\"";
        }
        return returnArray;
    }
}
GtdDev
  • 748
  • 6
  • 14