0

There are ~150 linked servers and I have to run a query in each linked server. I have to run this as fast as I can.

The way I'm doing today:

def get():
    p = Pool(20)
    stores_sales = p.starmap(function, parameters)
    p.close()

def funtion(parameters):
    engine = create_engine(connString)
    connection = engine.connect()
    session = sessionmaker(bind=self.engine)
    connection.execute(query)
    connection.invalidate()

The problem is that TI guys still want that we make less connections. If we reduce the number of pool (from 20 to 10, for example) we lose a lot of performance if we do that.

I'd like to know if there is a way to run theses queries in parallel using one connection? Or if you have any other suggestion?

Matheus Epifanio
  • 103
  • 1
  • 5
  • Most sql libraries are not thread safe and can't share connections. What database are you working with? Also, 20 connections really isn't much. – jordanm Dec 30 '20 at 16:40
  • SQL Server. We used to work with a pool of 70, but users were reporting problems to run queries. So we reduced to 20. 20 is not ideal for us (devs) but it's better for them (TI guys) – Matheus Epifanio Dec 31 '20 at 19:45

0 Answers0