0

I'm using PyRFC on Databricks and I'm trying to do around 20k queries to SAP. Due to the large volume of queries, I want to do this by utilizing multiprocessing. Here's what I have:

ASHOST='Some_Server_Name'
CLIENT='xx'
SYSNR='xx'
USER='xxxx'
PASSWD='xxxx'
conn = Connection(ashost=ASHOST, sysnr=SYSNR, client=CLIENT, user=USER, passwd=PASSWD)


q = Queue()

def worker(plant, material):
    print("called")
    q.put(conn.call(
        "Func_Name",
        **{"WERKS": plant, "MATNR": material.zfill(18)}
    ))

jobs = []
for i in plant_collection[:10]:
    p = Process(target=worker, args=(i["plant"], i["material"]))
    jobs.append(p)

for j in jobs:
    j.start()

print(q.get())

I'm using the multiprocessing library. The print function never gets executed and when I call q.get() it never returns back. It just hangs. What am I doing wrong and how can I fix this?

user2896120
  • 3,180
  • 4
  • 40
  • 100
  • is it print `called`? – rzlvmp Jun 01 '22 at 00:16
  • @rzlvmp Print never gets called – user2896120 Jun 01 '22 at 01:16
  • I can't reproduce this behavior. `called` should be printed before `PyRFC`'s `conn` is called. I removed `conn` related parts from your code and it at least printing `called`. – rzlvmp Jun 01 '22 at 02:28
  • @rzlvmp when you don't remove the conn part, is called printing? – user2896120 Jun 01 '22 at 02:29
  • If I don't remove it `Connection is undefined` error will be raised before main code start to work – rzlvmp Jun 01 '22 at 02:31
  • @rzlvmp What if you define the connection from outside of the multiprocessing? – user2896120 Jun 01 '22 at 02:33
  • `called` is printing even if I add `1/0` (division by zero error) directly after `print` line. Code that you provided working at least before first `called` print. I don't have `PyRFC` installed and can't check this library, but pretty sure that problem is somewhere else. – rzlvmp Jun 01 '22 at 02:38
  • probably code stuck at `conn = Connection(...)`. Try to add prints before and after it – rzlvmp Jun 01 '22 at 02:40

0 Answers0