How to call a parent variable in a nested function

Question

I've written a denoising function with cv2 and concurrent.futures, to be applied on both my training and test image data.

The functions (at current) are as follows:

def denoise_single_image(img_path):
    nonlocal data
    img = cv2.imread(f'../data/jpeg/{data}/{img_path}')
    dst = cv2.fastNlMeansDenoising(img, 10,10,7,21)
    cv2.imwrite(f'../processed_data/jpeg/{data}/{img_path}', dst)
    print(f'{img_path} denoised.')
 
def denoise(data):
    img_list = os.listdir(f'../data/jpeg/{data}')
    with concurrent.futures.ProcessPoolExecutor() as executor:
        tqdm.tqdm(executor.map(denoise_single_image, img_list))

data is to be either train or test, as necessary; img_list is a list of all image names in that directory.

I need to create the denoise_single_image() function before denoise(), otherwise denoise() won't recognize it; but I need to define data before I create denoise_single_image(). This seems to be a catch-22, unless I can figure out how to tell denoise_single_image() to refer to a variable that exists one level up.

nonlocal doesn't work because it assumes that data has already been defined in this environment. Is there any way to get this to work?

Any reason why data can't be globally scoped? you do not seem to need to modify it anywhere, so it's not a bad use case. — m.oulmakki, May 04 '21 at 19:11
@m.oulmakki I've tried both `global` and `nonlocal`. While it runs (technically) with `global`, it doesn't seem to acknowledge the existence of anything. Adding `print(data)` after `global data` didn't return anything. — Yehuda, May 04 '21 at 19:18
You do not need to define `denoise_signal_image` before defining `denoise`, only before *calling* `denoise`. Further, `nonlocal` applies to *lexical* scopes, not the scope in which `denoise_single_image` might be called. The parameter `data` is *not* the `nonlocal` variable `data` that `denoise_single_image` refers to. — chepner, May 04 '21 at 19:19
You have two choices: make `data` global so that `denoise` can set its value before calling `denoise_single_image`, or make `data` an argument to `denoise_single_image` and use something like `lambda x: denose_single_image("train", x)` as the first argument to `map`. — chepner, May 04 '21 at 19:22
@chepner I would think this would work, but `tqdm.tqdm(executor.map(lambda x: denoise_single_image(data, x), img_list))` seems to stall the function. I see nothing in the output directory after a few minutes, and I don't get any `{img_path} denoised` responses, even after defining `denoise_single_image(data, img_path)`. — Yehuda, May 04 '21 at 19:34

score 0 · Accepted Answer · answered May 04 '21 at 19:46

You can change the iterable in executor.map to be a tuple of arguments, which can then split in your other function.

executor.map(denoise_single_image, ((img_path, data) for img_path in img_list))

def denoise_single_image(inputs):
    img_path, data = inputs
    # etc

But in your case I would just modify the individual image paths like so

executor.map(denoise_single_image, (f'jpeg/{data}/{img_path}' for img_path in img_list))

def denoise_single_image(img_path):
    img = cv2.imread(f'../data/{img_path}')
    dst = cv2.fastNlMeansDenoising(img, 10,10,7,21)
    cv2.imwrite(f'../processed_data/{img_path}', dst)
    print(f'{img_path.split('/')[-1]} denoised.')

How to call a parent variable in a nested function

1 Answers1