0

I have a Python package with dozens of subpackages/modules. Nearly each of the modules uses open built-in Python function. I have written an own implementation of the file opening function and would like to "redirect" all open calls I have in the package modules to my_open function.

I am aware that it is possible to write open = my_open_file in the top of the module to shadow the open within the module, but this would imply editing each module. Alternatively, putting open = my_open_file in the __init__.py of the package and then doing from package_name import open which also implies adding a single line of code to each module.

Is it possible to define the my_open_file function for the package scope just in a single place? Or adding a single line of code in each module is my only option?

Alex Tereshenkov
  • 3,340
  • 8
  • 36
  • 61

2 Answers2

1

Think about what you're asking to do with this module: you've written your own package, a personal change to the way a built-in function is linked. You want to redefine a standard language feature. That's okay ... part of the power of a language is the capability to do things the way you define.

The problem comes when you want to do this by default, overriding that standard capability for code that hasn't explicitly requested that change. You cannot do this without some high-powered authority. The most straightforward way is to rebuild your Python system from sources, replacing the standard open with your own.

In reality, the "normal" way is to have each of your application modules "opt-in", declaring explicitly that it wants to use the new open instead of the one defined by the language. This is the approach you've outlined in your second paragraph. How is this a problem for you? Inserting a line in each of a list of parameterized files is a single system command.

Prune
  • 76,765
  • 14
  • 60
  • 81
  • Thanks for the post, useful. I don't mind editing the code, that's no problem. I was just wondering whether Python supports overwriting a built-in function in a single place and then making it available for the whole package. Out of curiosity, can one make a custom function available throughout the whole package modules without importing in each module (function `my_open_file` defined in some module or `__init__.py`)? – Alex Tereshenkov Mar 26 '19 at 16:54
  • Yes, but just as with the overwrite, you have to rebuild the Python interpreter to give your addition that "built-in" status. Keep in mind that you're asking to do is to *change the language definition*: you're trying to undercut the mechanism defined for exactly your problem, and make your changes part of the standard packages. – Prune Mar 26 '19 at 17:05
  • I am a bit afraid if I was unclear asking for help - the function I've written is just a wrapper around built-in `open` - it is written in Python, not C. Does it change anything in your answers to my questions? – Alex Tereshenkov Mar 26 '19 at 22:09
  • No, that doesn't change anything, except for obviously broadening the necessary recompilation steps for changing that definition. – Prune Mar 26 '19 at 22:34
0

A very neat way to do what you are asking, is by using the builtins module name during the override:

import builtins
_open = builtins.open 
def override(*args, **kwargs):
    # do whatever before or after open
    return _open(*args,**kwargs)
builtins.open = override

Ensure that this runs before everything, possibly putting it in your package __init__.py and you will have the functionality you want with zero extra overhead, no other lines are needed, only a single import wherever in your code. Of course pay attention that, if for example you are running code inside an IDE, the IDE is likely to try and import your module, which is going to cause the IDE to also use the modified open function.

Vasilis Lemonidis
  • 636
  • 1
  • 7
  • 25