2

I'm using a temporary file to exchange data between two processes:

  1. I create a temporary file and write some data into it
  2. I start a subprocess that reads and modifies the file
  3. I read the result from the file

For demonstration purposes, here's a piece of code that uses a subprocess to increment a number:

import subprocess
import sys
import tempfile

# create the file and write the data into it
with tempfile.NamedTemporaryFile('w', delete=False) as file_:
    file_.write('5')  # input: 5

    path = file_.name

# start the subprocess
code = r"""with open(r'{path}', 'r+') as f:
    num = int(f.read())
    f.seek(0)
    f.write(str(num + 1))""".format(path=path)
proc = subprocess.Popen([sys.executable, '-c', code])
proc.wait()

# read the result from the file
with open(path) as file_:
    print(file_.read())  # output: 6

As you can see above, I've used tempfile.NamedTemporaryFile(delete=False) to create the file, then closed it and later reopened it. My question is:

Is this reliable, or is it possible that the operating system deletes the temporary file after I close it? Or perhaps it's possible that the file is reused for another process that needs a temporary file? Is there any chance that something will destroy my data?

Aran-Fey
  • 39,665
  • 11
  • 104
  • 149
  • 1
    That strongly depends on the OS. Even different [Linux distributions differ in `/tmp` cleanup behaviour](https://serverfault.com/questions/377348/when-does-tmp-get-cleared). – Eli Korvigo Jul 15 '18 at 17:39
  • 1
    @EliKorvigo "It strongly depends on the OS" sounds an awful lot like "No, that's not reliable at all" to me (: Thanks for the info. – Aran-Fey Jul 15 '18 at 18:05
  • I haven't called it downright unreliable, because you are not bound to use `tempfile.gettempdir()`, hence you can theoretically store the file in a safe place. That being said, a temporary file that should safely survive past the lifetime of its process doesn't sound so temporary to me. If you need to exchange data with another process, you can use sockets or a named pipe. Or use a plain-old regular file (with some custom permissions, if you want to mimic a tempfile). – Eli Korvigo Jul 15 '18 at 18:16
  • @EliKorvigo The file doesn't need to persist after my program exits. And I can't use pipes because the subprocess only supports on-disk files. – Aran-Fey Jul 15 '18 at 18:33

1 Answers1

0

The documentation does not say. The operating system might automatically delete the file after some time, depending on any number of things about how it was set up and what directory is used. If you want persistence, code for persistence: use a regular file, not a temporary one.

Tom Zych
  • 13,329
  • 9
  • 36
  • 53
  • Any recommendations where I should create that file? In the current working directory? In the directory where my program is located? Somewhere else? – Aran-Fey Jul 15 '18 at 17:43
  • 1
    Depends on your use case. You probably want a fixed directory, not the CWD, since another process needs to access the file and that process might have a different CWD. Under Linux, I’d probably use a hidden file (`.filename`) in my home directory. – Tom Zych Jul 15 '18 at 17:45