Python: Fastest way to write large integer to file

Question

I want to write a really large integer from Python to a textfile, about 10 to 1000 Megabyte.

The following options have the same speed, unfortunately both are really slow:

import time
import pickle

num = 17**(10**7)

t1=time.time()
pickle.dump( num , open( "save2.p", "wb" ) )
t2=time.time()
print(str(t2-t1))

t3=time.time()
file = open("testfile2.txt","w") 
file.write(str(num))
file.close()
t4=time.time()
print(str(t4-t3))

(of course, the value of num is just a placeholder for another big integer)

My questions:

Is there a faster way to write a human-readable file with the decimal digits?
If not, how can I write it faster without human-readability?

Who can help?

I have to ask, *why* does it have to be human readable? Are you actually going to inspect those digits? The conversion to `str`, i.e. a human-readable string, is what's taking so long. If you used a raw binary representation it would be much faster. — juanpa.arrivillaga, Apr 25 '20 at 19:43
unless you can pickle it, you're kind of stuck with str(int) and if that's too slow, it's because it actually takes time to write a gigabyte. — Kenny Ostrom, Apr 25 '20 at 20:46
Is there a way to avoid binary numbers and to calculate in base 10 from the beginning? If yes, we could avoid the conversion... — L. Milla, Apr 25 '20 at 21:32
You have a lot of questions. Which one are we answering? Just how to write a specific format? Sounds like a job for the modulo operator (or just division). — Kenny Ostrom, Apr 25 '20 at 21:48
What do you mean? `int` objects are fundamentally represented as binary numbers, arbitrary sized in python. — juanpa.arrivillaga, Apr 26 '20 at 00:27

decipher · Answer 1 · 2022-09-14T22:22:11.273

0

Try this:

import random

f=open("filename.txt","w")
char=""
ints=["0","1","2","3","4","5","6","7","8","9"]
for i in range(10**10): #Ten to the power of ten
  char+=random.choice(ints)
if char.startswith("0"):
  char=char[1:]
f.write(char)
f.close()

If you want to read the integer, you write this:

myInt=int(open("filename.txt","r").read())

Hope this helps!

edited Sep 14 '22 at 22:22

answered Sep 14 '22 at 22:21

decipher

1
3

Python: Fastest way to write large integer to file

1 Answers1