0

How would I get python to work with values of the order of magnitude of 1099511627776 bits large (yeah. 137 gb)? I some what need to implement this (or if you can suggest a better way to do it, will change methods). apparently, pgp's new length types have 3 sections instead of 2. now they are: length type, value-of-length-type, and the length. length type is 2 bits, which translates to 191 bytes, 8383 bytes, 4294967296 bytes, or partial length. the length is then encoded in bytes. how would i check if a value is less than 4294967296 bytes large if i cant even do 1 << (4294967296 << 8)? it is too big to fit in even a long.

calccrypto
  • 8,583
  • 21
  • 68
  • 99
  • 4
    ...yikes! It's not sane, practical, or advisable to deal with a "number" this size. It needs to be broken down into pieces -- e.g. a 50GB database generally isn't loaded at once. –  Jan 03 '11 at 07:02
  • do you happen to know how pgp does it? – calccrypto Jan 03 '11 at 07:04
  • @calcrypto If it does deal with such huge numbers I'd imagine it stores it in a "compacted" form -- as on http://primes.utm.edu/largest.html :-) Is the source available? Although, looking at the primes ... the largest is "only" 12million digits long. –  Jan 03 '11 at 07:09
  • well, i dont think any pgp block has ever gotten that big... and yes the sourcecode is available, but im terrible at reading other peoples codes, so im unable find the part, much less figure out what they did – calccrypto Jan 03 '11 at 07:21
  • 3
    I'm trying to envision a practical use for a number accurate to a hundred billion significant figures and failing. – Robert Rossney Jan 03 '11 at 22:09
  • IIRC 256 bits is enough precision to number all the atoms in the universe. "the number of atoms in the observable universe to be close to 10**80". That seems like the largest number of any practical value. – S.Lott Jan 05 '11 at 02:27

4 Answers4

2

It's not only to big to fit in a long, it's too big to fit in the memory of any computer. I think you misunderstood something.

As I understand it, the largest key value is 4,294,967,295 bytes. That's 4GB, not 137 GB. You hold that key in memory not as one number, but as a string of bytes. So I don't know where you get a number that's 137 GB large from.

If PGP required that, it would be impossible to implement. Since there are implementations, I'm sure that's not how it is done.

(Also, I'm sure there are PGP modules for Python, but if you are doing this not because you need it, but for practice and because you want to learn, then keep it up!)

Lennart Regebro
  • 167,292
  • 41
  • 224
  • 251
  • 1
    4gb. oops. i probably did something weird in wolfram alpha. yeah im reading through rfc4880 one random section at a time. im the type of person who wants to know whats behind the scenes, rather than just use given commands – calccrypto Jan 03 '11 at 23:49
1

With the three-argument form of pow().

Ignacio Vazquez-Abrams
  • 776,304
  • 153
  • 1,341
  • 1,358
  • what? `pow(2, 4294967296 << 8)`? the problem isnt calculating the value. the problem is storing the value. and i dont have a number to mod by – calccrypto Jan 03 '11 at 07:00
  • 2
    @calccrypto: i think what Ignacio meant here is like in math you don't write 10.....0 , 1000 zero you just wrote 10^1000, so your numbers can be stored in this form :) , and i think the pattern that Ignacio have given you can be very helpful also for doing calculation ,look at RSA algorithm to see what i mean, it all based on arithmetic especially the modulo operator, think about it; +1 for Ignacio. – mouad Jan 03 '11 at 15:26
1

Use a big number library like GMPY.

highBandWidth
  • 16,751
  • 20
  • 84
  • 131
  • 1
    Nice link. However, this doesn't "solve" the issue of dealing with a number "requiring" 137GB of data to store ;-) –  Jan 03 '11 at 19:46
0

I interpret RFC4880 (November 2007) differently. Section 3.2 describes multiple precision integers as having a 2 octet length so the largest size would be 64KB. Section 4.2.2 describes the new packet format and documents numbers of the scale you are describing. But the packet format is not the same as the multiple precision integer format. If you are interpreting it differently, please update your question with the exact sections of the RFC that you are reading.

casevh
  • 11,093
  • 1
  • 24
  • 35