32

Obviously, Java doesn't support unsigned number types natively, and that's not going to change soon (comments starting in 2002). However, when working with databases, such as MySQL, they may come in handy every now and then. There are a lot of questions dealing with how to simulate unsigned numbers. For example:

All of them superficially describe how it could be done. But is there any library actually going all the way and implementing suitable wrappers for UByte, UShort, UInteger, ULong? Preferably, those wrappers would extend java.lang.Number and provide an arithmetic API similar to that of java.math.BigInteger.

As can be seen in this document, there's a lot to think of, and a lot that can go wrong (e.g. how to bitwise shift, how to multiply, etc), so I don't want to do it myself. Also, I don't want to just use the next higher type (e.g. Short instead of Byte, etc.). I want the notion of an 8-bit, 16-bit, 32-bit, 64-bit number preserved, for best interaction with databases, for instance.

UPDATE:

Before you answer! Consider that I know all the workarounds, but I'd really really like to have exactly those 4 types with exactly the above properties. And maybe someone has already done that, so that's why I ask. No need to remind me of the workarounds.

Community
  • 1
  • 1
Lukas Eder
  • 211,314
  • 129
  • 689
  • 1,509
  • 4
    All the caveats of bit fudging to play unsigned-in-signed-land, not having "numbers that acted like numbers" would generally be enough to turn me away. There are some areas were Java just missed, this being one of them. –  Nov 19 '11 at 09:14
  • @pst: I feel the same. In most cases I'm turned away, too. But in this case, such a library would really be a treat. – Lukas Eder Nov 19 '11 at 09:25
  • 1
    Some time ago I met [javolution](http://javolution.org/target/site/apidocs/javolution/io/Struct.html) library with Struct class containing implementation of Uints 8-,16-,32-bit. – 87element Nov 19 '11 at 12:37
  • @87element: That's a nice library, actually. Unforunately, in this case, the unsigned number classes neither extend `java.lang.Number`, nor do they provide arithmetic methods... But still something to keep in mind... – Lukas Eder Nov 19 '11 at 17:00
  • just curious, i've done a fair amount if bit/byte level work in java, and aside from a couple of util methods to cast a byte/short to an int as unsigned, i've never felt it to be such a huge deal. what do you need to do for which you need all this extra "support"? – jtahlborn Nov 19 '11 at 20:51
  • @jtahlborn: I want to find the most general solution for supporting some databases' unsigned data types in http://www.jooq.org. I won't be using that library on a bit/byte level, myself. But if I'm going to introduce `UInteger`-like wrappers, then they better be useful for jOOQ users. – Lukas Eder Nov 19 '11 at 22:00
  • @pst, i have quite significant experience w/ bit fiddling and custom compression in java and never needed the unsigned long. Unsigned long is the only unsigned type you actually may need. – bestsss Nov 19 '11 at 22:15
  • @Lukas, unless you need unsigned 64 bit, just use long and you're done. – bestsss Nov 19 '11 at 22:17
  • @LukasEder jooq reminds me of http://openhms.sourceforge.net/sqlbuilder/ Interesting though – A.H. Nov 19 '11 at 22:33
  • 1
    @A.H. I know OpenHMS's SqlBuilder. It seems to have lost traction, though. You'll find more similar tools here: http://stackoverflow.com/questions/1217228/what-is-the-java-equivalent-for-linq – Lukas Eder Nov 19 '11 at 22:39
  • @bestsss: With databases, it's often the (64-bit) `UNSIGNED BIGINT` type, that's needed. But for many users, correct primitive type casting and bitwise operations can be a major pain, just as user **pst** suggested. So I guess, this is a rather "religious thing", whether or not this is truly needed. I'd just like to keep the semantics and leave the processing up to the user. If they want to operate on `long`, they can still use `UInteger.longValue()` since `UInteger extends Number`... – Lukas Eder Nov 19 '11 at 22:41
  • @Luke, what exactly do you need unsigned bitint, Identity field? They are just ok w/ signed. My first real language (aside Basic) was assembler for 6502 and then assembler for 8808 and virtually everything was unsigned, still i do not miss it. – bestsss Nov 19 '11 at 22:54
  • @bestsss: I was reluctant as you to introduce `unsigned` types just for the sake of ID fields, as I agree with you. But I was made aware of this library: http://code.google.com/p/cityhash/ (which apparently can be used with MySQL), and I can imagine other remote use-cases. So why not just introduce support for those who *do need* it. For the others, it doesn't matter anyway. I don't remember having missed `unsigned` data types either, but there are many use-cases for many things in this world :-) – Lukas Eder Nov 20 '11 at 07:36
  • @Lukas, a quick look at the library and it seems it'd yield the same if signed int64 is used (in java as it defines the 2 complimentary form) – bestsss Nov 20 '11 at 14:17
  • @Lukas, this comes late, yet java does have one unsigned type - 16bit char. – bestsss Nov 20 '11 at 20:34
  • @bestsss: ;-) I know I know... But by now you might've guessed that that's not going to help me with MySQL's `TINYINT UNSIGNED`, `SMALLINT UNSIGNED`, `INT UNSIGNED`, `BIGINT UNSIGNED` – Lukas Eder Nov 20 '11 at 20:36

5 Answers5

35

When I needed this functionality inside of jOOQ, I haven't found anything like it, so I rolled my own Open Source library that I call jOOU (U for Unsigned):

http://github.com/jOOQ/jOOU

I understand that some may think this is overkill, but I'd really like to have precisely those wrappers wrapping what other languages call ubyte, ushort, uint, ulong. Hopefully with Valhalla, those wrappers can be turned into value types.

Of course, contributions to the arithmetics / bitwise operation implementations are very welcome!

Lukas Eder
  • 211,314
  • 129
  • 689
  • 1,509
20

There are some reasons why nobody created these wrappers in the way you want.

  • Performance
  • Garbage collector overhead
  • no autoboxing / unboxing
  • bad / useless interface.
  • easier ways to deal with it exists

The first four points are demonstrated by a small C example:

unsigned int x=42, y, m=5, t=18;
y = x * m + t;

This would be translated into:

UInteger m = new UInteger(5);
UInteger t = new UInteger(18);
UInteger x = new UInteger(42);

UInteger y = x.multiplyBy(m);
y = y.add(t);

Several wrapper objects must be created, multiplyBy and add will generate some more. This will put quite some burden on the garbage collector if many calculations are done this way. The wrapping and unwrapping will also eat up your CPUs for nothing.

That even simple arithmetic is a PITA to write or read is also obvious.

For the same reasons NOBODY does arithmetic using the signed wrapper types.

All this is unnecessary if you do the calculations using the next-bigger signed type and cut off the upper part like this:

long x=42, y, m=5, t=18
y = (x*m + t) & 0xFFFFFFFF;

Transfer between Java and a database can also be done using the next biggest signed type. And since JDBC will not create these unsigned wrapper types you would have to do exactly that by yourself only to transform the data into the unsigned wrappers thereafter.

I have done some CPU intensive data processing for myself and handled binary protocols. On these occasions I wished I had unsigned datatypes also. But emulating them in Java with wrapper types would have been more problematic than dealing with the problem directly on each single occasion.

A.H.
  • 63,967
  • 15
  • 92
  • 126
  • 2
    I agree, it only gets "ugly" when the binary protocol uses unsigned 64bit integers.. hello BigInteger :( – Voo Nov 19 '11 at 21:48
  • You have good points there. On the other hand, sometimes you *do need* `BigInteger` and/or `BigDecimal`. Then you don't have a choice but use that verbose API. And for the same reasons, if you *do need* `UInteger`, why not use it if it were available? And if you're in for the speed, then you can still do the calculations yourself... I'm currently evaluating whether I want to use such wrappers for http://www.jooq.org, or whether there is any other way... In my case, speed is probably not a problem. – Lukas Eder Nov 19 '11 at 21:57
  • @LukasEder you're aware that you can still do the arithmetic using the BigInteger wrapper (for 64bit unsigned) and the next larger value for everything else and just tell the DB to store the values correctly? I'm actually quite sure you can actually do that for some DBs at least. – Voo Nov 20 '11 at 21:12
  • @Voo: I'm aware of many many workarounds. I'd just really like a wrapper that communicates how many bits there are. In database-speak, `BigInteger` is the best match for `DECIMAL(n, 0)` with `n > 19`. That has a different semantics to me, than `unsigned long`... – Lukas Eder Nov 20 '11 at 21:25
  • @LukasEder In that case don't reimplement the arithmetic but just store a simple `int bitsize;` value in your wrapper (or just write wrappers for each type). Then just implement the "how do I store values of bitsize X in DB Y" logic for all important DBs (no idea if this is not even standardized?) and you're done. If performance is unimportant writing the abstract base class is trivial. The interesting part is in the DB communication. – Voo Nov 20 '11 at 22:15
  • @Voo: Not so interesting :-) JDBC's `PreparedStatement.setString()` and `ResultSet.getString()` will do for at least 12 databases... I already implemented the [simple wrappers](http://code.google.com/p/joou/). Just thought someone might've already done it before... – Lukas Eder Nov 20 '11 at 22:53
  • @LukasEder I wasn't thinking of strings there - because that has horrible performance characteristics both when accessing (having to parse every time) and storing (congratulations, you just turned a 4byte value into 20byte [assuming a UTF-16 encoding for strings in the db which is used often enough]) – Voo Nov 20 '11 at 23:35
  • @Voo: That's just JDBC (Java). The driver then turns the `String` back into the DB's `int unsigned` again, for instance. I think a modern JVM can handle the odd String-parse. But maybe, I could pass the unsigned version of the value as `int`, `long`, etc... – Lukas Eder Nov 21 '11 at 07:11
  • you don't need a full blown class for unsigne types because basic operations like addition, subtraction, non-widening multiplication and all bitwise operations except right shift is the same for signed and unsigned type. For the remaining operations you may write a function for those – phuclv Mar 20 '15 at 10:59
  • @LưuVĩnhPhúc: This is not about some utility methods, the Q is about a real wrapper type. Something which can be used in method signatures in order to make it clear that this value is really unsigned and where the compiler will give you errors if you mix signed and unsigned. If you are OK with helper methods - thats exactly the second part of my answer. – A.H. Mar 21 '15 at 15:52
3

Another library to consider is Google's Guava. It supports the following unsigned types:

they do extend Number and implement arithmetic operations.

Timofey Gorshkov
  • 4,987
  • 6
  • 41
  • 66
Andrejs
  • 26,885
  • 12
  • 107
  • 96
  • For some reason, Guava doesn't have `ushort` and `ubyte` types. Specifically `ubyte` is the most useful. Do you know why? – Lukas Eder Aug 06 '14 at 14:19
  • There's an issue raised for `short`: https://code.google.com/p/guava-libraries/issues/detail?id=670, for `byte` there's `UnsignedBytes` – Andrejs Aug 06 '14 at 14:54
  • Yeah, but `UnsignedBytes` only provides static methods for `byte` manipulation... – Lukas Eder Aug 06 '14 at 15:04
2
    // Java 8
    int vInt = Integer.parseUnsignedInt("4294967295");
    System.out.println(vInt); // -1
    String sInt = Integer.toUnsignedString(vInt);
    System.out.println(sInt); // 4294967295

    long vLong = Long.parseUnsignedLong("18446744073709551615");
    System.out.println(vLong); // -1
    String sLong = Long.toUnsignedString(vLong);
    System.out.println(sLong); // 18446744073709551615

    // Guava 18.0
    int vIntGu = UnsignedInts.parseUnsignedInt(UnsignedInteger.MAX_VALUE.toString());
    System.out.println(vIntGu); // -1
    String sIntGu = UnsignedInts.toString(vIntGu);
    System.out.println(sIntGu); // 4294967295

    long vLongGu = UnsignedLongs.parseUnsignedLong("18446744073709551615");
    System.out.println(vLongGu); // -1
    String sLongGu = UnsignedLongs.toString(vLongGu);
    System.out.println(sLongGu); // 18446744073709551615
blueberry0xff
  • 3,707
  • 30
  • 18
2

The solution that is used in commons-primitives for an array of unsignedInt it to pass around a long consider this as an unsigned int. You can read more here:

http://commons.apache.org/primitives/apidocs/org/apache/commons/collections/primitives/ArrayUnsignedIntList.html

Grouchal
  • 9,756
  • 6
  • 34
  • 46
  • Thanks, I'm aware of all of those partial solutions. Many libraries have taken 1-2 steps towards unsigned number types. But I'm really looking for a library doing precisely what I described in the question, one that goes all the way including arithmetics... – Lukas Eder Nov 19 '11 at 18:46