2

I haven't seen the common sense notion of converting an integer to network order and to write the resulting bytes into an indexable entity in a string - string database vs. writing the string representation of the number anywhere in the documentation of such databases.

Surely the size overhead of writing a 64-bit int as a string into a database must outweigh the trivial complexity of having to do a ntohl call before writing the bytes back into an integer type.

I am therefore missing something here, what are the downsides to using big-endian bytes vs. strings as indexable entities in string-string databases ?

(C++/C tags as I am talking about writing bytes into the memory location of a programatic type, BDB as that is the database I am using, could be kyotodb as well).

Hassan Syed
  • 20,075
  • 11
  • 87
  • 171

2 Answers2

2

If the database validates that the string data you send is valid in the expected encoding then you can't just give it any data you want. You'll only be able to send such integers as happen to look like a valid encoding. I don't know if BDB or kyotodb do such validation.

Also it seems to me like a hack to try to trick one data type to hold something else, and then rely on clients to all know the trick. Of course that applies whether you're using the string to hold a ascii-decimal representation of the integer or if you're using the string as a raw memory buffer to hold the integer. It seems to me that it'd be better to use a database that actually holds the types you want to hold, instead of just strings.

bames53
  • 86,085
  • 15
  • 179
  • 244
2

The advantage of big-endian in this case is that the strings would sort correctly in ascending order.

If the database architecture cannot natively store 64-bit integers, but you need to store them anyway, stringifying them this way is a way to do it.

Of course if you later upgrade the database to one that can store 64-bit integers natively, you will either be "stuck" with the implementation or have to go through a migration process.

CashCow
  • 30,981
  • 5
  • 61
  • 92
  • I'll read what you are saying as. If the architecture does not have a compatible native data type . But something like `boost::lexical_cast(std::string)` should still be possible. I guess such a conversion might prove to be too expensive on certain old-school hardware. – Hassan Syed Jan 11 '12 at 14:53
  • Converting the values is unlikely to be a very expensive operations in the scheme of things. (Although boost::lexical_cast is unlikely to be the optimal way to do it, and if you are converting large numbers of these, e.g. a big results set, could give you performance issues). – CashCow Jan 11 '12 at 17:00
  • I think we are both concluding that the ascending sort order, and the size and speed improvements are to good to ignore ? :D the database technology in question (BDB specifically) can indeed only function with 32-bit integers in its `recno` mode (perhaps it's what you mean by native). If you edit your answer to be a bit more assertive (i.e., if databases do not have a native type to use as a indexable value then big-endian integer bytes embedded into a string are better than string representations) I will accept it :D. – Hassan Syed Jan 12 '12 at 18:53
  • I mean that it is not in-built for the database to accept 64-bit integers as a type. Therefore to store them you need to come up with something it can store, and also sort correctly. What is being done here is a good way to do it. – CashCow Jan 12 '12 at 19:13