3

I'm trying to get an old-style VB6 application to read a large (greater than 2GB) binary file. I declare a buffer as:

Dim TCBuffer as String
TCPBuffer = String(4096, Chr(0))

And read the data in a loop using:

Get #FileNum, , TCPBuffer

But once the current position reaches 2GB, the line above raises an error.

Run-time error '63':

Bad record number

So I wondered about using Scripting.FileSystemObject instead. However, it appears this object is very limited. For one thing, it appears you can only create and open text files.

Any suggestions on how I can read a large binary file from VB6?

UPDATE:

Thinking about this some more, another option would be to use API functions. But one of the real limitations of VB6 here is the lack of unsigned data types. So even using API functions would be a trick.

Community
  • 1
  • 1
Jonathan Wood
  • 65,341
  • 71
  • 269
  • 466
  • VB6 is so old that when it was current there were no files allowed larger than 2GB, and I don't believe it supported more than 32-bit integers (which would max out at 2GB). No numeric support for the value + no large file support in the OS (Win95 at the time) = no files > 2GB in VB6. You might try a memory mapped file, but I'm not sure how you would use it from that dinosaur of a language. – Ken White May 24 '18 at 00:18
  • @KenWhite: Yes, correct. Functions like `FileLen` and `LOF` return a negative number, which I can convert to the correct value. But there is nothing I can do with the error described above. That's why I wondered about using something like `Scripting.FileSystemObject`, which I assume is much newer. – Jonathan Wood May 24 '18 at 00:20
  • Hmm... I don't know if that would work or not, given the inherent VB6 limitations. Might be worth a shot. If not, can you use something else to split the large file into smaller segments that VB6 can use? (I'm assuming you have a reason you have to use VB6, but can you split the task?) – Ken White May 24 '18 at 00:26
  • @KenWhite: Well, that's a thought if I can't find anything better. But, for that matter, I don't think VB can do that either! – Jonathan Wood May 24 '18 at 00:28
  • My thought would have been to see if there's a way to work with the API and a memory-mapped file, but I have no idea how I would do that in VB6. I'm also not sure that would work, either, because of the integer range issues. – Ken White May 24 '18 at 00:33
  • @KenWhite: Yes, I'm leaning towards an API solution. I can deal with the range issue by storing values in Doubles and convert to signed Longs as needed. Not ideal but I think it could work. – Jonathan Wood May 24 '18 at 01:03
  • `Currency` is a 64-bit integer with implicit scaling by 1/10000 and people use those with Win32 calls for reading and writing large binary and text files all the time. It is also possible to work with Variant variables containing LongLong subtype data (I64) in VB6, but using those with API calls can get a little tricky. – Bob77 May 24 '18 at 05:11
  • @Bob77: Actually, the 32-bit version of ReadFile() uses 32-bit values. All I need to do is convert between signed Longs and the unsigned value in a Double. – Jonathan Wood May 24 '18 at 05:16
  • An [ADODB.Stream](https://learn.microsoft.com/en-us/sql/ado/reference/ado-api/stream-object-ado?view=sql-server-2017) may be worth looking at, its a true stream that can .Read() in chunks although I have no idea what happens if you exceed 2GB. – Alex K. May 24 '18 at 11:25
  • Can you write your data access in a .NET language and then wrap that in a minimal interface (COM) that VB6 can call? – StayOnTarget May 24 '18 at 11:38
  • This may already be covered in other comments or the answer below, but I found this interesting link: https://www.codeguru.com/cpp/w-d/doc_view/fileopensave/article.php/c12917/Getting-Past-the-2-Gb-File-Limit.htm – Bill Hileman May 24 '18 at 19:05
  • @BillHileman: Thanks for the link. That's basically the approach I'm taking. But I will read through it more carefully to pick up any issues I hadn't considered. – Jonathan Wood May 24 '18 at 19:56
  • @JonathanWood just googling VBForums gives me this result: [VB6 - Huge (>2GB) File I/O Class](http://www.vbforums.com/showthread.php?531321-VB6-Huge-(-gt-2GB)-File-I-O-Class&highlight=huge) – deblocker May 24 '18 at 20:26
  • @deblocker: Thanks, that looks like another good link that I didn't see. – Jonathan Wood May 24 '18 at 21:08
  • I've used this API code from codereview with no issues - the only modification I made was to handle going backwards with FileSeek - that's in the last post of the thread. https://codereview.stackexchange.com/q/102209/106658 – dbmitch May 26 '18 at 18:43

2 Answers2

1

I ended up creating a file class that uses the Windows API internally.

Any function that accepts or returns a file size or offset is of type Double. A little code converts that value to and from an unsigned long, even though it must be stored in a signed Long when being passed to the API (because VB6/VBA doesn't support unsigned longs).

Jonathan Wood
  • 65,341
  • 71
  • 269
  • 466
0

You best bet is probably to use the Windows API to memory map the file, using the CreateFile, CreateFileMapping, and MapViewOfFile functions. Refer to the Microsoft documentation on "Creating a File Mapping object" for more information on how the API works, though obviously you'll need to translate the specifics to VB. I did a little searching and found some old articles still around with some pointers, and an archive of sample code from Visual Basic Programmer's Journal (see "The Persistence Of Memory (July 1996)"), so you're not the only person who's needed to do this and it seems to be an established technique. Keep in mind that for a file that big in a 32-bit process you'll need to map just part of the file at a time into your address space.

  • 1
    I don't understand what benefit comes from a memory mapped file. Why not just use the API file functions to read the file? Memory mapped files will take extra effort, and I don't see the win. – Jonathan Wood May 24 '18 at 13:52
  • @JonathanWood Mainly so that you can map just part of the file into memory at a time. Your process only has 32 bits of address space to work with. Though if you only need sequential access to the file, and there's an API you can use that won't try to load the whole file at once, maybe there is a better option. I just generally think of memory mapping when trying to read part of a file at once. –  May 24 '18 at 13:59
  • Yes, `ReadFile()` reads just the requested number of bytes. This is the approach I've started. There are a few issues, but I think I have them worked out. We'll see how it goes. – Jonathan Wood May 24 '18 at 14:00