Questions tagged [large-file-support]
25 questions
13
votes
2 answers
Is O_LARGEFILE needed just to write a large file?
Is the O_LARGEFILE flag needed if all that I want to do is write a large file (O_WRONLY) or append to a large file (O_APPEND | O_WRONLY)?
From a thread that I read titled "Cannot write >2gb index file" on the CLucene-dev mailing list, it appears…
Daniel Trebbien
- 38,421
- 18
- 121
- 193
9
votes
3 answers
How to enable large file support under Darwin?
I have a C application I am trying to compile for Mac OS X 10.6.4:
$ uname -v
Darwin Kernel Version 10.4.0: Fri Apr 23 18:28:53 PDT 2010; root:xnu-1504.7.4~1/RELEASE_I386
My gcc is as follows:
$ gcc --version
i686-apple-darwin10-gcc-4.2.1 (GCC)…
Alex Reynolds
- 95,983
- 54
- 240
- 345
9
votes
6 answers
Doing file operations with 64-bit addresses in C + MinGW32
I'm trying to read in a 24 GB XML file in C, but it won't work. I'm printing out the current position using ftell() as I read it in, but once it gets to a big enough number, it goes back to a small number and starts over, never even getting 20%…
zacaj
- 1,987
- 1
- 19
- 39
8
votes
2 answers
How to install Git Large File Storage on Windows?
According to its doc, double-click should be enough. But when I opened git bash and ran git lfs init, it doesn't work:
$ git lfs init
git: 'lfs' is not a git command. See 'git --help'.
laike9m
- 18,344
- 20
- 107
- 140
8
votes
2 answers
Python class to merge sorted files, how can this be improved?
Background:
I'm cleaning large (cannot be held in memory) tab-delimited files. As I clean the input file, I build up a list in memory; when it gets to 1,000,000 entries (about 1GB in memory) I sort it (using the default key below) and write the…
tgray
- 8,826
- 5
- 36
- 41
7
votes
2 answers
iostream and large file support
I'm trying to find a definitive answer and can't, so I'm hoping someone might know.
I'm developing a C++ app using GCC 4.x on Linux (32-bit OS). This app needs to be able to read files > 2GB in size.
I would really like to use iostream stuff vs.…
Joe
- 41,484
- 20
- 104
- 125
5
votes
2 answers
ftello() and fseeko() android build errors
I am trying to build Android L for 64-bit architecture.
My code goes like:
#if (HAS_LARGE_FILE_SUPPORT)
#define _FILE_OFFSET_BITS 64 //Defined in header file
/*Some File operations*/
#if HAS_LARGE_FILE_SUPPORT
return fseeko(iFile, offset,…
Android_Noob
- 51
- 2
5
votes
0 answers
Large file support not working on Android
I'm trying to build md5deep (http://md5deep.sourceforge.net/) for android, but with the default build setup, the assertion that sizeof(off_t) == 8 fails, indicating (AFAIK) that LFS isn't enabled. I've tried enabling by adding…
user2521200
- 83
- 6
5
votes
4 answers
Paperclip, large file uploads, and AWS
So, I'm using Paperclip and AWS-S3, which is awesome. And it works great. Just one problem, though: I need to upload really large files. As in over 50 Megabytes. And so, nginx dies. So apparently Paperclip stores things to disk before going to S3?
I…
Steve Klabnik
- 14,521
- 4
- 58
- 99
3
votes
1 answer
Convert largefiles to normal files
I have a repository where the trunk does not contain any largefiles. In my working branch, I previously added and committed some binaries as largefiles. Now, I don't want them to be largefiles anymore. I haven't merged into trunk yet.
I've tried…
Cryptovirus
- 271
- 2
- 6
3
votes
0 answers
OpenCV videocapture for large files
I am using OpenCV 2.4.3 with my own 64-bit binaries that are also built with QT 4.8.3. I am using DirectShow for videocapture and recently encountered a problem that I am unable to solve. Namely, the following code works perfectly on uncompressed…
Terry
- 31
- 3
2
votes
1 answer
C - creating new file with custom filename
I want to create new file with custom filename. I made this function to do this task:
int addfile(char* name)//return 0 if cannot create file
{
FILE* fn = fopen(name, "r");
if(fn == NULL)
{
return 0;
}
fclose(fn);
…
Konrad Stolarz
- 49
- 9
2
votes
1 answer
Understanding how Git LFS repository works for developers who don't have LFS extension installed
As the title already says, I am interested in how would a git repository with LFS files behave for developers who don't have LFS git extension installed (cloning of affected files, committing of affected files, etc). Would it be even possible for…
Emir
- 380
- 3
- 11
2
votes
2 answers
When writing C code involving files, should I define _FILE_OFFSET_BITS=64?
I've noticed some C projects compiling code which accesses files with _FILE_OFFSET_BITS=64. Now, on my system (which is 64-bit), adding or removing this doesn't seem to do much - but perhaps on other systems it does.
When should I use…
einpoklum
- 118,144
- 57
- 340
- 684
2
votes
1 answer
How can I portably turn on large file support?
I am currently writing a C program that reads and writes files that might be over 2 GiB in size. On linux feature_test_macros(7) specifies:
_LARGEFILE64_SOURCE
Expose definitions for the alternative API specified by the LFS (Large File…
fuz
- 88,405
- 25
- 200
- 352