Reading files larger than 606666752 bytes

I'm trying to read a large file in C using open() and read(). I am doing this is a non-blocking way using a kqueue. Everything works fine until 606666752 bytes have been read from the file descriptor, but at that point kevent stops returning (i.e., no bytes are ready to be read). For smaller files, this is not an issue. Is there any setting I have to change to be able to read large files? Thanks in advance.
 
Are you testing for EV_ERROR or similar?

FreeBSD off_t should handle up to OFF_MAX, which is nowhere near the value you're seeing. Would be easier to see your code...
 
Let us know if this is the same problem as EOF, you may want to warnx("%lli", ke.data) just before going to the next loop sequence.
 
Ah, it looks like that ke.data contains the value 606666752 in the first loop instead of the correct file size. Therefore, it only reads 606666752 bytes. I suspect this is because of the limited size of that struct field, but I have no idea how to get around this limitation. Anyone?
 
When ke.data - BUFSIZ reaches <=0, issue EV_CLEAR on the next invocation? Not sure it'll work, but from coarse reading of the code, EV_CLEAR should clear the kernel copy of the event. Then you need to solve the problem to detect when really EOF.
 
The file I'm using is 4901634048 bytes, which is 4.6 gigabytes. So the file is much bigger than INT_MAX.
 
It looks like the problem is with the file I am using for testing. Other large files work fine. I don't know exactly what's wrong with the culprit, but it seems as if the file size is much larger than the actual file contents. Weird.
 
Sparse file maybe? Tho, EOF should be at what stat(2) says. I'd copy it to another dir. It's possible the directory entry is lying. If so, do fsck -y in single user.
 
Back
Top