Why can't I create files larger than 2 GB on Linux?

Question

I wanted to share some knowledge I picked up when I ran into trouble using libpcap and snort to sniff a high-capacity (1 GB full duplex; 2 GB max aggregate) network link. The applications would sniff all traffic successfully, but would crash when the file size hit 2 GB captured.

If you're having issues with creating 2 GB files, even though you have a filesystem that supports it, and/or you know the kernel supports it, this is for you.

Answer 1

Large file support ( >2GB ) for Linux needs to be addressed in three separate locations to ensure you do not run into the 2 GB max file size limit:

1. Large file support enabled in the kernel

2. A filesystem that supports large file sizes (many Linux-based filesystems do (ext3, reiserfs >= 3.6, etc))

3. Large file support within the libraries or applications utilized

Kernel support for large files has been around since 2.4.0-test7; if you use a custom kernel, ensure you've included the large file options.

Most Linux filesystems support large files, but you may have issues using a network file system.

Lastly, libraries (i.e. libpcap) and applications need to be compiled with the gcc options -D _LARGEFILE64_SOURCE -D _FILE_OFFSET_BITS=64. Check to ensure the packages you are using either come precompiled with these options or roll your own.

See here for more details.

Answer 2

Looks like this is a problem with libpcap ... As stated in the first answer.

Recompile libpcap, or find another version.