13

In the documentation for the access_log directive, the nginx documentation says

The buffer size must not exceed the size of an atomic write to a disk file.

How can I determine what this size is on my system?

2
  • @mdpc From the linked document it is pretty clear, that it is not about sector sizes, which btw. has been 512 bytes on most media since the late 80s until now. There is a move towards 4K sector sizes on new drives.
    – kasperd
    May 29, 2014 at 17:39
  • This specification may be relevant. Though it doesn't seem to give any exact answer to the question: pubs.opengroup.org/onlinepubs/7908799/xsh/write.html
    – kasperd
    May 26, 2015 at 14:28

2 Answers 2

7

better late than never :)

quick answer is: "2,147,479,552 bytes, if kernel version is 3.14 or newer"

detailed answer:

As far as I understand, it is about write syscall:

http://man7.org/linux/man-pages/man2/write.2.html

1) any POSIX systems ( linux, bsd, all unix ) are guaranteed to be able to write up to MAX_SSIZE bytes

According to POSIX.1, if count is greater than SSIZE_MAX, the result is implementation-defined; see NOTES for the upper limit on Linux.

# getconf SSIZE_MAX
32767

2) linux guaranteed to be able to write up to 1.99 GiB ( and it's atomic operation for linux kernel version 3.14 and newer )

On Linux, write() (and similar system calls) will transfer at most 0x7ffff000 (2,147,479,552) bytes, returning the number of bytes actually transferred. (This is true on both 32-bit and 64-bit systems.)

But it's fair atomic operation only from linux kernel 3.14

According to POSIX.1-2008/SUSv4 Section XSI 2.9.7 ("Thread Interactions with Regular File Operations"):

All of the following functions shall be atomic with respect to each other in the effects specified in POSIX.1-2008 when they operate on regular files or symbolic links: ...

Among the APIs subsequently listed are write() and writev(2). And among the effects that should be atomic across threads (and processes) are updates of the file offset. However, on Linux before version 3.14, this was not the case: if two processes that share an open file description (see open(2)) perform a write() (or writev(2)) at the same time, then the I/O operations were not atomic with respect updating the file offset, with the result that the blocks of data output by the two processes might (incorrectly) overlap. This problem was fixed in Linux 3.14.

1

This Superuser answer had a good definition of what atomic write size is.

This is at least as large as the size of the hardware sector, which is the atomic read/write size.

2
  • 1
    OK, so how do I determine how big a disk sector is?
    – bdesham
    May 29, 2014 at 17:49
  • 10
    The nginx documentation and the superuser answer are not talking about the same layer in the storage stack. The nginx documentation talks about the largest atomic write at the file system layer, which is OS and FS dependent. The superuser answer is talking about the largest atomic write at the block level, which is hardware dependent.
    – kasperd
    May 29, 2014 at 17:54

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .