We have apache serving files and proxying requests through to our api and today there was a new thing in the logs that I can't explain.

I'm investigating an issue that caused downtime and looking at the access logs I can see the last request and the first one that worked after it got back online but in between there's a couple lines with @s and ^s that I can't explain. I'm looking for a an explanation of what could have caused these characters to be printed in the middle of the access log.

The only thing I can guess is there was a hardware error that could somehow have corrupted the next entry that was in the middle of being written in that file. Not sure what else I can look for here?

[29/May/2020:12:04:00 +0000] "PUT /api/orders/2657675?cacheBuster=1590753840180 HTTP/1.1" 200 "Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/83.0.4103.61 Safari/537.36"
[29/May/2020:12:17:47 +0000] "GET /favicon.ico HTTP/1.1" 200 5430 "-" "Mozilla/5.0 (iPhone; CPU iPhone OS 13_4 like Mac OS X) AppleWebKit/605.1.15 (KHTML, like Gecko) CriOS/79.0.3945.73 Mobile/15E148 Safari/604.1"

^@ is a typical visualization of the NUL (0x00) byte.

It's almost certainly the tool that you use to display the file that is visualizing a sequence of NUL bytes as this sequence of ^@, while the file itself has no ^ or @ characters there.
(Typically inverted colors are used for such visualizations of otherwise unprintable characters, just to highlight that there is something special going on.)

Probably this log file was being written just as the machine was abruptly turned off, leading to the file ending up with a sequence of NUL bytes at the very end as a result of some filesystem specific handling of a write operation that was interrupted part-way through.

| improve this answer | |
  • Thanks for that explanation. It was what we were thinking as well but that first line there in your answer is very helpful, thanks! – caiocpricci2 May 29 at 14:26

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.