Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I know that head and tail can take -c option to specify a byte offset. I'm looking for a way to efficiently extract a byte range from a large log file.

share|improve this question

4 Answers 4

up vote 3 down vote accepted

The DareDevil of the Unix commands, dd to the rescue!

dd if=yourfile ibs=1 skip=200 count=100

That would start from byte 200 and show 100 next bytes, or in other words, bytes 200-300. ibs means dd only reads one byte at a time instead of the default 512 bytes, but still writes out in default 512 byte chunks. Go and see if ibs harms the performance, I hope not.

share|improve this answer

You can use dd if=logfile of=pieceoflogfile skip=startingblock count=#ofblocks (possibly with bs=1 to get one-byte blocks, otherwise it uses 512 byte blocks). Not sure how efficient it is to tell it to write one byte at a time, though.

share|improve this answer
    
Well, if concerned regarding efficiency — 2 dd's can be pipe-chained together (the 1st might cut fat piece and the 2nd would do fine job over the pipe, not disk), but dd has not only bs but separate ibs and obs as well, so at least it could output with bigger blocks than reading. –  poige Jul 11 '12 at 15:25

If your interest is in the bytes, would od be of more interest.

-j, --skip-bytes=bytes
-N, --read-bytes=bytes

So to read the 16 bytes starting at byte 1024, and output in ascii

od -j 1024 -N 16 -a /bin/sh
share|improve this answer

Assuming the file isn't excessively large (e.g. several GB or so), piping from one to the next is as efficient as you'll get, short of writing your own program to do so.

head ... file | tail ...

(Or the other way around. Whichever.)

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.