Hot answers tagged

28

Or use curl, where it's the default behaviour. curl http://www.google.com/ http://curl.haxx.se/


26

wget -O - -o /dev/null http://google.com


25

You might have a look at zgrep. >$ zgrep -h grep through gzip files usage: zgrep [grep_options] pattern [files]


23

Similiar to other answers, but: lsof | grep 90222668 Will show you both ends, because both ends share the 'pipe number'.


22

Use find ... -print0 | xargs -0 ... e.g. find /path/to -name "*.html" -print0 | xargs -0 grep -l "rumpus" from the find man page -print0 True; print the full file name on the standard output, followed by a null character (instead of the newline character that ‘-print’ uses). This allows file names that ...


21

To do that, use one extra file descriptor to switch stderr and stdout: find /var/log 3>&1 1>&2 2>&3 | tee foo.file Basically, it works, or at least I think it works, as follows: The re-directions are evaluated left-to-right. 3>&1 Makes a new file descriptor, 3 a duplicate (copy) of fd 1 (stdout). 1>&2 Make stdout (1) ...


19

While a zip file is in fact a container format, there's no reason why it can't be read as a stream if the file can fit into memory easily enough. Here's a Python script that takes a zip file as standard input and extracts the contents to the current directory or to a specified directory if specified. import zipfile import sys import StringIO data = ...


18

Use pv the pipe viewer. It's a great tool. Once you know about it you'll never know how you lived without it. It can also show you a progress bar, and the 'speed' of transfering.


18

You can use the -d option in curl with a @- argument to accept input from a pipe. You will need to construct the key-value pairs yourself. Try this: echo "time=`uptime`" | curl -d @- http://URL The backticks (`) denote that the enclosed command (in this case uptime) should be executed, and the backtick-quoted text replaced with the output of the executed ...


16

This is unlikely to work how you expect. Zip is not just a compression format, but also a container format. It rolls up the jobs of both tar and gzip.bzip2 into one. Having said that, if your zip has a single file, you can use unzip -p to extract the files to stdout. If you have more than one file, there's no way for you to tell where they start and stop. ...


16

Edit: I ran this with your source file in my environment and have the following results: [root@xt ~]# time tiff2ps test.tif > test.ps real 0m0.795s user 0m0.659s sys 0m0.135s [root@xt ~]# time ps2pdf13 -sPAPERSIZE=a4 test.ps > test.pdf real 0m0.592s user 0m0.513s sys 0m0.075s [root@xt ~]# time tiff2ps test.tif | ps2pdf13 ...


15

The tool to use is probably the windows ports of netcat.


13

You do not need to use xargs, because find can execute commands itself. When doing this, you do do not have to worry about the shell interpreting characters in the name. find /path/to -name "*.html" -exec grep -l "rumpus" '{}' + from the find man page -exec command {} + This variant of the -exec action runs the specified command on the selected ...


11

Try using sponge from moreutils like this: sed "s/root/toor/" /etc/passwd | grep -v joey | sponge /etc/passwd It collects the whole input before writing to it's output.


11

You probably want sed 's/exp1/exp2/g' foo.txt > foo2.txt Read more at Sed tutorial, Another tutorial, and A small tutorial at Linux HOWTOs


11

You can use xargs with -n1 to run a command once for each piped argument $some_command | xargs -n 1 touch In the case of touch however which accepts multiple arguments touch `$some_command` will probably work for you.


11

You can use pv to do this e.g. pv file | processor_application As pv passes it's stdin directly to it's stdout you don't need to use cat. Edit As your program is already running then find the PID of the cat process and then look at the contents of /proc/<PID>/io which will tell you how many bytes it has written - wchar.


11

This is possible with s3cmd 1.5+ (link): $ mysqldump ... | s3cmd put - s3://bucket/file-name.sql


11

This happened because the first thing > does is to create the file it wants to write to - and if the file already exists, its contents will be deleted. (Also, there's no need at all to use cat in your statement since grep works on files, not just on STDIN.) The correct way to do this is to use a temporary file either to read from or to write to. So ...


10

Kyle's Unix/Linux command does the job of switching the STDERR with the STDOUT; however the explanation is not quite right. The redirecting operators do not do any copying or duplicating, they just redirect the flow to a different direction. Rewriting Kyle's command by temporary moving the 3>&1 to the end, would make it easier to understand the concept: ...


10

The zgrep program exists for this specific purpose. http://linux.about.com/library/cmd/blcmdl1_zgrep.htm


9

Pipe through dd. dd's default input is stdin and default output is stdout; when it finishes stdin/stdout I/O, it will report to stderr on how much data it transferred. If you want to capture the output of dd and the other programs already talk to stderr, then use another file-descriptor. Eg, $ exec 4>~/fred $ input-command | dd 2>&4 | ...


9

Absolutely! Pipe Viewer does exactlty that. Just insert it in your pipeline: cat myfile | pv | processor_application You can optimize away the cat in the above example: pv myfile | processor_application Which has the advantage of providing an actual progress indicator, since pv can determine the size of the input directly. If you do use pv in the ...


9

Have you tried $ python example.py | bash It ought to work, as it's a common enough trick. For example, the monitoring tool munin has a node configurator (munin-node-configure) that tells you what plugins you can run, and then takes a --shell flag that makes it spit out a bunch of ln -s commands to link in the plugins, when piped directly to bash. There ...


8

$ my_command | tail -n +3 In this case, the +3 means "start output at the third line of the file".


8

If I understand what you want to do properly, you can do it with bash's command substitution feature: foo <(somecommand | pv) This does something similar to what the mkfifo-based answers suggest, except that bash handles the details for you (and it winds up passing something like /dev/fd/63 to the command, rather than a regular named pipe). You might ...


7

I only use for ... do ... done for very simple cases. For more complicated/dangerous scenarios: command | sed 's/^/touch /' This does nothing but prints intended commands. Review the results, then perform the same thing piping to sh -x (the -x flag is for debugging): command | sed 's/^/touch /' | sh -x


7

You usually are only asked if you want to connect when ssh is performing host key checking. Instead of trying to disable by using expect or a pipe, perhaps you could just disable it in your ssh configuration. Add StrictHostKeyChecking no to your ~/.ssh/config, and ssh will no longer ask you if you want to connect, it will just connect.


7

Try psql ... | less -S The -S option to less enables horizontal scrolling instead of line wrapping.


7

The latter; it makes sure STDOUT and STDERR of the original command go to the same fd, then feeds them jointly into tee. In the former case, it's the STDERR of the tee command that you'd be joining with its STDOUT.



Only top voted, non community-wiki answers of a minimum length are eligible