The tag has no usage guidance.

learn more… | top users | synonyms

5
votes
0answers
42 views

Can you scp, sftp, or rsync, a pipe? [migrated]

I want to tar and send a ~700GiB directory to a remote drive I don't control. I don't have the HDD space locally to create the tarball and then copy that over. The remote is also protected by rssh - ...
4
votes
2answers
1k views

Apache errorlog piping fail

Trying to log to a central syslog server, either direct using Apache's ErrorLog to pipe to logger, or getting syslog to forward, but nothing is working and the errors are not making sense to me. I can ...
0
votes
1answer
23 views

gzip piped output into ssh command

I make backup over ssh like this: mysqldump -u dbuser -pXXXXXXXX dbname | ssh -i ~/.ssh/id_rsa_backup backup@my.server.com 'cat > /var/backups/services/my_service/db/$(date +"%Y-%m-%d").sql' How ...
0
votes
0answers
27 views

Starting and stopping piped processes with Supervisor

I have a situation where I pipe the output of ffmpeg into a PHP script for some processing. I might have a few of these processes running at any given time and I figured to start and stop them I might ...
5
votes
2answers
137 views

Cat, Grep, Redirect Output… Blank File?

I just ran cat /opt/webapplications/Word/readme.log | grep -v 'Apple' and I got the output on the cli that I was expecting, which was all the lines in readme.log that did not contain 'Apple'... ...
0
votes
6answers
3k views

How to recursively search and replace from command line on unix/linux system

So I want to change several files at one with one command. This is what I have so far: find -regex ".*\.ext$" | xargs grep -l searchText 2> /dev/null | xargs -i sed 's/searchText/replaceText/' >...
5
votes
7answers
2k views

Changing files on the fly in Linux (writing to input file on a pipe)

How to change a file on the fly along a pipe? I'm probably looking for a way to buffer a file at the beginning of a pipe, which in contrast to this: cat foo.txt | grep bar > foo.txt ... ...
1
vote
1answer
50 views

How do I pipe stdout and std error to a file and back to stdout on Windows?

On Windows when running a batch script I want to record the output of I run: myprogram.bat 1> mylog.log 2>&1 Which is great - but I have to run a separate tail process to read the output ...
0
votes
2answers
49 views

Streaming stdout

I have a script that outputs a lot of debug data that I would like to avoid to save on disk (specially since the file system is on a flash memory card). So I run it on the background with nohup + &...
7
votes
4answers
4k views

Count number of bytes piped from one process to another

I'm running a shell script that pipes data from one process to another process_a | process_b Does anyone know a way to find out how many bytes were passed between the two programs? The only ...
29
votes
8answers
29k views

Unzipping files that are flying in through a pipe

Can I make unzip or any similar programs work on the standard output? The situation is I'm downloading a zip file, which is supposed to be unzipped on the fly. Related issue: http://serverfault.com/...
0
votes
0answers
48 views

Postfix bouncing emails submitted through pipe

I have a Postfix with a piped content filter which tries to locally deliver the messages before sending them out, resulting in a bounce notification for each event. According to the Postfix ...
17
votes
6answers
25k views

How do I pipe a downloaded file to standard output in bash?

How is it possible to pipe out wget's downloaded file? If not what alternatives should I use?
1
vote
2answers
436 views

Continual tailing and sorting of multiple files on linux?

Is there a way to use pipes or FIFO in combination to continually monitor several (same format) log files whilst sorting on the first field and then being able to effectively do a tail -f on that ...
0
votes
2answers
459 views

Unzip from stdin to stdout - funzip, python

The goal is to read a zip file from stdin and uncompress to stdout. Funzip works and is the solution I am looking for, the zip contains a single file, unfortunately funzip fails when the compressed ...
0
votes
3answers
2k views

forward all mail on a specified domain to script

I run a disposable e-mail service that accepts all incoming mail and forwards it to a PHP script that stores it in a database for people to view. Before now, I have been on shared hosting with cPanel, ...
0
votes
0answers
429 views

nginx - open() failed (2: No such file or directory)

Our ubuntu server running nginx and uwsgi is causing lot's of open() failed (2: No such file or directory) errors by ngnix. Some web pages loading slowely or do not load completely. It seems to ...
2
votes
1answer
289 views

How to avoid “in use by another process” errors when piping output through a program?

In a Windows environment, are executable programs that are used to process piped output guaranteed to be unique tasks if called from different CMD environments? For example, suppose script1.cmd ...
1
vote
1answer
60 views

How to pipe a file being written to?

I have a binary file that is being written to continuously. I need to read the full content of that file to a pipe (either to nc or to a FIFO) and keep reading it while its is written (i.e. the ...
5
votes
3answers
505 views

Postfix - Pipe multiple recipient message as a single message

I am trying to pipe emails through Postfix to a Python script I have. Currently, messages with multiple recipients are piped to the script once for each recipient. I would like these messages to be ...
3
votes
2answers
3k views

postfix pipe to script: execvp permissions denied error

I am new to postfix and am trying to pipe a message to a particular email address to a bash script. I am running CentOS 6 in case that matters. My script has 777 permission (for testing), and when I ...
-1
votes
1answer
68 views

Cron: Execute python script, capture stdout & stderr and send via msmtp [duplicate]

I try this for almost 4 hours now, in different ways. I googled a lot, but nothing I try work as expected. I have a simple .msmtprpc file, that allows me to send mails like this: echo -e "Subject: ...
0
votes
1answer
356 views

wget and pipe to tar in the background

I am trying to download a large file which is .tar.gz and (> 2GB) in size. The bash script also does lots of other things, so I want to kick off the download and then continue processing other ...
11
votes
5answers
4k views

Pass a pipe to a command that expects a filename

Say I have a command foo which takes a filename argument: foo myfile.txt. Annoyingly, foo doesn't read from standard input. Instead of an actual file, I'd like to pass it the result of another command ...
0
votes
1answer
2k views

syslog-ng not flushing the pipe to external program

I have written a python script that takes log entries from syslog-ng and writes them to MongoDB (I couldn't do it with afmongodb driver because I need to do some special processing). This is how it ...
1
vote
2answers
1k views

Non-blocking bash redirection

Does anyone know how to redirect the output of a program, running from a bash script, to a named pipe IN NON BLOCKING MODE (having 'O_NONBLOCK' flag set)?
1
vote
2answers
329 views

How to pipe postfix into Codeigniter controller

I currently have postfix piping into a PHP script to process the email via an alias. catchall: |/var/www/vhosts/website/httpdocs/scripts/incoming_mail.php However, I wish to use the DB config etc ...
2
votes
3answers
2k views

how to pipe a mysql dump to s3cmd

I want to transfer a mysql dump, compressed, to s3. I tried: mysqldump -u root -ppassword -all-databases | gzip -9 | s3cmd put s3://bucket/sql/databases.sql.gz but then I get: ERROR: Not enough ...
0
votes
0answers
109 views

Remote packet capturing using fifo and dumpcap

I'm currently working on two linux servers where one server is running in 2.6.18-274.el5(Server A) and the second one in CentOS 7 (Server B). My requirement is to capture the packets of the network ...
7
votes
1answer
2k views

using temporary files vs pipes advantages and disadvantages

Say I have a file named jobs.csv and I would like to get the top 50k jobs done by Foo I can either do: # cat jobs.csv | sort -u | head -n 50000 > /tmp/jobs.csv # cat /tmp/jobs.csv | while read ...
18
votes
3answers
9k views

How can I handle spaces in file names when using xargs on find results?

One of my common practices is to perform greps on all files of a certain type, e.g., find all the HTML files that have the word "rumpus" in them. To do it, I use find /path/to -name "*.html" | xargs ...
2
votes
2answers
176 views

Adding a progress bar to a data pipe

Are there any alternatives to pv? I'd like to see the current transfer rate of a data pipe, a progress bar and an estimated completion time. Any other tools that can do that?
18
votes
1answer
2k views

Shell command slow when using pipe, fast with intermediate file

Does anyone understand this huge difference in processing time, when using an intermediate file, or when using a pipe? I'm converting tiff to pdf using standard tools on a fresh debian squeeze server. ...
1
vote
1answer
81 views

Postfix pipe mail at user level

I am trying to filter mail at the user level. I have CentOS and Postfix installed. I managed to install a filter on the server and its working great, except, instead of filtering by user, all emails (...
1
vote
1answer
32 views

pipe rrd to a new directory

I am chrooted and running out of drive space, so I mounted back to the host drive which has plenty of space but cannot seem to redirect the outputted data to my empty drive [root@localhost rra]# ls /...
0
votes
1answer
197 views

Pipe multiple files into one tar gz

I'd like to call mysqldump to create multiple backup files and then archive them in one tar.gz. I am following this tutorial on mysqldump part: http://dev.mensfeld.pl/2013/04/backup-mysql-dump-all-...
0
votes
2answers
71 views

Unzip piping order

I have a file, foo.zip which looks something like this: foo.zip -- fileA.txt -- fileB.txt I can pipe the output of the extracted files like this: unzip -p foo.zip *.txt | something My problem is ...
3
votes
1answer
201 views

does email pipe to program cause problems with unicode characters?

I'm piping incoming mail into a PHP script, immediately storing the RAW email in a MySQL db. It works very well, except ~0.7% of emails arrive with a truncated message body. I found someone whose ...
-1
votes
3answers
4k views

Can't pipe echo to netcat?

I have the following command: echo 'HTTP/1.1 200 OK\r\n' | nc -l -p 8000 -c and when I curl localhost:8000 I am not seeing HTTP/1.1 200 .. being printed. I am on mac os x with netcat 0.7.1 Any ...
0
votes
2answers
381 views

Powershell pipe exporting to csv is loosing data

I'm trying to write a script that will take input from a .txt file of IP address. Do a reverse DNS lookup to get the hostname of the IP address and then export that data into .csv file. Here is what ...
1
vote
1answer
275 views

Exim4 setup not working with piping

I'm trying to setup exim4 on the server, so that it can receive mail for a couple sub domains, and I want to pipe that to a PHP script. I want all incoming mail to be processed through my custom ...
1
vote
1answer
223 views

Cat pipe mysql : Stopped. What stopped and how can I fix that?

I imported a large database from remote using a cat + mysql command : cat my_db.sql | mysql --host=external_host --user=my_user --password=some_password my_database And after some time, it returned ...
1
vote
5answers
338 views

How to split the output into different files?

I have a input file like foo xxx yyy zzz foo xxx yyy zzz foo xxx yyy zzz foo xxx yyy zzz bar xxx yyy zzz bar xxx yyy zzz foo xxx yyy zzz .. How to split the input file by line, into foo.txt ...
1
vote
1answer
194 views

Postfix Pipe then forward to original recipient

I may well be on the wrong track here with what I'm trying to do, so I'd appreciate any advice. Is it possible to pipe email to a script (which I have working), and then have Postfix forward it on ...
4
votes
0answers
1k views

DSGET escapes hash signs (#) in outputted Distinguished Names (DN)s but expects unescaped hash signs for input

Lets say I want to do the very simple query in AD dsquery user -name "John Smith" | dsget user -memberof -expand This will output the DNs of the AD groups that this user belongs to. I can make it ...
2
votes
2answers
874 views

SELinux preventing /etc/aliases email pipe to script from working

When somebody sends an email to my server, e.g. somestringthatisnotnecessarilyauser@myserver.com I want it to pipe to a PHP script. So in my /etc/aliases file I have: ...
0
votes
1answer
100 views

How to comma-demilit the output from cut?

So far, I have this: grep -B 36 'pattern1' Mapper.1.Configuration.MapRules | grep "pattern2" | cut -d'"' -f2 which give me a new-line delimited list of values that I need. I could put this into a ...
1
vote
2answers
186 views

how to pipe results from LS to CD in ubuntu

I'd like to ls for the latest created directory in a given directory (we create a new folder for each release) and then cd to that directory. I'd like to create an alias for this so I don't have to ...
-1
votes
1answer
208 views

Make a copy of email using pipe in postfix alongside 1 content_filter

I am using postfix and already usHi. I am using postfix and already using a script as a content_filter which re-injects back to smtpd to all users, but before this happens, I would like to make a copy ...
0
votes
2answers
792 views

pipe raw email message to python script

In exim4, I use following pipe to send a message to a python script, process it, and upload it to django: send_to_django_mailbox: driver = pipe command = /usr/local/bin/python /etc/exim4/conf.d/...