16

for exmaple, using the command

cat foo.txt | xargs -I{} -n 1 -P 1 sh -c "echo {} | echo"

The foo.txt contains two lines

foo
bar

The above command print nothing.

0

2 Answers 2

6
cat foo.txt | xargs -J % -n 1 sh -c "echo % | bar.sh" 

Tricky part is that xargs performs implicit subshell invocation. Here sh invoked explicitly and pipe not becomes the part of parent conveyor

6
  • 1
    Thanks, I have updated my question to provide a more concret example. but it does not work as you suggested..
    – Ryan
    Dec 24, 2013 at 9:13
  • 1
    echo cant read from stdin, so piping to it has no sense. compare this: cat foo.bar | wc -l and cat foo.bar | xargs -J % -n 1 sh -c "echo % | wc -l"
    – Kondybas
    Dec 24, 2013 at 9:50
  • 1
    I think you mean -I instead of -J; there is no -J option to xargs Jun 1, 2017 at 2:18
  • @Hitechcomputergeek FreeBSD's version of xargs have -J option that is equivalent to the -i of the linux xargs
    – Kondybas
    Jun 1, 2017 at 13:15
  • @Kondybas Thanks for telling me that; I was not aware that there was a difference between the two. You can trust GNU to not follow POSIX lol. (-J isn't defined in POSIX but -I is and has a different use than GNU's.) Jun 1, 2017 at 13:59
2

If you want to process all the lines of foo.txt you will have to use a loop. Use & to put the process to background

while read line; do
   echo $line | bar.sh &
done < foo.txt

If your input contain spaces temporarily set the internal field separator to the newline

# save the field separator
OLD_IFS=$IFS

# new field separator, the end of line 
IFS=$'\n'

for line in $(cat foo.txt) ; do
   echo $line | bar.sh &
done

# restore default field separator  
IFS=$OLD_IFS     
3
  • No-no, I'm stuck with it too. TS want to split file into seperate lines and feed them into script independently
    – Kondybas
    Dec 24, 2013 at 7:19
  • 1
    I need to use xargs to parallel the process..
    – Ryan
    Dec 24, 2013 at 7:21
  • OK :-) I just looked up the options of xargs the OP used.
    – Matteo
    Dec 24, 2013 at 7:25

Not the answer you're looking for? Browse other questions tagged or ask your own question.