72

Is it possible to make xargs use only newline as separator? (in bash on Linux and OS X if that matters)

I know -0 can be used, but it's PITA as not every command supports NUL-delimited output.

5
  • What is it that you're trying to accomplish. There may be another way to go about it. Mar 31, 2010 at 4:50
  • @Dennis I use it for various things, often in pipeline with grep, sed, very basic awk etc., mostly to parallelize execution. find -print0 -name \*.foo -maxdepth 1 | xargs -0 -P4 is way too much to type compared with ls *.foo | xargs -P4.
    – Kornel
    Mar 31, 2010 at 13:21
  • Scripts and functions are two excellent ways to reduce typing. Mar 31, 2010 at 13:49
  • This is an SO question, isn't it? Apr 1, 2010 at 11:22
  • Only upvoted it so that it reaches 69. Nice
    – Valdrinium
    Jun 20, 2023 at 19:12

6 Answers 6

23

Something along the lines of

alias myxargs='perl -p -e "s/\n/\0/;" | xargs -0'
cat nonzerofile | myxargs command

should work.

4
  • 59
    tr '\n' '\0' works too.
    – Kornel
    Apr 5, 2010 at 15:26
  • 4
    perl is overkill for a single character. tr is faster both for the cpu and for your fingers. Apr 23, 2015 at 20:09
  • 4
    When the only tool you know is radioactive decay, everything looks like a nuclear reactor
    – Ray Foss
    Sep 11, 2020 at 8:49
  • 1
    UGH! You want me to use perl? the hell i won't!!!
    – Michael
    Apr 20, 2023 at 22:50
80

GNU xargs (default on Linux; install findutils from MacPorts on OS X to get it) supports -d which lets you specify a custom delimiter for input, so you can do

ls *foo | xargs -d '\n' -P4 foo 
4
  • I have alias xxargs="xargs -d '\n'" in my bashrc. So I can just do things like this: grep -IRl foo | xxargs sed -i s/foo/bar/g
    – tylerl
    Feb 13, 2011 at 0:45
  • 6
    This really should be the accepted answer. No need to break out perl. Though tr is a good idea too. Dec 3, 2012 at 18:36
  • 7
    brew install findutils and gxargs was exactly what I needed on OS X, thanks.
    – Brad Koch
    Mar 22, 2016 at 19:00
  • @cmcginty's answer shows how to do this with xargs on macOS as well, without additional tools installed.
    – tresf
    May 22 at 15:03
6

With Bash, I generally prefer to avoid xargs for anything the least bit tricky, in favour of while-read loops. For your question, while read -ar LINE; do ...; done does the job (remember to use array syntax with LINE, e.g., ${LINE[@]} for the whole line). This doesn't need any trickery: by default read uses just \n as the line terminator character.

I should post a question on SO about the pros & cons of xargs vs. while-read loops... done!

6
  • 3
    This can only emulate xargs -n1 -d'\n', and not the whole xargs. Apr 23, 2015 at 20:12
  • @HelloWorld - You can bunch arguments together easily enough, so you can simulate xargs -n$N as well with only a bit of extra scripting. May 5, 2015 at 16:33
  • Yes, I'm among the upvoters, it's just a remark worth mentioning. May 9, 2015 at 10:43
  • This is exactly what I do! Plus, you can use the read options, like reading more words in separate variables, or change delimiter with IFS=' '.
    – caesarsol
    Mar 13, 2017 at 14:14
  • 1
    Reading the line into an array when you're expecting single lines is overkill at best, but more likely bug-inducing. When Bash reads into an array, it splits input on $IFS (normally whitespace), and concatenating the array with ${LINE[*]} or ${LINE[@]} does not restore that original whitespace. Using while read -r line; do ... is better for most situations, and lowercase variables avoid conflict with built-in/environment variables.
    – Walf
    Oct 26, 2021 at 2:56
6
$ echo "1\n2 3\n4 5 6" | xargs -L1 echo  "#"
# 1
# 2 3
# 4 5 6
3
  • 1
    Since echo is xarg's default command, you can just do echo "..." | xargs -L1. Nov 23, 2022 at 10:59
  • 1
    yes but what if I want each line in the file to be one argument to a single command invocation?
    – Michael
    Apr 20, 2023 at 22:55
  • 1
    This answer should be the accepted answer, as it does what the OP describes and works on Linux and macOS.
    – tresf
    May 22 at 15:04
2

please use the -d option of xargs

  --delimiter=delim, -d delim

Input items are terminated by the specified character. The specified delimiter may be a single character, a C-style character escape such as \n, or an octal or hexadecimal escape code. Octal and hexadecimal escape codes are understood as for the printf command. Multibyte characters are not supported. When processing the input, quotes and backslash are not special; every character in the input is taken literally. The -d option disables any end-of-file string, which is treated like any other argument. You can use this option when the input consists of simply newline-separated items, although it is almost always better to design your program to use --null where this is possible.

Example:

$ echo 'for arg in "$@"; do echo "arg: <$arg>"; done' > show_args
$ printf "a a\nb b\nc c\n"
a a
b b
c c
$ printf "a a\nb b\nc c\n" | xargs -d '\n' bash show_args
arg: <a a>
arg: <b b>
arg: <c c>
-3

What about cat file | xargs | sed 's/ /\n/ig' this will convert spaces to newlines, using standard Linux bash tools.

1
  • 2
    If file contains spaces and newlines, then the result of this command will make it impossible to distinguish between them, which seems to be going in the opposite direction of what this question asks for
    – Max Nanasy
    Sep 24, 2018 at 19:07

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .