Take the 2-minute tour ×
Server Fault is a question and answer site for system and network administrators. It's 100% free, no registration required.

I have a bash script encoded in utf8 .

Within the script i use sed command using § as a separator .

Now when i run execute this script sed complains about the separator.

If i use normal char as a separator for ex @ then everything works.

I have viewed the script in putty[set utf8 in putty] and the character appears fine.

Also Linux default char set from locale command shows

 LC_CSET=en_US.UTF-8 

What could have gone wrong?

Earlier i used to have windows-1252 encoding for the shell scripts and this used to work.

share|improve this question

1 Answer 1

up vote 5 down vote accepted

Probably your version of sed does not support multibyte separator characters. If you look at the way § is encoded in the two character sets, you'll see the difference:

% locale
LANG="en_CA.UTF-8"
LC_COLLATE="en_CA.UTF-8"
LC_CTYPE="en_CA.UTF-8"
LC_MESSAGES="en_CA.UTF-8"
LC_MONETARY="en_CA.UTF-8"
LC_NUMERIC="en_CA.UTF-8"
LC_TIME="en_CA.UTF-8"
LC_ALL=
% printf § > section.utf8
% hexdump -C section.utf8
00000000  c2 a7                                             |..|
00000002
% iconv -f UTF-8 -t WINDOWS-1252 < section.utf8 > section.win1252
% hexdump -C section.win1252
00000000  a7                                                |.|
00000001

Various versions of sed will give you more or less helpful messages. On my OS X 10.6 system, I get the somewhat cryptic:

% sed 's§foo§bar§' 
sed: 1: "s§foo§bar§": RE error: illegal byte sequence

The version of sed that Ubuntu 10.04 LTS uses is more helpful:

% sed 's§foo§bar§'
sed: -e expression #1, char 2: delimiter character is not a single-byte character
share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.