Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

CentOS

Is there an easy way to convert HTML special entities from a data stream? I'm passing data to a bash script and sometimes that data includes special entities. For example:

"test" & test $test ! test @ # $ % ^ & *

I'm not sure why some characters show up fine and other don't but unfortunately, I don't have control over the data coming in.

I'm thinking I might be able to use SED here but that seems like it would be cumbersome and possibly prone to false positives. Is there a Linux command I could pipe to that specializes in decoding this type of data?

share|improve this question

3 Answers 3

up vote 5 down vote accepted

PHP is well suited to this. This example requires PHP 5:

cat file.html | php -R 'echo html_entity_decode($argn);'
share|improve this answer
    
Thanks for the great idea but for some reason I'm getting an error: PHP Warning: html_entity_decode() expects at least 1 parameter, 0 given in Command line run code on line 1 –  Mike B Oct 22 '12 at 4:30
1  
Of course. Shell quoting strikes again. Try it with single quotes. –  Michael Hampton Oct 22 '12 at 13:19
    
Thanks. That resolved it. –  Mike B Oct 23 '12 at 16:51

Perl is (as always) your friend. I think this will do it:

perl -n -mHTML::Entities -e ' ; print HTML::Entities::decode_entities($_) ;'

E.g.:

echo '"test" & test $test ! test @ # $ % ^ & *' |perl -n -mHTML::Entities -e ' ; print HTML::Entities::decode_entities($_) ;'

With output:

someguy@somehost ~]$ echo '"test" & test $test ! test @ # $ % ^ & *' |perl -n -mHTML::Entities -e ' ; print HTML::Entities::decode_entities($_) ;'
"test" & test $test ! test @ # $ % ^ & *
share|improve this answer
    
This works on my OSX10.8 laptop and a RHEL5.something host. –  Jason Tan Oct 22 '12 at 5:52
    
In order to output the file in UTF-8, use binmode: echo "«" | perl -n -mHTML::Entities -mutf8 -e 'binmode(STDOUT, ":utf8"); print HTML::Entities::decode_entities($_) ;' –  falstaff Nov 23 '13 at 22:46

Takes text file from stdin:

#!/bin/bash
#
while read lin; do
  newl=${lin//>/>}
  newl=${newl//&lt;/<}
  newl=${newl//&amp;/<}
  # ...other entites
  echo "$newl"
done

It probably needs bash >= version 4

share|improve this answer

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.