Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

I'm maintaining two generally parallel sites based on a recent release of a well-known php-based CMS. One site is in English, one in Polish. (Polish localization is a standard option for the CMS.) Both are operating normally.

In particular, the Polish site correctly renders Polish diacritic characters as well as a sprinkling of "special" German and Cyrillic characters. When I examine the CMS-generated headers, I see

<meta http-equiv='Content-Type' content='text/html; charset=utf-8' />

exactly as I would expect. Unicode is the way to go.

The English site renders English characters correctly, of course, plus a similar sprinkling of "special" German and Cyrillic characters are rendered correctly. When I examine the CMS-generated headers, I see

<meta http-equiv='Content-Type' content='text/html; charset=iso-8859-1' />

which is not what I expect, as iso-8859-1 --as far as I can tell-- is incapable of rendering Polish diacritics and any Cyrillic. (I suppose I must except the non-diacritic Polish characters and Cyrillic characters that look like Latin ones, but the overlaps are beside the point.)

Q1: On a page declared in the header to be iso-8859-1 encoded, how is it that the Polish diacritics and Cyrillic characters render correctly? Could the browser be reading the BOM or doing an analysis of the actual content and overriding the header declaration? Or what?

Q2: Is there a good technical reason that the default English installation of the CMS should still use iso-8859-1 encoding instead of utf-8? I think all installations should use utf-8 encoding, but there's no pressing reason to convert the English version. Maybe someone can here can think of a good reason?

share|improve this question

3 Answers 3

A1: Probably your web server is configured to send UTF-8 encoding in HTTP header, before HTML is sent. I think you can inspect HTTP headers with Firebug or Chrome developer tools (Resources->http://...->Headers->Response Headers).

A2: Maybe they're still using 8859-1 because they didn't have time to switch to UTF8?

share|improve this answer
    
I've had the same problem and the reason was the app was sending "wrong" headers - i.e. iso-8859-1 encoding –  Jure1873 Nov 15 '10 at 15:10

Q1: CMS might use HTML entities to encode characters from outside ISO 8859-1 code range.

Q2: I'm not aware of any reasons to choose ISO 8859-1 over UTF 8 in this case.

share|improve this answer

The challenge with unicode is that all links in the chain must support it or it will be garbled.

Here's a common problem. Is the content being stored in a database? This must be UTF8 compatible. For mysql, log in at the command line and issue the command

show table status

Every table will show the collation/character set encoding.

You can see more on php utf8 encoding here

http://stackoverflow.com/questions/1344692/i-need-help-fixing-broken-utf8-encoding

and more on php/mysql here

http://stackoverflow.com/questions/405684/php-mysql-with-encoding-problems

To answer your second question -- U+0000 to U+00FF in UTF8 is identical to ISO 8859-1 (Latin-1). We use UTF-8 for encoding in all our websites and have not had any difficulties.

share|improve this answer
    
Right, all links must be working right... I inherited the site 16 months ago and spent a lot of time converting the DB to all UTF8, no problems there. The issue is with a moderate amount of textual material that is stored as ordinary HTML. –  hen3ry Oct 11 '10 at 18:50

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.