0

I'm maintaining two generally parallel sites based on a recent release of a well-known php-based CMS. One site is in English, one in Polish. (Polish localization is a standard option for the CMS.) Both are operating normally.

In particular, the Polish site correctly renders Polish diacritic characters as well as a sprinkling of "special" German and Cyrillic characters. When I examine the CMS-generated headers, I see

<meta http-equiv='Content-Type' content='text/html; charset=utf-8' />

exactly as I would expect. Unicode is the way to go.

The English site renders English characters correctly, of course, plus a similar sprinkling of "special" German and Cyrillic characters are rendered correctly. When I examine the CMS-generated headers, I see

<meta http-equiv='Content-Type' content='text/html; charset=iso-8859-1' />

which is not what I expect, as iso-8859-1 --as far as I can tell-- is incapable of rendering Polish diacritics and any Cyrillic. (I suppose I must except the non-diacritic Polish characters and Cyrillic characters that look like Latin ones, but the overlaps are beside the point.)

Q1: On a page declared in the header to be iso-8859-1 encoded, how is it that the Polish diacritics and Cyrillic characters render correctly? Could the browser be reading the BOM or doing an analysis of the actual content and overriding the header declaration? Or what?

Q2: Is there a good technical reason that the default English installation of the CMS should still use iso-8859-1 encoding instead of utf-8? I think all installations should use utf-8 encoding, but there's no pressing reason to convert the English version. Maybe someone can here can think of a good reason?

3 Answers 3

1

A1: Probably your web server is configured to send UTF-8 encoding in HTTP header, before HTML is sent. I think you can inspect HTTP headers with Firebug or Chrome developer tools (Resources->http://...->Headers->Response Headers).

A2: Maybe they're still using 8859-1 because they didn't have time to switch to UTF8?

1
  • I've had the same problem and the reason was the app was sending "wrong" headers - i.e. iso-8859-1 encoding
    – Jure1873
    Nov 15, 2010 at 15:10
1

Q1: CMS might use HTML entities to encode characters from outside ISO 8859-1 code range.

Q2: I'm not aware of any reasons to choose ISO 8859-1 over UTF 8 in this case.

0

The challenge with unicode is that all links in the chain must support it or it will be garbled.

Here's a common problem. Is the content being stored in a database? This must be UTF8 compatible. For mysql, log in at the command line and issue the command

show table status

Every table will show the collation/character set encoding.

You can see more on php utf8 encoding here

https://stackoverflow.com/questions/1344692/i-need-help-fixing-broken-utf8-encoding

and more on php/mysql here

https://stackoverflow.com/questions/405684/php-mysql-with-encoding-problems

To answer your second question -- U+0000 to U+00FF in UTF8 is identical to ISO 8859-1 (Latin-1). We use UTF-8 for encoding in all our websites and have not had any difficulties.

1
  • Right, all links must be working right... I inherited the site 16 months ago and spent a lot of time converting the DB to all UTF8, no problems there. The issue is with a moderate amount of textual material that is stored as ordinary HTML.
    – hen3ry
    Oct 11, 2010 at 18:50

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .