Take the 2-minute tour ×
Server Fault is a question and answer site for professional system and network administrators. It's 100% free, no registration required.

This is relating to http://stackoverflow.com/questions/1791082/utf-8-php-and-xml-mysql, which I am still trying to get my head around.

I Have a couple of separate questions that will hopefully help me understand how to resolve the issues I am having.

I am trying to read values from a database and output into a file in UTF-8 format. But I am having encoding issues, so i thought I would strip back all my code ans start with:

$string = "Otivägen";
// then output to a file.

But in vim i cant even enter the that string, every time I paste it in I get Otivägen

I tried to create a blank PHP file with only that string and upload it, but when I cat the file again I get Otivägen.

My questions are ...

1) Why is `vim` displaying it like this?
2) If the file is downloaded would it display correctly if an application was expecting UTF-8?
3) How can I output this string into a file that will eventually be an XML file in UTF-8 encoding.

My understanding of encoding is limited at the moment, and I am trying to understand it.

share|improve this question
add comment

2 Answers 2

up vote 2 down vote accepted

1) Why is vim displaying it like this?

This looks like vim is displaying UTF-8-encoded data as ISO 8859-1. Copy&Paste can be problematic (you don't write what system you are on), so I'd advise to type in the text directly.

To properly edit the file in vim, first set vim to use UTF-8:

:set encoding=utf-8

Then type in the text, make sure it's correctly displayed, and save. That will give you a file encoded in UTF-8.

2) If the file is downloaded would it display correctly if an application was expecting UTF-8?

Depends on the encoding. If you save it as above, then yes.

3) How can I output this string into a file that will eventually be an XML file in UTF-8 encoding.

That is apparently very difficult. I'm not that familiar with PHP, but according to Wikipedia:

PHP currently does not have native support for Unicode or multibyte strings; Unicode support will be included in PHP 6[...]

So you'll probably have to google for a workaround. There are also a few UTF-8 helper libraries for PHP & UTF-8. Otherwise it might be better to choose a different language, e.g. Java which has solid Unicode support.

share|improve this answer
1  
PHP can do multibyte strings, you just have to use the right functions for the job; the default string functions aren't multibyte-aware: php.net/manual/en/book.mbstring.php –  quack quixote Nov 27 '09 at 3:29
add comment

UTF8 is fun. Once it works. :-/ If anything in the chain is expecting something else and doesn't check, then it all goes pear-shaped.

  • You need to use a terminal program that supports UTF8. Gnome-terminal does. KTerm does. ETerm doesn't.
  • Check your LANG variable in your shell. Mine is en_AU.UTF-8 which means English (Australian) in UTF8.
  • vim should inherit this from the shell. You should be able to verify this with :set encoding

I just tried it and that made everything work.

The key is that everything needs to be in UTF8 mode.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.