Tag Info

Hot answers tagged

15

You can use rsync's --iconv option to convert between UTF-8 NFC & NFD, at least if you're on a Mac. There is a special utf-8-mac character set that stands for UTF-8 NFD. So to copy files from your Mac to your NAS, you'd need to run something like: rsync -a --iconv=utf-8-mac,utf-8 localdir/ mynas:remotedir/ This will convert all the local filenames ...


12

Use convmv: Package: convmv Priority: optional Section: utils Installed-Size: 88 Maintainer: Raphael Zimmerer <killekulla@rdrz.de> Architecture: all Version: 1.12-1 Depends: perl Filename: pool/main/c/convmv/convmv_1.12-1_all.deb Size: 20052 MD5sum: dcc45d5b8517026f588d769d81d67768 SHA1: 55da9650cfee5c64d8a4fdf278aaf9401a5e5dec SHA256: ...


10

The problem here doesn't seem to be on your browser nor your Apache configuration. You need to double check the locale settings of your system. You need to check if the locale apache is running is UTF-8 enabled. To do so you may run the command: $ sudo su -l -c locale www-data where www-data is the apache user. Check if the locale returned doesn't ...


8

Really, the surefire way to test is to download a text file and cat it in the terminal and see if everything looks ok. or, if you can, recompile the terminal enabling the unicode option (assuming it has one). what does $TERM and $LANG look like?


8

Type this in your terminal: echo -e '\xe2\x82\xac' If your terminal supports UTF-8 it will output the euro sign: €


7

But watch out : Do not use the character set named UTF8 as the database character set unless required for compatibility with Oracle Database clients and servers in version 8.1.7 and earlier, or unless explicitly requested by your application vendor. Despite having a very similar name, UTF8 is not a proper implementation of the Unicode encoding UTF-8. If the ...


7

There's not a specific command (or at least, not one that I'm aware of) to get this information, but you can find it between those provided by systeminfo.exe.


6

Rory, First of all, you are correct for wanting to monitor what gets created in you databases. While we all implement steps to prevent mistakes, you cannot assume that the mistakes won't creep in. I do a very similar thing as most of our infrastructure demands UTF8. The following queries are good for checking stats: SELECT DEFAULT_CHARACTER_SET_NAME, ...


6

I'm fairly sure bash pays attention to your locale setting, so if it's UTF-8, you should be good to go.


6

The default lighttpd.conf file seems to load MIME types from /etc/mime.types using a perl script: include_shell "/usr/share/lighttpd/create-mime.assign.pl" In my /etc/mime.types my html files had been set to be served as text/html. I commented out the include_shell perl script and added the following mimetype.assign entry to lighttpd.conf: ...


5

Probably your version of sed does not support multibyte separator characters. If you look at the way § is encoded in the two character sets, you'll see the difference: % locale LANG="en_CA.UTF-8" LC_COLLATE="en_CA.UTF-8" LC_CTYPE="en_CA.UTF-8" LC_MESSAGES="en_CA.UTF-8" LC_MONETARY="en_CA.UTF-8" LC_NUMERIC="en_CA.UTF-8" LC_TIME="en_CA.UTF-8" LC_ALL= % printf ...


5

Bitstream Vera Sans Mono does not support the Chess Symbols section of UTF8. Courier New also does not currently support the Chess Symbols section of UTF8. You'll have to find a Unicode font that supports chess symbols. Consider DejaVu Sans Mono which is Bitsream Vera Sans Mono, just with more unicode characters (including the chess symbols). FYI, these are ...


4

You should have two choices to make : Choose your database character set (used by VARCHAR2, CHAR, CLOB datatypes). Choose your national character set (used by NVARCHAR2, NCHAR, NCLOB datatypes). As seen here : Oracle recommends using Unicode for all new system deployments. National character sets can only be Unicode : UTF-8 or UTF-16. So ...


4

Okay, I found the answer after some googling. Apparently, LESSCHARSET needs to be set like this: export LESSCHARSET=utf-8 Now less works fine for me.


4

You can switch awstats to use 4.01 Transitional with the following configuration: (from awstats.model.conf) # If you prefer having the report output pages be built as XML compliant pages # instead of simple HTML pages, you can set this to 'xhtml' (May not work # properly with old browsers). # Change : Effective immediatly # Possible values: html or xhtml # ...


3

On Unix-like systems, the encoding of file names is not set at the filesystem level, but rather in the user environment. Check the output of locale and look at the stuff after the dot — for example, in my case LANG=en_US.UTF-8, so the file names in my environment are interpreted as UTF-8. This is the default setting in Ubuntu. The answer from Dennis ...


3

Note for readers who come across this page before they transferred the files: you can do the transfer with a recent rsync and the --iconv option: rsync -va --iconv=utf8,iso88591 /source/latin1/ /destination/utf8 (yes, the ordering of the iconv charsets is not intuitive!) For when the transfer has already been done, indeed convmv is the solution.


3

The lamest way: run following and check the output. It will be a capital O with circumflex if the terminal displays UTF-8. perl -le 'print "\x{c3}\x{94}"'


3

Well, I did not find any benchmarks in the Internet, so I decided to made benchmarks myself. I created a very simple table with 500000 rows: CREATE TABLE test( ID INT(11) DEFAULT NULL, Description VARCHAR(20) DEFAULT NULL ) ENGINE = INNODB CHARACTER SET utf8 COLLATE utf8_general_ci; Then I filled it with random data by running this stored procedure: ...


3

The GECOS field is in /etc/passwd, not /etc/shadown. I've been using UTF-8 realnames there with no adverse effects for years. (The adduser utility on Debian used to let me specify UTF-8 usernames a long time ago, but later started rejecting them. I sometimes wonder why.)


3

Some legacy application might not like it, but you should definitely use UTF-8 for new installations. for converting files (the contents) you can use iconv(1) for converting filenames you can use convmv(1) Other special applications (application generated data) might require other considerations.


3

The mongo version 1.6.4 was installed by the package management system, which includes the mongo init scripts. The solution I came up with was to download the newest Mongo(2.0.3) binaries from http://www.mongodb.org/downloads. Then I replaced all the binaries from the rpm /usr/bin with the ones I downloaded. Everything now works perfectly!


3

The file command uses only the beginning of the file to examine its content (for performance reasons). If your file contains only ascii characters in the beginning, the file command reports the file as ASCII. If the extracted file contains UTF-8 characters in the beginning (or a BOM-Header), the command reports the file as UTF-8 (as in your second example). ...


3

I doubt very much that the name of the file will affect the read/write speed of the system to a file. The reason being that once the file is open all actions on it take place through the file descriptor (fd) and the name is not used.


3

No. You'd either have to modify Linux or the filesystem implementation, or use a pass-through filter filesystem (perhaps implemented with fuse) that enforces the restriction. It's a nice idea, but probably very difficult to get consensus on: The old-school purists will insist that a filename should be able to be any nul-terminated byte string. Others will ...


3

Yes, the maximal file name length is 255 bytes, resulting in varying max length for filenames depending on code points used and their encoding length.


2

As long as you don't use the UTF-8 BOM, you should be ok.


2

The most sure fire way is to use the ‘locale’ command. It will print out all the various and sundry variables that dictate what character set to use. For instance, this is my output on RHEL5.3, set to only use UTF-8 by default. LANG="en_US.UTF-8" LC_COLLATE="en_US.UTF-8" LC_CTYPE="en_US.UTF-8" LC_MESSAGES="en_US.UTF-8" LC_MONETARY="en_US.UTF-8" ...


2

in /etc/my.cnf default-character-set=utf8 if you will use only this charset for databases already created which use latin1 charset, do the following alter database database_name charset=utf8; for each table: alter table table_name charset=utf8; for each varchar/char type column: alter table table_name alter column column_name charset=utf8;


2

Are there any motivations for NOT using UTF8 or other unicode character set? Just the one; you have an existing dataset of which you can't guarantee the current charset encoding. In which case you probably want to remedy that and still use UTF8.



Only top voted, non community-wiki answers of a minimum length are eligible