Server Fault is a question and answer site for system and network administrators. Join them; it only takes a minute:

Sign up
Here's how it works:
  1. Anybody can ask a question
  2. Anybody can answer
  3. The best answers are voted up and rise to the top

I'm trying to migrate a bunch (300GB+) of files from a FAT32 drive to my freeNas ZFS filesystem but every command I throw at it (tar,pax,mv,cp) throws an 'invalid argument' when it encounters a non-ASCII filename - it's usually something that's been created under Windows and it reads something along the lines of "foo?s bar.mp3..." where the ? may have been an apostrophe or such.

Can anyone help with a few lines of code to recursively go through the directory tree and rename files to remove the offending characters.

Much appreciated.

share|improve this question
You shouldn't have any problem with odd filenames. Any script would likely use mv anyway, perhaps with sed or Bash equivalents. Please post the specific commands you've tried. – Dennis Williamson Jan 1 '10 at 22:56
Amongst others, I've tried something like: pax -w -f elements.tar /mnt/Elements mv -R /mnt/Elements /mnt/ZfsPool/ZfsDataset – Dan Jan 4 '10 at 18:53
See my edited answer. – Dennis Williamson Jan 6 '10 at 17:47
Anyone have an answer for this? Everythign thus far suggesed hasn't worked. – Dan Jan 11 '10 at 13:18
up vote 0 down vote accepted

Try mounting the filesystem with the iocharset option set to the encoding it uses.

From man mount under the "Mount options for fat" section:

          Character set to use for converting between 8 bit characters and
          16 bit Unicode characters. The default is iso8859-1.  Long file‐
          names are stored on disk in Unicode format.

See also under the "Mount options for vfat" section:

          Translate  unhandled  Unicode  characters  to  special   escaped
          sequences.   This lets you backup and restore filenames that are
          created with any Unicode characters. Without this option, a  '?'
          is used when no translation is possible. The escape character is
          ':' because it is otherwise illegal on the vfat filesystem.  The
          escape  sequence  that gets used, where u is the unicode charac‐
          ter, is: ':', (u & 0x3f), ((u>>6) & 0x3f), (u>>12).


   utf8   UTF8  is  the  filesystem safe 8-bit encoding of Unicode that is
          used by the console. It can be be  enabled  for  the  filesystem
          with this option or disabled with utf8=0, utf8=no or utf8=false.
          If `uni_xlate' gets set, UTF8 gets disabled.


I'm sorry, that was Linux, this is for BSD (from man mount_msdosfs:

 -L locale
     Specify locale name used for file name conversions for DOS and
     Win'95 names.  By default ISO 8859-1 assumed as local character

 -D DOS_codepage
     Specify the MS-DOS code page (aka IBM/OEM code page) name used
     for file name conversions for DOS names.
share|improve this answer
Thanks for the reply, here's what I tried: <pre>oracle:/mnt# mount -t msdos -o iocharset=utf8 /dev/ad6s1 /mnt/Elements </pre> But this failed with: <pre> mount: Using "-t msdosfs", since "-t msdos" is deprecated. mount_msdosfs: /dev/ad6s1: mount option <iocharset> is unknown: Invalid argument <pre> – Dan Jan 6 '10 at 16:01
<pre> doesn't work in comments, use backticks instead. – Dennis Williamson Jan 6 '10 at 17:45
Okay I tried with mount_msdosfs but I'm unlear what to specify for the L or D switches as the docs don't go into any specific detail. I figured the drive meant I needed the large option and here's what I tried: mount_msdosfs -o large /dev/ad6s1 /mnt/Elements. Unfortunately this didn't work either, I'm still getting pax (and others) choking on certain filenames containing 'special' characters. – Dan Jan 7 '10 at 21:35
What is the origin of the disk? Was it from a Windows system? What language version? Likely values for -L or -D would include CP437 or IBM437, CP1252, ASCII, ISO-646, other ISO-8859-* or variations of those names or others. On my Ubuntu system there's a directory at /usr/share/i18n/charmaps/ with files of character maps (the filenames and the header text in them) can be informative. See also: – Dennis Williamson Jan 8 '10 at 1:03
It was a Western Digital "Elements" External USB HDD, of which the external case failed, which I removed it from and popped it into a spare SATA port in my NAS box. OS X showed it as a FAT drive so made the assumption of it being FAT32. I've used it with UK versions of Windows XP and OS X. – Dan Jan 8 '10 at 9:06

Rename can do this..

try something like

find dir -depth -exec rename -n 's/[^[:ascii:]]/_/g' {} \; | cat -v

you may need the cat -v to properly display any weird characters without your terminal getting screwed.

if that prints acceptable substitutions change the -n to -v.

That said, it sounds like the charset on your filesystem is wrong(mount -o utf8 ?), since this sort of thing should really work...

share|improve this answer
Thank you for the reply-- I have read that I should be able to mount my filesystem as something different but the web seemed to indicate that it doesn't apply to FAT32 partitions? I'd like to be corrected though if this isn't the case? FreeNAS auto-mounts the drive when it starts but I do believe there's the option to override/re-mount etc. – Dan Jan 4 '10 at 13:49
Hmm... I don't seem to have the rename command on this box? I've tried man rename with no luck – Dan Jan 4 '10 at 18:52

This is a one correct way to apply recursively:

find . -depth -execdir rename 'y/[\:\;\>\<\@\$\#\&\(\)\?\\\%\ ]/_/' {} \;

change all of this symbols for underscore. Be carefull, is considering all white spaces.

why it works? take this test:

mkdir test

cd test

mkdir -p a$/b$/c$/d$ f%/g%/h%/i% j%/k%/l%/m%

find . -depth -execdir rename 'y/[\:\;\>\<\@\$\#\&\(\)\?\\\%\ ]/_/' {} \;

ls -R

(as you can see, all files were changed)

share|improve this answer
hey, I read that you need a "bsd" solution, I tested in GNU/Linux Ubuntu 12.04. – bash_newbie Oct 24 '12 at 0:55

Use convmv to convert the file names if they are really incorrectly encoded. You should prefer mounting the filesystem with the correct encoding in the first place.

share|improve this answer

Replacing by underscores :

find . | perl -ane '{ if(m/[[:^ascii:]]/) { print } }' | rename -n 's/[^[:ascii:]]/_/g'
share|improve this answer

Your Answer


By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.