.\" Automatically generated by Pod::Man 4.14 (Pod::Simple 3.40) .\" .\" Standard preamble: .\" ======================================================================== .de Sp \" Vertical space (when we can't use .PP) .if t .sp .5v .if n .sp .. .de Vb \" Begin verbatim text .ft CW .nf .ne \\$1 .. .de Ve \" End verbatim text .ft R .fi .. .\" Set up some character translations and predefined strings. \*(-- will .\" give an unbreakable dash, \*(PI will give pi, \*(L" will give a left .\" double quote, and \*(R" will give a right double quote. \*(C+ will .\" give a nicer C++. Capital omega is used to do unbreakable dashes and .\" therefore won't be available. \*(C` and \*(C' expand to `' in nroff, .\" nothing in troff, for use with C<>. .tr \(*W- .ds C+ C\v'-.1v'\h'-1p'\s-2+\h'-1p'+\s0\v'.1v'\h'-1p' .ie n \{\ . ds -- \(*W- . ds PI pi . if (\n(.H=4u)&(1m=24u) .ds -- \(*W\h'-12u'\(*W\h'-12u'-\" diablo 10 pitch . if (\n(.H=4u)&(1m=20u) .ds -- \(*W\h'-12u'\(*W\h'-8u'-\" diablo 12 pitch . ds L" "" . ds R" "" . ds C` "" . ds C' "" 'br\} .el\{\ . ds -- \|\(em\| . ds PI \(*p . ds L" `` . ds R" '' . ds C` . ds C' 'br\} .\" .\" Escape single quotes in literal strings from groff's Unicode transform. .ie \n(.g .ds Aq \(aq .el .ds Aq ' .\" .\" If the F register is >0, we'll generate index entries on stderr for .\" titles (.TH), headers (.SH), subsections (.SS), items (.Ip), and index .\" entries marked with X<> in POD. Of course, you'll have to process the .\" output yourself in some meaningful fashion. .\" .\" Avoid warning from groff about undefined register 'F'. .de IX .. .nr rF 0 .if \n(.g .if rF .nr rF 1 .if (\n(rF:(\n(.g==0)) \{\ . if \nF \{\ . de IX . tm Index:\\$1\t\\n%\t"\\$2" .. . if !\nF==2 \{\ . nr % 0 . nr F 2 . \} . \} .\} .rr rF .\" .\" Accent mark definitions (@(#)ms.acc 1.5 88/02/08 SMI; from UCB 4.2). .\" Fear. Run. Save yourself. No user-serviceable parts. . \" fudge factors for nroff and troff .if n \{\ . ds #H 0 . ds #V .8m . ds #F .3m . ds #[ \f1 . ds #] \fP .\} .if t \{\ . ds #H ((1u-(\\\\n(.fu%2u))*.13m) . ds #V .6m . ds #F 0 . ds #[ \& . ds #] \& .\} . \" simple accents for nroff and troff .if n \{\ . ds ' \& . ds ` \& . ds ^ \& . ds , \& . ds ~ ~ . ds / .\} .if t \{\ . ds ' \\k:\h'-(\\n(.wu*8/10-\*(#H)'\'\h"|\\n:u" . ds ` \\k:\h'-(\\n(.wu*8/10-\*(#H)'\`\h'|\\n:u' . ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'^\h'|\\n:u' . ds , \\k:\h'-(\\n(.wu*8/10)',\h'|\\n:u' . ds ~ \\k:\h'-(\\n(.wu-\*(#H-.1m)'~\h'|\\n:u' . ds / \\k:\h'-(\\n(.wu*8/10-\*(#H)'\z\(sl\h'|\\n:u' .\} . \" troff and (daisy-wheel) nroff accents .ds : \\k:\h'-(\\n(.wu*8/10-\*(#H+.1m+\*(#F)'\v'-\*(#V'\z.\h'.2m+\*(#F'.\h'|\\n:u'\v'\*(#V' .ds 8 \h'\*(#H'\(*b\h'-\*(#H' .ds o \\k:\h'-(\\n(.wu+\w'\(de'u-\*(#H)/2u'\v'-.3n'\*(#[\z\(de\v'.3n'\h'|\\n:u'\*(#] .ds d- \h'\*(#H'\(pd\h'-\w'~'u'\v'-.25m'\f2\(hy\fP\v'.25m'\h'-\*(#H' .ds D- D\\k:\h'-\w'D'u'\v'-.11m'\z\(hy\v'.11m'\h'|\\n:u' .ds th \*(#[\v'.3m'\s+1I\s-1\v'-.3m'\h'-(\w'I'u*2/3)'\s-1o\s+1\*(#] .ds Th \*(#[\s+2I\s-2\h'-\w'I'u*3/5'\v'-.3m'o\v'.3m'\*(#] .ds ae a\h'-(\w'a'u*4/10)'e .ds Ae A\h'-(\w'A'u*4/10)'E . \" corrections for vroff .if v .ds ~ \\k:\h'-(\\n(.wu*9/10-\*(#H)'\s-2\u~\d\s+2\h'|\\n:u' .if v .ds ^ \\k:\h'-(\\n(.wu*10/11-\*(#H)'\v'-.4m'^\v'.4m'\h'|\\n:u' . \" for low resolution devices (crt and lpr) .if \n(.H>23 .if \n(.V>19 \ \{\ . ds : e . ds 8 ss . ds o a . ds d- d\h'-1'\(ga . ds D- D\h'-1'\(hy . ds th \o'bp' . ds Th \o'LP' . ds ae ae . ds Ae AE .\} .rm #[ #] #H #V #F C .\" ======================================================================== .\" .IX Title "CONVMV 1" .TH CONVMV 1 "2021-01-01" "perl v5.32.0" " " .\" For nroff, turn off justification. Always turn off hyphenation; it makes .\" way too many mistakes in technical documents. .if n .ad l .nh .SH "NAME" convmv \- converts filenames from one encoding to another .SH "SYNOPSIS" .IX Header "SYNOPSIS" \&\fBconvmv\fR [\fBoptions\fR] \s-1FILE\s0(S) ... \s-1DIRECTORY\s0(S) .SH "OPTIONS" .IX Header "OPTIONS" .IP "\fB\-f \s-1ENCODING\s0\fR" 4 .IX Item "-f ENCODING" specify the current encoding of the filename(s) from which should be converted .IP "\fB\-t \s-1ENCODING\s0\fR" 4 .IX Item "-t ENCODING" specify the encoding to which the filename(s) should be converted .IP "\fB\-i\fR" 4 .IX Item "-i" interactive mode (ask y/n for each action) .IP "\fB\-r\fR" 4 .IX Item "-r" recursively go through directories .IP "\fB\-\-nfc\fR" 4 .IX Item "--nfc" target files will be normalization form C for \s-1UTF\-8\s0 (Linux etc.) .IP "\fB\-\-nfd\fR" 4 .IX Item "--nfd" target files will be normalization form D for \s-1UTF\-8\s0 (\s-1OS X\s0 etc.). .IP "\fB\-\-qfrom\fR , \fB\-\-qto\fR" 4 .IX Item "--qfrom , --qto" be more quiet about the \*(L"from\*(R" or \*(L"to\*(R" of a rename (if it screws up your terminal e.g.). This will in fact do nothing else than replace any non-ASCII character (bytewise) with ? and any control character with * on printout, this does not affect rename operation itself. .IP "\fB\-\-exec\fR command" 4 .IX Item "--exec command" execute the given command. You have to quote the command and #1 will be substituted by the old, #2 by the new filename. Using this option link targets will stay untouched. Have in mind that #1 and #2 will be quoted by convmv already, you must not add extra quotation marks around them. .Sp Example: .Sp convmv \-f latin1 \-t utf\-8 \-r \-\-exec \*(L"echo #1 should be renamed to #2\*(R" path/to/files .IP "\fB\-\-list\fR" 4 .IX Item "--list" list all available encodings. To get support for more Chinese or Japanese encodings install the Perl HanExtra or \s-1JIS2K\s0 Encode packages. .IP "\fB\-\-lowmem\fR" 4 .IX Item "--lowmem" keep memory footprint low by not creating a hash of all files. This disables checking if symlink targets are in subtree. Symlink target pointers will be converted regardlessly. If you convert multiple hundredthousands or millions of files the memory usage of convmv might grow quite high. This option would help you out in that case. .IP "\fB\-\-nosmart\fR" 4 .IX Item "--nosmart" by default convmv will detect if a filename is already \s-1UTF8\s0 encoded and will skip this file if conversion from some charset to \s-1UTF8\s0 should be performed. \&\f(CW\*(C`\-\-nosmart\*(C'\fR will also force conversion to \s-1UTF\-8\s0 for such files, which might result in \*(L"double encoded \s-1UTF\-8\*(R"\s0 (see section below). .IP "\fB\-\-fixdouble\fR" 4 .IX Item "--fixdouble" using the \f(CW\*(C`\-\-fixdouble\*(C'\fR option convmv does only convert files which will still be \s-1UTF\-8\s0 encoded after conversion. That's useful for fixing double-encoded \&\s-1UTF\-8\s0 files. All files which are not \s-1UTF\-8\s0 or will not result in \s-1UTF\-8\s0 after conversion will not be touched. Also see chapter \*(L"How to undo double \s-1UTF\-8 ...\*(R"\s0 below. .IP "\fB\-\-notest\fR" 4 .IX Item "--notest" Needed to actually rename the files. By default convmv will just print what it wants to do. .IP "\fB\-\-parsable\fR" 4 .IX Item "--parsable" This is an advanced option that people who want to write a \s-1GUI\s0 front end will find useful (some others maybe, too). It will convmv make print out what it would do in an easy parsable way. The first column contains the action or some kind of information, the second column mostly contains the file that is to be modified and if appropriate the third column contains the modified value. Each column is separated by \e0\en (nullbyte newline). Each row (one action) is separated by \e0\e0\en (nullbyte nullbyte newline). .IP "\fB\-\-run\-parsable\fR" 4 .IX Item "--run-parsable" This option can be used to blindly execute the output of a previous \&\fB\-\-parsable\fR run. This way it's possible to rename a huge amount of file in a minimum of time. .IP "\fB\-\-no\-preserve\-mtimes\fR" 4 .IX Item "--no-preserve-mtimes" modifying filenames usually causes the parent directory's mtime being updated. Since version 2 convmv by default resets the mtime to the old value. If your filesystem supports sub-second resolution the sub-second part of the atime and mtime will be lost as Perl does not yet support that. With this option you can \&\fBdisable\fR the preservation of the mtimes. .IP "\fB\-\-replace\fR" 4 .IX Item "--replace" if the file to which shall be renamed already exists, it will be overwritten if the other file content is equal. .IP "\fB\-\-unescape\fR" 4 .IX Item "--unescape" this option will remove this ugly % hex sequences from filenames and turn them into (hopefully) nicer 8\-bit characters. After \-\-unescape you might want to do a charset conversion. This sequences like \f(CW%20\fR etc. are sometimes produced when downloading via http or ftp. .IP "\fB\-\-upper\fR , \fB\-\-lower\fR" 4 .IX Item "--upper , --lower" turn filenames into all upper or all lower case. When the file is not ASCII-encoded, convmv expects a charset to be entered via the \-f switch. .IP "\fB\-\-map=\fRsome-extra-mapping" 4 .IX Item "--map=some-extra-mapping" apply some custom character mappings, currently supported are: .Sp ntfs\-sfm(\-undo), ntfs\-sfu(\-undo) for the mapping of illegal ntfs characters for Linux or Macintosh cifs clients (see \s-1MS KB 117258\s0 also mapchars mount option of mount.cifs on Linux). .Sp ntfs\-pretty(\-undo) for for the mapping of illegal ntfs characters to pretty legal Japanese versions of them. .Sp See the \fBmap_get_newname()\fR function how to easily add own mappings if needed. Let me know if you think convmv is missing some useful mapping here. .IP "\fB\-\-dotlessi\fR" 4 .IX Item "--dotlessi" care about the dotless i/I issue. A lowercase version of \*(L"I\*(R" will also be dotless while an uppercase version of \*(L"i\*(R" will also be dotted. This is an issue for Turkish and Azeri. .Sp By the way: The superscript dot of the letter i was added in the Middle Ages to distinguish the letter (in manuscripts) from adjacent vertical strokes in such letters as u, m, and n. J is a variant form of i which emerged at this time and subsequently became a separate letter. .IP "\fB\-\-caseful\-sz\fR" 4 .IX Item "--caseful-sz" let convmv convert the sz ligature (U+00DF) to the uppercase version (U+1E9E) and vice versa. As of 2017 most fs case mapping tables don't treat those two code points as case equivalents. Thus the default of convmv is to treat it caseless for now also (unless this option is used). .IP "\fB\-\-help\fR" 4 .IX Item "--help" print a short summary of available options .IP "\fB\-\-dump\-options\fR" 4 .IX Item "--dump-options" print a list of all available options .SH "DESCRIPTION" .IX Header "DESCRIPTION" \&\fBconvmv\fR is meant to help convert a single filename, a directory tree and the contained files or a whole filesystem into a different encoding. It just converts the filenames, not the content of the files. A special feature of convmv is that it also takes care of symlinks, also converts the symlink target pointer in case the symlink target is being converted, too. .PP All this comes in very handy when one wants to switch over from old 8\-bit locales to \s-1UTF\-8\s0 locales. It is also possible to convert directories to \s-1UTF\-8\s0 which are already partly \s-1UTF\-8\s0 encoded. convmv is able to detect if certain files are \s-1UTF\-8\s0 encoded and will skip them by default. To turn this smartness off use the \f(CW\*(C`\-\-nosmart\*(C'\fR switch. .SS "Filesystem issues" .IX Subsection "Filesystem issues" Almost all \s-1POSIX\s0 filesystems do not care about how filenames are encoded, here are some exceptions: .PP \fI\s-1HFS+\s0 on \s-1OS X /\s0 Darwin\fR .IX Subsection "HFS+ on OS X / Darwin" .PP Linux and (most?) other Unix-like operating systems use the so called normalization form C (\s-1NFC\s0) for its \s-1UTF\-8\s0 encoding by default but do not enforce this. \s-1HFS+\s0 on the Macintosh \s-1OS\s0 enforces normalization form D (\s-1NFD\s0), where a few characters are encoded in a different way. On \s-1OS X\s0 it's not possible to create \s-1NFC UTF\-8\s0 filenames because this is prevented at filesystem layer. On \s-1HFS+\s0 filenames are internally stored in \s-1UTF\-16\s0 and when converted back to \s-1UTF\-8\s0 (because the Unix based \s-1OS\s0 can't deal with \s-1UTF\-16\s0 directly), \s-1NFD\s0 is created for whatever reason. See http://developer.apple.com/qa/qa2001/qa1173.html for defails. I think it was a very bad idea and breaks many things under \s-1OS X\s0 which expect a normal \s-1POSIX\s0 conforming system. Anywhere else convmv is able to convert files from \s-1NFC\s0 to \&\s-1NFD\s0 or vice versa which makes interoperability with such systems a lot easier. .PP \fI\s-1APFS\s0 on macOS\fR .IX Subsection "APFS on macOS" .PP Apple, with the introduction of \s-1APFS\s0 in macOS 10.3, gave up to impose \s-1NFD\s0 on user space. But once you enforced \s-1NFD\s0 there is no easy way back without breaking existing applications. So they had to make \s-1APFS\s0 normalization-insensitive, that means a file can be created in \s-1NFC\s0 or \s-1NFD\s0 in the filesystem and it can be accessed with both forms also. Under the hood they store hashes of the normalized form of the filename to provide normalization insensitivity. Sounds like a great idea? Let's see: If you readddir a directory, you will get back the files in the the normalization form that was used when those files were created. If you stat a file in \s-1NFC\s0 or in \s-1NFD\s0 form you will get back whatever normalization form you used in the stat call. So user space applications can't expect that a file that can be stat'ed and accessed successfully, is also part of directory listings because the returned normalization form is faked to match what the user asked for. Theoretically also user space will have to normalize strings all the time. This is the same problem as for the case insensitivity of filenames before, which still breaks many user space applications. Just that the latter one was much more obvious to spot and to implement than this thing. So long, and thanks for all the fish. .PP \fI\s-1JFS\s0\fR .IX Subsection "JFS" .PP If people mount \s-1JFS\s0 partitions with iocharset=utf8, there is a similar problem, because \s-1JFS\s0 is designed to store filenames internally in \s-1UTF\-16,\s0 too; that is because Linux' \s-1JFS\s0 is really \s-1JFS2,\s0 which was a rewrite of \s-1JFS\s0 for \s-1OS/2. JFS\s0 partitions should always be mounted with iocharset=iso8859\-1, which is also the default with recent 2.6.6 kernels. If this is not done, \s-1JFS\s0 does not behave like a \s-1POSIX\s0 filesystem and it might happen that certain files cannot be created at all, for example filenames in \s-1ISO\-8859\-1\s0 encoding. Only when interoperation with \s-1OS/2\s0 is needed iocharset should be set according to your used locale charmap. .PP \fI\s-1NFS4\s0\fR .IX Subsection "NFS4" .PP Despite other \s-1POSIX\s0 filesystems \s-1RFC3530\s0 (\s-1NFS 4\s0) mandates \s-1UTF\-8\s0 but also says: \&\*(L"The nfs4_cs_prep profile does not specify a normalization form. A later revision of this specification may specify a particular normalization form.\*(R" In other words, if you want to use \s-1NFS4\s0 you might find the conversion and normalization features of convmv quite useful. .PP \fI\s-1FAT/VFAT\s0 and \s-1NTFS\s0\fR .IX Subsection "FAT/VFAT and NTFS" .PP \&\s-1NTFS\s0 and \s-1VFAT\s0 (for long filenames) use \s-1UTF\-16\s0 internally to store filenames. You should not need to convert filenames if you mount one of those filesystems. Use appropriate mount options instead! .SS "How to undo double \s-1UTF\-8\s0 (or other) encoded filenames" .IX Subsection "How to undo double UTF-8 (or other) encoded filenames" Sometimes it might happen that you \*(L"double-encoded\*(R" certain filenames, for example the file names already were \s-1UTF\-8\s0 encoded and you accidentlly did another conversion from some charset to \s-1UTF\-8.\s0 You can simply undo that by converting that the other way round. The from-charset has to be \s-1UTF\-8\s0 and the to-charset has to be the from-charset you previously accidentlly used. If you use the \f(CW\*(C`\-\-fixdouble\*(C'\fR option convmv will make sure that only files will be processed that will still be \s-1UTF\-8\s0 encoded after conversion and it will leave non\-UTF\-8 files untouched. You should check to get the correct results by doing the conversion without \f(CW\*(C`\-\-notest\*(C'\fR before, also the \f(CW\*(C`\-\-qfrom\*(C'\fR option might be helpful, because the double utf\-8 file names might screw up your terminal if they are being printed \- they often contain control sequences which do funny things with your terminal window. If you are not sure about the charset which was accidentlly converted from, using \f(CW\*(C`\-\-qfrom\*(C'\fR is a good way to fiddle out the required encoding without destroying the file names finally. .SS "How to repair Samba files" .IX Subsection "How to repair Samba files" When in the smb.conf (of Samba 2.x) there hasn't been set a correct \*(L"character set\*(R" variable, files which are created from Win* clients are being created in the client's codepage, e.g. cp850 for western european languages. As a result of that the files which contain non-ASCII characters are screwed up if you \*(L"ls\*(R" them on the Unix server. If you change the \*(L"character set\*(R" variable afterwards to iso8859\-1, newly created files are okay, but the old files are still screwed up in the Windows encoding. In this case convmv can also be used to convert the old Samba-shared files from cp850 to iso8859\-1. .PP By the way: Samba 3.x finally maps to \s-1UTF\-8\s0 filenames by default, so also when you migrate from Samba 2 to Samba 3 you might have to convert your file names. .SS "Netatalk interoperability issues" .IX Subsection "Netatalk interoperability issues" When Netatalk is being switched to \s-1UTF\-8\s0 which is supported in version 2 then it is \s-1NOT\s0 sufficient to rename the file names. There needs to be done more. See http://netatalk.sourceforge.net/2.0/htmldocs/upgrade.html#volumes\-and\-filenames and the uniconv utility of Netatalk for details. .SH "SEE ALSO" .IX Header "SEE ALSO" \&\fBlocale\fR\|(1) \fButf\-8\fR\|(7) \fBcharsets\fR\|(7) .SH "BUGS" .IX Header "BUGS" no bugs or fleas known .SH "DONATE" .IX Header "DONATE" You can support convmv by doing a donation, see .SH "AUTHOR" .IX Header "AUTHOR" Bjoern \s-1JACKE\s0 .PP Send mail to bjoern [at] j3e.de for bug reports and suggestions.