Convert ASCII chars to Unicode FULLWIDTH latin letters in Python?
The range of fullwidth ASCII replacements starts at U+FF01, not U+FF00. U+FF00 is (strangely) not defined. To get a fullwidth SPACE, you need to use U+3000 IDEOGRAPHIC SPACE. Don't rely on typing what appears to be what you want followed by visual inspection of characters to check your mapping -- unicodedata.name
is your friend. This code:
# coding: utf-8
from unicodedata import name as ucname
# OP
normal = u"""0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!"#$%&()*+,-./:;<=>?@[\\]^_`{|}~"""
wide = u"""0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!゛#$%&()*+、ー。/:;〈=〉?@[\\]^_‘{|}~"""
# above after editing (had = twice)
widemapOP = dict((ord(x[0]), x[1]) for x in zip(normal, wide))
# Ingacio V
normal = u' 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!"#$%&()*+,-./:;<=>?@[\\]^_`{|}~'
wide = u' 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!゛#$%&()*+、ー。/:;〈=〉?@[\\]^_‘{|}~'
widemapIV = dict((ord(x[0]), x[1]) for x in zip(normal, wide))
# JM
widemapJM = dict((i, i + 0xFF00 - 0x20) for i in xrange(0x21, 0x7F))
widemapJM[0x20] = 0x3000 # IDEOGRAPHIC SPACE
maps = {'OP': widemapOP, 'IV': widemapIV, 'JM': widemapJM}.items()
for i in xrange(0x20, 0x7F):
a = unichr(i)
na = ucname(a, '?')
for tag, widemap in maps:
w = a.translate(widemap)
nw = ucname(w, '?')
if nw != "FULLWIDTH " + na:
print "%s: %04X %s => %04X %s" % (tag, i, na, ord(w), nw)
when run shows what you have really got: some missing mappings and some idiosyncratic mappings:
JM: 0020 SPACE => 3000 IDEOGRAPHIC SPACE
IV: 0020 SPACE => 3000 IDEOGRAPHIC SPACE
OP: 0020 SPACE => 0020 SPACE
IV: 0022 QUOTATION MARK => 309B KATAKANA-HIRAGANA VOICED SOUND MARK
OP: 0022 QUOTATION MARK => 309B KATAKANA-HIRAGANA VOICED SOUND MARK
IV: 0027 APOSTROPHE => 0027 APOSTROPHE
OP: 0027 APOSTROPHE => 0027 APOSTROPHE
IV: 002C COMMA => 3001 IDEOGRAPHIC COMMA
OP: 002C COMMA => 3001 IDEOGRAPHIC COMMA
IV: 002D HYPHEN-MINUS => 30FC KATAKANA-HIRAGANA PROLONGED SOUND MARK
OP: 002D HYPHEN-MINUS => 30FC KATAKANA-HIRAGANA PROLONGED SOUND MARK
IV: 002E FULL STOP => 3002 IDEOGRAPHIC FULL STOP
OP: 002E FULL STOP => 3002 IDEOGRAPHIC FULL STOP
IV: 003C LESS-THAN SIGN => 3008 LEFT ANGLE BRACKET
OP: 003C LESS-THAN SIGN => 3008 LEFT ANGLE BRACKET
IV: 003E GREATER-THAN SIGN => 3009 RIGHT ANGLE BRACKET
OP: 003E GREATER-THAN SIGN => 3009 RIGHT ANGLE BRACKET
IV: 005C REVERSE SOLIDUS => 005C REVERSE SOLIDUS
OP: 005C REVERSE SOLIDUS => 005C REVERSE SOLIDUS
IV: 0060 GRAVE ACCENT => 2018 LEFT SINGLE QUOTATION MARK
OP: 0060 GRAVE ACCENT => 2018 LEFT SINGLE QUOTATION MARK
Those "wide" characters are named FULLWIDTH LATIN LETTER
: http://www.unicodemap.org/range/87/Halfwidth%20and%20Fullwidth%20Forms/
They have range 0xFF00 - -0xFFEF. You can make look-up table or just add 0xFEE0 to ASCII code.
Yes; in Python 3, cleanest is to use str.translate and str.maketrans:
HALFWIDTH_TO_FULLWIDTH = str.maketrans(
'0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!"#$%&()*+,-./:;<=>?@[]^_`{|}~',
'0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!゛#$%&()*+、ー。/:;〈=〉?@[]^_‘{|}~')
def halfwidth_to_fullwidth(s):
return s.translate(HALFWIDTH_TO_FULLWIDTH)
In Python 2, str.maketrans is instead string.maketrans and doesn’t work with Unicode characters, so you need to make a dictionary, as Ignacio Vazquez notes above.
Yes.
>>> normal = u' 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!"#$%&()*+,-./:;<=>?@[\\]^_`{|}~'
>>> wide = u' 0123456789abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!゛#$%&()*+、ー。/:;〈=〉?@[\\]^_‘{|}~'
>>> widemap = dict((ord(x[0]), x[1]) for x in zip(normal, wide))
>>> print u'Hello, world!'.translate(widemap)
Hello、 world!