unidecode
Unidecode turns Unicode text into a readable ASCII approximation using hand-tuned, context-free character tables. It is useful for legacy systems, search keys, filenames, and stored URL slugs when simply deleting accents loses too much information. It does not detect language, preserve pronunciation reliably, or replace proper Unicode support, and its own README warns against showing the output directly to users.
Install it for lossy machine-facing ASCII conversion, especially with Latin-script input. Do not use it as a user-facing transliterator, and settle the GPL question before it enters a product dependency tree.
Use it if
- You must send readable text to an ASCII-only protocol, device, or legacy database
- You generate machine identifiers or URL slugs once, store them, and can tolerate a lossy approximation
- You want one dependency-free Python function with explicit policies for unmapped characters
- Your input is mostly Latin-script text, where the README says result quality is generally good
- The result is displayed to users: the README warns that language-agnostic mappings can be wrong or offensive, especially for characters shared by languages
- You need correct Japanese, Chinese, or Korean romanization: mappings are context-free and some Japanese Kanji deliberately receive Chinese transliterations
- Your distribution cannot comply with GPL-2.0-or-later terms; the maintainer explicitly points license-sensitive users to text-unidecode
- You need stable generated URLs across dependency updates: the README says transliteration tables change and can turn old links into 404s unless slugs are stored or the version is pinned
- You only need accent removal while keeping Unicode letters: Python's built-in unicodedata normalization is narrower and avoids Unidecode's guesses for symbols and non-Latin scripts
Setup reality
Installation is only `pip install unidecode`; version 1.4.0 has no runtime dependencies and supports Python 3.7 or later. The setup risk is semantic, not operational. The default `errors='ignore'` silently removes characters missing from its tables, including private-use characters, so production code should choose whether to reject, replace, or preserve them. `errors='preserve'` breaks the usual promise that the result can be encoded as ASCII. Language and locale are never detected. German umlauts become `a`, `o`, and `u`, not `ae`, `oe`, and `ue`, while shared Japanese and Chinese characters may get Chinese-oriented output. The README also requires a wide Unicode build for characters outside the Basic Multilingual Plane; modern mainstream Python builds normally satisfy that, but embedded or unusual runtimes deserve the documented `sys.maxunicode` check. If output becomes a URL or external ID, generate it once and persist it because table improvements may change future output. Finally, review the GPL license before shipping it inside redistributed software rather than discovering that constraint during release review.
Patterns
Convert Unicode text to ASCIItransliterate-text
from unidecode import unidecode
result = unidecode('kožušček')
print(result) # kozuscekThe conversion is an approximation, not reversible normalization; keep the original text as your source of truth.
Create and persist a URL slugcreate-stored-slug
import re
from unidecode import unidecode
def make_slug(title: str) -> str:
ascii_title = unidecode(title).lower()
return re.sub(r'[^a-z0-9]+', '-', ascii_title).strip('-')
article.slug = make_slug(article.title) # store this value onceDo not recompute historical slugs after upgrades; the README says mapping changes can otherwise break existing URLs.
Fail on an unmapped characterreject-unknown-character
from unidecode import UnidecodeError, unidecode
try:
value = unidecode(input_text, errors='strict')
except UnidecodeError as exc:
bad_character = input_text[exc.index]
raise ValueError(f'Unsupported character: {bad_character!r}') from excStrict mode exposes the offending index; the default mode would silently delete the character.
Replace characters missing from the tablesreplace-unknown-character
from unidecode import unidecode
clean = unidecode('name\ue000', errors='replace', replace_str='[?]')A visible replacement is safer than the default silent deletion when field length and auditability matter.
Preserve characters without mappingspreserve-unknown-character
from unidecode import unidecode
result = unidecode('name\ue000', errors='preserve')
print(result.encode('utf-8'))Preserve mode can return non-ASCII text, so do not call `encode('ascii')` or promise an ASCII-only result.
Preprocess German umlautsapply-german-rules
from unidecode import unidecode
def german_ascii(text: str) -> str:
replacements = str.maketrans({'ä': 'ae', 'ö': 'oe', 'ü': 'ue', 'Ä': 'Ae', 'Ö': 'Oe', 'Ü': 'Ue', 'ß': 'ss'})
return unidecode(text.translate(replacements))Unidecode intentionally does not infer German; apply known language rules before its context-free mapping.
Optimize an ASCII-heavy hot pathoptimize-ascii-input
from unidecode import unidecode_expect_ascii
values = [unidecode_expect_ascii(value) for value in incoming_values]This variant returns the same text as `unidecode`; it is tuned for inputs that are usually already ASCII.
Optimize a non-ASCII-heavy hot pathoptimize-nonascii-input
from unidecode import unidecode_expect_nonascii
values = [unidecode_expect_nonascii(value) for value in localized_values]Benchmark with representative text before switching; this variant is only slightly faster for non-ASCII input.
Transliterate a command-line valuetransliterate-command-input
unidecode -c '30 𝗄𝗆/𝗁'The `-c` form reads the following argument; without it the CLI reads a file or standard input.
Transliterate a file with an explicit encodingtransliterate-file
unidecode -e utf-8 input.txt > output.txtThe CLI otherwise chooses its input encoding from the system locale, which can differ between a shell, CI, and a container.
Check support for characters outside the BMPcheck-wide-unicode
import sys
if sys.maxunicode <= 0xFFFF:
raise RuntimeError('Unidecode requires a wide Unicode Python build')The README says narrow builds do not correctly support surrogate pairs outside the Basic Multilingual Plane.
Enforce an ASCII boundaryverify-ascii-output
from unidecode import unidecode
def to_ascii_bytes(text: str) -> bytes:
converted = unidecode(text, errors='replace')
return converted.encode('ascii', errors='strict')The final strict encode catches accidental use of preserve mode or later code that reintroduces Unicode.
Alternatives
| Package | Registry | Pick it when |
|---|---|---|
| text-unidecode | PyPI | You need similar table-driven transliteration under a more permissive license |
| Unihandecode | PyPI | You need language-aware handling for Chinese, Japanese, Korean, or Vietnamese text |
| anyascii | PyPI | You want a permissively licensed Unicode-to-ASCII library with broad script coverage |