PyPIDataupdated 07 Aug 2026

html5lib

Item: html5lib
Rating: 3.3
Author: Keyoor

html5lib is a pure Python HTML parser that follows the WHATWG HTML parsing algorithm, the same tree-construction rules browsers implement. That matters because real-world HTML is broken: unclosed tags, misnested elements, tables with stray text, implied <html> and <body> that were never written. Where a regex or a lenient parser guesses, html5lib applies the spec's error recovery and produces the tree a browser would produce. It hands that tree back as xml.etree elements by default, or as minidom or lxml if you ask, and it can serialize a tree back to HTML. It is also the most spec-faithful backend BeautifulSoup can use, which is where most of its download volume comes from.

html5lib · v1.1 · 9.0M/wk · ★ 1,223 · pushed 21 Apr 2026 · html5lib/html5lib-python · docs · reviewed by Keyoor

Verdict

Still the reference-grade HTML parser in Python when you need browser-identical trees from broken markup, and worth keeping installed as a BeautifulSoup fallback. Do not choose it for volume, and do not use its sanitizer for anything.

API stability5/5No release since 1.1 in June 2020, so nothing has changed under anyone; parse, parseFragment, HTMLParser, getTreeBuilder, and serialize behave exactly as they did, deprecated sanitizer aside.

Docs3/5html5lib.readthedocs.io plus a clear README cover parsing, treebuilders, and serializing with runnable examples; the namespace default, the encoding options, and the treewalker and filter layer get much less explanation than they need.

Maintenance1/5Last release June 2020, last commit on master February 2024, 84 open issues (104 counting PRs), and the sanitizer's recommended successor Bleach was itself retired in June 2026.

Ecosystem4/5Around 9M weekly downloads and 1.2k stars, driven mostly by being a BeautifulSoup backend and a transitive dependency of packaging tooling; almost nobody imports it directly any more.

Use it if

You need the tree a browser would build from mangled markup, for example scraping pages with unclosed <p> and <li> tags, misnested inline elements, or content dropped outside <tbody>
You are already using BeautifulSoup and lxml gives you the wrong structure on a specific page; BeautifulSoup(html, "html5lib") swaps in the spec-conformant parse without changing the rest of your code
You cannot install compiled dependencies. html5lib is pure Python, runs on PyPy, and needs no C toolchain, unlike lxml or the faster C parsers
You want the parse errors, not just the tree: HTMLParser().errors gives you a list of (position, error code, data) tuples, which is useful for HTML linting or validating generated markup

Skip it if

Throughput matters. It is pure Python and parses far slower than lxml on the same document; if you are processing a crawl rather than a handful of pages, that difference dominates your runtime
You want the sanitizer. html5lib.filters.sanitizer has been deprecated since version 1.1 and prints a DeprecationWarning the moment you import it. Its recommended replacement, Bleach, announced in June 2026 that it is also no longer maintained, including for security issues. Use nh3 and treat any html5lib-based sanitizing code as something to migrate
You expect ongoing releases. Version 1.1 shipped in June 2020 and there has been none since; the last commit on the default branch is from February 2024. The WHATWG spec is a living standard, so the gap only widens
You do not want Python 2 baggage. It still depends on six, still declares support back to Python 2.7, and its API uses camelCase names like parseFragment, getTreeBuilder, and namespaceHTMLElements
You just need to pull a few fields out of well-formed HTML. selectolax or lxml with CSS selectors does that in a fraction of the time, and stdlib html.parser handles the truly simple cases with nothing installed

Setup reality

pip install html5lib pulls six and webencodings, both pure Python, so there is no build step anywhere. The surprises are in the output rather than the install. By default namespaceHTMLElements is True, so every tag comes back as {http://www.w3.org/1999/xhtml}p and your ElementTree find() calls silently match nothing until you either pass namespaceHTMLElements=False or write the namespace into every query. The lxml treebuilder needs lxml installed and the README warns it is not supported on PyPy, where it is known to segfault. Character encoding detection from bytes works out of the box but improves with the chardet extra (pip install html5lib[chardet]), and you should pass transport_encoding when you have a charset from an HTTP header. Serialization defaults are opinionated too: omit_optional_tags is on, so a round trip drops the <html> and <head> tags you just parsed.

Patterns

Parse a full HTML documentparse-document

import html5lib

document = html5lib.parse("<p>Hello World!")
print(document.tag)
# {http://www.w3.org/1999/xhtml}html

with open("page.html", "rb") as f:
    document = html5lib.parse(f)

You get an xml.etree element with implied html, head, and body inserted per the spec. Open files in binary mode so html5lib can do its own encoding detection instead of Python guessing for it.

Get plain tag names instead of namespaced onesdisable-namespaces

import html5lib

tree = html5lib.parse("<ul><li>a<li>b", namespaceHTMLElements=False)
print([li.text for li in tree.iter("li")])   # ['a', 'b']

# with the default you would need:
ns = "{http://www.w3.org/1999/xhtml}"
tree2 = html5lib.parse("<ul><li>a<li>b")
print([li.text for li in tree2.iter(ns + "li")])

This is the single biggest time sink for new users. namespaceHTMLElements defaults to True, so tree.find("body") returns None and nothing tells you why. Set it False unless you actually need XHTML namespaces.

Parse a snippet without wrapping it in a documentparse-fragment

import html5lib

frag = html5lib.parseFragment("<b>bold</b> text")
print(frag.tag)                      # DOCUMENT_FRAGMENT
print([e.tag for e in frag])

row = html5lib.parseFragment("<tr><td>x", container="table")

Without container the fragment is parsed as if inside a div, which throws away table rows and cells because they are not valid there. Pass the real parent element name when parsing table or select internals.

Build a minidom or lxml tree instead of etreechoose-treebuilder

import html5lib

parser = html5lib.HTMLParser(tree=html5lib.getTreeBuilder("dom"))
minidom_doc = parser.parse("<p>Hello")
print(minidom_doc.toxml())

lxml_parser = html5lib.HTMLParser(tree=html5lib.getTreeBuilder("lxml"))
lxml_tree = lxml_parser.parse("<p>Hello")   # needs lxml installed

The lxml builder gives you XPath and CSS selection over a spec-conformant tree, which is the usual reason to pick it. The README warns it is unsupported on PyPy, where it is known to segfault.

See everything wrong with a documentcollect-parse-errors

import html5lib

parser = html5lib.HTMLParser()
parser.parse("<p>x</p></br>")

for position, code, data in parser.errors:
    print(position, code, data)
# (1, 3) expected-doctype-but-got-start-tag {'name': 'p'}
# (1, 13) unexpected-end-tag-treated-as {'originalName': 'br', ...}

errors accumulates on the parser instance, so reuse across documents keeps appending. Create a fresh HTMLParser per document, or read and clear the list between parses.

Raise on the first parse errorstrict-mode

import html5lib

parser = html5lib.HTMLParser(strict=True)
try:
    parser.parse("<p>unclosed<div>")
except html5lib.html5parser.ParseError as exc:
    print(exc)   # Unexpected start tag (p). Expected DOCTYPE.

strict=True is a conformance checker, not a scraping mode. It fires on a missing doctype, so almost every real page fails immediately; use it on HTML you generate, not HTML you fetch.

Write a parsed tree back out as HTMLserialize-tree

import html5lib

doc = html5lib.parse("<p>Hello <b>World")

print(html5lib.serialize(doc, tree="etree"))
# <p>Hello <b>World</b>

print(html5lib.serialize(doc, tree="etree", omit_optional_tags=False))
# <html><head></head><body><p>Hello <b>World</b></body></html>

serialize takes the tree and picks the walker itself, so do not pass it a treewalker result. omit_optional_tags defaults to True, which is why the default round trip loses the html and head wrappers html5lib just inserted.

Normalize attributes and whitespace on outputserializer-options

html5lib.serialize(
    doc,
    tree="etree",
    quote_attr_values="always",
    alphabetical_attributes=True,
    strip_whitespace=True,
)
# '<p class="x">a b'

quote_attr_values defaults to "legacy", which leaves some values unquoted. Setting it to "always" plus alphabetical_attributes gives byte-stable output, which is what you want if you diff serialized HTML in tests.

Handle encoding when parsing raw bytesparse-bytes-with-encoding

from urllib.request import urlopen
import html5lib

with urlopen("https://example.com/") as f:
    document = html5lib.parse(
        f, transport_encoding=f.info().get_content_charset()
    )

Feed bytes, not a decoded string, so html5lib can apply the spec's sniffing rules: transport encoding first, then a BOM, then a meta charset. Install the chardet extra if you scrape pages that declare nothing.

Use html5lib through BeautifulSoupbeautifulsoup-backend

from bs4 import BeautifulSoup

soup = BeautifulSoup(broken_html, "html5lib")
print(soup.p.get_text())

# compare against the faster, less conformant option:
soup_fast = BeautifulSoup(broken_html, "lxml")

This is how most projects actually use html5lib. Keep "lxml" as your default and switch a specific parse to "html5lib" when a page's structure comes out wrong; the API above them is identical.

Do not sanitize untrusted HTML with html5libavoid-sanitizer

# Deprecated since html5lib 1.1, warns on import:
# from html5lib.filters.sanitizer import Filter
# html5lib.serialize(doc, tree="etree", sanitize=True)

import nh3
print(nh3.clean("<script>bad()</script><b>ok</b>"))
# <b>ok</b>

Importing html5lib.filters.sanitizer emits a DeprecationWarning naming Bleach as the replacement, and Bleach itself stopped being maintained in June 2026. nh3 wraps the Rust ammonia crate and is the maintained choice.

Pull the visible text out of a pageextract-text

import html5lib

tree = html5lib.parse(html_bytes, namespaceHTMLElements=False)
for tag in ("script", "style"):
    for node in tree.iter(tag):
        node.text = None

text = " ".join(t.strip() for t in tree.itertext() if t.strip())

itertext walks every descendant including script and style bodies, so clear those first or your output picks up JavaScript. ElementTree offers no parent pointers, which is why this blanks text in place rather than removing nodes.

Alternatives

Package	Registry	Pick it when
html5-parser	PyPI	You want the same HTML5 parsing algorithm but in C, producing lxml trees, when spec conformance and speed both matter.
lxml	PyPI	Speed is the priority and libxml2's lenient HTML recovery is close enough for the pages you actually parse.
selectolax	PyPI	You are extracting fields with CSS selectors at scale and want the fastest option available in Python.
nh3	PyPI	Your real goal is sanitizing untrusted HTML; this is a maintained Rust-backed sanitizer, unlike html5lib's deprecated filter or Bleach.