tree-sitter review
tree-sitter 0.26.0 is the Python binding for Tree-sitter's incremental concrete-syntax parser. It returns a navigable tree even when a file is half-written, records ERROR and MISSING nodes for recovery, and can reuse an edited old tree to limit later parse work. Query patterns match grammar structure across the result. No language grammar is bundled, so every Python, JavaScript, Rust, or other parser comes from another wheel. Version 0.26 adds editable Point and Range objects plus containing-range query filters, while removing Language.query, Language.version, and two timeout properties.
tree-sitter 0.26.0 installed as one 3 MB package in 0.3 seconds and imported in 0.13 seconds in our sandbox, with bundled typing and no audit findings. Use it for error-tolerant multi-language syntax work, but budget strict grammar pins and migration tests for its still-breaking 0.x API.
We installed it
| Install | ✓ · 0.3s | 1 package on disk · 3 MB |
| Import | ✓ | import tree_sitter in 0.13s · compiled extensions · py.typed · requires Python >=3.10 |
| Known vulns | 0 | (pip-audit) |
Answers from our run
Does tree-sitter install cleanly?
Yes. In a fresh container with an empty cache, pip install tree-sitter finished in 0.3s, leaving 1 package and 3 MB on disk. pip-audit reported no known vulnerabilities.
What does tree-sitter need to run?
Python >=3.10, and a platform wheel with compiled extensions. In our run import tree_sitter succeeded in 0.13s, and the package ships py.typed for type checkers.
tree-sitter or libcst: which should you use?
libcst: Choose it for Python codemods that must retain comments, whitespace, and formatting. tree-sitter 0.26.0 installed as one 3 MB package in 0.3 seconds and imported in 0.13 seconds in our sandbox, with bundled typing and no audit findings.
When should you not use tree-sitter?
Your work is Python-only and needs Python semantics. ast, parso, or LibCST gives domain-specific nodes without a separate grammar ABI decision.
Use it if
- An editor, indexer, or code search tool needs the same node and query APIs across several programming languages.
- Incomplete source must still yield a useful structure with explicit recovery nodes instead of a fatal parse exception.
- Frequent small edits can be described precisely and reparsed against the previous tree.
- A structural query is clearer than maintaining a custom recursive visitor for every match shape.
- Your work is Python-only and needs Python semantics. ast, parso, or LibCST gives domain-specific nodes without a separate grammar ABI decision.
- The output must include cross-file symbols, types, definitions, or references. Tree-sitter describes syntax and does not provide a language server's semantic model.
- A 0.x minor release may not remove public APIs. Version 0.26 deletes Language.query, Language.version, Parser.timeout_micros, and QueryCursor.timeout_micros.
- You cannot pin grammar wheels beside the binding. Each grammar releases independently and must expose an ABI the installed binding accepts.
- The application assumes Python character indexes equal parser offsets. Node offsets count encoded bytes, which diverge as soon as source contains multibyte Unicode.
Setup reality
We installed tree-sitter 0.26.0 in a fresh, unprivileged Python 3.12 Bookworm container. pip completed in 0.3 seconds, left one package, and used 3 MB. The measured metadata records 7 direct dependencies and requires Python 3.10 or newer. The wheel includes compiled .so extensions and py.typed, identifies the MIT License, and imported as tree_sitter in 0.13 seconds. pip-audit found zero known vulnerabilities.
The binding alone cannot parse any programming language. Install a grammar such as tree-sitter-python, pass its capsule into Language(), and give that Language to Parser. Grammar packages use their own release schedule and ABI. Pin the binding and every grammar as one tested set. The project publishes wheels for major platforms; an unsupported platform is a native build problem, not a missing optional feature.
Parser input is bytes or a callback returning bytes in UTF-8 or UTF-16. start_byte and end_byte index the encoded buffer, while Point holds a row and encoded column. Retain the exact source bytes beside the tree and decode only extracted slices. Incremental parsing works only after tree.edit receives correct old and new byte plus point coordinates, followed by parser.parse(new_source, old_tree). Wrong coordinates can produce a plausible but incorrect tree.
Version 0.26 constructs Query(language, source) directly and executes it through QueryCursor. captures() groups nodes by capture name; matches() keeps related captures together. Cancellation now uses progress_callback instead of timeout_micros. The new containing-range methods require a capture to fit fully inside the selected range, which differs from intersection filtering. Parsers, cursors, and edited trees should not be mutated concurrently without application-level isolation.
Patterns
Parse bytes with a separate grammar wheel load-python-grammar
import tree_sitter_python as tspython
from tree_sitter import Language, Parser
PYTHON = Language(tspython.language())
parser = Parser(PYTHON)
source = b'def total(a, b):\n return a + b\n'
tree = parser.parse(source)Install tree-sitter-python separately and pin it with tree-sitter. Parser.parse accepts bytes, not a Python str.
Extract a named grammar field read-node-by-field
function = tree.root_node.named_children[0]
name = function.child_by_field_name('name')
if name is not None:
text = source[name.start_byte:name.end_byte].decode('utf8')
print(text)A field lookup is less brittle than a numeric child position, but grammars may omit the field and return None.
Traverse without building child lists walk-with-cursor
cursor = tree.walk()
visited_children = False
while True:
if not visited_children:
visit(cursor.node)
if cursor.goto_first_child():
continue
if cursor.goto_next_sibling():
visited_children = False
continue
if not cursor.goto_parent():
break
visited_children = TrueTreeCursor cannot move above the node where it started. It avoids allocating a Python list for every node's children.
Query every Python function definition capture-function-names
from tree_sitter import Query, QueryCursor
query = Query(
PYTHON,
'(function_definition name: (identifier) @function.name)',
)
captures = QueryCursor(query).captures(tree.root_node)
for node in captures.get('function.name', []):
print(node.text.decode('utf8'))Version 0.26 requires Query(language, source). The older Language.query(source) shortcut has been removed.
Pair each function name with its body keep-related-captures
query = Query(PYTHON, '''
(function_definition
name: (identifier) @name
body: (block) @body)
''')
for _, captures in QueryCursor(query).matches(tree.root_node):
name = captures['name'][0]
body = captures['body'][0]
consume(name, body)matches preserves the relation between captures from one pattern. captures flattens all nodes into lists keyed only by capture name.
Find syntax the parser recovered from detect-recovered-errors
tree = parser.parse(b'def broken(:\n pass\n')
if tree.root_node.has_error:
cursor = tree.walk()
collect_error_and_missing_nodes(cursor)Invalid source usually yields ERROR or MISSING nodes instead of raising from parse. Inspect those nodes before treating the tree as valid code.
Describe an edit before reusing the tree incremental-reparse
tree.edit(
start_byte=4,
old_end_byte=4,
new_end_byte=8,
start_point=(0, 4),
old_end_point=(0, 4),
new_end_point=(0, 8),
)
new_source = source[:4] + b'fast' + source[4:]
new_tree = parser.parse(new_source, tree)
changed = list(tree.changed_ranges(new_tree))All six coordinates must describe the same byte edit. Call changed_ranges on the edited old tree after parsing the new buffer.
Supply source in chunks parse-buffer-callback
def read(byte_offset, point):
chunk = source[byte_offset:byte_offset + 4096]
return chunk or None
tree = parser.parse(read, encoding='utf8')The callback returns bytes and ends with b'' or None. Very small chunks multiply Python callback overhead.
Search only captures fully inside a region limit-captures-to-range
cursor = QueryCursor(query)
cursor.set_containing_byte_range(start_byte, end_byte)
captures = cursor.captures(tree.root_node)This 0.26 method requires the whole capture inside the interval. Cursor range state remains active for later calls on that cursor.
Stop callback parsing at a deadline cancel-long-parse
import time
deadline = time.monotonic() + 0.25
def stop_when_due(_state):
return time.monotonic() >= deadline
tree = parser.parse(
read,
encoding='utf8',
progress_callback=stop_when_due,
)Version 0.26 removed Parser.timeout_micros. The progress callback is available with callback-based input.
Record the loaded language versions inspect-grammar-abi
language = Language(tspython.language())
print(language.abi_version)
print(language.semantic_version)Language.version no longer exists in 0.26. Capture abi_version when diagnosing a binding and grammar mismatch.
Choose a pinned grammar by file suffix map-file-to-language
from pathlib import Path
languages = {
'.py': PYTHON,
'.js': JAVASCRIPT,
}
for path, source in files:
language = languages.get(Path(path).suffix)
if language is None:
continue
parser.language = language
index(parser.parse(source))For incremental editor work, keep one old tree per file and avoid sharing a mutable Parser across concurrent requests.
Alternatives
| Package | Registry | Pick it when |
|---|---|---|
| libcst | PyPI | Choose it for Python codemods that must retain comments, whitespace, and formatting. |
| parso | PyPI | Choose it for error-tolerant Python parsing across several Python grammar versions. |
| Pygments | PyPI | Choose it when lexical tokens for highlighting are enough and a syntax tree would add unused work. |
| tree-sitter-language-pack | PyPI | Choose it when one prebuilt wheel set for many grammars is preferable to managing each grammar package. |
More utils guides
lru-cache · ajv · type-fest · p-limit · find-up · js-yaml · the whole shelf →
How this guide is made: grounded in the library's documentation, release notes, changelog, and issue history, on a fixed rubric — not a hands-on install of every release. The 50 most-downloaded entries are additionally install-verified in clean containers. Corrections: contact the desk.