mrkeyoor.com_
Tue 08 Sept 14:59 UTC
Open Source6 min read

Microsoft's tgrep Adds 1,353 Stars for Indexed Code Search

Microsoft's tgrep is drawing developers with indexed regex search for huge repositories, though its biggest speed claims come with real storage and workflow costs.

Microsoft's tgrep repository picked up 1,353 GitHub stars in a daily snapshot captured on September 8, the largest one-day repository gain among the projects reviewed for this report. The rush is attached to a specific developer pain: coding tools and agents may search the same large checkout dozens of times, while a conventional recursive search starts another scan each time. tgrep pays for an index up front, keeps it current through a local server, and reuses it across queries.

That makes the timing more interesting than a routine command-line release. Microsoft says tgrep now powers fast grep searches in GitHub Copilot CLI, so the project is already part of an agent workflow rather than a detached performance experiment. Its public repository was created on April 3, is written in Rust, and carries the MIT license. The latest tagged build, version 1.0.4, arrived on September 7 with Homebrew installation support and a correction that lets .svg and .raw files pass the extension filter.

A star burst measures attention, not installed seats or retained users. Still, 1,353 stars in a day is enough to show that repeated code search has become its own optimization target. The useful question is where tgrep saves time, and what a team has to maintain in exchange.

The index changes the unit of work

ripgrep is a fast recursive scanner. It walks eligible files for each query, respects ignore rules, skips hidden and binary files by default, and uses optimized regex machinery to search the bytes it reads. That model is simple and works especially well on warm Linux filesystems or repositories small enough to scan before a person notices. There is no persistent search database to build or refresh.

tgrep moves part of that work ahead of the query. During indexing, it extracts every overlapping three-byte sequence, or trigram, from admitted text files and writes posting lists that map each trigram to the files containing it. A search for a regex with required literal fragments can intersect those lists, produce a much smaller candidate set, and run the full regex only against those files. The project's explanation of its query path says alternations become unions of candidate lists. Patterns that cannot be narrowed safely fall back to scanning all eligible files.

The basic workflow is short enough to try without changing editor configuration:

tgrep index .
tgrep serve .
tgrep "fn main" .

The client automatically connects to the server associated with the repository. The server combines a memory-mapped disk index with an in-memory overlay for changed files, while a filesystem watcher feeds updates into that overlay. It also reconciles the tree on a timer to catch events lost during a branch switch, build, queue overflow, or unreliable network filesystem. According to the architecture notes, clients speak newline-delimited JSON-RPC over TCP, and multiple clients can search the same index.

This design fits an interactive shell, but the agent connection is sharper. A coding agent routinely asks variations of the same question: where a symbol is defined, which tests mention a fixture, which call sites use an older API. Re-scanning 100,000 files for every step spends I/O on files that could never match. An index lets that repeated cost collapse into candidate lookup and verification. GitHub's decision to put tgrep inside Copilot CLI supplies the first concrete product use for that idea.

Microsoft's 52x claim needs its denominator

The top number in Microsoft's README is 51.9 times faster than ripgrep on Mozilla's 387,841-file Gecko repository on macOS. In that run, ripgrep averaged 33.4 seconds per query and tgrep 643 milliseconds. The full benchmark table reports geometric-mean gains across six repositories of 14.6 times on Windows, 8.61 times on macOS, and 2.82 times on Linux. tgrep won 17 of the 18 operating-system and repository combinations.

Those are project-run measurements, and the setup matters. The trigram index was already built before search timing began. Every tgrep measurement did include a fresh client process and its TCP round trip, but both tools ran on shared GitHub-hosted machines, where CPU, storage, and page-cache behavior vary between jobs. The repository warns readers to compare tools within a row instead of treating the absolute milliseconds as controlled-machine results. No query hit the benchmark's 120-second timeout.

The least flattering row is useful. On the 31,300-file Kubernetes repository under Linux, tgrep averaged 101.8 milliseconds while ripgrep took 94.4 milliseconds, leaving tgrep at 0.93 times ripgrep's speed. The project attributes the loss to a warm page cache and the overhead of serializing and delivering matches through the client-server path. Common patterns that return tens of thousands of results can spend more time moving output than the index saved by pruning files. ripgrep's own performance notes make the same general point: high match counts can erase much of the difference between search engines because result handling dominates.

Repository size changes the result too. Microsoft's Linux figures range from 1.29 times on the 15,833-file Go tree to 9.38 times on the 95,831-file Linux kernel tree. Windows shows larger and steadier gains in the published matrix. That is plausible because avoiding file opens matters more where per-file overhead is higher, but independent runs on developer workstations would tell us more than another shared-runner sweep.

Fast queries leave an index bill

Chromium gives the clearest view of the trade. Microsoft's benchmark indexed 504,351 files into roughly 2.58GB. Building that index took about 52 seconds on Linux, 73 seconds on Windows, and 248 seconds on macOS. The measured peak memory stayed below 470MiB on all three platforms after the project changed its default builder to spill sorted segments to disk. Those costs can make sense for a checkout searched all day; they are hard to justify for a small repository or a one-off query.

The shipped tool also differs from ripgrep in ways that can affect results. tgrep skips files larger than 64MiB during normal indexing and directory searches, while ripgrep has no default size cap. Users can pass --no-max-filesize, and an explicitly named large file is searched unless the user supplied a limit. Encoding flags, hidden-file searches, binary searches, and options that disable ignore rules bypass the index and walk the filesystem instead. Compressed-file search through -z is unsupported and exits with code 2. These exceptions are documented in the CLI compatibility section, but a drop-in replacement script still deserves testing against its actual corpus.

Freshness is the other bill. A scanner sees the files that exist when it runs. tgrep depends on watcher events, an overlay, periodic reconciliation, and matching file-selection flags between index and serve. The documentation warns that serving an index with a different --max-filesize, --exclude, or ignore configuration can remove entries the server no longer sees. The current server source binds its TCP listener to loopback, which keeps the service local, yet teams still need a clean way to start it, stop it, and discard stale index directories.

Version 1.0.4 is evidence that file parity remains active work. Its .svg and .raw fix followed version 1.0.3 changes for watcher reconciliation, duplicate events, and indexed filename listing. That is normal for a young filesystem tool, though it argues for running tgrep beside ripgrep before making it the only search path in build scripts or editor automation. The repository includes a --no-index escape hatch and ripgrep-compatible JSON output, which should make comparison easier.

What to watch after the star spike

The next useful evidence will come from independent benchmarks on repositories people search every day, with cold-start time and index storage included. It will also matter whether integrations beyond Copilot CLI keep a server alive long enough to amortize the build. If users report consistent gains on selective queries without stale-result bugs after large branch changes, tgrep has a credible place beside ripgrep in big checkouts. If most of the advantage remains confined to prebuilt indexes on unusually large trees, its audience will be narrower. The star count has already answered whether developers are curious; the issue tracker and repeatable outside tests will answer whether they stay.

We reviewed this

  1. checkout — our honest review
  2. kubernetes — our honest review
  3. fresh — our honest review

Sources

  1. Microsoft tgrep repository
  2. tgrep README and architecture
  3. tgrep benchmark methodology and results
  4. tgrep v1.0.4 release
  5. ripgrep repository and performance notes
  6. tgrep server source