mrkeyoor.comAsk

kyotofin/tax-doc-classifiertested 10 Nov 2026commit 3e95a77

tax-doc-classifier review

Our sandbox install of tax-doc-classifier failed after 11 seconds at commit 3e95a77 because pnpm reported ignored esbuild build scripts.

    Failedinstall
      Dataevaluationupdated 14 Nov 2026
      +2stars / 7d

      tax-doc-classifier is a TypeScript library that identifies US federal tax forms from the text of individual pages using the external Jev API. It returns a form name, page kind and confidence information so your application can accept an answer or send it to a fallback.

      Verdict

      Our sandbox install of tax-doc-classifier failed after 11 seconds at commit 3e95a77 because pnpm reported ignored esbuild build scripts. Consider it for a developer-owned US federal-form routing stage with an explicit fallback and confidence handling. The author reports zero strict errors on 314 filled pages, but its 261-form blank corpus has 38 low-confidence strict errors, so the headline needs that qualification.

      tax-doc-classifier infographic: identifies US federal tax forms from page text, plus our test result and who it suits
      Tested 13 November 2026: npm install github failed after 3.9 s. Before that: 10 Nov: source install failed.

      We ran it

      Lab card: what happened when we ran tax-doc-classifier
      Lab card from our sandbox run
      Screenshot of tax-doc-classifier (github.com/kyotofin/tax-doc-classifier)
      github.com/kyotofin/tax-doc-classifier
      Install✗ · 11s
      Build—
      Repo19 files~698 lines of source · 0.2 MB · 0 CI workflows

      Answers from our run

      Does tax-doc-classifier build from source?

      The dependency install failed, and the project has no separate build step. We cloned commit 3e95a77 into a clean Debian container with 3 CPUs and no project-specific setup.

      Who should not use tax-doc-classifier?

      Teams needing built-in OCR: the README says image-only scans are reported as blank until OCR supplies text.

      What are the alternatives to tax-doc-classifier?

      Donut IRS tax document classifier, Azure Document Intelligence tax model, Google Document AI. Our sandbox install of tax-doc-classifier failed after 11 seconds at commit 3e95a77 because pnpm reported ignored esbuild build scripts.

      Setup1/5In our 2026-11-13 run the README’s “npm install github:org/tax-doc-classifier@latest” failed with “npm error A complete log of this run can be found in: /work/home/.npm…”, and our source install failed.Editor’s note: Installation stopped at pnpm build-script approval in our sandbox; a working configured classifier was not assessed.
      Docs4/5The README has an install section (“Setup”) and usage examples in code; we did not run its quickstart ourselves.Editor’s note: The README was read for scope and integration instructions, but documentation correctness across its execution paths was not assessed after installation stopped.
      Community2/5Last push on 2026-09-29, 520 stars and maintainers answered or closed 1 of the 3 most-discussed issues we sampled.Editor’s note: The supplied issues show one repaired packaging report and two open notices; that sample cannot support a numerical support-quality assessment.
      Maturity1/5No tagged GitHub release and repository created 2026-09-18.Editor’s note: Our run ended during installation and supplies no production reliability assessment or classification workload result.

      How we score

      Who it’s for

      Developers replacing a federal-form classification stage in an existing tax-document pipeline.
      Teams prepared to calibrate confidence thresholds on their own pages and handle rejected answers.
      Integrators who can supply extracted English text and approve sending it to Jev.

      Who it’s NOT for

      Teams needing built-in OCR: the README says image-only scans are reported as blank until OCR supplies text.
      Developers who want to route on a form name alone: MrJev reports unrelated federal identifiers on out-of-scope pages whose gate rejected them.
      Teams seeking Indian tax-document identification from the shipped registry: its criteria describe US federal IRS forms.
      Operators requiring a backend they can run locally without Jev: only the Jev implementation ships.

      Setup reality

      Our November 10, 2026 sandbox install at commit 3e95a77 failed with exit 1 after 11 seconds. The log reports ERR_PNPM_IGNORED_BUILDS for esbuild@0.21.5 and esbuild@0.28.2, and suggests pnpm approve-builds. The recorded sandbox had 3 CPUs, 8 GB RAM and image label lab-node:22. That label does not establish the actual runtime or operating-system version.

      The README requires Node 20 or newer, pnpm and a TypeSafe API key for the external Jev service. PDF helpers also require poppler, providing pdftotext and pdfinfo. Calling classifyPage with text lines avoids those PDF binaries.

      Issue #1 records two older packaging defects repaired in 3e95a77, separately from our script-approval error. The scan found 19 files, approximately 698 source lines and a 0.2 MB checkout, with no CI workflow files, Dockerfile or tests directory. A later harness attempt retained an organization placeholder and failed before classification.

      Our sandbox install of tax-doc-classifier failed after 11 seconds at commit 3e95a77, with pnpm refusing ignored esbuild build scripts. That leaves a narrower recommendation than the headline accuracy claims suggest: consider this library if you already operate a US federal-tax intake pipeline and can own its confidence handling. Choose another starting point if you need image ingestion or a finished filing workflow.

      The useful design is a classifier that can decline to give you an actionable answer. A returned form name alone does not tell you whether the library accepted that answer. That distinction explains both the published accuracy figures and the most consequential integration mistake reported by another reviewer.

      What people use tax-doc-classifier for

      The documented uses center on replacing a federal-form classification stage and inspecting its decisions, rather than preparing a complete tax return.

      • Nakshatra Saxena says his team replaced its existing LLM classifier with Jev for its tax-document corpus. He reports about $0.001 per page, 34 times cheaper and six times faster than the previous setup. These are the author's results, and the comparison concerns that particular pipeline. Author's announcement.
      • The project author evaluated filled TaxCalcBench pages and blank IRS forms. The filled corpus has 314 pages covering 15 forms and zero strict errors. The blank corpus has 753 pages covering 261 forms and 38 strict errors. This is evidence about two distinct corpora. Published evaluation.
      • MrJev's reviewer sent downloaded IRS pages through real Jev calls in Docker. All six listed federal-form pages cleared the confidence gate, including Schedule J's second-call path. That report gives a developer a concrete example of the hierarchy working. MrJev's hands-on account.
      • The same reviewer deliberately supplied a California return and ordinary prose. Both received appropriate broad page kinds, alongside unrelated federal form names that the confidence gate rejected. That exercise exposed why callers must handle rejection explicitly. Out-of-scope examples.

      The README describes a TypeScript library that sends page text to the external Jev API. Its generated registry describes 261 IRS forms. You supply extracted text directly, or use its PDF helpers, then consume the returned form identifier, page kind and probabilities. Those are useful inputs to a routing system you control.

      GitHub records the last push as September 29, 2026. The tracker contains a closed packaging repair and the open accuracy and security notices discussed below. That mix shows why a last-push date alone cannot settle a maintenance judgment: read what was fixed and what still needs a response.

      What happened when we ran it

      Our November 10, 2026 sandbox install failed with exit 1 after 11 seconds at commit 3e95a77.

      Recorded observation Result
      Sandbox configuration 3 CPUs, 8 GB RAM, image label lab-node:22
      Checkout 19 files, approximately 698 source lines, 0.2 MB
      Dependency installation Failed, exit 1, 11 seconds
      Error in the log ERR_PNPM_IGNORED_BUILDS
      Build scripts named by pnpm esbuild@0.21.5, esbuild@0.28.2
      Repository scan 0 CI workflow files, no Dockerfile, no tests directory

      The log says "Ignored build scripts" and suggests pnpm approve-builds to choose which dependencies may run scripts. It also says the lockfile passes supply-chain policies. Those messages describe different checks: the positive lockfile message does not erase the installation failure. Review the named packages before approving their scripts, then retry setup in your own environment. This log alone does not explain why both esbuild versions were present.

      A separate November 13 harness record also stopped at installation. Its command retained github:<org>/tax-doc-classifier@latest, and the log attempted ls-remote against ssh://git@github.com/null/latest.git. The record labels the failure install-failed and points to an npm debug log. That attempt did not reach a classifier task, and its unresolved repository placeholder makes it poor evidence about the library's packaging. The run record names 3e95a77f763c; teardown is recorded as successful.

      The setup durations above come solely from our November 10 measurement block. Neither the image label nor the failed command establishes an actual Node or operating-system version. The missing tests directory is a scan observation, not a count of tests or proof that no test files exist elsewhere.

      Where people get stuck: three setup failures need different responses

      The pnpm script-approval error in our run is separate from the two packaging defects closed in issue #1.

      Symptom Evidence and status Response
      ERR_PNPM_IGNORED_BUILDS Our November 10 log names two esbuild versions Inspect the scripts and use pnpm's approval mechanism deliberately
      ERR_PACKAGE_PATH_NOT_EXPORTED for criteria Issue #1 reports the missing data-file export at 6afcf70, fixed in 3e95a77 Select a revision containing the export repair
      ERR_MODULE_NOT_FOUND after a GitHub dependency install Issue #1 reports absent build output, fixed by a prepare script in 3e95a77 Select the repaired revision rather than assuming the old defect persists
      Git lookup targets null/latest.git Our later harness retained the README's organization placeholder Supply the actual repository identity before treating the attempt as a product check

      In issue #1, MrJev reported both original defects, and maintainer Nedwize replied that both were reproduced and repaired. The maintainer reports checking the repaired commit in an empty project. MrJev's review also records a successful post-repair check. Those are attributed repair reports, separate from our failed sandbox setup.

      The README requires Node 20 or newer and pnpm. PDF helpers require poppler's pdftotext and pdfinfo on the path. Calling classifyPage with lines of text avoids those PDF binaries. The shipped Jev backend still needs a TypeSafe API key, so this small repository has an external service dependency even when PDF extraction happens on your machine.

      A plausible form name can accompany a rejected answer

      Check gated before consuming result.form, because an out-of-scope page can still receive a federal form name.

      MrJev's California Form 540 example makes the problem easy to picture. The broad kind was state_tax_form, and not_in_this_list took essentially all the form probability. Yet result.form was form-982, with confidence 0.000 and gated: false. An application that stores only the name could put a state return in a federal-form folder despite the classifier's own rejection. MrJev's California example.

      Ordinary newsletter prose produced the same shape: letter_or_other was the kind, while form-8082 appeared in the form field. The reviewer explains that the code selects a federal-form winner while excluding not_in_this_list. Keep rejected results on a separate path, and preserve the probabilities if a reviewer needs to understand the decision. A readable identifier is not permission to route a document automatically.

      The same report says an instruction-booklet page was labeled as a form page. The form confidence gate does not also gate the page kind. If your workflow treats instructions differently, use kindConfidence as a separate decision input. A passed form gate answers a narrower question than "is every part of this result safe to act on?"

      At volume, count rejections and second calls

      Blank pages require no Jev call, while five corporate or foreign form families can require a second call to identify a schedule.

      The README names those families as 5471, 8865, 8933, 1118 and 5713. It uses a parent-first question where the page announces the parent more clearly than the schedule, then asks a smaller follow-up question. formConfidence takes the minimum confidence across the steps. A budget based on exactly one request for every page will miss that behavior. Classification flow.

      More important for an intake team is the rejected-page workload. The author's blank-form evaluation includes 38 pages whose labels were correct but whose confidence fell below the strict threshold. The README describes instruction pages, deep corporate-form pages and schedule-versus-parent ambiguity among them. Your fallback still has work to do when the classifier chooses the right label without enough confidence to release it.

      For an evaluation, record accepted correct labels, accepted wrong labels and rejected pages separately. Then price the fallback you actually use, whether another classifier or a human queue. This is our recommended decision method, not an additional result from the repository. It connects the explicit confidence gate to the operational question: how much work reaches your existing process after classification?

      Security: page text goes to a hosted decision service

      The shipped backend sends page text to Jev, and an open notice warns about attacker-influenced decisions from untrusted context.

      The README reads the TypeSafe key from TYPESAFE_API_KEY and says the library does not write it to disk or log it. Keep the distinction between protecting that key and approving a service to receive tax-document text. Before sending client material, have the person responsible for that data approve the provider and handling policy. Open-source classification code does not remove the external API from that decision. README requirements.

      Issue #4 is a notice from researchers identifying themselves with Beijing University of Posts and Telecommunications. They describe a risk in which manipulated context produces structurally valid, attacker-influenced judgments. Their recommendations include independent validation before consequential actions. The supplied issue remains open and explicitly says the notice does not imply that this repository is necessarily exploitable. Treat that as a risk notice with that stated boundary, rather than a demonstrated vulnerability or a completed mitigation.

      tax-doc-classifier vs Claude Sonnet, Donut and Azure Document Intelligence

      The author's Sonnet comparison measures a specific previous PDF-page pipeline whose prompts did not share a cache across pages.

      Author-reported measure Previous Sonnet classifier Jev classifier
      Cost per page $0.039 $0.00115
      Warm latency per page Approximately 3.3 seconds Approximately 0.5 seconds
      Form identities available 30 in its phrase table 261 in the registry

      Those are classification results reported by the author, not our install timings or an independent benchmark. The README says the comparison used the same bench pages, machine and hour. Its no-shared-cache detail belongs next to the price claim because it describes the baseline being replaced. Use that comparison to ask whether your current pipeline has the same cost structure. Author's comparison.

      For a different input path, the hsarfraz Donut classifier model card supplies image-based inference code and describes a model fine-tuned for IRS documents. Its narrower label set and image processing make it a different candidate from a text registry read by Jev. Prefer evaluating it when running a classifier over document images is central to your requirements. Its model card is a capability description, not evidence that it beats Jev on your forms.

      Azure Document Intelligence's tax model documents OCR and field extraction for selected English tax formats, including scanned documents and phone-captured images. Evaluate that service when extracting values from supported forms is part of the job. For broader workflow context, Google's Document AI tax-pipeline example connects classification to specialized parsers. It is a dated example, not a current coverage list or a measured competitor result.

      What the other reviews get wrong

      Calibrated Decisions' claim of 100% strict accuracy across 261 IRS forms conflicts with the README's blank-corpus table.

      The Calibrated Decisions page repeats that headline, and the supplied X roundup records repeat the same conflation. The README reports zero wrong labels on the blank corpus, but 38 strict errors because low confidence also counts. Its zero-strict-error result belongs to the separate filled corpus covering 15 forms. Issue #3 requests clarification and is open in the supplied record. Keep coverage, wrong labels and strict errors separate when quoting these results.

      "One request per page" is also shorthand that needs qualification. The documented blank-page path makes no call, and the corporate-family path can make two. That changes both an implementation diagram and a request budget. The README itself contains the detail needed to correct its opening summary.

      MrJev's closing advice to fix packaging or vendor the source is stale within its own account. Earlier on the same page, it reports that 3e95a77 fixed both defects and that the repaired installation was checked. The closed issue corroborates the repair report. Read the resolution before turning an old error into current setup advice.

      Questions people ask

      The published evaluation distinguishes wrong labels from low-confidence answers, which resolves several questions raised under the author's announcement.

      How does tax-doc-classifier's accuracy compare with frontier models?

      The README reports cost and warm latency against the author's previous Claude Sonnet pipeline, alongside separate Jev corpus results. It supplies no broad frontier-model accuracy ranking. The useful comparison is the documented baseline and corpus, rather than a claim about every current model. Author's evaluation.

      Does 100% classification mean exact top-1 accuracy on unseen documents, or does it include confidence thresholds and human review?

      The README defines a strict error as a wrong answer or confidence below 0.95. Its filled corpus has zero strict errors, while the blank corpus has 38. Those tables describe classifier scoring, not a human-review-adjusted success rate. Scoring definition.

      Does the quoted $0.001 per page include OCR and preprocessing?

      The README describes text classification and separately requires OCR for scanned pages. Treat the quoted price as the author's classifier figure, rather than an all-in scanned-document processing price. Input limits and comparison.

      How does tax-doc-classifier handle new document formats?

      The registry builder reads IRS accepted forms and PDFs, and the README instructs users to regenerate it when the IRS publishes revisions. Adding support therefore involves the criteria and the evaluation corpus. A new layout is not automatically covered by an unchanged accuracy headline. Registry generation.

      Would this approach work for Indian GST invoices, GSTR filings, TDS certificates, Form 16/26AS, AIS/TIS, challans and tax notices?

      The shipped registry identifies US federal IRS forms, and the README specifies English-only input. Those Indian document types need their own criteria and evaluation before this approach could answer the question raised by a reader of the author's announcement. Project scope, reader's question.

      Were evaluation documents split by page, issuer or form version?

      The published tables identify pages, forms and corpus sources. They do not describe an issuer- or version-held-out split. A reader explicitly requested that distinction under the announcement. Corpus coverage alone cannot answer it. Evaluation description, split question.

      Was an entire unseen template family held out during evaluation?

      No supplied evaluation description identifies an entirely held-out template family. The announcement's commenter asks for exactly that safeguard against familiar layouts making the result look stronger. It is a useful requirement for your own evaluation set. Template-family question.

      How can a developer obtain access to Jev?

      The README directs developers to TypeSafe for an API key, which the shipped backend reads through TYPESAFE_API_KEY. Obtain service access before designing an integration around this backend. Access instructions.

      What metric or version supports the claim of 100% strict accuracy across 261 forms?

      Issue #3 asks the maintainer to identify that metric or clarify the description. The supplied record contains no resolution. The defensible figures are the two README corpus rows, each with its own coverage and strict-error count. Accuracy clarification request.

      What we have not tested yet

      Our failed setup provides no classification-accuracy, inference-latency or operating-cost result for tax-doc-classifier.

      We have not reproduced the author's corpora or cost comparison, and MrJev's six accepted federal pages remain a small independent sample. The records leave issuer, form-version and template-family generalization unresolved. They also lack an aggregate evaluation for state forms, broker or bank statements, and letters. Naming those page kinds in an API does not measure their reliability.

      Scans without a text layer are reported as blank until OCR supplies text. We have not measured the combined OCR and classification path on scans, degraded pages or phone photos. Nor have we assessed multi-page grouping, missing-page detection, tax treatment of transactions, or the replacement backend interface with an implementation other than Jev. The shipped backend is the only documented implementation.

      The open decision-hijacking notice supplies neither a repository-specific exploit demonstration nor a mitigation test or maintainer resolution. Provider data handling and production reliability also sit outside our run. For a deployment decision, the next useful evidence is your own labeled intake set with the rejected pages kept in the results. Measure what happens to those pages before replacing the fallback you already trust.

      Alternatives

      ProjectWhat it isPick it when
      Donut IRS tax document classifierAn image-based IRS classifier with inference code in its Hugging Face model card.Evaluate it when classifying page images with a model you run is central to the job, rather than sending extracted text to Jev.
      Azure Document Intelligence tax modelA hosted OCR and field-extraction service for selected English tax formats.Evaluate it when scans or phone images and extracting values from supported forms are part of the same requirement.
      Google Document AIA hosted document-processing service whose published tax example connects classification with specialized parsers.Explore it when classification must feed field extraction in a managed pipeline; the linked tax example is historical.

      Sources

      1. raw.githubusercontent.com/kyotofin/tax-doc-classifier/main/README.md
      2. tax-doc-classifier Review: 261 IRS Forms, No Model | MrJev
      3. Both README install paths fail: data/criteria.json isn't exported, and a GitHub install has no dist/
      4. Clarify the repository description strict-accuracy claim
      5. Security Notice: Potential Decision Hijacking Risk in Jev-Based Applications
      6. hsarfraz/donut-irs-tax-docs-classifier · Hugging Face
      7. Document Intelligence US tax documents data extraction - Foundry Tools
      8. Automating income taxes with Document AI | Google Cloud Blog

      More data reviews