chevrotain-allstar
chevrotain-allstar is an alternative lookahead strategy for Chevrotain parsers. Normal Chevrotain prediction examines a configured maximum number of tokens; this plugin builds an augmented transition network and adaptive DFA so a decision can keep looking until alternatives separate, following the ALL(*) approach introduced by ANTLR4. You pass LLStarLookaheadStrategy into the parser constructor and keep writing ordinary Chevrotain rules. It is useful for programming-language and DSL grammars with long or recursive shared prefixes, not a parser generator by itself.
Use chevrotain-allstar when an otherwise sound Chevrotain grammar genuinely needs adaptive unbounded lookahead. Do not install it to avoid understanding an ambiguity: it moves some validation to runtime, adds performance uncertainty, and resolves a real tie by alternative order.
Use it if
- Your Chevrotain 13 grammar has legitimate alternatives that cannot be distinguished by a practical fixed maxLookahead
- You are building Langium-style language tooling and need adaptive prediction for recursive or long shared prefixes
- You need an incomplete-input mode that can make a best-effort alternative choice at EOF for editor features
- You want to keep Chevrotain's lexer, rule DSL, error recovery, and parser actions while replacing only lookahead
- Your grammar can be left-factored or resolved with a small maxLookahead: Chevrotain's built-in strategy has fewer moving parts, while this package builds ATN and DFA structures and may inspect an unbounded token prefix
- You are not on Chevrotain 13: version 0.5.0 declares chevrotain ^13.0.0 as a peer dependency, so older parser projects need their own major upgrade first
- You expect ambiguous grammars to fail during performSelfAnalysis: the source returns empty validation results for ambiguous and empty OR alternatives, reports ambiguity at runtime, and selects the lowest-numbered viable alternative
- You need CommonJS output: the package is type: module and its exports map provides one ESM JavaScript entry plus declarations, with no require condition
- You want a well-documented standalone parser toolkit: the README contains one constructor example and no API section for logging or incomplete mode; Chevrotain grammar design and lexer setup remain prerequisites
Setup reality
Install both chevrotain-allstar and a compatible chevrotain ^13.0.0; Chevrotain is a peer rather than bundled. The package is ESM-only, includes TypeScript declarations, and has lodash-es as its one direct runtime dependency. There is no native build, credential, file-based configuration, or generated parser step. The required code change is passing new LLStarLookaheadStrategy() as lookaheadStrategy to super, before defining rules and calling performSelfAnalysis. Do not omit performSelfAnalysis; the strategy receives the grammar there and builds its ATN and DFA caches. Use a fresh strategy for each different parser grammar because initialize replaces its rule-derived state. Easy LL(1) decisions still take a fast single-token path, while conflicting decisions use adaptive prediction and may read far ahead, so benchmark representative worst-case inputs and guard input size in services. The plugin deliberately suppresses Chevrotain's static ambiguous-alternative and empty-alternative validation errors. A true runtime ambiguity calls logging, which defaults to console.log, then chooses the smallest alternative number; provide a logger that goes to tests or diagnostics rather than letting production parsers print. Set incomplete only for editor or completion input where EOF commonly means unfinished text, because it permits a best guess instead of insisting on a full match. Semantic GATE predicates participate in separate DFA caches, while lexer errors, parser errors, CST or embedded actions, and recoveryEnabled are still ordinary Chevrotain responsibilities.
Patterns
Enable ALL(*) lookahead in a parserconfigure-allstar-parser
import { EmbeddedActionsParser } from 'chevrotain';
import { LLStarLookaheadStrategy } from 'chevrotain-allstar';
class Parser extends EmbeddedActionsParser {
constructor(tokens) {
super(tokens, {
lookaheadStrategy: new LLStarLookaheadStrategy(),
});
// Define RULE calls here.
this.performSelfAnalysis();
}
}The strategy must be passed to super before rules are analyzed, and performSelfAnalysis is still mandatory after all rule definitions.
Run the Chevrotain lexer and parser togethertokenize-and-parse
const lexed = lexer.tokenize(source);
if (lexed.errors.length) throw new Error(lexed.errors[0].message);
const parser = new Parser(allTokens);
parser.input = lexed.tokens;
const value = parser.program();
if (parser.errors.length) throw new Error(parser.errors[0].message);The plugin changes prediction only. Lexing, assigning parser.input, invoking the entry rule, and checking both error arrays remain application work.
Keep long shared-prefix alternatives readableparse-shared-prefixes
this.RULE('statement', () => {
this.OR([
{ ALT: () => {
this.CONSUME(Identifier);
this.CONSUME(Dot);
this.CONSUME1(Identifier);
this.SUBRULE(this.assignmentTail);
} },
{ ALT: () => {
this.CONSUME2(Identifier);
this.CONSUME1(Dot);
this.CONSUME3(Identifier);
this.SUBRULE(this.callTail);
} },
]);
});ALL(*) can look beyond the shared prefix, but Chevrotain occurrence indexes such as CONSUME1 and CONSUME2 are still required for repeated DSL calls in one top-level rule.
Send ambiguity reports to application diagnosticscapture-ambiguity-diagnostics
const ambiguityMessages = [];
const strategy = new LLStarLookaheadStrategy({
logging(message) {
ambiguityMessages.push(message);
},
});
super(tokens, { lookaheadStrategy: strategy });The default logger calls console.log. A runtime ambiguity is reported through this callback and then resolved by choosing the lowest-numbered viable alternative.
Allow best guesses for editor inputparse-incomplete-input
super(tokens, {
lookaheadStrategy: new LLStarLookaheadStrategy({
incomplete: true,
logging: reportGrammarAmbiguity,
}),
recoveryEnabled: true,
});incomplete mode makes an educated alternative choice when lookahead reaches EOF mid-decision. Reserve it for editors and completion, not strict file validation.
Combine adaptive lookahead with semantic gatesgate-alternatives
this.OR([
{
GATE: () => this.languageVersion >= 2,
ALT: () => this.SUBRULE(this.modernDeclaration),
},
{ ALT: () => this.SUBRULE(this.legacyDeclaration) },
]);The strategy evaluates GATE predicates and maintains DFA caches by predicate set. Gates should be fast and deterministic for the parser state.
Keep Chevrotain error recovery enabledcombine-error-recovery
super(tokens, {
lookaheadStrategy: new LLStarLookaheadStrategy(),
recoveryEnabled: true,
});ALL(*) chooses grammar paths; it does not replace token insertion, deletion, resynchronization, or parser.errors handling provided by Chevrotain recovery.
Use the strategy with a CST parserbuild-cst-parser
import { CstParser } from 'chevrotain';
class QueryCstParser extends CstParser {
constructor(tokens) {
super(tokens, {
lookaheadStrategy: new LLStarLookaheadStrategy(),
});
this.RULE('expression', () => {
this.CONSUME(Identifier);
});
this.RULE('query', () => {
this.SUBRULE(this.expression);
});
this.performSelfAnalysis();
}
}The lookahead strategy is independent of CST construction. Visitors and CST typing are still configured through normal Chevrotain APIs.
Use dynamic token vocabulariessupport-dynamic-tokens
super(tokens, {
dynamicTokensEnabled: true,
lookaheadStrategy: new LLStarLookaheadStrategy(),
});The implementation avoids its static LL(1) shortcut when dynamicTokensEnabled is true and routes decisions through adaptive prediction. Benchmark this mode on real input.
Create one strategy per parser grammarisolate-parser-grammars
const allstarOptions = () => ({
lookaheadStrategy: new LLStarLookaheadStrategy(),
});
class ExpressionParser extends EmbeddedActionsParser {
constructor(tokens) {
super(tokens, allstarOptions());
// expression rules, then performSelfAnalysis()
}
}
class TemplateParser extends EmbeddedActionsParser {
constructor(tokens) {
super(tokens, allstarOptions());
// template rules, then performSelfAnalysis()
}
}initialize builds rule-specific ATN and DFA state on the strategy instance. Do not share one instance across different grammar definitions.
Turn ambiguity logs into test failurestest-no-runtime-ambiguity
const ambiguities = [];
class CorpusParser extends EmbeddedActionsParser {
constructor(tokens) {
super(tokens, {
lookaheadStrategy: new LLStarLookaheadStrategy({
logging: (message) => ambiguities.push(message),
}),
});
// corpus grammar rules, then performSelfAnalysis()
}
}
parseCorpus(new CorpusParser(allTokens), fixtures);
expect(ambiguities).toEqual([]);The strategy disables Chevrotain's normal static ambiguous-alternative errors, so representative corpus tests are the practical way to catch runtime ties.
Put the intended fallback alternative firstprefer-ordered-fallback
this.OR([
{ ALT: () => this.SUBRULE(this.preferredInterpretation) },
{ ALT: () => this.SUBRULE(this.compatibilityInterpretation) },
]);If the full token path remains ambiguous, implementation source chooses the minimum alternative index after logging. Order is therefore observable behavior, not cosmetic.
Alternatives
| Package | Registry | Pick it when |
|---|---|---|
| chevrotain | npm | Your grammar works with bounded lookahead and you want the standard validator to reject ambiguities during self-analysis |
| antlr4 | npm | You prefer grammar files and generated parsers built around the original ANTLR4 ALL(*) toolchain |
| peggy | npm | A PEG grammar with ordered choice is a clearer model than Chevrotain's token DSL and adaptive ambiguity handling |