Cleanup: word and identifier are the same regex - pointless conflict #24
Clément DELALANDE opened 2 days ago

Summary

word and identifier are defined with the same regex:

identifier: () => /[a-zA-Z_][a-zA-Z0-9_]*/,
...
word: () => /[a-zA-Z_][a-zA-Z0-9_]*/,

This is a deliberate-looking ambiguity, and it is listed in conflicts:

conflicts: ($) => [
  [$.source_file, $.typed_variable],
  [$.identifier, $.word],
],

The two are used in different contexts (identifier for $ref variable references, word for names), but since they match identically, the conflict forces tree-sitter to try both productions. This generates spurious ambiguity and can produce surprising parse trees.

Expected

Pick one canonical token and delete the duplicate. If identifier and word must remain semantically distinct (e.g. for highlighting or transpiler consumers), give them different productions or mark one as an alias.

Files

  • grammar.js — identifier, word, conflicts
  • Clément DELALANDE changed title 2 days ago
    Previous Value Current Value
    Cleanup:  and  are the same regex — pointless conflict
    Cleanup: word and identifier are the same regex - pointless conflict
1/1
Type
Improvement
Priority
Minor
Assignees
Not assigned
Issue Votes (0)
Watchers (1)
Reference
gate/tree-sitter-gate#24
Please wait...
Connection lost or session expired, reload to recover
Page is in error, reload to recover