Report EmptyStackException as error

[zwille]

I have written a C-like language grammar using '{' '}' for nesting blocks.
The lexer pushes the DEFAULT_MODE on '{' and pops it on '}'
I'm using this "trick" for parsing embedded expressions in strings (island language),
like so: "xxx{a+b}xx".

Now when the input is not nested well, a EmptyStackException will fire during parsing.
Consider the following pseudo token stream:

{ -> pushMode DEFAULT_MODE
    ...
    { -> pushMode DEFAULT_MODE
    ...
    } -> popMode
    } -> popMode // too many '}'s
} -> popMode -> EmptyStackException

Here is the call stack:

        at org.antlr.v4.runtime.Lexer.popMode(Lexer.java:192)
	at org.antlr.v4.runtime.atn.LexerPopModeAction.execute(LexerPopModeAction.java:58)
	at org.antlr.v4.runtime.atn.LexerActionExecutor.execute(LexerActionExecutor.java:168)
	at org.antlr.v4.runtime.atn.LexerATNSimulator.accept(LexerATNSimulator.java:369)
	at org.antlr.v4.runtime.atn.LexerATNSimulator.failOrAccept(LexerATNSimulator.java:302)
	at org.antlr.v4.runtime.atn.LexerATNSimulator.execATN(LexerATNSimulator.java:233)
	at org.antlr.v4.runtime.atn.LexerATNSimulator.match(LexerATNSimulator.java:117)
	at org.antlr.v4.runtime.Lexer.nextToken(Lexer.java:141)
	at org.antlr.v4.runtime.BufferedTokenStream.fetch(BufferedTokenStream.java:169)
	at org.antlr.v4.runtime.BufferedTokenStream.fill(BufferedTokenStream.java:485)
	at org.antlr.v4.gui.TestRig.process(TestRig.java:174)
	at org.antlr.v4.gui.TestRig.process(TestRig.java:166)
	at org.antlr.v4.gui.TestRig.main(TestRig.java:119)

An error message with the token position would be helpful, instead of the exception.
Maybe the lexer should recover after the exception (in DEFAULT_MODE).

[ericvergnaud]

This looks more like a parser activity than a lexer one, and using a parser you can customise error handling to your articular needs.

[KvanTTT]

I think adding an error message with correct recovery to the outermost mode (DEFAULT) is more expected and pleasant behavior.

[KvanTTT]

pushMode and popMode commands are native grammar syntax, they are not custom actions or predicates. That's why their using should not cause any runtime exceptions in any combination.

[parrt]

ok, i'll hang until you have a proposed fix.