Error when preprocessing verilog header with dos line endings

[nxpMartin]

It looks like, there is a problem with the Verilog preprocessor for files that uses dos line endings.

Verilog multiple line macro definition works well in a header file with Unix line endings and causes an error like "SyntaxError:mismatched input 8'b0010_0100 expecting ...."

See params.v for details.

You can also clone (git clone https://github.com/nxpMartin/hdl_analyzer.git -b verilog_preprocessor) and run pytest in the main directory to see it.

[Thomasb81]

Ieee1800.1 2017 does not specify carriage return (CR) as part in section A.9.4:
white_space ::= space | tab | newline | eof

Similarly the standard specify that source code should be ascii text... but some tool vendor support unicode at least in comments ...

[Thomasb81]

The issue is probably in lexer of the pre-processor grammar : https://github.com/Nic30/hdlConvertor/blob/master/grammars/verilogPreprocLexer.g4

[nxpMartin]

I went into this error when I try to parse the Verilog description of the processor. The description is normally synthesizable by some commercial tools.
So, I believe that it is a good idea to improve/fix it.

[Nic30]

@nxpMartin It is possible to hotfix this by normalization of newlines on input, but lets add it to grammar directly as it should not break anything and it should greatly improve user experience.
Also do not be afraid to send pull request with suggested change, it greatly shortens the time required to fix issue.

[Nic30]

Closing this as it is fixed in mesonbuild branch and will be merged to master soon.