Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

No longer necessary too generate sets or single atom transit that are bigger than 16bits. #3620

Merged
merged 1 commit into from
Apr 3, 2022
Merged

No longer necessary too generate sets or single atom transit that are bigger than 16bits. #3620

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
33 changes: 9 additions & 24 deletions runtime/Java/src/org/antlr/v4/runtime/atn/CodePointTransitions.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,40 +6,25 @@

package org.antlr.v4.runtime.atn;

import org.antlr.v4.runtime.misc.IntervalSet;

/**
* Utility class to create {@link AtomTransition}, {@link RangeTransition},
* and {@link SetTransition} appropriately based on the range of the input.
*
* To keep the serialized ATN size small, we only inline atom and
* range transitions for Unicode code points <= U+FFFF.
*
* Whenever we encounter a Unicode code point > U+FFFF, we represent that
* as a set transition (even if it is logically an atom or a range).
* Previously, we distinguished between atom and range transitions for
* Unicode code points <= U+FFFF and those above. We used a set
* transition for a Unicode code point > U+FFFF. Now that we can serialize
* 32-bit int/chars in the ATN serialization, this is no longer necessary.
parrt marked this conversation as resolved.
Show resolved Hide resolved
*/
public abstract class CodePointTransitions {
/**
* If {@code codePoint} is <= U+FFFF, returns a new {@link AtomTransition}.
* Otherwise, returns a new {@link SetTransition}.
*/
/** Return new {@link AtomTransition} */
public static Transition createWithCodePoint(ATNState target, int codePoint) {
return createWithCodePointRange(target, codePoint, codePoint);
}

/**
* If {@code codePointFrom} and {@code codePointTo} are both
* <= U+FFFF, returns a new {@link RangeTransition}.
* Otherwise, returns a new {@link SetTransition}.
*/
/** Return new {@link AtomTransition} if range represents one atom else {@link SetTransition}. */
public static Transition createWithCodePointRange(ATNState target, int codePointFrom, int codePointTo) {
if (Character.isSupplementaryCodePoint(codePointFrom) || Character.isSupplementaryCodePoint(codePointTo)) {
return new SetTransition(target, IntervalSet.of(codePointFrom, codePointTo));
}
else {
return codePointFrom == codePointTo
? new AtomTransition(target, codePointFrom)
: new RangeTransition(target, codePointFrom, codePointTo);
}
return codePointFrom == codePointTo
? new AtomTransition(target, codePointFrom)
: new RangeTransition(target, codePointFrom, codePointTo);
}
}
6 changes: 2 additions & 4 deletions tool-testsuite/test/org/antlr/v4/test/tool/TestATNSerialization.java
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -293,10 +293,9 @@ public void testSetUp() throws Exception {
"4:BASIC 0\n" +
"rule 0:1 1\n" +
"mode 0:0\n" +
"0:128169..128169\n" +
"0->1 EPSILON 0,0,0\n" +
"1->3 EPSILON 0,0,0\n" +
"3->4 SET 0,0,0\n" +
"3->4 ATOM 128169,0,0\n" +
"4->2 EPSILON 0,0,0\n" +
"0:0\n";
ATN atn = createATN(lg, true);
Expand All @@ -318,10 +317,9 @@ public void testSetUp() throws Exception {
"4:BASIC 0\n" +
"rule 0:1 1\n" +
"mode 0:0\n" +
"0:'a'..128169\n" +
"0->1 EPSILON 0,0,0\n" +
"1->3 EPSILON 0,0,0\n" +
"3->4 SET 0,0,0\n" +
"3->4 RANGE 97,128169,0\n" +
"4->2 EPSILON 0,0,0\n" +
"0:0\n";
ATN atn = createATN(lg, true);
Expand Down