-
Notifications
You must be signed in to change notification settings - Fork 886
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support octal and hex within regex character class pattern #11112
Support octal and hex within regex character class pattern #11112
Conversation
yy = handle_hex(); | ||
break; | ||
} | ||
case 'w': { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Codecov Report
@@ Coverage Diff @@
## branch-22.08 #11112 +/- ##
===============================================
Coverage ? 86.30%
===============================================
Files ? 144
Lines ? 22701
Branches ? 0
===============================================
Hits ? 19592
Misses ? 3109
Partials ? 0 Continue to review full report at Codecov.
|
rerun tests |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
It took me a while to convince myself of the correctness of this code. I came in with an incorrect assumption that octals in regular expressions always begin with \0
so I had to work through that and re-review. I have a few comments/notes that might improve the PR.
case '0' ... '7': { | ||
chr = handle_octal(chr); |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
My prior belief was that a prefix of \0
was required to denote octals in a string, just like \x
is needed to denote hexadecimals. However, I was wrong. Python, C++, and Java do not require a \0
prefix, so this implementation appears to be aligned with all the octal conventions we care about. TIL.
See also:
- https://en.cppreference.com/w/cpp/language/escape
- https://docs.python.org/3/library/re.html?highlight=octal#regular-expression-syntax Note that there's some rules about group references:
\number
Matches the contents of the group of the same number. Groups are numbered starting from 1. For example,(.+) \1
matches'the the'
or'55 55'
, but not'thethe'
(note the space after the group). This special sequence can only be used to match one of the first 99 groups. If the first digit of number is 0, or number is 3 octal digits long, it will not be interpreted as a group match, but as the character with octal value number. Inside the'['
and']'
of a character class, all numeric escapes are treated as characters.
Octal escapes are included in a limited form. If the first digit is a 0, or if there are three octal digits, it is considered an octal escape. Otherwise, it is a group reference. As for string literals, octal escapes are always at most three digits in length.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Of note, Python supports three-digit octals that exceed one byte. \777
maps to character 511 'ǿ'
. C++ does not allow this.
@gpucibot merge |
Closes #11109
Adds support for
\0
octal and\x
hex patterns within a regex character class[ ]
pattern. Refactors the existing octal and hex parsing in non-class expression so it can be reused when building a character class instruction. The refactored code was also simplified as well.This change fixes the linked issue by supporting
\0
and\x00
in the expression which identify embedded null characters.Additional gtests were added to check for octal and hex within an
[ ]
expression.