Fix handling of backtracking stack with some loops #79353
Conversation
With both RegexOptions.Compiled and the Regex source generator, Regex greedy loops with - a minimum bound of at least 2 - no child constructs that backtrack - and a child that's more than a one/notone/set (aka things that match a single character) are possibly leaving state on the backtracking stack when: - at least one iteration of the loop successfully matches - but not enough iterations match to make the loop successful such that matching the loop fails In that case, if a previous construct in the pattern pushed any state onto the backtracking stack such that it expects to be able to pop off and use that state upon backtracking to it, it will potentially pop the erroneously leftover state. This can then cause execution to go awry, as it's getting back an unexpected value. That can lead to false positives, false negatives, or exceptions such as an IndexOutOfRangeException due to trying to pop too much from the backtracking stack. We already have the ability to remember the backtracking stack position when we initially enter the loop so that we can reset to that position later on. The fix is simply to extend that to also perform that reset when failing the match of such a loop in such circumstances.
ghost
assigned 
|
Tagging subscribers to this area: @dotnet/area-system-text-regularexpressions Issue DetailsWith both RegexOptions.Compiled and the Regex source generator, Regex greedy loops with
are possibly leaving state on the backtracking stack when:
In that case, if a previous construct in the pattern pushed any state onto the backtracking stack such that it expects to be able to pop off and use that state upon backtracking to it, it will potentially pop the erroneously leftover state. This can then cause execution to go awry, as it's getting back an unexpected value. That can lead to false positives, false negatives, or exceptions such as an IndexOutOfRangeException due to trying to pop too much from the backtracking stack. We already have the ability to remember the backtracking stack position when we initially enter the loop so that we can reset to that position later on. The fix is simply to extend that to also perform that reset when failing the match of such a loop in such circumstances. Fixes #79298, but I think we should backport this.
|
|
/backport to release/7.0 |
|
Started backporting to release/7.0: https://github.com/dotnet/runtime/actions/runs/3642612581 |
With both RegexOptions.Compiled and the Regex source generator, Regex greedy loops with
are possibly leaving state on the backtracking stack when:
In that case, if a previous construct in the pattern pushed any state onto the backtracking stack such that it expects to be able to pop off and use that state upon backtracking to it, it will potentially pop the erroneously leftover state. This can then cause execution to go awry, as it's getting back an unexpected value. That can lead to false positives, false negatives, or exceptions such as an IndexOutOfRangeException due to trying to pop too much from the backtracking stack.
We already have the ability to remember the backtracking stack position when we initially enter the loop so that we can reset to that position later on. The fix is simply to extend that to also perform that reset when failing the match of such a loop in such circumstances.
Fixes #79298, but I think we should backport this to release/7.0.