Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Optimize Styles.isStylePattern() to avoid StackOverflowError #817

Merged
merged 1 commit into from
Feb 22, 2023

Conversation

exceptionfactory
Copy link
Contributor

Changes for JLine 3.19.0 in commit a27bcd1 adjusted pattern matching to support named colors in style declarations. These changes expanded the scope of valid style declarations, but also increased the possibility of performance problems when parsing long or malformed styles.

Fedora 37 has a default setting for the LS_COLORS environment variable that includes style elements which cause performance problems using Styles.STYLE_PATTERN.matches(). Calling Styles.lsStyle() with Azul Zulu Java 8 Update 352 on Fedora 37 causes the JVM to hang due to excessive iteration in regular expression matching. This issue does not occur on Java 11 or 17.

Changes in this pull request optimize style evaluation using a compiled Pattern for one key-value style element. Styles.isStylePattern() splits the input style string using the standard separator character, and then evaluates all elements against the compiled STYLE_ELEMENT_PATTERN. This approach avoids compiling the style pattern for every call to Styles.isStylePattern(), and also avoids a StackOverflowError that occurs with the current implementation when evaluating a test style constructed of 1000 style elements.

For reference, the default value of LS_COLORS on Fedora 37 is as follows:

rs=0:di=01;34:ln=01;36:mh=00:pi=40;33:so=01;35:do=01;35:bd=40;33;01:cd=40;33;01:or=40;31;01:mi=01;37;41:su=37;41:sg=30;43:ca=00:tw=30;42:ow=34;42:st=37;44:ex=01;32:*.tar=01;31:*.tgz=01;31:*.arc=01;31:*.arj=01;31:*.taz=01;31:*.lha=01;31:*.lz4=01;31:*.lzh=01;31:*.lzma=01;31:*.tlz=01;31:*.txz=01;31:*.tzo=01;31:*.t7z=01;31:*.zip=01;31:*.z=01;31:*.dz=01;31:*.gz=01;31:*.lrz=01;31:*.lz=01;31:*.lzo=01;31:*.xz=01;31:*.zst=01;31:*.tzst=01;31:*.bz2=01;31:*.bz=01;31:*.tbz=01;31:*.tbz2=01;31:*.tz=01;31:*.deb=01;31:*.rpm=01;31:*.jar=01;31:*.war=01;31:*.ear=01;31:*.sar=01;31:*.rar=01;31:*.alz=01;31:*.ace=01;31:*.zoo=01;31:*.cpio=01;31:*.7z=01;31:*.rz=01;31:*.cab=01;31:*.wim=01;31:*.swm=01;31:*.dwm=01;31:*.esd=01;31:*.avif=01;35:*.jpg=01;35:*.jpeg=01;35:*.mjpg=01;35:*.mjpeg=01;35:*.gif=01;35:*.bmp=01;35:*.pbm=01;35:*.pgm=01;35:*.ppm=01;35:*.tga=01;35:*.xbm=01;35:*.xpm=01;35:*.tif=01;35:*.tiff=01;35:*.png=01;35:*.svg=01;35:*.svgz=01;35:*.mng=01;35:*.pcx=01;35:*.mov=01;35:*.mpg=01;35:*.mpeg=01;35:*.m2v=01;35:*.mkv=01;35:*.webm=01;35:*.webp=01;35:*.ogm=01;35:*.mp4=01;35:*.m4v=01;35:*.mp4v=01;35:*.vob=01;35:*.qt=01;35:*.nuv=01;35:*.wmv=01;35:*.asf=01;35:*.rm=01;35:*.rmvb=01;35:*.flc=01;35:*.avi=01;35:*.fli=01;35:*.flv=01;35:*.gl=01;35:*.dl=01;35:*.xcf=01;35:*.xwd=01;35:*.yuv=01;35:*.cgm=01;35:*.emf=01;35:*.ogv=01;35:*.ogx=01;35:*.aac=01;36:*.au=01;36:*.flac=01;36:*.m4a=01;36:*.mid=01;36:*.midi=01;36:*.mka=01;36:*.mp3=01;36:*.mpc=01;36:*.ogg=01;36:*.ra=01;36:*.wav=01;36:*.oga=01;36:*.opus=01;36:*.spx=01;36:*.xspf=01;36:*~=00;90:*#=00;90:*.bak=00;90:*.old=00;90:*.orig=00;90:*.part=00;90:*.rej=00;90:*.swp=00;90:*.tmp=00;90:*.dpkg-dist=00;90:*.dpkg-old=00;90:*.ucf-dist=00;90:*.ucf-new=00;90:*.ucf-old=00;90:*.rpmnew=00;90:*.rpmorig=00;90:*.rpmsave=00;90:

@@ -32,7 +33,8 @@ public class Styles {
private static final String KEY = "([a-z]{2}|\\*\\.[a-zA-Z0-9]+)";
private static final String VALUE = "(([!~#]?[a-zA-Z0-9]+[a-z0-9-;]*)?|" + REGEX_TOKEN_NAME + ")";
private static final String VALUES = VALUE + "(," + VALUE + ")*";
private static final String STYLE_PATTERN = KEY + "=" + VALUES + "(:" + KEY + "=" + VALUES + ")*(:|)";
private static final Pattern STYLE_ELEMENT_SEPARATOR = Pattern.compile(":");
private static final Pattern STYLE_ELEMENT_PATTERN = Pattern.compile(KEY + "=" + VALUES);
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

REDOS: The regular expression "([a-z]{2}|\\.[a-zA-Z0-9]+)=(([!~#]?[a-zA-Z0-9]+[a-z0-9-;])?|[A-Z_]+)(,(([!~#]?[a-zA-Z0-9]+[a-z0-9-;])?|[A-Z_]+))" is vulnerable to a denial of service attack (ReDOS)


ℹ️ Learn about @sonatype-lift commands

You can reply with the following commands. For example, reply with @sonatype-lift ignoreall to leave out all findings.

Command Usage
@sonatype-lift ignore Leave out the above finding from this PR
@sonatype-lift ignoreall Leave out all the existing findings from this PR
@sonatype-lift exclude <file|issue|path|tool> Exclude specified file|issue|path|tool from Lift findings by updating your config.toml file

Note: When talking to LiftBot, you need to refresh the page to see its response.
Click here to add LiftBot to another repo.


Was this a good recommendation?
[ 🙁 Not relevant ] - [ 😕 Won't fix ] - [ 😑 Not critical, will fix ] - [ 🙂 Critical, will fix ] - [ 😊 Critical, fixing now ]

@exceptionfactory
Copy link
Contributor Author

The REDOS warning from Sonatype Lift provides some additional explanation as to why a StackOverflowError can occur with the current regular expression pattern. The updates to Styles.isStylePattern() with the initial split on the : separator mitigate some issues without changing the fundamental behavior of the expression. A separate issue may be necessary to evaluate the expression pattern itself, but the initial set of changes should maintain compatibility with current matching behavior.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants