-
Notifications
You must be signed in to change notification settings - Fork 1k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[perflint
] Add PERF101
with autofix
#5121
Merged
Merged
Changes from 4 commits
Commits
Show all changes
7 commits
Select commit
Hold shift + click to select a range
deb99c3
Add PERF101, add first two cases
qdegraaf ea3e136
Remove testfile
qdegraaf 5ace48a
Fix docs formatting
qdegraaf a2c5b06
Add few more testcases review snapshot
qdegraaf 4f3900f
Update autofix to use range of iterable
qdegraaf fed86de
Tweaks
charliermarsh 0672230
Merge branch 'main' into feature/PER101
charliermarsh File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
foo_tuple = (1, 2, 3) | ||
foo_list = [1, 2, 3] | ||
foo_set = {1, 2, 3} | ||
foo_dict = {1: 2, 3: 4} | ||
foo_int = 123 | ||
|
||
for i in list(foo_tuple): # PERF101 | ||
pass | ||
|
||
for i in list(foo_list): # PERF101 | ||
pass | ||
|
||
for i in list(foo_set): # PERF101 | ||
pass | ||
|
||
|
||
for i in list((1, 2, 3)): # PERF101 | ||
pass | ||
|
||
for i in list([1, 2, 3]): # PERF101 | ||
pass | ||
|
||
for i in list({1, 2, 3}): # PERF101 | ||
pass | ||
|
||
|
||
for i in list(foo_dict): # Ok | ||
pass | ||
|
||
|
||
for i in list(1): # Ok | ||
pass | ||
|
||
for i in list(foo_int): # Ok | ||
pass |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -1,3 +1,5 @@ | ||
pub(crate) use incorrect_dict_iterator::{incorrect_dict_iterator, IncorrectDictIterator}; | ||
pub(crate) use unnecessary_list_cast::{unnecessary_list_cast, UnnecessaryListCast}; | ||
|
||
mod incorrect_dict_iterator; | ||
mod unnecessary_list_cast; |
100 changes: 100 additions & 0 deletions
100
crates/ruff/src/rules/perflint/rules/unnecessary_list_cast.rs
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,100 @@ | ||
use ruff_text_size::{TextRange, TextSize}; | ||
use rustpython_parser::ast; | ||
use rustpython_parser::ast::Expr; | ||
|
||
use ruff_diagnostics::{AlwaysAutofixableViolation, Diagnostic, Edit, Fix}; | ||
use ruff_macros::{derive_message_formats, violation}; | ||
use ruff_python_ast::prelude::Stmt; | ||
|
||
use crate::checkers::ast::Checker; | ||
use crate::registry::AsRule; | ||
|
||
/// ## What it does | ||
/// Checks for explicit usages of `list()` on an iterable to iterate over them in a for-loop | ||
/// | ||
/// ## Why is this bad? | ||
/// Using a `list()` call to eagerly iterate over an already iterable type is inefficient as a | ||
/// second list iterator is created, after first iterating the value: | ||
/// | ||
/// ## Example | ||
/// ```python | ||
/// items = (1, 2, 3) | ||
/// for i in list(items): | ||
/// print(i) | ||
/// ``` | ||
/// | ||
/// Use instead: | ||
/// ```python | ||
/// items = (1, 2, 3) | ||
/// for i in items: | ||
/// print(i) | ||
/// ``` | ||
#[violation] | ||
pub struct UnnecessaryListCast; | ||
|
||
impl AlwaysAutofixableViolation for UnnecessaryListCast { | ||
#[derive_message_formats] | ||
fn message(&self) -> String { | ||
format!("Do not cast an iterable to `list` before iterating over it") | ||
} | ||
|
||
fn autofix_title(&self) -> String { | ||
format!("Remove `list()` cast") | ||
} | ||
} | ||
|
||
/// PERF101 | ||
pub(crate) fn unnecessary_list_cast(checker: &mut Checker, iter: &Expr) { | ||
let Expr::Call(ast::ExprCall{ func, args, range, ..}) = iter else { | ||
return; | ||
}; | ||
|
||
if args.is_empty() { | ||
return; | ||
} | ||
|
||
if let Expr::Name(ast::ExprName { id, .. }) = func.as_ref() { | ||
if id != "list" || !checker.semantic().is_builtin("list") { | ||
return; | ||
} | ||
}; | ||
|
||
match &args[0] { | ||
Expr::Tuple(_) | Expr::List(_) | Expr::Set(_) => fix_incorrect_list_cast(checker, *range), | ||
Expr::Name(ast::ExprName { id, .. }) => { | ||
let scope = checker.semantic().scope(); | ||
if let Some(binding_id) = scope.get(id) { | ||
let binding = &checker.semantic().bindings[binding_id]; | ||
if binding.kind.is_assignment() || binding.kind.is_named_expr_assignment() { | ||
if let Some(parent_id) = binding.source { | ||
let parent = checker.semantic().stmts[parent_id]; | ||
match parent { | ||
Stmt::Assign(ast::StmtAssign { value, .. }) | ||
| Stmt::AnnAssign(ast::StmtAnnAssign { | ||
value: Some(value), .. | ||
}) => match value.as_ref() { | ||
Expr::Tuple(_) | Expr::List(_) | Expr::Set(_) => { | ||
fix_incorrect_list_cast(checker, *range); | ||
} | ||
_ => {} | ||
}, | ||
_ => (), | ||
} | ||
} | ||
} | ||
} | ||
} | ||
_ => {} | ||
} | ||
} | ||
|
||
fn fix_incorrect_list_cast(checker: &mut Checker, range: TextRange) { | ||
let mut diagnostic = Diagnostic::new(UnnecessaryListCast, range); | ||
if checker.patch(diagnostic.kind.rule()) { | ||
diagnostic.set_fix(Fix::automatic_edits( | ||
Edit::deletion(range.start(), range.start() + TextSize::from(5)), | ||
[Edit::deletion(range.end() - TextSize::from(1), range.end())], | ||
)); | ||
} | ||
checker.diagnostics.push(diagnostic); | ||
} |
122 changes: 122 additions & 0 deletions
122
...s/ruff/src/rules/perflint/snapshots/ruff__rules__perflint__tests__PERF101_PERF101.py.snap
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,122 @@ | ||
--- | ||
source: crates/ruff/src/rules/perflint/mod.rs | ||
--- | ||
PERF101.py:7:10: PERF101 [*] Do not cast an iterable to `list` before iterating over it | ||
| | ||
5 | foo_int = 123 | ||
6 | | ||
7 | for i in list(foo_tuple): # PERF101 | ||
| ^^^^^^^^^^^^^^^ PERF101 | ||
8 | pass | ||
| | ||
= help: Remove `list()` cast | ||
|
||
ℹ Fix | ||
4 4 | foo_dict = {1: 2, 3: 4} | ||
5 5 | foo_int = 123 | ||
6 6 | | ||
7 |-for i in list(foo_tuple): # PERF101 | ||
7 |+for i in foo_tuple: # PERF101 | ||
8 8 | pass | ||
9 9 | | ||
10 10 | for i in list(foo_list): # PERF101 | ||
|
||
PERF101.py:10:10: PERF101 [*] Do not cast an iterable to `list` before iterating over it | ||
| | ||
8 | pass | ||
9 | | ||
10 | for i in list(foo_list): # PERF101 | ||
| ^^^^^^^^^^^^^^ PERF101 | ||
11 | pass | ||
| | ||
= help: Remove `list()` cast | ||
|
||
ℹ Fix | ||
7 7 | for i in list(foo_tuple): # PERF101 | ||
8 8 | pass | ||
9 9 | | ||
10 |-for i in list(foo_list): # PERF101 | ||
10 |+for i in foo_list: # PERF101 | ||
11 11 | pass | ||
12 12 | | ||
13 13 | for i in list(foo_set): # PERF101 | ||
|
||
PERF101.py:13:10: PERF101 [*] Do not cast an iterable to `list` before iterating over it | ||
| | ||
11 | pass | ||
12 | | ||
13 | for i in list(foo_set): # PERF101 | ||
| ^^^^^^^^^^^^^ PERF101 | ||
14 | pass | ||
| | ||
= help: Remove `list()` cast | ||
|
||
ℹ Fix | ||
10 10 | for i in list(foo_list): # PERF101 | ||
11 11 | pass | ||
12 12 | | ||
13 |-for i in list(foo_set): # PERF101 | ||
13 |+for i in foo_set: # PERF101 | ||
14 14 | pass | ||
15 15 | | ||
16 16 | | ||
|
||
PERF101.py:17:10: PERF101 [*] Do not cast an iterable to `list` before iterating over it | ||
| | ||
17 | for i in list((1, 2, 3)): # PERF101 | ||
| ^^^^^^^^^^^^^^^ PERF101 | ||
18 | pass | ||
| | ||
= help: Remove `list()` cast | ||
|
||
ℹ Fix | ||
14 14 | pass | ||
15 15 | | ||
16 16 | | ||
17 |-for i in list((1, 2, 3)): # PERF101 | ||
17 |+for i in (1, 2, 3): # PERF101 | ||
18 18 | pass | ||
19 19 | | ||
20 20 | for i in list([1, 2, 3]): # PERF101 | ||
|
||
PERF101.py:20:10: PERF101 [*] Do not cast an iterable to `list` before iterating over it | ||
| | ||
18 | pass | ||
19 | | ||
20 | for i in list([1, 2, 3]): # PERF101 | ||
| ^^^^^^^^^^^^^^^ PERF101 | ||
21 | pass | ||
| | ||
= help: Remove `list()` cast | ||
|
||
ℹ Fix | ||
17 17 | for i in list((1, 2, 3)): # PERF101 | ||
18 18 | pass | ||
19 19 | | ||
20 |-for i in list([1, 2, 3]): # PERF101 | ||
20 |+for i in [1, 2, 3]: # PERF101 | ||
21 21 | pass | ||
22 22 | | ||
23 23 | for i in list({1, 2, 3}): # PERF101 | ||
|
||
PERF101.py:23:10: PERF101 [*] Do not cast an iterable to `list` before iterating over it | ||
| | ||
21 | pass | ||
22 | | ||
23 | for i in list({1, 2, 3}): # PERF101 | ||
| ^^^^^^^^^^^^^^^ PERF101 | ||
24 | pass | ||
| | ||
= help: Remove `list()` cast | ||
|
||
ℹ Fix | ||
20 20 | for i in list([1, 2, 3]): # PERF101 | ||
21 21 | pass | ||
22 22 | | ||
23 |-for i in list({1, 2, 3}): # PERF101 | ||
23 |+for i in {1, 2, 3}: # PERF101 | ||
24 24 | pass | ||
25 25 | | ||
26 26 | | ||
|
||
|
Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If there's a more elegant way to extract the string representation of the value in the
list()
call to replace the whole thing instead of messing about with set ranges, I'm happy for the input.There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Can you use the statt of the iterator? It has the downside that it truncates comments but produces a correct fix if there's a space (or even comment between list and the (
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Sure! I'm not sure I exactly understand what you mean, though. Can you maybe give an example that would break in this implementation that I can add to the fixture to check after I have made the tweak if I understood?
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Because the range being passed in now is already the range of the
iter: &Expr
iterator. And if you mean the iterable then I don't really see how that helps with comments (aside from the issue of getting the range from the iterable depending on whether it's a binding or literal etc.)There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah wait I think I get it. Do you mean taking the deletion range to be from the start of the iterator to the start of the iterable?
Now that I'm looking it over that still leaves another scenario that would be unfixable namely:
The most complete way would be to somehow extract whatever is in the
list()
call in its entirety, trim the whitespace, and replace the whole range of the iterator with that value. That would catch multiline args but would still leave the issue of comments in something like:I'll see if I can figure something out tomorrow and otherwise I might leave the autofix for later
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
You would probably need to use libCST if you want to retain comments. Many of Ruff's fixes do not handle comments.
I think it's otherwise fine to delete the ranges from the
list(
call expression start to the start of the iterable and from the end of the iterable to the end of thelist call
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Ah in that case I'll just delete the comments. Thanks for the feedback, pushed change!