Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add autofix for PIE800 #8668

Merged
merged 5 commits into from
Nov 15, 2023
Merged
Show file tree
Hide file tree
Changes from 2 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions crates/ruff_linter/resources/test/fixtures/flake8_pie/PIE800.py
Original file line number Diff line number Diff line change
@@ -1,9 +1,21 @@
{"foo": 1, **{"bar": 1}} # PIE800

{**{"bar": 10}, "a": "b"} # PIE800

foo({**foo, **{"bar": True}}) # PIE800

{**foo, **{"bar": 10}} # PIE800

{ # PIE800
"a": "b",
# Preserve
**{
# all
"bar": 10, # the
# comments
},
}

{**foo, **buzz, **{bar: 10}} # PIE800

{**foo, "bar": True } # OK
Expand Down
1 change: 1 addition & 0 deletions crates/ruff_linter/src/rules/flake8_pie/mod.rs
Original file line number Diff line number Diff line change
Expand Up @@ -32,6 +32,7 @@ mod tests {
}

#[test_case(Rule::UnnecessaryPlaceholder, Path::new("PIE790.py"))]
#[test_case(Rule::UnnecessarySpread, Path::new("PIE800.py"))]
#[test_case(Rule::ReimplementedContainerBuiltin, Path::new("PIE807.py"))]
fn preview_rules(rule_code: Rule, path: &Path) -> Result<()> {
let snapshot = format!(
Expand Down
Original file line number Diff line number Diff line change
@@ -1,9 +1,9 @@
use ruff_python_ast::Expr;

use ruff_diagnostics::Diagnostic;
use ruff_diagnostics::Violation;
use ruff_diagnostics::{Diagnostic, Edit, Fix, FixAvailability, Violation};
use ruff_macros::{derive_message_formats, violation};
use ruff_text_size::Ranged;
use ruff_python_trivia::{BackwardsTokenizer, SimpleTokenKind};
use ruff_text_size::{Ranged, TextSize};

use crate::checkers::ast::Checker;

Expand Down Expand Up @@ -32,10 +32,16 @@ use crate::checkers::ast::Checker;
pub struct UnnecessarySpread;

impl Violation for UnnecessarySpread {
const FIX_AVAILABILITY: FixAvailability = FixAvailability::Sometimes;

#[derive_message_formats]
fn message(&self) -> String {
format!("Unnecessary spread `**`")
}

fn fix_title(&self) -> Option<String> {
Some(format!("Remove unnecessary dict"))
}
}

/// PIE800
Expand All @@ -44,8 +50,57 @@ pub(crate) fn unnecessary_spread(checker: &mut Checker, keys: &[Option<Expr>], v
if let (None, value) = item {
// We only care about when the key is None which indicates a spread `**`
// inside a dict.
if let Expr::Dict(_) = value {
let diagnostic = Diagnostic::new(UnnecessarySpread, value.range());
if let Expr::Dict(dict) = value {
let mut diagnostic = Diagnostic::new(UnnecessarySpread, value.range());
if checker.settings.preview.is_enabled() {
// Delete the `**{`
let tokenizer = BackwardsTokenizer::up_to(
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is there any way to restructure this to use SimpleTokenizer, i.e., forwards lexing? We've considered removing the backwards lexer in the past since it's quite complex, so I'd prefer to minimize usages if possible.

Copy link
Contributor Author

@alanhdu alanhdu Nov 14, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah, I think that should be possible:

  • For deleting **{, I can lex forward from the end of the previous value in the outer dictionary.
  • For deleting the last }, I can lex forward from the last value of the inner dictionary.

I might have to change the function signature to passi n the whole dict (and not just the keys + values) to handle the case where {**{"a": "b"}} case where there's no previous value (in which case I'd lex forward from the dict start).

(might not get to this until tomorrow though).

Copy link
Contributor Author

@alanhdu alanhdu Nov 15, 2023

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ok -- I think this should work.

I ended up leaving the formatting fairly ugly -- the more I tried to make it look nice, the less confident I became that I handled all the edge-cases correctly (e.g. handling comments correctly, avoiding double commas, getting the indentation right, etc) so I ended up going for something simple and just letting an autoformatter figure it out afterwards.

I might try and take another crack at having the fix provide better formatted code later.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for doing a second pass here!

dict.range.start(),
checker.locator().contents(),
&[],
);
let mut start = None;
for tok in tokenizer {
if let SimpleTokenKind::DoubleStar = tok.kind() {
start = Some(tok.range.start());
break;
}
}
// unwrap is ok, b/c item.0 can't be None without a DoubleStar
let first =
Edit::deletion(start.unwrap(), dict.range.start() + TextSize::from(1));

// Delete the `}` (and possibly a trailing comma) but preserve comments
let mut edits = Vec::with_capacity(1);
let mut end = dict.range.end();

let tokenizer = BackwardsTokenizer::up_to(
dict.range.end() - TextSize::from(1),
checker.locator().contents(),
&[],
);
for tok in tokenizer {
match tok.kind() {
SimpleTokenKind::Comment => {
if tok.range.end() != end {
edits.push(Edit::deletion(tok.range.end(), end));
}
end = tok.range.start();
}
SimpleTokenKind::Comma
| SimpleTokenKind::Whitespace
| SimpleTokenKind::Newline
| SimpleTokenKind::Continuation => {}
_ => {
if tok.range.end() != end {
edits.push(Edit::deletion(tok.range.end(), end));
}
break;
}
}
}
diagnostic.set_fix(Fix::safe_edits(first, edits));
}
checker.diagnostics.push(diagnostic);
}
}
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -6,37 +6,67 @@ PIE800.py:1:14: PIE800 Unnecessary spread `**`
1 | {"foo": 1, **{"bar": 1}} # PIE800
| ^^^^^^^^^^ PIE800
2 |
3 | foo({**foo, **{"bar": True}}) # PIE800
3 | {**{"bar": 10}, "a": "b"} # PIE800
|
= help: Remove unnecessary dict

PIE800.py:3:15: PIE800 Unnecessary spread `**`
PIE800.py:3:4: PIE800 Unnecessary spread `**`
|
1 | {"foo": 1, **{"bar": 1}} # PIE800
2 |
3 | foo({**foo, **{"bar": True}}) # PIE800
| ^^^^^^^^^^^^^ PIE800
3 | {**{"bar": 10}, "a": "b"} # PIE800
| ^^^^^^^^^^^ PIE800
4 |
5 | {**foo, **{"bar": 10}} # PIE800
5 | foo({**foo, **{"bar": True}}) # PIE800
|
= help: Remove unnecessary dict

PIE800.py:5:11: PIE800 Unnecessary spread `**`
PIE800.py:5:15: PIE800 Unnecessary spread `**`
|
3 | foo({**foo, **{"bar": True}}) # PIE800
3 | {**{"bar": 10}, "a": "b"} # PIE800
4 |
5 | {**foo, **{"bar": 10}} # PIE800
| ^^^^^^^^^^^ PIE800
5 | foo({**foo, **{"bar": True}}) # PIE800
| ^^^^^^^^^^^^^ PIE800
6 |
7 | {**foo, **buzz, **{bar: 10}} # PIE800
7 | {**foo, **{"bar": 10}} # PIE800
|
= help: Remove unnecessary dict

PIE800.py:7:19: PIE800 Unnecessary spread `**`
PIE800.py:7:11: PIE800 Unnecessary spread `**`
|
5 | {**foo, **{"bar": 10}} # PIE800
5 | foo({**foo, **{"bar": True}}) # PIE800
6 |
7 | {**foo, **buzz, **{bar: 10}} # PIE800
| ^^^^^^^^^ PIE800
7 | {**foo, **{"bar": 10}} # PIE800
| ^^^^^^^^^^^ PIE800
8 |
9 | {**foo, "bar": True } # OK
9 | { # PIE800
|
= help: Remove unnecessary dict

PIE800.py:12:7: PIE800 Unnecessary spread `**`
|
10 | "a": "b",
11 | # Preserve
12 | **{
| _______^
13 | | # all
14 | | "bar": 10, # the
15 | | # comments
16 | | },
| |_____^ PIE800
17 | }
|
= help: Remove unnecessary dict

PIE800.py:19:19: PIE800 Unnecessary spread `**`
|
17 | }
18 |
19 | {**foo, **buzz, **{bar: 10}} # PIE800
| ^^^^^^^^^ PIE800
20 |
21 | {**foo, "bar": True } # OK
|
= help: Remove unnecessary dict


Original file line number Diff line number Diff line change
@@ -0,0 +1,133 @@
---
source: crates/ruff_linter/src/rules/flake8_pie/mod.rs
---
PIE800.py:1:14: PIE800 [*] Unnecessary spread `**`
|
1 | {"foo": 1, **{"bar": 1}} # PIE800
| ^^^^^^^^^^ PIE800
2 |
3 | {**{"bar": 10}, "a": "b"} # PIE800
|
= help: Remove unnecessary dict

ℹ Safe fix
1 |-{"foo": 1, **{"bar": 1}} # PIE800
1 |+{"foo": 1, "bar": 1} # PIE800
2 2 |
3 3 | {**{"bar": 10}, "a": "b"} # PIE800
4 4 |

PIE800.py:3:4: PIE800 [*] Unnecessary spread `**`
|
1 | {"foo": 1, **{"bar": 1}} # PIE800
2 |
3 | {**{"bar": 10}, "a": "b"} # PIE800
| ^^^^^^^^^^^ PIE800
4 |
5 | foo({**foo, **{"bar": True}}) # PIE800
|
= help: Remove unnecessary dict

ℹ Safe fix
1 1 | {"foo": 1, **{"bar": 1}} # PIE800
2 2 |
3 |-{**{"bar": 10}, "a": "b"} # PIE800
3 |+{"bar": 10, "a": "b"} # PIE800
4 4 |
5 5 | foo({**foo, **{"bar": True}}) # PIE800
6 6 |

PIE800.py:5:15: PIE800 [*] Unnecessary spread `**`
|
3 | {**{"bar": 10}, "a": "b"} # PIE800
4 |
5 | foo({**foo, **{"bar": True}}) # PIE800
| ^^^^^^^^^^^^^ PIE800
6 |
7 | {**foo, **{"bar": 10}} # PIE800
|
= help: Remove unnecessary dict

ℹ Safe fix
2 2 |
3 3 | {**{"bar": 10}, "a": "b"} # PIE800
4 4 |
5 |-foo({**foo, **{"bar": True}}) # PIE800
5 |+foo({**foo, "bar": True}) # PIE800
6 6 |
7 7 | {**foo, **{"bar": 10}} # PIE800
8 8 |

PIE800.py:7:11: PIE800 [*] Unnecessary spread `**`
|
5 | foo({**foo, **{"bar": True}}) # PIE800
6 |
7 | {**foo, **{"bar": 10}} # PIE800
| ^^^^^^^^^^^ PIE800
8 |
9 | { # PIE800
|
= help: Remove unnecessary dict

ℹ Safe fix
4 4 |
5 5 | foo({**foo, **{"bar": True}}) # PIE800
6 6 |
7 |-{**foo, **{"bar": 10}} # PIE800
7 |+{**foo, "bar": 10} # PIE800
8 8 |
9 9 | { # PIE800
10 10 | "a": "b",

PIE800.py:12:7: PIE800 [*] Unnecessary spread `**`
|
10 | "a": "b",
11 | # Preserve
12 | **{
| _______^
13 | | # all
14 | | "bar": 10, # the
15 | | # comments
16 | | },
| |_____^ PIE800
17 | }
|
= help: Remove unnecessary dict

ℹ Safe fix
9 9 | { # PIE800
10 10 | "a": "b",
11 11 | # Preserve
12 |- **{
12 |+
13 13 | # all
14 14 | "bar": 10, # the
15 |- # comments
16 |- },
15 |+ # comments,
17 16 | }
18 17 |
19 18 | {**foo, **buzz, **{bar: 10}} # PIE800

PIE800.py:19:19: PIE800 [*] Unnecessary spread `**`
|
17 | }
18 |
19 | {**foo, **buzz, **{bar: 10}} # PIE800
| ^^^^^^^^^ PIE800
20 |
21 | {**foo, "bar": True } # OK
|
= help: Remove unnecessary dict

ℹ Safe fix
16 16 | },
17 17 | }
18 18 |
19 |-{**foo, **buzz, **{bar: 10}} # PIE800
19 |+{**foo, **buzz, bar: 10} # PIE800
20 20 |
21 21 | {**foo, "bar": True } # OK
22 22 |


Loading