Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Update Indexer to use new tokens to compute the ranges #7290

Closed
Tracked by #7299
dhruvmanila opened this issue Sep 12, 2023 · 0 comments · Fixed by #7325
Closed
Tracked by #7299

Update Indexer to use new tokens to compute the ranges #7290

dhruvmanila opened this issue Sep 12, 2023 · 0 comments · Fixed by #7325
Assignees
@dhruvmanila
Labels
core Related to core functionality python312 Related to Python 3.12

Comments

@dhruvmanila
Copy link
Member

The Indexer needs to be updated to use the new f-string tokens to compute the following ranges:

  1. F-string ranges
  2. Triple-quoted string ranges

The f-string ranges (1) can be computed using the start range of a FStringStart token to the end range of a FStringEnd token. Here, a stack would probably be required as f-strings can be nested and we need to pick up the correct start value for the current end value. It would be similar to computing the ranges of parentheses pair in (foo, (bar, baz), another, (start, (nested, done), nope), last).

A FStringStart/FStringEnd token consists of prefixes (f, fr, etc.) and quotes. Using this information we can detect if a f-string is tripled-quoted or not using either of the following proposed solutions:

  1. Using the Locator, we can extract out the source for the FStringStart token using the token range, extract the quotes using leading_quote and use the .text_len method to check the number of quotes.
  2. We can store additional information in the FStringStart token itself in the form of bitflags or boolean values similar to FStringMiddle. This can then be used to check if it's a tripled-quoted f-string or not.
@dhruvmanila dhruvmanila mentioned this issue Sep 12, 2023
Closed
9 tasks
@dhruvmanila dhruvmanila added rule Implementing or modifying a lint rule python312 Related to Python 3.12 core Related to core functionality and removed rule Implementing or modifying a lint rule labels Sep 12, 2023
@dhruvmanila dhruvmanila self-assigned this Sep 13, 2023
@dhruvmanila dhruvmanila mentioned this issue Sep 13, 2023
Merged
@dhruvmanila dhruvmanila linked a pull request Sep 13, 2023 that will close this issue
Update Indexer to use new f-string tokens #7325
Merged
dhruvmanila added a commit that referenced this issue Sep 19, 2023
@dhruvmanila
Update Indexer to use new f-string tokens (#7325)
1b2ef3e 
## Summary

This PR updates the `Indexer` to use the new f-string tokens to compute
the `f_string_ranges` for f-strings. It adds a new abstraction which
exposes two methods to support extracting the range for the surrounding
innermost and outermost f-string. It uses the builder pattern to build
the f-string ranges which is similar to how the comment ranges are
built.

## Test Plan

Add new test cases for f-strings for:
* Tab indentation rule
* Line continuation detection in the indexer
* To get the innermost / outermost f-string range
* All detected f-string ranges

fixes: #7290
dhruvmanila added a commit that referenced this issue Sep 20, 2023
@dhruvmanila
Update Indexer to use new f-string tokens (#7325)
6fe4584 
## Summary

This PR updates the `Indexer` to use the new f-string tokens to compute
the `f_string_ranges` for f-strings. It adds a new abstraction which
exposes two methods to support extracting the range for the surrounding
innermost and outermost f-string. It uses the builder pattern to build
the f-string ranges which is similar to how the comment ranges are
built.

## Test Plan

Add new test cases for f-strings for:
* Tab indentation rule
* Line continuation detection in the indexer
* To get the innermost / outermost f-string range
* All detected f-string ranges

fixes: #7290
dhruvmanila added a commit that referenced this issue Sep 21, 2023
@dhruvmanila
Update Indexer to use new f-string tokens (#7325)
1b34b41 
## Summary

This PR updates the `Indexer` to use the new f-string tokens to compute
the `f_string_ranges` for f-strings. It adds a new abstraction which
exposes two methods to support extracting the range for the surrounding
innermost and outermost f-string. It uses the builder pattern to build
the f-string ranges which is similar to how the comment ranges are
built.

## Test Plan

Add new test cases for f-strings for:
* Tab indentation rule
* Line continuation detection in the indexer
* To get the innermost / outermost f-string range
* All detected f-string ranges

fixes: #7290
dhruvmanila added a commit that referenced this issue Sep 22, 2023
@dhruvmanila
Update Indexer to use new f-string tokens (#7325)
56a1b49 
## Summary

This PR updates the `Indexer` to use the new f-string tokens to compute
the `f_string_ranges` for f-strings. It adds a new abstraction which
exposes two methods to support extracting the range for the surrounding
innermost and outermost f-string. It uses the builder pattern to build
the f-string ranges which is similar to how the comment ranges are
built.

## Test Plan

Add new test cases for f-strings for:
* Tab indentation rule
* Line continuation detection in the indexer
* To get the innermost / outermost f-string range
* All detected f-string ranges

fixes: #7290
dhruvmanila added a commit that referenced this issue Sep 22, 2023
@dhruvmanila
Update Indexer to use new f-string tokens (#7325)
cc054c7 
## Summary

This PR updates the `Indexer` to use the new f-string tokens to compute
the `f_string_ranges` for f-strings. It adds a new abstraction which
exposes two methods to support extracting the range for the surrounding
innermost and outermost f-string. It uses the builder pattern to build
the f-string ranges which is similar to how the comment ranges are
built.

## Test Plan

Add new test cases for f-strings for:
* Tab indentation rule
* Line continuation detection in the indexer
* To get the innermost / outermost f-string range
* All detected f-string ranges

fixes: #7290
dhruvmanila added a commit that referenced this issue Sep 22, 2023
@dhruvmanila
Update Indexer to use new f-string tokens (#7325)
ebd1e6b 
## Summary

This PR updates the `Indexer` to use the new f-string tokens to compute
the `f_string_ranges` for f-strings. It adds a new abstraction which
exposes two methods to support extracting the range for the surrounding
innermost and outermost f-string. It uses the builder pattern to build
the f-string ranges which is similar to how the comment ranges are
built.

## Test Plan

Add new test cases for f-strings for:
* Tab indentation rule
* Line continuation detection in the indexer
* To get the innermost / outermost f-string range
* All detected f-string ranges

fixes: #7290
dhruvmanila added a commit that referenced this issue Sep 26, 2023
@dhruvmanila
Update Indexer to use new f-string tokens (#7325)
6e63bdf 
## Summary

This PR updates the `Indexer` to use the new f-string tokens to compute
the `f_string_ranges` for f-strings. It adds a new abstraction which
exposes two methods to support extracting the range for the surrounding
innermost and outermost f-string. It uses the builder pattern to build
the f-string ranges which is similar to how the comment ranges are
built.

## Test Plan

Add new test cases for f-strings for:
* Tab indentation rule
* Line continuation detection in the indexer
* To get the innermost / outermost f-string range
* All detected f-string ranges

fixes: #7290
dhruvmanila added a commit that referenced this issue Sep 27, 2023
@dhruvmanila
Update Indexer to use new f-string tokens (#7325)
dce90d0 
## Summary

This PR updates the `Indexer` to use the new f-string tokens to compute
the `f_string_ranges` for f-strings. It adds a new abstraction which
exposes two methods to support extracting the range for the surrounding
innermost and outermost f-string. It uses the builder pattern to build
the f-string ranges which is similar to how the comment ranges are
built.

## Test Plan

Add new test cases for f-strings for:
* Tab indentation rule
* Line continuation detection in the indexer
* To get the innermost / outermost f-string range
* All detected f-string ranges

fixes: #7290
dhruvmanila added a commit that referenced this issue Sep 28, 2023
@dhruvmanila
Update Indexer to use new f-string tokens (#7325)
8692c8a 
## Summary

This PR updates the `Indexer` to use the new f-string tokens to compute
the `f_string_ranges` for f-strings. It adds a new abstraction which
exposes two methods to support extracting the range for the surrounding
innermost and outermost f-string. It uses the builder pattern to build
the f-string ranges which is similar to how the comment ranges are
built.

## Test Plan

Add new test cases for f-strings for:
* Tab indentation rule
* Line continuation detection in the indexer
* To get the innermost / outermost f-string range
* All detected f-string ranges

fixes: #7290
dhruvmanila added a commit that referenced this issue Sep 29, 2023
@dhruvmanila
Update Indexer to use new f-string tokens (#7325)
57fac75 
## Summary

This PR updates the `Indexer` to use the new f-string tokens to compute
the `f_string_ranges` for f-strings. It adds a new abstraction which
exposes two methods to support extracting the range for the surrounding
innermost and outermost f-string. It uses the builder pattern to build
the f-string ranges which is similar to how the comment ranges are
built.

## Test Plan

Add new test cases for f-strings for:
* Tab indentation rule
* Line continuation detection in the indexer
* To get the innermost / outermost f-string range
* All detected f-string ranges

fixes: #7290
dhruvmanila added a commit that referenced this issue Sep 29, 2023
@dhruvmanila
Update Indexer to use new f-string tokens (#7325)
124cd4a 
## Summary

This PR updates the `Indexer` to use the new f-string tokens to compute
the `f_string_ranges` for f-strings. It adds a new abstraction which
exposes two methods to support extracting the range for the surrounding
innermost and outermost f-string. It uses the builder pattern to build
the f-string ranges which is similar to how the comment ranges are
built.

## Test Plan

Add new test cases for f-strings for:
* Tab indentation rule
* Line continuation detection in the indexer
* To get the innermost / outermost f-string range
* All detected f-string ranges

fixes: #7290
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@dhruvmanila dhruvmanila

Labels
core Related to core functionality python312 Related to Python 3.12
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

Update Indexer to use new f-string tokens astral-sh/ruff
1 participant
@dhruvmanila