Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support str_to_map [databricks] #4636

Merged
merged 77 commits into from
Feb 23, 2022
Merged

Support str_to_map [databricks] #4636

merged 77 commits into from
Feb 23, 2022

Conversation

ttnghia
Copy link
Collaborator

@ttnghia ttnghia commented Jan 26, 2022 •
edited
Loading

@ttnghia
Add GpuOverride
77a760d 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Implement skeleton
02cf8b8 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Fix type check
6d49cf6 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
WIP
0ff0e3f 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Simplify code
5e6d23b 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Fix delimiter check
7832760 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Add comments
6b4ffe5 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Implement GpuCreateMap.createMapFromKeysValuesAsStructs
2a64e97 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Using GpuCreateMap.createMapFromKeysValuesAsStructs to create map f…
08bb1b2 
…rom split strings

Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia ttnghia added feature request New feature or request SQL part of the SQL/Dataframe plugin cudf_dependency An issue or PR with this label depends on a new feature in cudf labels Jan 26, 2022
@ttnghia ttnghia self-assigned this Jan 26, 2022
@ttnghia
Update comment
ac99eff 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Change input for createMapFromKeysValuesAsStructs
657fcc3 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Fix toMap method
fca136a 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
MISC
5c3e755 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Merge branch 'branch-22.04' into str_to_map
a654cb2
@ttnghia
Update copyright year
310f331 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Remove unused constructors
a6d70ac 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Rename variables
395e41f 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Fix type check
f3074e4 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Rename variable
c5d6780 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@sameerz sameerz linked an issue Jan 31, 2022 that may be closed by this pull request
[FEA] Support str_to_map function #3542
Closed
@ttnghia
Use makeStructView instead of makeStruct to avoid copying data
afd1f16 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Add regex tests
6231d5b 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Update compatibility.md
dbfb5ac 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Update description
b05de28 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Update docs
e610f9b 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia ttnghia marked this pull request as ready for review February 15, 2022 17:55
@ttnghia ttnghia requested a review from revans2 February 15, 2022 17:55
@ttnghia
Update test
656df3d 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Merge branch 'branch-22.04' into str_to_map
db65868 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>

# Conflicts:
#	integration_tests/src/main/python/map_test.py
@ttnghia
Rename test
57ba669 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Copy link
Collaborator Author

ttnghia commented Feb 18, 2022

build

@ttnghia
Copy link
Collaborator Author

ttnghia commented Feb 18, 2022 •
edited
Loading

@revans2 @andygrove Should we test build this with [databricks]?

@revans2
Copy link
Collaborator

revans2 commented Feb 18, 2022

@revans2 @andygrove Should we test build this with [databricks]?

I think databricks would be good just so we can be sure that it compiles. It looks like you are getting a compile error because GpuStringToMap is not inheriting from the shimmed class that lets us support newer versions of Spark ShimExpression.

@ttnghia
Inherit ShimExpression
0bab1db 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia ttnghia changed the title Support str_to_map Support str_to_map [databricks] Feb 18, 2022
@ttnghia
Copy link
Collaborator Author

ttnghia commented Feb 18, 2022

build

revans2
revans2 previously approved these changes Feb 18, 2022
View reviewed changes
integration_tests/src/main/python/map_test.py Show resolved Hide resolved
@ttnghia
Update special pattern to almost always generate duplicate keys
c237cb9 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Merge branch 'branch-22.04' into str_to_map
f08c093
@ttnghia
Fix python import
4ef4a8e 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>
@ttnghia
Copy link
Collaborator Author

ttnghia commented Feb 18, 2022

build

@ttnghia ttnghia requested a review from revans2 February 22, 2022 20:29
revans2
revans2 previously approved these changes Feb 23, 2022
View reviewed changes
@ttnghia
Merge branch 'branch-22.04' into str_to_map
7c05021 
Signed-off-by: Nghia Truong <nghiatruong.vn@gmail.com>

# Conflicts:
#	integration_tests/src/main/python/map_test.py
@ttnghia
Copy link
Collaborator Author

ttnghia commented Feb 23, 2022

build

@ttnghia ttnghia merged commit 7a87978 into NVIDIA:branch-22.04 Feb 23, 2022
@ttnghia ttnghia deleted the str_to_map branch February 23, 2022 20:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@revans2 revans2 revans2 approved these changes

@andygrove andygrove Awaiting requested review from andygrove

Assignees

@ttnghia ttnghia

Labels
cudf_dependency An issue or PR with this label depends on a new feature in cudf feature request New feature or request SQL part of the SQL/Dataframe plugin
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

[FEA] Support regular expression delimiters for str_to_map [FEA] Support str_to_map function
3 participants
@ttnghia @revans2 @andygrove