Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add UDF compiler implementations #497

Merged
merged 46 commits into from
Aug 12, 2020
Merged

Add UDF compiler implementations #497

merged 46 commits into from
Aug 12, 2020

Commits on Aug 11, 2020

  1. Add javassist as a dependency

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    41e61bc View commit details
    Browse the repository at this point in the history

  2. Simple implementation of the udf -> expression translation

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    2792201 View commit details
    Browse the repository at this point in the history

  3. Use the udf -> expressions translation when possible

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    66e64db View commit details
    Browse the repository at this point in the history

  4. Replace javassist with jvmci and broaden opcode coverage

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    db4441a View commit details
    Browse the repository at this point in the history

  5. Add support for ISUB.

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    532c855 View commit details
    Browse the repository at this point in the history

  6. Restructure the code.

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    50c40bb View commit details
    Browse the repository at this point in the history

  7. Add an example for UDF to expr conversion

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    79ee9ef View commit details
    Browse the repository at this point in the history

  8. Correctly handle loads and stores with IFLT and GOTO

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    3beb37d View commit details
    Browse the repository at this point in the history

  9. Remove combineExpr 

    As Scala lambdas cannot have multiple returns, we can simply return the
expression from the return instruction without worrying about combining
multiple return values.
    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    a6dffd6 View commit details
    Browse the repository at this point in the history

  10. Fix a compile error and a traversal error

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    4b0117d View commit details
    Browse the repository at this point in the history

  11. Simplify condition expressions.

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    fdb8782 View commit details
    Browse the repository at this point in the history

  12. Remove unnecessary import lines

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    5fc0184 View commit details
    Browse the repository at this point in the history

  13. Make CatalystExpressionBuilder a case class

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    7208a98 View commit details
    Browse the repository at this point in the history

  14. Update connectBasicBlocks to detect implicit fallthrough edges

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    3896f36 View commit details
    Browse the repository at this point in the history

  15. Remove the unnecessary LocalVariables class and allow adding conditio… 

    …nals to stack elements
    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    797501e View commit details
    Browse the repository at this point in the history

  16. Correctly update and propagate the entry condition for basic block

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    653b894 View commit details
    Browse the repository at this point in the history

  17. DCMPG, DCONST_0, DCONST_1, *RETURN, IFLE, IFGT, IFGE, IFEQ

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    b70cdfb View commit details
    Browse the repository at this point in the history

  18. Add more condition simplification rules

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    e869565 View commit details
    Browse the repository at this point in the history

  19. Change the traversal order with the conditional branches. 

    The false path is now traversed before the true path so we can generate
the conditions in the same evaluation order in the source.
    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    59ac31d View commit details
    Browse the repository at this point in the history

  20. Simplify cond repeatedly until it can't be simplified. 

    Also add somem more simplification rules.
    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    bfc765c View commit details
    Browse the repository at this point in the history

  21. Rename simplifyCond to simplifyExpr

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    14ac09e View commit details
    Browse the repository at this point in the history

  22. DLOAD_2, DSTORE_2, DADD

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    e6f1cf5 View commit details
    Browse the repository at this point in the history

  23. Add an example with short-circuit conditionals

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    e3c6452 View commit details
    Browse the repository at this point in the history

  24. Update the short circuit conditional example

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    5acc26d View commit details
    Browse the repository at this point in the history

  25. Acos and Asin

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    22d8134 View commit details
    Browse the repository at this point in the history

  26. Add the Apache-2 license text to the header

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    54db041 View commit details
    Browse the repository at this point in the history

  27. Add the jvm flags to enable JVMCI to scalatest

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    c2de1e0 View commit details
    Browse the repository at this point in the history

  28. Use SparkException instead of Exception

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    acc17ed View commit details
    Browse the repository at this point in the history

  29. syntax/scalastyle edits

    @nvnedelman @wjxiz1992
    nvnedelman authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    f0150e9 View commit details
    Browse the repository at this point in the history

  30. Fix the brace style

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    5ee3801 View commit details
    Browse the repository at this point in the history

  31. ScalaUDF.scala:1065, remove braces from case statement

    @nvnedelman @wjxiz1992
    nvnedelman authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    d256dc5 View commit details
    Browse the repository at this point in the history

  32. Add the support for the rest of *LOAD_* and *STORE_* instructions

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    766638b View commit details
    Browse the repository at this point in the history

  33. Add support for *MUL and the rest of *ADD and *SUB

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    239c540 View commit details
    Browse the repository at this point in the history

  34. Adding tags to tests in UDFSuite.scala for individual test execution … 

    …support
    @nvnedelman @wjxiz1992
    nvnedelman authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    fdbb089 View commit details
    Browse the repository at this point in the history

  35. add OpcodeSuite.scala for UDF to expression catalyst builder testing

    @nvnedelman @wjxiz1992
    nvnedelman authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    fb7ac69 View commit details
    Browse the repository at this point in the history

  36. minor edits to IFEQ test

    @nvnedelman @wjxiz1992
    nvnedelman authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    d429bf2 View commit details
    Browse the repository at this point in the history

  37. Add support for ?LOAD, ?CONST_<n>, DUP, ?2?, IFNE, IFNULL, and IFNONNULL

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    2d43aff View commit details
    Browse the repository at this point in the history

  38. Adding script to generate OpcodeSuite reports, minor suite edits

    @nvnedelman @wjxiz1992
    nvnedelman authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    2f3f82a View commit details
    Browse the repository at this point in the history

  39. Add support for more math ops 

    abs, atan, cos, cosh, sin, tan, tanh, ceil, floor, exp, log,
log10, and sqrt
    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    2eec237 View commit details
    Browse the repository at this point in the history

  40. expanding OpcodeSuite to cover more math, casting, and loading ops

    @nvnedelman @wjxiz1992
    nvnedelman authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    76ef25c View commit details
    Browse the repository at this point in the history

  41. Replace SharedSQLContext with SharedSparkSession in a test

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    174bbbe View commit details
    Browse the repository at this point in the history

  42. use foldLeft instead of deprecated /:

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    b374e83 View commit details
    Browse the repository at this point in the history

  43. Use SerializedLambda and javassist instead of JVMCI

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    9ba0a8e View commit details
    Browse the repository at this point in the history

  44. Fix a comment

    @seanprime7 @wjxiz1992
    seanprime7 authored and wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    c8601d9 View commit details
    Browse the repository at this point in the history

  45. Move codes from spark to rapids repo

    @wjxiz1992
    wjxiz1992 committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    a7b1059 View commit details
    Browse the repository at this point in the history

  46. Add UDF compiler implementations 

    * Add UDF compiler implementations
* Update related docs

Signed-off-by: Allen Xu <allxu@nvidia.com>

Co-authored-by: Sean Lee <selee@nvidia.com>
Co-authored-by: Nicholas Edelman <nedelman@nvidia.com>
Co-authored-by: Alessandro Bellina <abellina@nvidia.com>
    @wjxiz1992 @seanprime7
    @nvnedelman @abellina
    4 people committed Aug 11, 2020
    Configuration menu
    Copy the full SHA
    83f92eb View commit details
    Browse the repository at this point in the history