Skip to content

Commit

Permalink
[SPARK-23159][PYTHON] Update cloudpickle to v0.4.3
Browse files Browse the repository at this point in the history
## What changes were proposed in this pull request?

The version of cloudpickle in PySpark was close to version 0.4.0 with some additional backported fixes and some minor additions for Spark related things.  This update removes Spark related changes and matches cloudpickle [v0.4.3](https://github.com/cloudpipe/cloudpickle/releases/tag/v0.4.3):

Changes by updating to 0.4.3 include:
* Fix pickling of named tuples cloudpipe/cloudpickle#113
* Built in type constructors for PyPy compatibility [here](cloudpipe/cloudpickle@d84980c)
* Fix memoryview support cloudpipe/cloudpickle#122
* Improved compatibility with other cloudpickle versions cloudpipe/cloudpickle#128
* Several cleanups cloudpipe/cloudpickle#121 and [here](cloudpipe/cloudpickle@c91aaf1)
* [MRG] Regression on pickling classes from the __main__ module cloudpipe/cloudpickle#149
* BUG: Handle instance methods of builtin types cloudpipe/cloudpickle#154
* Fix <span>#</span>129 : do not silence RuntimeError in dump() cloudpipe/cloudpickle#153

## How was this patch tested?

Existing pyspark.tests using python 2.7.14, 3.5.2, 3.6.3

Author: Bryan Cutler <cutlerb@gmail.com>

Closes apache#20373 from BryanCutler/pyspark-update-cloudpickle-42-SPARK-23159.
  • Loading branch information
BryanCutler authored and HyukjinKwon committed Mar 8, 2018
1 parent fe22f32 commit 9bb239c
Show file tree
Hide file tree
Showing 3 changed files with 151 additions and 184 deletions.
1 change: 0 additions & 1 deletion python/pyspark/accumulators.py
Original file line number Diff line number Diff line change
Expand Up @@ -94,7 +94,6 @@
else:
import socketserver as SocketServer
import threading
from pyspark.cloudpickle import CloudPickler
from pyspark.serializers import read_int, PickleSerializer


Expand Down
Loading

0 comments on commit 9bb239c

Please sign in to comment.