Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Download MXNet MNIST data from FTP in integration #2564

Merged
merged 7 commits into from
Jul 21, 2020
Merged

Download MXNet MNIST data from FTP in integration #2564

merged 7 commits into from
Jul 21, 2020

Conversation

hkvision
Copy link
Contributor

No description provided.

@hkvision hkvision requested a review from glorysdj July 20, 2020 08:50
@glorysdj
Copy link
Contributor

also need to upload $FTP_URI/analytics-zoo-data/mnist.zip to http

qiyuangong
qiyuangong approved these changes Jul 20, 2020
View reviewed changes
Copy link
Contributor

@qiyuangong qiyuangong left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hkvision
Copy link
Contributor Author

Succeeded: http://10.239.47.210:18888/view/ZOO-PR/job/ZOO-PR-Python-ExampleTests-py36-ray/31/console

2020-07-20 12:51:56 URL: ftp://zoo:*password*@10.239.47.210/analytics-zoo-data/mnist.zip [11595270] -> "/opt/work/jenkins/workspace/ZOO-PR-Python-ExampleTests-py36-ray/data/mnist.zip" [1]

�[2m�[36m(pid=37978)�[0m INFO:filelock:Lock 140205914266592 acquired on data.lock
�[2m�[36m(pid=37978)�[0m [12:52:23] src/io/iter_mnist.cc:113: MNISTIter: load 30000 images, shuffle=1, shape=[256,1,28,28]
�[2m�[36m(pid=37978)�[0m [12:52:24] src/io/iter_mnist.cc:113: MNISTIter: load 5000 images, shuffle=1, shape=[256,1,28,28]
�[2m�[36m(pid=37978)�[0m INFO:filelock:Lock 140205914266592 released on data.lock
�[2m�[36m(pid=37978)�[0m INFO:filelock:Lock 140205914266592 acquired on data.lock
�[2m�[36m(pid=37978)�[0m [12:52:24] src/io/iter_mnist.cc:113: MNISTIter: load 30000 images, shuffle=1, shape=[256,1,28,28]
�[2m�[36m(pid=37980)�[0m INFO:filelock:Lock 140048523926496 acquired on data.lock
�[2m�[36m(pid=37978)�[0m [12:52:24] src/io/iter_mnist.cc:113: MNISTIter: load 5000 images, shuffle=1, shape=[256,1,28,28]
�[2m�[36m(pid=37978)�[0m INFO:filelock:Lock 140205914266592 released on data.lock
�[2m�[36m(pid=37980)�[0m [12:52:25] src/io/iter_mnist.cc:113: MNISTIter: load 30000 images, shuffle=1, shape=[256,1,28,28]
�[2m�[36m(pid=37980)�[0m [12:52:25] src/io/iter_mnist.cc:113: MNISTIter: load 5000 images, shuffle=1, shape=[256,1,28,28]

Not downloading any more.

@hkvision
Copy link
Contributor Author

Previously, http://10.239.47.210:18888/view/ZOO-PR/job/ZOO-PR-Python-ExampleTests-py36-ray/17/console

�[2m�[36m(pid=110557)�[0m INFO:filelock:Lock 139717059750768 acquired on data.lock
�[2m�[36m(pid=110557)�[0m INFO:root:downloaded http://data.mxnet.io/mxnet/data/mnist.zip into data/mnist.zip successfully
�[2m�[36m(pid=110557)�[0m [03:30:48] src/io/iter_mnist.cc:113: MNISTIter: load 30000 images, shuffle=1, shape=[256,1,28,28]
�[2m�[36m(pid=110557)�[0m [03:30:48] src/io/iter_mnist.cc:113: MNISTIter: load 5000 images, shuffle=1, shape=[256,1,28,28]
�[2m�[36m(pid=110557)�[0m INFO:filelock:Lock 139717059750768 released on data.lock
�[2m�[36m(pid=110557)�[0m INFO:filelock:Lock 139717059750768 acquired on data.lock
�[2m�[36m(pid=110557)�[0m [03:30:49] src/io/iter_mnist.cc:113: MNISTIter: load 30000 images, shuffle=1, shape=[256,1,28,28]
�[2m�[36m(pid=110557)�[0m [03:30:49] src/io/iter_mnist.cc:113: MNISTIter: load 5000 images, shuffle=1, shape=[256,1,28,28]

@hkvision
Copy link
Contributor Author

http://10.239.47.210:18888/view/ZOO-PR/job/ZOO-PR-Validation/3818/ One job failure is due to network.
Thanks so much for your review @qiyuangong

@hkvision hkvision merged commit 29e2bf6 into intel-analytics:master Jul 21, 2020
@hkvision hkvision deleted the fix-integration branch July 21, 2020 02:43
sgwhat pushed a commit to sgwhat/analytics-zoo that referenced this pull request Oct 9, 2021
@hkvision
Download MXNet MNIST data from FTP in integration (intel-analytics#2564)
c4c5281 
* download from ftp

* fix

* fix ftp

* tempoary remove

* revert

* fix path

* update
dding3 pushed a commit that referenced this pull request Oct 13, 2021
@hkvision
Download MXNet MNIST data from FTP in integration (#2564)
3cf7ee0 
* download from ftp

* fix

* fix ftp

* tempoary remove

* revert

* fix path

* update
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@qiyuangong qiyuangong qiyuangong approved these changes

@glorysdj glorysdj Awaiting requested review from glorysdj

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
3 participants
@hkvision @glorysdj @qiyuangong