Add sphand ros #2707

pazeshun · 2020-01-23T13:04:58Z

Add updated version of https://github.com/pazeshun/sphand_ros
Those packages are required for my demo.

wkentaro · 2020-01-23T19:15:22Z

Are there some specific reasons why you don’t add your sphand_ros to the rosinstall?

pazeshun · 2020-01-24T05:37:05Z

Are there some specific reasons why you don’t add your sphand_ros to the rosinstall?

I just want to place my code under a repository of our lab, not to let my code forgotten.
This placing also makes maintenance easy, because sphand_ros packages depend on jsk_apc packages.
Someone who changes jsk_apc drastically may adjust sphand_ros if it is included in jsk_apc.

wkentaro · 2020-01-24T13:44:49Z

Thanks. That makes sense.

On Fri, 24 Jan 2020 at 05:37, Shun Hasegawa ***@***.***> wrote: Are there some specific reasons why you don’t add your sphand_ros to the rosinstall? I just want to place my code under a repository of our lab, not to let my code forgotten. This placing also makes maintenance easy, because sphand_ros packages depend on jsk_apc packages. Someone who changes jsk_apc drastically may adjust sphand_ros if it is included in jsk_apc. — You are receiving this because you commented. Reply to this email directly, view it on GitHub <#2707?email_source=notifications&email_token=ABA4LE7DSNWSVUZY6EWMX2DQ7J5ADA5CNFSM4KKWDYR2YY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEJZY35A#issuecomment-577998324>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABA4LE4MIK6WCCQNCROJU6DQ7J5ADANCNFSM4KKWDYRQ> .

-- Kentaro Wada / 和田健太郎 https://wkentaro.com <http://wkentaro.com>

pazeshun · 2020-05-09T05:11:14Z

Travis got timeout in cache setting:

fetching PR.2707/cache--linux-trusty-7471d3ccbbdeaec95cea57133066e84e8dedc6006c1eff07012e6f7e0064ef76--compiler-gpp.tgz

fetching PR.2707/cache-linux-trusty-7471d3ccbbdeaec95cea57133066e84e8dedc6006c1eff07012e6f7e0064ef76--compiler-gpp.tgz

fetching PR.2707/cache--compiler-gpp.tgz

fetching master/cache--linux-trusty-7471d3ccbbdeaec95cea57133066e84e8dedc6006c1eff07012e6f7e0064ef76--compiler-gpp.tgz

found cache

0.00sadding /home/travis/.ccache to cache

adding /home/travis/.cache/pip to cache

adding /home/travis/.ros/data to cache

creating directory /home/travis/.ros/data

No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.

Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#build-times-out-because-no-output-was-received

The build has been terminated

I assume the master cache is too large and it cannot be expanded in the time frame.
https://travis-ci.community/t/builds-timeout-during-or-shortly-after-creating-directory-home-travis-cache-sccache/5001

I'll try deleting the master cache. If that trial succeeded, we have to consider changing the cache setting or commenting out allow_failures job to reduce cache size.

pazeshun · 2020-05-09T08:32:50Z

I'll try deleting the master cache.

Travis becomes passed after this deletion, so my assumption (travis failed as cache is too large) may be correct.
And cache of this PR is 22372.95MB, which is more than the previous master cache and seems too heavy:

I'll try to reduce this.

See start-jsk#2707 (comment)

pazeshun · 2020-05-09T09:54:21Z

Reached to download limit. Now waiting.

$ gdown https://drive.google.com/uc?id=0B9P1L--7Wd2vS1VaWWVFNDVFQ1k
/usr/local/lib/python2.7/dist-packages/requests/__init__.py:83: RequestsDependencyWarning: Old version of cryptography ([1, 2, 3]) may cause slowdown.
  warnings.warn(warning, RequestsDependencyWarning)
Access denied with the following error:

 	Too many users have viewed or downloaded this file recently. Please
	try accessing the file again later. If the file you are trying to
	access is particularly large or is shared with many people, it may
	take up to 24 hours to be able to view or download the file. If you
	still can't access a file after 24 hours, contact your domain
	administrator. 

You may still be able to access the file from the browser:

	 https://drive.google.com/uc?id=0B9P1L--7Wd2vS1VaWWVFNDVFQ1k

knorth55 · 2020-05-09T10:07:04Z

can you try pip install gdown="3.11.0"?
i heard new gdown suppress the error.

pazeshun · 2020-05-09T10:16:20Z

can you try pip install gdown="3.11.0"?

I tried, but faced the same error.

pazeshun · 2020-05-09T10:29:13Z

Currently, there seems no solution: wkentaro/gdown#26 (comment)
Perhaps there is a tricky solution, but I don't want to spend my time on finding it.

wkentaro · 2020-05-09T14:49:19Z

It seems this file is downloaded extremely many times before, so I can't access the file even from the browser without logging in. (incognito mode on Chrome)

wkentaro · 2020-05-09T15:20:05Z

Currently most of the large files in arc2017_competition directory (in which the original file is located) are not downloadable. (so maybe the drive directory (arc2017_competition) may be in a tricky condition)

pazeshun · 2020-05-11T01:05:32Z

commenting out allow_failures job to reduce cache size

This is tried at 98c558f and cache size is reduced to 18304.37MB:

Now I'll restart travis and check if current cache can be expanded.

wkentaro · 2020-05-12T08:56:22Z

Now it's back to normal, so we can see the caching helps to keep the file downloadable for a longer period.

% gdown  https://drive.google.com/uc?id=0B9P1L--7Wd2vS1VaWWVFNDVFQ1k                                                                                                                                                                  
Downloading...                                                                                                                                                                                                                        
From: https://drive.google.com/uc?id=0B9P1L--7Wd2vS1VaWWVFNDVFQ1k                                                                                                                                                                     
To: /home/wkentaro/fcn32s_arc2017_datasetv2_cfg003_20170612.npz                                                                                                                                                                       
52.0MB [00:05, 9.18MB/s]

BTW, is file caching enabled for ~/.ros (or something similar) directory? If it's enabled, I don't think this problem won't happen that frequently.

wkentaro · 2020-05-12T09:02:34Z

It was enabled

jsk_apc/.travis.yml

Line 15 in d0a238a

- $HOME/.ros/data

But I found that the file in the issue is not used in the system because datasetv3 is used instead of v2. So probably we can just comment it out.

jsk_apc/jsk_arc2017_common/scripts/install_data.py

Lines 84 to 93 in 76db5dd

    
           symlink_src = os.path.join( 
        
               PKG_PATH, 'data/models/fcn32s_arc2017_datasetv3_cfg009_20170724.npz') 
        
           symlink_dst = os.path.join(PKG_PATH, 'data/models/fcn32s.npz') 
        
           if os.path.exists(symlink_dst): 
        
               print('[%s] File already exists, so skipping.' % symlink_dst) 
        
           else: 
        
               if os.path.islink(symlink_dst): 
        
                   os.remove(symlink_dst) 
        
               print('[%s] Creating symlink to: %s' % (symlink_dst, symlink_src)) 
        
               os.symlink(symlink_src, symlink_dst)

pazeshun · 2020-05-12T09:36:58Z

Now I'll restart travis and check if current cache can be expanded.

This trial failed, and I realized that the problem is not about the whole cache size.
And further debugging on https://github.com/pazeshun/jsk_apc/tree/add-sphand_ros-cache told me:

On both kinetic test and indigo test, the following files exist under $HOME/.ros/data and cached:

/home/travis/.ros/data/jsk_arc2017_common/fcn32s_arc2017_datasetv2_cfg003_20170612.npz
/home/travis/.ros/data/jsk_arc2017_common/fcn32s_arc2017_datasetv2_cfg003_20170721.npz
/home/travis/.ros/data/jsk_arc2017_common/fcn32s_arc2017_datasetv3_cfg009_20170724.npz
/home/travis/.ros/data/jsk_2016_01_baxter_apc/sib_right_softkinetic.bag.tar.gz
/home/travis/.ros/data/jsk_2016_01_baxter_apc/sib_kinect2.bag.tar.gz
/home/travis/.ros/data/jsk_2016_01_baxter_apc/sib_right_softkinetic.bag
/home/travis/.ros/data/jsk_2016_01_baxter_apc/sib_kinect2.bag
/home/travis/.ros/data/jsk_2016_01_baxter_apc/2016-04-30-16-33-54_apc2016-bin-boxes.bag
/home/travis/.ros/data/jsk_apc2016_common/vgg16_rotation_translation_brightness_372000.chainermodel
/home/travis/.ros/data/jsk_apc2016_common/tabletop_objects.bag
/home/travis/.ros/data/jsk_apc2016_common/vgg16_96000.chainermodel
/home/travis/.ros/data/jsk_apc2016_common/tabletop_objects.bag.tgz
/home/travis/.ros/data/jsk_apc2016_common/fcn32s_v2_148000.chainermodel
/home/travis/.ros/data/jsk_apc2016_common/fcn32s_6000.chainermodel

Only on kinetic test, extracting /home/travis/.ros/data/jsk_arc2017_common doesn't finish (even though this directory is the same as indigo test and is lighter than /home/travis/.ros/data/jsk_apc2016_common (I confirmed file size on local) ):

fetching add-sphand_ros-cache/cache--linux-trusty-8defbc9ec54de23ff9c8cc77fb06464a913b1a901ef3ad077d9f613bfc5f48ae--compiler-gpp.tgz
found cache
adding /home/travis/.ccache to cache
adding /home/travis/.cache/pip to cache
adding /home/travis/.ros/data/jsk_2016_01_baxter_apc to cache
creating directory /home/travis/.ros/data/jsk_2016_01_baxter_apc
adding /home/travis/.ros/data/jsk_apc2016_common to cache
creating directory /home/travis/.ros/data/jsk_apc2016_common
adding /home/travis/.ros/data/jsk_arc2017_common to cache
creating directory /home/travis/.ros/data/jsk_arc2017_common

No output has been received in the last 10m0s, this potentially indicates a stalled build or something wrong with the build itself.
Check the details on how to adjust your build configuration on: https://docs.travis-ci.com/user/common-build-problems/#build-times-out-because-no-output-was-received
The build has been terminated

Setting cache timeout doesn't work, because this is timeout for uploading cache, not extracting cache.

@wkentaro Do you mean we can comment out here and here?
Then this will be the best solution I know.

wkentaro · 2020-05-12T10:27:11Z

Sent a PR #2712

…/catkin/html/howto/format1/index.html

…nition is too heavy

Far part: ToF distance, middle & close part: distance generated from intensity

… after calibration

When subscribing compressedDepth from kinetic UP Board, hz of image_raw drops. This is because usage of CPU processing camera becomes 100%. Even on baxter-c1, hz of image_raw drops if png_level is 9 or 8.

…Board

…ure on travis due to googledrive access limit I tried travis caching of those data, but cache couldn't be expanded before travis timeout: https://travis-ci.org/github/start-jsk/jsk_apc/jobs/686632934 https://travis-ci.org/github/start-jsk/jsk_apc/jobs/686826882

pazeshun · 2020-05-14T07:30:29Z

Changes to existing code by this PR are 6f7e1d9 and 40c5cea.
6f7e1d9 simply adds new packages added by this PR to indigo test.
40c5cea changes grasp_fusion package not to download heavy data from GoogleDrive on travis for avoiding access limit and build failure.
(As described in the commit message, travis cache doesn't work in this case.)
@knorth55 Could you review this?

knorth55 · 2020-05-14T15:49:08Z

@pazeshun Good work! Thank you!

pazeshun · 2020-05-14T23:19:43Z

Thank you!

pazeshun mentioned this pull request Jan 23, 2020

Add grasp_prediction_arc2017 #2708

Merged

pazeshun force-pushed the add-sphand_ros branch from 04cbcd4 to b886cf8 Compare February 8, 2020 10:05

pazeshun changed the title ~~Add sphand ros~~ [WIP] Add sphand ros Feb 8, 2020

pazeshun force-pushed the add-sphand_ros branch from b886cf8 to 36e4188 Compare February 11, 2020 07:38

knorth55 added this to the 4.3.0 milestone Apr 1, 2020

pazeshun force-pushed the add-sphand_ros branch from 811e482 to 2bb655a Compare April 28, 2020 05:55

pazeshun changed the title ~~[WIP] Add sphand ros~~ Add sphand ros Apr 28, 2020

pazeshun closed this May 8, 2020

pazeshun reopened this May 8, 2020

pazeshun changed the title ~~Add sphand ros~~ [WIP] Add sphand ros May 9, 2020

pazeshun added a commit to pazeshun/jsk_apc that referenced this pull request May 9, 2020

[travis] Comment out allow_failures to reduce cache size

98c558f

See start-jsk#2707 (comment)

wkentaro mentioned this pull request May 12, 2020

Stop downloading unused model checkpoints #2712

Merged

pazeshun added 2 commits May 13, 2020 15:48

Copy sphand_ros from https://github.com/pazeshun/sphand_ros

d4c1a49

Copy vl53l0x_mraa_ros from https://github.com/pazeshun/vl53l0x_mraa_ros

e3886ac

pazeshun added 16 commits May 13, 2020 15:48

[sphand_ros] Mention --ignore-src in README to reduce disk usage

c1d4e52

[sphand_ros] Add launch recording rosbag

14ea803

[sphand_ros] Fix installing to follow http://docs.ros.org/melodic/api…

9976ccf

…/catkin/html/howto/format1/index.html

[sphand_ros] Add note about catkin build in README

1976e3e

[sphand_ros] Use .rosinstall of other UP Boards as building jsk_recog…

41c48dd

…nition is too heavy

[sphand_ros] ws_jsk_apc -> kinetic

59e4f60

[sphand_ros] Publish distance combined with ToF output

b8e9758

Far part: ToF distance, middle & close part: distance generated from intensity

[sphand_ros] Fix to use ToF in combined distance when no obj is found…

ef865b8

… after calibration

[sphand_ros] Add euslisp interface of combined distance

dee66a9

[sphand_ros] Don't use compressedDepth from UP Board

6bffeaf

When subscribing compressedDepth from kinetic UP Board, hz of image_raw drops. This is because usage of CPU processing camera becomes 100%. Even on baxter-c1, hz of image_raw drops if png_level is 9 or 8.

[sphand_ros] Record compressedDepth as it is now not relayed from UP …

dee0d45

…Board

[sphand_ros] Add euslisp interface to get prop_const

dbb81b8

[sphand_ros] prop_const -> reflectance_param

3a9cad6

[sphand_ros] Speed up dynamixel loop to 40Hz

8fdc1e3

[sphand_ros] Add reflectance_param to format_printed_prox_to_csv

1ef2207

Add sphand_ros packages to travis indigo test

6f7e1d9

pazeshun force-pushed the add-sphand_ros branch 3 times, most recently from c899bf5 to dcc34a4 Compare May 14, 2020 01:55

pazeshun force-pushed the add-sphand_ros branch from dcc34a4 to 40c5cea Compare May 14, 2020 04:31

pazeshun changed the title ~~[WIP] Add sphand ros~~ Add sphand ros May 14, 2020

knorth55 self-requested a review May 14, 2020 15:49

knorth55 approved these changes May 14, 2020

View reviewed changes

knorth55 merged commit 129f6cc into start-jsk:master May 14, 2020

pazeshun deleted the add-sphand_ros branch May 14, 2020 23:19

pazeshun mentioned this pull request Jul 17, 2020

add selective_dualarm_grasping #2720

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Add sphand ros #2707

Add sphand ros #2707

pazeshun commented Jan 23, 2020

wkentaro commented Jan 23, 2020 •

edited

Loading

pazeshun commented Jan 24, 2020

wkentaro commented Jan 24, 2020 via email

pazeshun commented May 9, 2020 •

edited

Loading

pazeshun commented May 9, 2020 •

edited

Loading

pazeshun commented May 9, 2020

knorth55 commented May 9, 2020

pazeshun commented May 9, 2020

pazeshun commented May 9, 2020

wkentaro commented May 9, 2020

wkentaro commented May 9, 2020

pazeshun commented May 11, 2020 •

edited

Loading

wkentaro commented May 12, 2020

wkentaro commented May 12, 2020

pazeshun commented May 12, 2020 •

edited

Loading

wkentaro commented May 12, 2020

pazeshun commented May 14, 2020

knorth55 commented May 14, 2020

pazeshun commented May 14, 2020

Add sphand ros #2707

Add sphand ros #2707

Conversation

pazeshun commented Jan 23, 2020

wkentaro commented Jan 23, 2020 • edited Loading

pazeshun commented Jan 24, 2020

wkentaro commented Jan 24, 2020 via email

pazeshun commented May 9, 2020 • edited Loading

pazeshun commented May 9, 2020 • edited Loading

pazeshun commented May 9, 2020

knorth55 commented May 9, 2020

pazeshun commented May 9, 2020

pazeshun commented May 9, 2020

wkentaro commented May 9, 2020

wkentaro commented May 9, 2020

pazeshun commented May 11, 2020 • edited Loading

wkentaro commented May 12, 2020

wkentaro commented May 12, 2020

pazeshun commented May 12, 2020 • edited Loading

wkentaro commented May 12, 2020

pazeshun commented May 14, 2020

knorth55 commented May 14, 2020

pazeshun commented May 14, 2020

wkentaro commented Jan 23, 2020 •

edited

Loading

pazeshun commented May 9, 2020 •

edited

Loading

pazeshun commented May 9, 2020 •

edited

Loading

pazeshun commented May 11, 2020 •

edited

Loading

pazeshun commented May 12, 2020 •

edited

Loading