Support for FS100 controllers #227

gavanderhoorn · 2024-03-15T11:49:14Z

This is for #19 (and a resubmit of #226 from a branch here on the main repository).

A draft currently, but first steps towards FS100 support.

gavanderhoorn · 2024-03-15T11:49:56Z

CI will fail as we currently have no release of M+ libmicroros for FS100.

We'll deal with that later.

Edit: and of course because of #212 (comment).

gavanderhoorn · 2024-03-15T14:12:33Z

Something to decide: the current work-around/hack mentioned in #19 (comment) only works for Agents running on little-endian machines.

I expect that to be the typical setup for use with MotoROS2 (ie: amd64 based ROS PC running the Agent), but I could see someone trying to use an RPi or similar aarch64 or aarch32 platform to delegate the Agent to.

That would not work for the FS100.

Personally, I feel we could document this, but:

the asymmetry with the other controllers is not nice (™)
MotoROS1 had no such limitation
it's likely such a note in our documentation will be overlooked, leading to users posting issues and/or opening discussions about their installation not working

Given the fact the FS100 is a legacy platform at this point this may all be acceptable, but we need to decide how we want to approach this.

ted-miller · 2024-03-18T15:46:20Z

There's only AUTO, CYCLE and STEP. If I set it to AUTO, SOUT#035 becomes high, but as soon as I switch the TP to Remote, cycle gets reset to CYCLE.
Any ideas?

Go to Setup > Operate Cond. Then change the CYCLE SWITCH IN REMOTE MODE setting.

Given the fact the FS100 is a legacy platform at this point this may all be acceptable, but we need to decide how we want to approach this

I'm of this opinion. Just document the limitation and move on.

There will be a limited set of users using MR2 on FS100 and a much smaller set of those using a bigendian cpu.

ted-miller · 2024-03-25T12:27:34Z

@gavanderhoorn Beyond the potential endianness issue, what concerns do you have? Given the success of the previous testing, I'm not really sure why this is a 'draft' PR.

My biggest concern is that we haven't validated on a non-dummy controller. I'm going to try to scrounge up one of the old educational carts.

Now that the rebase is complete, I'm happy to test all topics/services again. Please provide a binary of the libmicroros. (I'll also validate the VS settings for building MR2.)

ted-miller · 2024-03-25T13:57:46Z

I've located an FS100 with an arm. It should be here tomorrow afternoon.

gavanderhoorn · 2024-03-25T17:35:07Z

It's partly a draft because I haven't actually moved anything, neither real hw nor virtual.

It also lacks documentation updates, and I still need to verify which parameters need to be set to which values.

ted-miller · 2024-03-25T19:56:56Z

I still need to verify which parameters need to be set to which values.

From my personal notes regarding the FS100:

I don't think that S2C1402 ~~exists~~ (EDIT) is used.

Everything else should be covered by our existing setup-instructions.

ted-miller · 2024-03-27T20:32:52Z

@gavanderhoorn
Can you provide the latest libmicroros or .out?
Or would you prefer I validate using <outdated_binary> ?

gavanderhoorn · 2024-03-28T11:35:39Z

Or would you prefer I validate using <outdated_binary> ?

that wouldn't work, as it has DUMMY_SERVO_MODE enabled.

I'll build a fresh alpha.

gavanderhoorn · 2024-03-28T12:48:10Z

@ted-miller: here you go: <outdated_binary>.

ted-miller · 2024-03-28T19:20:43Z

I'll do some more thorough testing next week. But doing a quick test, I was able to get the robot to move with the point-queue service.

Other quick notes:

effort data in joint_states is garbage data
velocity fields in joint_states are always zero
The feedback position is 'laggy'. If I jog the robot, the position immediately updates. But when I release the jogging button, the position coasts for about a second before coming to a stop. I see this same behavior on the pendant, so it's not MR2. But it is something different with the FS100.

gavanderhoorn · 2024-03-30T12:39:26Z

I'll do some more thorough testing next week. But doing a quick test, I was able to get the robot to move with the point-queue service.

👍

Other quick notes:

effort data in joint_states is garbage data

hm. Is this a known issue? I'm running FS4.00 here.

velocity fields in joint_states are always zero

Probably due to the M-register for velocity feedback not being supported on FS100? At least that's what logged on my controller when it boots up MotoROS2.

The feedback position is 'laggy'. If I jog the robot, the position immediately updates. But when I release the jogging button, the position coasts for about a second before coming to a stop. I see this same behavior on the pendant, so it's not MR2. But it is something different with the FS100.

I believe I've seen something similar, but wasn't sure whether it was caused by the fact I don't have any manipulator connected.

ted-miller · 2024-04-01T15:16:01Z

effort data in joint_states is garbage data

hm. Is this a known issue? I'm running FS4.00 here.

No. The M+ manual says the API should work. But I looked at MR1 code and see that Ros_CtrlGroup_GetTorque never actually gets called anywhere. So, I don't have any known examples of the API working on the FS100.

velocity fields in joint_states are always zero

Probably due to the M-register for velocity feedback not being supported on FS100? At least that's what logged on my controller when it boots up MotoROS2.

Yup. That'll do it.

I guess we need to convert Ros_CtrlGroup_GetFBServoSpeed to use mpSvsGetVelTrqFb. But that's assuming mpSvsGetVelTrqFb actually works... (It's what is being used for effort data.)

gavanderhoorn · 2024-04-01T15:34:25Z

I guess we need to convert Ros_CtrlGroup_GetFBServoSpeed to use mpSvsGetVelTrqFb. But that's assuming mpSvsGetVelTrqFb actually works... (It's what is being used for effort data.)

I believe we have an open issue on the internal tracker (#547) where in the end I commented:

According to yourself (here), M-registers are better ;)

I was confused by the comments in that issue as you also suggested changing to mpSvsGetVelTrqFb(..), but that was/is already used. At least in MotoROS1.

ted-miller · 2024-04-01T20:57:36Z

(Not sure if this note is a repost. I know I had something typed up, but I think I closed the window prematurely.)

I made a small app that just polls mpSvsGetVelTrqFb continuously.

For velocity:
With servos off and the brakes engaged, I'm getting random +/- 1248 on one or more axes every few cycles. But while jogging, the numbers look reasonable.

For effort:
I don't know what to think of the numbers being displayed. I need to do more testing.
The pendant only displays percent of rated max torque, so I don't have a good correlation between this and Nm.

But it seems like there's a lot of fluctuation on multiple axes for even a very simple motion, such as jogging the S axis. Also, the resting-position seems unusual. If I jog S+ the value spikes. When motion is complete, the value settles around 15000. But then if I jog S- and stop, the value settles around -10000. I understand the sign inversion, but I don't get why the absolute value is significantly different.

I tested a motion with the FJT script. The motion worked, but the final position was outside of tolerance on both axes that I moved by 0.012 rad.

I'm wondering if this is related to the 'laggy' response I get when jogging.

In the error message about the trajectory being outside tolerance, the joint names were non-ascii characters.

motoros2/src/ActionServer_FJT.c

Lines 571 to 573 in 7f95156

    
           snprintf(formatBuffer, 64, " [%s: %.4f deviation]", 
        
               feedback_FollowJointTrajectory.feedback.joint_names.data[axis], 
        
               violators[axis]);

This has me questioning if something is overflowing an array. Or perhaps this isn't working properly.

gavanderhoorn · 2024-04-02T16:05:30Z

Hm. Sounds like something isn't working.

re: velocities & torques: could this be rate-dependent?

gavanderhoorn · 2024-04-16T09:52:50Z

@ted-miller: I've added dd13f53 which (im)ports the MotoROS1 version of Ros_CtrlGroup_GetFBServoSpeed(..) to use mpSvsGetVelTrqFb(..) instead of the M-registers.

gavanderhoorn · 2024-04-25T11:45:57Z

@ted-miller: I just remembered I haven't actually built this branch after I added d4fb2fd.

gavanderhoorn · 2024-04-25T11:51:06Z

@ted-miller wrote:

In the error message about the trajectory being outside tolerance, the joint names were non-ascii characters.

motoros2/src/ActionServer_FJT.c

Lines 571 to 573 in 7f95156

snprintf(formatBuffer, 64, " [%s: %.4f deviation]",

feedback_FollowJointTrajectory.feedback.joint_names.data[axis],

violators[axis]);

Should that be feedback_FollowJointTrajectory.feedback.joint_names.data[axis].data? Technically feedback_FollowJointTrajectory.feedback.joint_names.data[axis] is a rosidl_runtime_c__String, not a const char*.

This has me questioning if something is overflowing an array. Or perhaps this isn't working properly.

Technically I believe we could use rosidl_runtime_c__String__copy(..) instead of 'abusing' _assign(..), but it wouldn't really make much difference IIUC.

ted-miller · 2024-04-25T12:29:59Z

I just remembered I haven't actually built this branch after I added d4fb2fd.

I was able to build this yesterday. Though, I did have to shrink SIZEOF_BUFFER_FJT_GOAL yet again to make it load without 1020 [6]. (Admittedly, I didn't strip the binary after building and it was 300-400k larger than alpha50. So perhaps that would have made a difference.)

Also, there were a bunch of annoying warnings about an unneeded const identifier. I'm thinking that we should remove the -Wall from the compiler arguments for FS100. Presumably, any "important" warnings would also be raised by the DX/YRC compilers.

Should that be feedback_FollowJointTrajectory.feedback.joint_names.data[axis].data

Technically, yes. But since the string pointer is the first entry in that structure, it should be ok.

gavanderhoorn · 2024-04-25T12:55:48Z

Though, I did have to shrink SIZEOF_BUFFER_FJT_GOAL yet again to make it load without 1020 [6]. (Admittedly, I didn't strip the binary after building and it was 300-400k larger than alpha50. So perhaps that would have made a difference.)

that should indeed not really be needed.

I may be doing something 'different' when building it here. I'll take a look at my settings.

Also, there were a bunch of annoying warnings about an unneeded const identifier. I'm thinking that we should remove the -Wall from the compiler arguments for FS100. Presumably, any "important" warnings would also be raised by the DX/YRC compilers.

I don't remember anything about const modifier (I guess?), I do have warnings about return types, for which I added -Wno-return-type. But only for the FS100 builds.

Should that be feedback_FollowJointTrajectory.feedback.joint_names.data[axis].data

Technically, yes. But since the string pointer is the first entry in that structure, it should be ok.

technically, yes.

But after I changed that when working on #233, the issues with doubles and Ros_Debug_BroadcastMsg(..) stopped.

gavanderhoorn · 2024-05-22T19:10:05Z

@ted-miller: have you had any time to take a further look at this?

ted-miller · 2024-05-22T20:46:44Z

No I haven't.
I'm definitely not going to be able to get to it until at least the end of the month.

Sanchitkedia · 2024-06-20T22:07:20Z

@gavanderhoorn I have a Yaskawa Motoman SIA20F with an FS100 controller and can try to run this on my end. Could you help me with the necessary files and instructions on how to get started?

Humble only for now.

Update use of constants and add conditionals to account for this.

So update relevant constants to account for this.

So gate it.

So introduce a custom implementation for FS100 and call that instead.

Gate it on controller platforms which are known to support it (similar to how we do this in MotoROS1).

Limit use to controller series known to have it and have FS100 just fail the function completely.

`mpFstat(..)` doesn't appear to work reliably, or at least not for files on USB sticks and SRAM drive (not sure about main FS). Use (a custom implementation of) `tell(..)` to determine actual file size and use it to override `stat::st_size`.

Previous conditional was too conservative. Add FS100 to supported controllers.

Otherwise the binary won't load, and this should still be sufficient for the maximum size of the goals we want/need to support.

It doesn't support the M-register approach, so make use of mpSvsGetvelTrqFb(..) instead.

Remove all other controllers + M+ `libmicroros` tuples.

gavanderhoorn · 2024-08-19T15:16:18Z

@jfernandez37: I started work to try and reproduce what you've reported last week, but haven't been able so far. My FS100 isn't setup with physical hw, which may be why. I'll try some more ways to 'break' it.

gavanderhoorn · 2024-08-27T08:00:35Z

@jfernandez37: I'm starting to suspect the sleep/timer implementation on FS100 platforms (not the base OS, but MotoROS2's wrapper/use of it), but haven't been able to pin-point anything yet.

Have you observed any more issues in the meantime?

And thank you for testing the alpha. This is valuable.

jfernandez37 · 2024-08-27T15:31:30Z

No, we haven't noticed any more issues with it. Thank you for your help.

gavanderhoorn · 2024-08-28T13:05:20Z

#296 fixes at least one case of "strange chars" in the debug log of the FS100 build I'm testing.

gavanderhoorn · 2024-08-29T14:38:53Z

Just to provide an update: @jfernandez37: due to the way my FS100 is configured I can't seem to reproduce the problem(s) you encountered.

I've tried to break things by executing many different trajectories all over the (combined) joint position ranges of all joints without running into any issues so far. I've identified a couple of other issues with the port which we'll be addressing later, but we're going to need a different setup to debug.

I believe @ted-miller may have an opportunity soon to assist.

ted-miller · 2024-09-04T14:00:43Z

I have not been able to reproduce this issue either. Using a small 6-axis arm, I'm able to execute any trajectories repeatedly.

I also tried reconfiguring as a 7-axis SIA (with a virtual arm). Still not able to cause this issue.

It seems the problem is with the micro-ros agent. Please ensure that the latest humble docker image is being used.

Other oddities:

I didn't initially follow the setup instructions properly and forgot to set some of the parameters. I got fatal alarm [10] (couldn't create publisher) until I set these and rebooted.
I don't understand why this alarm occurred.
When setup as a virtual arm, I got a whole lot of Failed to get pulse position (command). But the next day, that went away. I don't think I changed anything. But I can't explain this. (Probably a moot point, since I was configured for DUMMY mode.)
start_traj_mode didn't work in DUMMY mode either. It wouldn't enable the servos automatically. I had to manually turn them on and then switch to REMOTE mode. (Again, moot since this is DUMMY)

gavanderhoorn · 2024-09-06T12:31:56Z

@ted-miller wrote:

I have not been able to reproduce this issue either. Using a small 6-axis arm, I'm able to execute any trajectories repeatedly.

did you encounter any other issues while executing those trajectories?

This was with actual HW right?

I also tried reconfiguring as a 7-axis SIA (with a virtual arm). Still not able to cause this issue.

It seems the problem is with the micro-ros agent. Please ensure that the latest humble docker image is being used.

Just making sure (again? perhaps) @jfernandez37: you are running this on an amd64 desktop/laptop and are indeed using the latest Agent Docker image?

Could (probably is) something else, but just to rule things out.

One possible other cause of the error mentioned in #227 (comment) would be a buffer overflow (on the MR2 side) and/or memory corruption.

Other oddities:

I didn't initially follow the setup instructions properly and forgot to set some of the parameters. I got fatal alarm [10] (couldn't create publisher) until I set these and rebooted.
I don't understand why this alarm occurred.

Yes, I remember running into this myself. I'm also unsure what causes that. I've considered it a red herring for now.

When setup as a virtual arm, I got a whole lot of Failed to get pulse position (command). But the next day, that went away. I don't think I changed anything. But I can't explain this. (Probably a moot point, since I was configured for DUMMY mode.)

was this with DUMMY_SERVO_MODE on the MR2 side, or without?

start_traj_mode didn't work in DUMMY mode either. It wouldn't enable the servos automatically. I had to manually turn them on and then switch to REMOTE mode. (Again, moot since this is DUMMY)

Also ran into this. Had to use the same work-around.

thanks for testing.

At this point I'll see if can create a heavily instrumented version of MotoROS2 which logs just about everything around the trajectory processing, re-ordering and execution queues.

jfernandez37 · 2024-09-06T13:23:12Z

Yes. We are running the most recent version of the docker image and on an am64 laptop.

gavanderhoorn · 2024-09-06T13:26:26Z

Apologies, that was supposed to read amd64 (ie: x86_64). But thank you for confirming.

jfernandez37 · 2024-09-06T13:31:14Z

I just checked the laptop that we have been using, and it has an Intel cpu

gavanderhoorn · 2024-09-06T13:34:35Z

I just checked the laptop that we have been using, and it has an Intel cpu

that's ok. amd64 is an architecture, not a brand. As long as it's not an ARM based processor (such as in an RPi), or something more exotic. The current version of MotoROS2 for FS100 is only compatible with x86_64.

ted-miller · 2024-09-06T13:34:46Z

did you encounter any other issues while executing those trajectories?

I got the "outside tolerance" failure that was mentioned previously. I think there's something "laggy" with the FS100.

But the arm executed the motion just fine.

This was with actual HW right?

The 6 axis tests were with a physical arm. The 7 axis tests were dummy-mode.

When setup as a virtual arm, I got a whole lot of Failed to get pulse position (command). But the next day, that went away. I don't think I changed anything. But I can't explain this. (Probably a moot point, since I was configured for DUMMY mode.)

was this with DUMMY_SERVO_MODE on the MR2 side, or without?

Yes, DUMMY SERVO MODE was enabled when compiling MR2. Ros_CtrlGroup_GetPulsePosCmd doesn't look at the DUMMY flag because it's not trying to read feedback position.

This comment was marked as outdated.

Sign in to view

gavanderhoorn mentioned this pull request Mar 15, 2024

Add support for FS100 #19

Open

18 tasks

gavanderhoorn force-pushed the fs100_support branch from 1989c13 to 7f95156 Compare March 15, 2024 13:37

gavanderhoorn marked this pull request as ready for review March 28, 2024 14:33

gavanderhoorn force-pushed the fs100_support branch 2 times, most recently from dd13f53 to d4fb2fd Compare April 16, 2024 10:09

gavanderhoorn added 17 commits August 13, 2024 14:08

Import FS parameter extraction lib

9d2ffe3

Import MotoROS1 FS100 mpBuilder setting files

6bbc8e2

mpBuilder: rename for Humble on FS100

1892994

mpBuilder: update settings for MotoROS2

855b83c

Extend VS project with FS100 settings

d207312

Humble only for now.

Ignore FS100 libmicroros

fac61f5

Humble only for now.

Max 4 motion groups on FS100

4109676

Update use of constants and add conditionals to account for this.

Max 16 tool files on FS100

be52a42

So update relevant constants to account for this.

No mpGetToolNo(..) available on FS100

060d10a

So gate it.

No mpGetCalendar(..) available on FS100

d2d2055

So introduce a custom implementation for FS100 and call that instead.

No mpApplicationInfoNotify(..) available on FS100

7959fe8

Gate it on controller platforms which are known to support it (similar to how we do this in MotoROS1).

No mpGetEncoderTemp(..) available on FS100

2357c7e

Limit use to controller series known to have it and have FS100 just fail the function completely.

cfg: correct file size on FS100

5779b38

`mpFstat(..)` doesn't appear to work reliably, or at least not for files on USB sticks and SRAM drive (not sure about main FS). Use (a custom implementation of) `tell(..)` to determine actual file size and use it to override `stat::st_size`.

Use SRAM drive for FS100 cfg too

13023c0

Previous conditional was too conservative. Add FS100 to supported controllers.

Make FJT goal static buffer smaller on FS100

646d07b

Otherwise the binary won't load, and this should still be sufficient for the maximum size of the goals we want/need to support.

(Im)port MotoROS1 GetFBServoSpeed(..) for FS100

2806e44

It doesn't support the M-register approach, so make use of mpSvsGetvelTrqFb(..) instead.

ci: temp: tweak CI for FS100 draft

da05b1c

Remove all other controllers + M+ `libmicroros` tuples.

gavanderhoorn force-pushed the fs100_support branch from f0bb782 to da05b1c Compare August 13, 2024 12:11

Support for FS100 controllers #227

Are you sure you want to change the base?

Support for FS100 controllers #227

Conversation

gavanderhoorn commented Mar 15, 2024

gavanderhoorn commented Mar 15, 2024 • edited Loading

This comment was marked as outdated.

gavanderhoorn commented Mar 15, 2024

ted-miller commented Mar 18, 2024

ted-miller commented Mar 25, 2024 • edited Loading

ted-miller commented Mar 25, 2024

gavanderhoorn commented Mar 25, 2024

ted-miller commented Mar 25, 2024 • edited Loading

ted-miller commented Mar 27, 2024 • edited by gavanderhoorn Loading

gavanderhoorn commented Mar 28, 2024 • edited Loading

gavanderhoorn commented Mar 28, 2024 • edited Loading

ted-miller commented Mar 28, 2024

gavanderhoorn commented Mar 30, 2024

ted-miller commented Apr 1, 2024

gavanderhoorn commented Apr 1, 2024

ted-miller commented Apr 1, 2024

gavanderhoorn commented Apr 2, 2024

gavanderhoorn commented Apr 16, 2024 • edited Loading

gavanderhoorn commented Apr 25, 2024

gavanderhoorn commented Apr 25, 2024

ted-miller commented Apr 25, 2024

gavanderhoorn commented Apr 25, 2024 • edited Loading

gavanderhoorn commented May 22, 2024

ted-miller commented May 22, 2024

Sanchitkedia commented Jun 20, 2024

gavanderhoorn commented Aug 19, 2024

gavanderhoorn commented Aug 27, 2024

jfernandez37 commented Aug 27, 2024

gavanderhoorn commented Aug 28, 2024

gavanderhoorn commented Aug 29, 2024

ted-miller commented Sep 4, 2024

gavanderhoorn commented Sep 6, 2024 • edited Loading

jfernandez37 commented Sep 6, 2024

gavanderhoorn commented Sep 6, 2024

jfernandez37 commented Sep 6, 2024

gavanderhoorn commented Sep 6, 2024 • edited Loading

ted-miller commented Sep 6, 2024

gavanderhoorn commented Mar 15, 2024 •

edited

Loading

ted-miller commented Mar 25, 2024 •

edited

Loading

ted-miller commented Mar 25, 2024 •

edited

Loading

ted-miller commented Mar 27, 2024 •

edited by gavanderhoorn

Loading

gavanderhoorn commented Mar 28, 2024 •

edited

Loading

gavanderhoorn commented Mar 28, 2024 •

edited

Loading

gavanderhoorn commented Apr 16, 2024 •

edited

Loading

gavanderhoorn commented Apr 25, 2024 •

edited

Loading

gavanderhoorn commented Sep 6, 2024 •

edited

Loading

gavanderhoorn commented Sep 6, 2024 •

edited

Loading