Implement Domain Randomization

[diegoferigo]

Quick TODO list of quantities to randomize:

Physics Engine

• Gravity
• (Lights)

Model

• Inertial parameters (inertia matrix, mass)
• Joint friction
• Contacts parameters
• External forces
• Link geometry

[diegoferigo]

From the SDF side, we could make use of the new support of custom elements and attributes introduced in osrf/sdf_tutorials#32.

[diegoferigo]

For what concerns the model, probably the most straightforward way to randomize its dynamics is:

1. Deserialize the sdf / urdf model
2. Modify the DOM
3. Serialize to a string / file
4. Insert a new simulated model from the modified file

Considering the structure of the project, we can decide whether to use C++ or Python. C++ will be faster and we can use SDFormat and the resources provided by <random>. From Python, instead, we could use the default XML parser and numpy.

I tried to develop a first C++ prototype, but it turns out that SDFormat is not really suitable for this task. It provides a great flexibility for what concerns deserialization, but it has severe serialization limits. More specifically, all the helpers like sdf::Model, sdf::Joint, etc do not store the values changed with the set* methods in the DOM. This means that working on raw Elements is necessary.

At this point, I'm considering to switch to Python. The default xml module (and many other alternatives) support OOTB XPath, that provides a very compact way to access fields.

The only drawback is that in Python we should convert URDF models to SDF beforehand, since the randomizer Python class will work on just one of the two formats (it will be SDF). This choice could lead to small model differences particularly when dealing with fixed joints, and users should be warned to be careful. I'm not really concerned about the speed of the process, most likely the rollout (and model insertion for big models) will be at least one or two order of magnitude longer than the deserialization / serialization process.

cc @traversaro

Resources:

• Support for dynamics modding and randomization (DynamicsModder) openai/mujoco-py#148
• Support for dynamics randomization ryanjulian/rllab#61
• Dynamics randomization for MuJoCo rlworkgroup/garage#51

[traversaro]

Tagging a few model-based control guys here, as domain randomization can also be really useful to test the robustness of model-based control algorithms to model uncertainties: @robotology/iit-dynamic-interaction-control .

[DanielePucci]

@diegoferigo @traversaro

Inertial parameters (inertia matrix, mass)
Joint friction
Contacts parameters
External forces
Link geometry

I do not know if we can find a n easy way (like randomize the densities) to link link geometry and inertial parameters

[traversaro]

@diegoferigo @traversaro

Inertial parameters (inertia matrix, mass)
Joint friction
Contacts parameters
External forces
Link geometry

I do not know if we can find a n easy way (like randomize the densities) to link link geometry and inertial parameters

From the point of view of URDF and SDF representation, inertia parameters and geometry are completely decoupled parameters. An alternative representation would be to use geometry + density, and randomize that, that would require some code to automatically compute the inertial parameters given the geometry and the density, see these related issues:

• Automatically compute inertial parameters from link shapes  idyntree#462
• Allow a subset of <geometry> instead of <inertia> gazebosim/sdformat#176

Note that for primitive shapes the amount of parameter to randomize over would probably remain similar (10 for inertial parameters vs 7/8 for primitive shapes) while for meshes the parameters that could be changed are much more (potentially every point in the mesh), unless you fix the shape and assume that you randomize over some kind of deformation (such as scaling, rotation, translation) that would limit the number of parameters.

[diegoferigo]

Thanks @traversaro for the insight. I think that the best we can achieve is exactly what you wrote:

unless you fix the shape and assume that you randomize over some kind of deformation (such as scaling, rotation, translation)

Actually, already achieving a proper meshes scaling would be an excellent achievement. We have to remember that implementing this feature means that also the corresponding collision should be scaled accordingly, and not always the geometry is the same (e.g. when simplified collisions are used). Plus, there could be multiple visual and collision elements associated to a link.

I propose to start with simple parameters randomization, and then discuss if also geometry is necessary. From my early tests with the trained policies stored in the xbpeng/DeepMimic repository, they were already quite robust to geometry changes without the need to randomize link geometry during training.