feat: support device profile operation

[reubenmiller]

Proposed changes

An initial proof of concept to demonstrate one way of processing a device profile operations on the tedge /cmd/ topic. This PR can be taken over by someone else to work on Rust implementation.

The current example device profile workflow uses a complex set of actions (to push the limits of this feature), the following highlights the nested workflows which are called through-out the process:

• device_profile
  • firmware_update
    • restart (device)
    • software_update (to update thin-edge.io components)
  • software_update (install other packages)
  • config_update

Types of changes

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Improvement (general improvements like code refactoring that doesn't explicitly fix a bug or add any new functionality)
• Documentation Update (if none of the other choices apply)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Paste Link to the issue

• support device profile operation #2921

Checklist

• I have read the CONTRIBUTING doc
• I have signed the CLA (in all commits with git commit -s)
• I ran cargo fmt as mentioned in CODING_GUIDELINES
• I used cargo clippy as mentioned in CODING_GUIDELINES
• I have added tests that prove my fix is effective or that my feature works
• I have added necessary documentation (if appropriate)

Further comments

[didier-wenzek]

From where are coming these name and version?

I guess from a previous successful device profile update. But what if meantime, some configuration files have been updated?

[albinsuresh]

Yes, the last applied device profile or the device profile with which the device was initially flashed in the factory. If such a factory profile is unavailable, that's when the agent can declare the empty JSON payload.

[reubenmiller]

I guess from a previous successful device profile update. But what if meantime, some configuration files have been updated?

The question of "has anything changed since applying the device profile" is a tricky one, as I don't think there is one-way here, as there are multiple ways that the device profile state could get "out of sync" (e.g. sending individual firmware/software/config operations, user running manual commands via ssh etc.)

Though I don't think that device profiles are always used to control the whole on-device setup (as this usually leads to have an unwieldy amount of device profiles to manage, especially if a single device only needs 1 extra software package or configuration)

Below are some options (not exhaustive) that thin-edge.io could do:

Option 1: Do nothing

thin-edge.io doesn't do anything, and it is up to the device maintainer to control how the device is managed

Option 2: Flag profile as "dirty" after other operations

We introduce a "dirty" flag to the device profile current state (e.g. in the /twin setting), which is set anytime a single firmware/software/configuration operation is sent after a device profile is applied. The "dirty" flag would work similar to git, where it just indicates that something has changed (but does not say what).

There could also be a possible option made to even disable individual cloud operations, e.g. only accept cloud device profile operations, and reject cloud based firmware/software/configuration operations.

Option 3: Add interface which can be executed to check if the device profile is still

Similar to the sm-plugin "list" command, a device profile handler could be used to indicate whether a device profile is still in the desired state (e.g. has not been affected by individual firmware/software/configuration operations). This would give a lot of control over the user deciding what is correct to do here (which could be nice).

[github-actions]

Robot Results

✅ Passed	❌ Failed	⏭️ Skipped	Total	Pass %	⏱️ Duration
495	0	2	495	100	1h22m36.179478999s

[didier-wenzek]

I'm not happy with the current proposal, even if I appreciate the intent.

Yes, it would be good to liberate the user from defining the looping logic, but not at the price of a fragile and complicated logic scattered over all the states.

I would address the problem starting from the tests/device_profile.sh script and tests/device_profile.toml workflow. Can the script be built-in, producing the next operation at each invocation? The user would have then only to provide a preparation script that groups and sequences the operations.

[didier-wenzek]

The current proposal is far better but still a bit too adhoc.

• The iterator action implicitly works on a list named operations yielding operation in turn. Why not iterating over any list yielding the items of this list in turn. For example action = "iterate @next_operation ${.operations}" where the user can choose for the list to iterate over as well as the current item name.
• Introducing a specific on_next handler makes impossible to substitute a script for iterate action.

[albinsuresh]

The iterator action implicitly works on a list named operations yielding operation in turn. Why not iterating over any list yielding the items of this list in turn. For example action = "iterate @next_operation ${.operations}" where the user can choose for the list to iterate over as well as the current item name.

Yeah, I can make this a generic iterator that takes any array as input, with the @next value generated on each iteration.

Introducing a specific on_next handler makes impossible to substitute a script for iterate action.

I didn't fully get that point. If a user overrides this action with a script, then they don't need to use the on_next handle at all, but can use any number of target states using either the on_stdout directive or on_exit.n directives, right? So, I didn't really understand how this special on_next handle exclusive to the iterator action can hinder any overriding possibilities.

[didier-wenzek]

I didn't really understand how this special on_next handle exclusive to the iterator action can hinder any overriding possibilities.

You are correct, my wording was a bit too strong. This is very specific but doesn't prevent the user using another approach.

[reubenmiller]

Though the idea of reusing the on_exit.n properties is to also reduce the API, so we're not adding yet another property (which is only used in a very specific use-case).

[gligorisaev]

LGTM

[didier-wenzek]

Approved. Great achievement!

[didier-wenzek]

Just realised that I missed the "skipping" logic in the iterator impl. Will do.

This can be done is a follow-up PR.