Paring of shared and sos_variable

[BoPeng]

The following workflow works

[1: shared = 'data']
data = [1, 2]

[2]
input: for_each = dict(par=data)
print(par)

because shared expose variables to the context of the steps that depends on step 1 in a forward workflow. In contrast, the following workflow

[data: shared = 'data']
data = [1, 2]

[default]
input: for_each = dict(par=data)
print(par)

will not work because default does not depend on data.

Now the tricky part is that

[1: shared = 'data']
data = [1, 2]

[2]
input: '1.txt', for_each = dict(par=data)
print(par)

will also not work because 2 does not depend on 1 on the surface so data is not available when 2 is executed.

To make things clear, I propose that we require, at least document explicit use of sos_variable() for any share. That is to say, we not only use sos_variable to fix the above example,

[1: shared = 'data']
data = [1, 2]

[2]
depends: sos_variable('data')
input: '1.txt', for_each = dict(par=data)
print(par)

but also document the use of sos_variable in the working example:

[1: shared = 'data']
data = [1, 2]

[2]
depends: sos_variable('data')
input: for_each = dict(par=data)
print(par)

In this way at least we do not have to explain why data sometimes exists and sometimes does not exist in examples of this ticket.

[gaow]

Thanks for the clarification. I agree with what you proposed above but I'd suggest we keep a pointer to this ticket for a note when sos_variable can be omitted under very restricted settings.

[BoPeng]

The first example works because of our particular implementation but it is quite confusing that adding 1.txt will suddenly invalidate data. Instead of explaining what is actually happening to users, I would rather say as a simple rule that sos_variable is required for any variable that is not defined in the global section (and in step itself), because the latter is conceptually clearer.

[BoPeng]

https://vatlab.github.io/sos-docs/doc/user_guide/step_variables.html#Variablessharedfromothersteps is updated to describe pairing of shared and sos_variable is the only way to share variables between steps.

[BoPeng]

I have enforced the use of sos_variable so the first example will no longer work. This removes a lot of guess work as what should be passed to a step and will make SoS run faster. I mean, previous,

1. we parse input and say: these variable might be needed to be passed from outside
2. we pass variables to step, but ignore variables that do not exist and assume they will be available somehow in the step.

Right now, only variables in sos_variable will be passed as external variable to a step.