Refactor preprocessing

[APJansen]

This PR simplifies the preprocessing layer, improving readability without changing any results. I think these changes are harmless and uncontroversial and can easily be merged.

Additional changes proposed

However my goal was a slightly bigger change, that may cause issues so I put the last commit in a different branch.
What this does is unite all the preprocessing factors into a single alpha vector and a single beta vector, rather than having tons of scalars. This layer is the only place in the model where flavors are treated individually. Each parameter has flavor-dependent min and max values, that go into both the initializer and a constraint, but this can be done as a vector as well.

The reason this may be controversial is that it doesn't allow setting weights as trainable on a flavor-by-flavor basis, only all alphas and/or all betas. I don't see why this would be necessary, but I think I did see a runcard that does this.

Timing

I did some timing tests as well, creating a model with only the preprocessing layer and training it on random targets for 10_000 epochs:

branch	time (s)
master	38
refactor_preprocessing	36
prepro_join_weights	24

The timing script is something like:

    input = Input(shape=(200, 1), batch_size=1)
    prepro = Preprocessing(flav_info=flav_info, seed=0)
    output = prepro(input)
    model = Model(inputs=input, outputs=output)
    test_x = tf.random.uniform(shape=(1, 200, 1), seed=42)
    test_y = tf.random.uniform(shape=(1, 200, 8), seed=43)

    model.compile(loss="mse", optimizer='adam')

    start = datetime.now()
    model.fit(test_x, test_y, epochs=10_000)
    end = datetime.now()
    diff = end - start

checks

• refactor_preprocessing: Regression test passes, so identical results (note though, even when specifying a seed, the initialization depends on the tensorflow version, the regression test passes on Snellius but not on my laptop)
• prepro_join_weights: initialization is different when the weights are a vector, so the regression test needs to be updated. But I checked that manually setting the weights to what they were in master, the results are still the same. Other regression tests and the structure of saved models will need to be changed as well. I'll do that only if these changes are approved.

[scarlehoff]

The reason this may be controversial is that it doesn't allow setting weights as trainable on a flavor-by-flavor basis, only all alphas and/or all betas. I don't see why this would be necessary

Please don't remove options that currently exist. Actually one of the basic runcards does exactly that. I'm ok with the changes but I would like to keep that option.

[Radonirinaunimi]

This will also remove the possibility of having a fixed-functional form (the one we briefly talked about at some point) that @peterkrack is working on.

[APJansen]

@Radonirinaunimi Why exactly? You mean only the changes in the separate prepro_join_weights branch?

[Radonirinaunimi]

@Radonirinaunimi Why exactly? You mean only the changes in the separate prepro_join_weights branch?

Sorry if I was unclear but my comment refers to the changes you propose here:

The reason this may be controversial is that it doesn't allow setting weights as trainable on a flavor-by-flavor basis, only all alphas and/or all betas. I don't see why this would be necessary, but I think I did see a runcard that does this.

(If I am not mistaken), having trainable weights on flavor-by-flavor basis is exactly what is required for the fixed functional form. Therefore if you restrict that option, this may no longer be possible.

[APJansen]

Ok, so two good reasons to keep this option. Keeping this option and having the weights as vectors will probably make things more complicated rather than simpler, so let's discard the prepro_join_weights branch then.
Are the changes in this branch ok?

[RoyStegeman]

Thanks @APJansen, this looks good! For as far as I'm concerned this can be merged after you've addressed a few minor points.

[APJansen]

@RoyStegeman As you saw I was having some issues with the test, thanks for fixing it. Did you understand what was going wrong? It was passing for me locally but not in the CI after incorporating your comment.

[RoyStegeman]

To be honest I don't know, for me this passes locally (of course, since I also generated the test_prefactors locally). The numpy version shouldn't make a difference for the rng so assuming you used the same seeds I find it actualy quite surprising that you found different numbers. Does this still fail locally for you now?

[github-actions]

Greetings from your nice fit 🤖 !
I have good news for you, I just finished my tasks:

• Fit Name: NNBOT-20d3c6885-2023-07-28
• Fit Report: https://vp.nnpdf.science/3Fbp6t_OTq2GFpWtIs9oXg==
• Fit Data: https://data.nnpdf.science/fits/NNBOT-20d3c6885-2023-07-28.tar.gz

Check the report carefully, and please buy me a ☕ , or better, a GPU 😉!

[scarlehoff]

Please check that I didn't made any typos in the suggestion before accepting any of them ¡!

RE the differences, by changing to using np.testing we'll have extra information on the failures, it might have just been numerics.

[APJansen]

@RoyStegeman It was passing locally for me before your change, and failing after. I think it's because of tensorflow, I had this issue before. The tensorflow version I have locally doesn't handle seeds properly. i.e. initializing a layer with say seed=42 will be consistent if you run it multiple times, but if you do it twice, initialize two layers with the same seed, they will actually be different.

Now I changed the seed at Juan's suggestion, which I agree is better, but it will break the tests of course, and if I update the numbers it will probably again pass for me locally but fail in the CI, so could you change them again? Sorry about that.

[RoyStegeman]

No problem!