Release v2.2.0

[amakropoulos]

• closes Hot-swap LoRA with updated llama.cpp #212
• closes The editor crashes when exiting playmode while it is creating the LLM service #171
• closes [Regression] Custom Model Path not working anymore #206

[ltoniazzi]

You can use multiple adapters at the same time, they are all initialised with scale 1.
Then you can use the SetLoraScale function to adjust the scale how you want.
I'm adapting the code to make it possible to set weights before the LLM starts

Does this mean that the Lora weights are fixed once the server starts? In that case hot-swapping is not really allowed, as one cannot change the adapters weights after the server started.

I think the current design in llama.cpp for hot swapping is to:

1. Start a server with all the desired adapters loaded,
2. With POST /lora-adapters change the scale to use only the desired adapter.

So, say, characters with different adapters share the same LLM object with all the adapters loaded, and before performing a call to the server each character runs POST /lora-adapters <my lora weight = 1, all the others = 0 >.

Let me know if I am misunderstanding something!

[amakropoulos]

@ltoniazzi ah no I just didn't phrase it properly.
I meant that you can now both set the weights before the LLM service starts but also adapt them afterwards.

To use a specific lora for a character, one would call the SetLoraWeight multiple times at the moment (0 for all except the character lora with 1).
I'll implement SetLoraWeights to allow a single call as well.