.Net: Handlebars syntax example

[joslat]

This example shows how to use Handlebars different syntax options

Motivation and Context

This example shows how to use Handlebars different syntax options like:

• set (works)
• function calling (works)
• loops (each) - not working at the moment, but tries to do so over a complex object generated by one of the functions: a JSON array-
• array - not working at the moment - to accumulate the results of the loop in an array
• conditionals (works)
• concatenation (works)

In order to create a Prompt Function that fully benefits from the Handlebars syntax power.
The example also shows how to use the HandlebarsPlanner to generate a plan (and persist it) which was used to generate the initial Handlebar template.
The example also shows how to create two prompt functions and a plugin to group them together.

1. Why is this change required?
   • there is no example that shows anything but the simple syntax of handlebars
2. What problem does it solve?
   • help others understand handlebars and how to do "all the things" possible ;)
3. What scenario does it contribute to?
   • Learning Semantic Kernel and enabling its usage.
4. If it fixes an open issue, please link to the issue here.
   • issue: no handlebars example with sets, function calling, conditionals and loops. ;)

Description

Contribution Checklist

• The code builds clean without any errors or warnings
• The PR follows the SK Contribution Guidelines and the pre-submission formatting script raises no violations
• All unit tests pass, and I have added new tests where possible
• I didn't break anyone 😄

[joslat]

@matthewbolanos - if you want to go over this I am in anytime.

[joslat]

"Maybe" the issue I am having with the loops is due to this issue: #4596

[joslat]

I did not realize I had to "resolve" the conversation - just did...

[markwallace-microsoft]

Can you fix the typo please

[markwallace-microsoft]

@joslat Thanks for the contribution, this is a useful addition. Can you please fix the typo and formatting changes.

[joslat]

@joslat Thanks for the contribution, this is a useful addition. Can you please fix the typo and formatting changes.

Thanks @markwallace-microsoft - Unsure you want to accept the PR as it is as mentions a couple of things that did not work - haven't seen if they are fixed though.
Typhos are fixed though :)
And now format issues too.

I put some code in there that is meant to be left as an example on how to do things/use handlebars... you tell me if you want me to adapt, remove any section... we can have a sync or you can tell me here - or go and do it.

[joslat]

Hi @teresaqhoang, have resolved I believe all of your comments but unsure the implementation is ideal (or the prompt improvement suggestions are fully applied) - so I have not labelled them as "resolved" - also would love to get samples on two of your suggestions, if you have any code links to what you suggest it would be perfect.
Thanks for the great feedback - I give you the🥇 for that!!

Also, those issues might be affecting/impacting the outcome/function of this sample:

• .Net: Same plugin behaves differently inside a prompt template execution. #4895
• .Net: Enhancements to Handlebars syntax #4892 @markwallace-microsoft I understand that this issue by you means that a JSON parsing will not work once this is fixed, right?
• .Net: Same plugin behaves differently inside a prompt template execution. #4895 - is this something complex/hard to solve?
• .Net Planners with models other than openai #4870 @teresaqhoang according to this it seems OpenAI is best for "planning things" - thoughts? - i am usually using Microsoft OpenAI services as I've got an Azure Sponsorship subscription... I also have an OpenAI paid subscription and paid credits too so can do both - is one better than the other at the moment?
• .Net: Handlebars Planner Failing to Parse JSON Array #4721 - I guess this is triggering your handlebars enhancements @markwallace-microsoft - yes? or is this another separate issue?

[joslat]

@teresaqhoang, any news on this? :)

[teresaqhoang]

Hi @teresaqhoang, have resolved I believe all of your comments but unsure the implementation is ideal (or the prompt improvement suggestions are fully applied) - so I have not labelled them as "resolved" - also would love to get samples on two of your suggestions, if you have any code links to what you suggest it would be perfect. Thanks for the great feedback - I give you the🥇 for that!!

Also, those issues might be affecting/impacting the outcome/function of this sample:

• .Net: Same plugin behaves differently inside a prompt template execution. #4895
• .Net: Enhancements to Handlebars syntax #4892 @markwallace-microsoft I understand that this issue by you means that a JSON parsing will not work once this is fixed, right?
• .Net: Same plugin behaves differently inside a prompt template execution. #4895 - is this something complex/hard to solve?
• .Net Planners with models other than openai #4870 @teresaqhoang according to this it seems OpenAI is best for "planning things" - thoughts? - i am usually using Microsoft OpenAI services as I've got an Azure Sponsorship subscription... I also have an OpenAI paid subscription and paid credits too so can do both - is one better than the other at the moment?
• .Net: Handlebars Planner Failing to Parse JSON Array #4721 - I guess this is triggering your handlebars enhancements @markwallace-microsoft - yes? or is this another separate issue?

Hey @joslat,

Apologies for the delayed response - I haven't been getting notifications for changes on this PR for some reason, will be more diligent in checking the PR directly moving forward.

Thanks for iterating! I can happily help you with more guidance! Can you reply on the unresolved conversation threads directly with your questions / points of concerns? And also call out the ones you want code samples for?

I'll also pull this example and test it locally to see if I can better identify some pain points the model's hitting.

[joslat]

@teresaqhoang "Pong" ;) - ball is in your ground :D
Let me know if there is anything more I can help here, happy to...

[teresaqhoang]

Nit: Please omit the number and give them more descriptive names based on what the example you're demonstrating

i.e., RunSimpleHandlebarsTemplateSample vs. RunHandlebarsTemplateWithLoopsSample

[teresaqhoang]

Move this to the top since it's the main scaffolding of this example (This is where you're directly invoking the Template factory and invoking the template)

[teresaqhoang]

Wanted to follow-up on my question here - why not just return an array of products? Is there a need to wrap in the outer JSON object? If not, you can remove the outer object to reduce complexity and probability of model error.

I recommend rather than creating the prompts inline, you leverage the YAML format so users (and the model) have sufficient context for the inputs and outputs of the function.

Here's an example of how I refactored one of your functions to make it more digestible to an LLM: 3438ec5 (Ignore the changes to use AOI)

Sone highlights of prompt improvements:

• Isolated definition on what a product should look like.
• Defined example output before instructing the model to begin.
• Buttoned up description to clear redundancy and be straightforward with expectations.

Also, I renamed the function from GenerateJSONProductNames to GenerateJSONProducts since it's not returning just product names.

[teresaqhoang]

I think the way your functions are set up are too complex for this example.

Either:

1. Have a function that generates the products (with name and description fields) and then a second function that can iterate through those products and produce more detailed descriptions.

i.e.

var products = GenerateNewProducts(string companyDescription) // outputs `[ product1, product2, ...]`

// Either pass in the entire result and have the model handle the products array for you or parse `products` array and loop through each object
GenerateDetailedDescription(product[0]) // outputs "Some compelling description"

2. Have a function that returns a list of product names (output: a list of strings) and then iterate over that list to generate descriptions for each name (output: a list of products containing name and desc)

[teresaqhoang]

In terms of showcase a HB prompt template, these are much too complex.

Example of how you can reduce complexity:

{{!-- Omit the initial variable mapping of inputs (this is required for the planner to reduce error) but as a dev executing their own HB template, you should have full context over how input are maps. --}}
{{set ""productNames"" (productMagician-GenerateJSONProducts input)}}

{{#if generateEngagingDescriptions}} 
{{!-- Since SK doesn't support mutable arrays, you must concatenate each result onto a string. --}}
    {{set ""finalProducts"" """"}}

    {{#each productNames}}
{{!-- Refactor your GenerateJSONProducts function to return just an array of products, then remove the inner {{#each}}} block. It's unnecessarily complex. --}}
        {{set ""productDescription"" (concat ""Product Name: "" this.name "" Description: "" this.description)}}
        {{set ""compellingDescription"" (productMagician-GenerateProductCompellingDescription this)}}
        {{set ""outputProduct"" (concat ""PRODUCT :"" this.name "" Engaging Description: "" compellingDescription)}}

{{!-- Check if you're on the first object; if not, concatenate to end of string.  --}}
        {{#if @first}}{{set ""finalProducts"" outputProduct}}
        {{else}}{{set ""finalProducts"" (concat finalProducts ""\n\n"" outputProduct)}}
        {{/if}}
    {{/each}}
{{!--  OUTPUT The following product descriptions as is, do not modify anything --}}   
    {{json finalProducts}}
{{else}} 
{{!-- Return products as is since the conditional is checking whether descriptions should be regenerated. --}}
    {{json productNames}}
{{/if}}

[teresaqhoang]

@joslat I see you resolved some of the conversations (thanks for that) but I'm not seeing any replies to the conversations that are still open so I'm not sure what kind of clarity you need where. I tried to reply best I could.

Do you see the option to reply directly on those conversation threads? If not, we can try to troubleshoot your access issues

IMO I think this example is too complex for what it's trying to showcase.

We try to keep the Syntax examples as simple as possible to help users grasp the concept easier, and I think this one (since it's derived from the planner) can be hard to follow.

I left more comments with suggestions on how to simplify these samples

Also happy to schedule a meeting so we can parallel program this together if you'd like :)

[teresaqhoang]

Hey @joslat, I discussed your PR with the team today and we think there's a lot of value in examples shown here, but some ideas are getting lost in translation and complexity.

What do you think about pivoting this example to show the difference between Handlebars Plan and a Handlebars Template? It would still fit your goal of showing the Handlebars syntax.

Some differences:

• a Handlebars Plan runs a sequence of steps to achieve a defined goal, so it can be more dynamic and doesn't produce concrete results. When a HB plan is invoked, the template is simply rendered using a HB instance, and the raw render result is the output.
• whereas a pure Handlebars Prompt template is invoked as a Semantic function using SK - a template can call multiple functions within it, but when a HB prompt template is invoked, it's first rendered using a HB instance and then SK passes the render result to the LLM. Then the completion result is the final output.

I created an example here that refactors your GenerateProductCompellingDescription function into a new semantic function called GenerateDetailedProducts that leverages a HB prompt template:
teresaqhoang@78bb77b

I left some prompt engineering pointers directly on that commit. Let me know if you have any questions

[joslat]

Hi @teresaqhoang would be my honor to do this in "extreme programming" way, that would be fantastic for "getting it done" - also this would ensure the focus in showcasing as well the differences between Handlebars Plan and Handlebars Template.

[teresaqhoang]

Hi @joslat, @matthewbolanos will handle follow-up on this moving forward! Let me know if I can help provide code reviews.

[markwallace-microsoft]

@joslat Thanks for the PR. We have decided we're not going to invest in Handlebars for planning and instead focus on providing first class support for planning with Function Calling and Python code execution using the new Azure Container Applications functionality for serverless code interpreter sessions.

[joslat]

That makes a lot of sense, but will we be able - read supported - to run python on other places like docker, locally if we dare as well?
Will this mean that the StepWisePlanner V2 will also be deprecated? I know it is mostly a wrapper over Function Calling and theoretically better...