Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

.Net: Get Method missing in Handlebars planner to parse JSON output #4661

Closed
rithvikb23 opened this issue Jan 17, 2024 · 2 comments · Fixed by #4804
Closed

.Net: Get Method missing in Handlebars planner to parse JSON output #4661

rithvikb23 opened this issue Jan 17, 2024 · 2 comments · Fixed by #4804
Assignees
Labels
bug Something isn't working .NET Issue or Pull requests regarding .NET code

Comments

@rithvikb23
Copy link

When using the Handlebars planner in semantic kernel, I noticed an issue when the planner was parsing the actual JSON output. For example: I have an API that calculates the price target for a given stock ticker. This API is documented in an OpenAPI spec. After loading the spec as a plugin using the ImportPluginFromOpenApiAsync method and asking the query "what is the price target of GOOGL" the planner produces this plan:

**Original plan:
{{!-- Step 1: Set the ticker symbol for which the price target is to be retrieved --}}
{{set "ticker" "GOOGL"}}

{{!-- Step 2: Use the custom helper to retrieve the price target for the specified ticker symbol --}}
{{set "priceTarget" (PriceTargetPlugin-getPriceTarget Ticker=ticker)}}

{{!-- Step 3: Output the result --}}
{{json (concat "The price target of " ticker " is: $" priceTarget)}}

Result:
The price target of GOOGL is: ${"value":180}**

The problem in the result is that the json helper is not getting the actual value, instead it is returning the actual json object.

Ideally it should look like: The price target of GOOGL is 180. I am using Semantic Kernel 1.0.1 in .NET 6.

Is there a setting allow getting the actual value, I have attempted to tweak the response section of my OpenAPI spec but it is still not retrieving the exact value.

@shawncal shawncal added .NET Issue or Pull requests regarding .NET code triage labels Jan 17, 2024
@github-actions github-actions bot changed the title Get Method missing in Handlebars planner to parse JSON output .Net: Get Method missing in Handlebars planner to parse JSON output Jan 17, 2024
@joslat
Copy link
Contributor

joslat commented Jan 18, 2024

Isn't this the same as this one: #4596 ".Net: get helper needs to be added back to the handlebars planner #4596"?
(found this after being mentioned during yesterday's Office Hours)

@matthewbolanos matthewbolanos added bug Something isn't working and removed triage labels Jan 19, 2024
@gitri-ms gitri-ms self-assigned this Jan 19, 2024
@gitri-ms
Copy link
Contributor

This appears to be a different issue than #4596. I think the real issue is that we are not including the response schema for custom helpers in the handlebars prompt template, so the planner does not know how to parse data out of the response object. I am working on a fix for this.

@gitri-ms gitri-ms linked a pull request Jan 30, 2024 that will close this issue
4 tasks
@gitri-ms gitri-ms linked a pull request Feb 1, 2024 that will close this issue
4 tasks
github-merge-queue bot pushed a commit that referenced this issue Feb 1, 2024
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

The custom `or` helper had a bug where it was checking for null values,
rather than checking for non-null values. This PR changes the condition
to `not null`.

Also, renamed a heading in the CreatePlan prompt to better guide the
model in identifying Handlebars built-in _block_ helpers.

### Description

Additional changes to the `GetArgumentValue` method in
`KernelHelperUtils.cs` to return null if variable isn't defined.

Before, if the template referenced a variable that wasn't defined in
`KernelArguments`, the method would just return the key as a string.
Now, it just returns null (as expected).
> I had it returning a string before because I thought Handlebars would
default the argument to its string representation since it templates
regular text, but I double checked [HB
docs](https://handlebarsjs.com/guide/expressions.html#helpers-with-multiple-parameters),
and generally, string literals are surrounded by quotes when a raw value
is passed as an argument.

Should resolve these issues:
- #4661
- #4732

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Gina Triolo <51341242+gitri-ms@users.noreply.github.com>
Bryan-Roe pushed a commit to Bryan-Roe/semantic-kernel that referenced this issue Oct 6, 2024
### Motivation and Context

<!-- Thank you for your contribution to the semantic-kernel repo!
Please help reviewers and future users, providing the following
information:
  1. Why is this change required?
  2. What problem does it solve?
  3. What scenario does it contribute to?
  4. If it fixes an open issue, please link to the issue here.
-->

The custom `or` helper had a bug where it was checking for null values,
rather than checking for non-null values. This PR changes the condition
to `not null`.

Also, renamed a heading in the CreatePlan prompt to better guide the
model in identifying Handlebars built-in _block_ helpers.

### Description

Additional changes to the `GetArgumentValue` method in
`KernelHelperUtils.cs` to return null if variable isn't defined.

Before, if the template referenced a variable that wasn't defined in
`KernelArguments`, the method would just return the key as a string.
Now, it just returns null (as expected).
> I had it returning a string before because I thought Handlebars would
default the argument to its string representation since it templates
regular text, but I double checked [HB
docs](https://handlebarsjs.com/guide/expressions.html#helpers-with-multiple-parameters),
and generally, string literals are surrounded by quotes when a raw value
is passed as an argument.

Should resolve these issues:
- microsoft#4661
- microsoft#4732

### Contribution Checklist

<!-- Before submitting this PR, please make sure: -->

- [x] The code builds clean without any errors or warnings
- [x] The PR follows the [SK Contribution
Guidelines](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md)
and the [pre-submission formatting
script](https://github.com/microsoft/semantic-kernel/blob/main/CONTRIBUTING.md#development-scripts)
raises no violations
- [x] All unit tests pass, and I have added new tests where possible
- [x] I didn't break anyone 😄

---------

Co-authored-by: Gina Triolo <51341242+gitri-ms@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working .NET Issue or Pull requests regarding .NET code
Projects
Archived in project
5 participants