Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Response with text media type is parsed to a JSON object instead of a string #11649

Closed
nguerrera opened this issue Oct 5, 2020 · 6 comments
Closed
Assignees
Labels
Schema Registry Service Attention Workflow: This issue is responsible by Azure service team.

Comments

@nguerrera
Copy link
Contributor

nguerrera commented Oct 5, 2020

The schema registry service returns schemas as text/plain. Avro schemas happen to be JSON, but future serialization formats could represent their schema in text that is not JSON.

Prior version of schema registry swagger indicated to send the schema as a string in JSON. So the body was quoted and escaped. It was very confusing. Now the body is just text/plain, UTF-8. However, upon regenerating the code from the new swagger, I started getting back the body as parsed JSON object, even though the generated code typed it as string.

I'm not sure if this is an issue with the core ServiceClient or with the code that autorest has generated. I do see mediaType: "text" in the generated code FWIW.

EDIT: The issue appears to be that the service doesn't actually response with Content-Type: text/plain. Self assigning, tagging schema registry and following up with service owners.

Repro steps

  1. Check out [schema registry] Update to latest swagger #11650
  2. Apply the following patch
diff --git a/sdk/schemaregistry/schema-registry/src/conversions.ts b/sdk/schemaregistry/schema-registry/src/conversions.ts
index 35c60a467..09121d5b8 100644
--- a/sdk/schemaregistry/schema-registry/src/conversions.ts
+++ b/sdk/schemaregistry/schema-registry/src/conversions.ts
@@ -30,10 +30,7 @@ type GeneratedResponse = GeneratedSchemaResponse | GeneratedSchemaIdResponse;
  * @internal
  */
 export function convertSchemaResponse(response: GeneratedSchemaResponse): Schema {
-  // https://github.com/Azure/azure-sdk-for-js/issues/11649
-  // Although response.body is typed as string, it is a parsed JSON object,
-  // so we use _response.bodyAsText instead as a workaround.
-  return convertResponse(response, { content: response._response.bodyAsText });
+  return convertResponse(response, { content: response.body });
 }
  1. rush update && rush build
  2. cd sdk/schemaregistry/schema-registry
  3. npm run test

Expected result

Tests pass

Actual result

Tests fail:

  1 failing

  1) SchemaRegistryClient
       gets schema by ID:
     AssertionError: expected { Object (type, name, ...) } to equal '{"type":"record","name":"User","namespace":"com.azure.schemaregistry.samples","fields":[{"name":"name","type":"string"},{"name":"favoriteNumber","type":"int"}]}'
@nguerrera nguerrera added Client This issue points to a problem in the data-plane of the library. Schema Registry labels Oct 5, 2020
@nguerrera nguerrera added this to the [2020] November milestone Oct 5, 2020
@nguerrera nguerrera self-assigned this Oct 5, 2020
@nguerrera nguerrera removed this from the [2020] November milestone Oct 5, 2020
@nguerrera nguerrera removed their assignment Oct 5, 2020
@jeremymeng
Copy link
Member

@nguerrera Looks like this is a recorded test, and the recorded response has content-type of application/json? So core-http will de-serialize it from JSON to object.

@nguerrera
Copy link
Contributor Author

Ah! You’re right. I should have checked that!. Looks like the server is not actually responding with text/plain like the swagger says it will! Thanks, I’ll follow up with the service folks.

@nguerrera nguerrera self-assigned this Oct 6, 2020
@nguerrera nguerrera removed the Client This issue points to a problem in the data-plane of the library. label Oct 6, 2020
@nguerrera nguerrera added this to the [2020] November milestone Oct 6, 2020
@nguerrera
Copy link
Contributor Author

cc @arerlend @hmlam

@ramya-rao-a ramya-rao-a modified the milestones: [2021] February, Backlog Feb 10, 2021
@nguerrera nguerrera added the Service Attention Workflow: This issue is responsible by Azure service team. label May 6, 2021
@ghost
Copy link

ghost commented May 6, 2021

Thanks for the feedback! We are routing this to the appropriate team for follow-up. cc @arerlend, @alzimmermsft.

Issue Details

The schema registry service returns schemas as text/plain. Avro schemas happen to be JSON, but future serialization formats could represent their schema in text that is not JSON.

Prior version of schema registry swagger indicated to send the schema as a string in JSON. So the body was quoted and escaped. It was very confusing. Now the body is just text/plain, UTF-8. However, upon regenerating the code from the new swagger, I started getting back the body as parsed JSON object, even though the generated code typed it as string.

I'm not sure if this is an issue with the core ServiceClient or with the code that autorest has generated. I do see mediaType: "text" in the generated code FWIW.

EDIT: The issue appears to be that the service doesn't actually response with Content-Type: text/plain. Self assigning, tagging schema registry and following up with service owners.

Repro steps

  1. Check out [schema registry] Update to latest swagger #11650
  2. Apply the following patch
diff --git a/sdk/schemaregistry/schema-registry/src/conversions.ts b/sdk/schemaregistry/schema-registry/src/conversions.ts
index 35c60a467..09121d5b8 100644
--- a/sdk/schemaregistry/schema-registry/src/conversions.ts
+++ b/sdk/schemaregistry/schema-registry/src/conversions.ts
@@ -30,10 +30,7 @@ type GeneratedResponse = GeneratedSchemaResponse | GeneratedSchemaIdResponse;
  * @internal
  */
 export function convertSchemaResponse(response: GeneratedSchemaResponse): Schema {
-  // https://github.com/Azure/azure-sdk-for-js/issues/11649
-  // Although response.body is typed as string, it is a parsed JSON object,
-  // so we use _response.bodyAsText instead as a workaround.
-  return convertResponse(response, { content: response._response.bodyAsText });
+  return convertResponse(response, { content: response.body });
 }
  1. rush update && rush build
  2. cd sdk/schemaregistry/schema-registry
  3. npm run test

Expected result

Tests pass

Actual result

Tests fail:

  1 failing

  1) SchemaRegistryClient
       gets schema by ID:
     AssertionError: expected { Object (type, name, ...) } to equal '{"type":"record","name":"User","namespace":"com.azure.schemaregistry.samples","fields":[{"name":"name","type":"string"},{"name":"favoriteNumber","type":"int"}]}'
Author: nguerrera
Assignees: nguerrera
Labels:

Schema Registry, Service Attention

Milestone: Backlog

@deyaaeldeen
Copy link
Member

Quick update: @hmlam and @nickghardwick will update the service to return responses with content-type of text/plain;charset=UTF-8.

@nguerrera nguerrera removed their assignment Aug 18, 2021
@deyaaeldeen deyaaeldeen modified the milestones: Backlog, [2021] October Aug 18, 2021
@deyaaeldeen deyaaeldeen self-assigned this Aug 18, 2021
@deyaaeldeen deyaaeldeen removed this from the [2021] October milestone Oct 1, 2021
@deyaaeldeen deyaaeldeen added this to the [2021] November milestone Oct 1, 2021
@deyaaeldeen
Copy link
Member

The official swagger defines the schema to be a file which gets compiled by autorest as a stream. In the SDK, this stream gets read and converted to text, so this is no longer an issue.

@github-actions github-actions bot locked and limited conversation to collaborators Apr 12, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
Schema Registry Service Attention Workflow: This issue is responsible by Azure service team.
Projects
None yet
Development

No branches or pull requests

4 participants