Batching h2o2 request with bassmaster yields timeout/504

[palmerjc]

I am using the h2o2 module to proxy calls to an external web service and the individual service works just fine. However, when I try to batch several of these calls together I end up getting a 504 error for each request. Looking for any guidance as to why proxy request might have issues with Bassmaster (https://github.com/hapijs/bassmaster).

[spanditcaa]

Hi @palmerjc - hapi 17/latest h2o2 and bassmaster ? Can you provide a repro sample?

edit: I was unable to repro this as I was not setting passThrough:true on my sample. That setting triggers the behavior.

[palmerjc]

Hi @spanditcaa - thank you for the prompt reply.

I am using hapi 17.3.1, h2o2 8.0.1, and bassmaster 3.1.1. I can provide you my endpoint definitions but can't provide a repo because of security for the service we are proxying.

This is my non-batch endpoint:

        {
            method: 'GET',
            path: '/license/v3',    
            options: {
                auth: 'jwt',   
                validate: {
                    headers: CommonValidation.target_enviornment_schema,
                    query: LicenseValidation.schemas['get.license-v3.query-parameters']
                },                             
                pre: [ {method: AuthenticationController.setupAuthHeader, assign: 'setupResult'} ]  
            },                                
            handler: {
                proxy: {
                    passThrough: true,
                    mapUri: LicenseController.getLicense_v3.mapUri
                }
            }
        }

This is my batch endpoint:

        {
            method: 'GET',
            path: '/license/v3/batch',
            options: {
                auth: 'jwt',
                handler: async function (request, h) {     

                    const payload = 
                    { "requests": [
                        {"method": "GET", "path": "/license/v3", "query": { "serialNumbers": "399-42672768", "includeRelatedAssets": "true" }},
                        {"method": "GET", "path": "/license/v3", "query": { "serialNumbers": "399-42672867", "includeRelatedAssets": "true" }}
                    ] };

                    return (await server.inject({
                        method: 'POST',
                        url: '/batch',
                        headers: request.headers,
                        payload: payload
                    })).result;
                }   
            }                   
        }

Does this help?

[spanditcaa]

Yes @palmerjc - I'll check it out.

[palmerjc]

@spanditcaa come across anything yet?

[spanditcaa]

@palmerjc I was able to reproduce the issue with a simplified version of your routes. It seems more likely a bassmaster issue but I'll dig further tonight and let you know if I find anything.

const Hapi = require('hapi')

let proxyMapUri = () => {
  return { uri: 'http://www.caa.com/healthcheck' }
}

const server = Hapi.server({ port: 7777 })

let startServer = async function () {
  try {
    await server.register({ plugin: require('h2o2') })
    await server.register({ plugin: require('bassmaster') })

    server.route({
      method: 'GET',
      path: '/license/v3',
      options: {
        //      auth: 'jwt',   
        //      validate: {
        //         headers: CommonValidation.target_enviornment_schema,
        //         query: LicenseValidation.schemas['get.license-v3.query-parameters']
        //     },                             
        //     pre: [ {method: AuthenticationController.setupAuthHeader, assign: 'setupResult'} ]  
      },
      handler: {
        proxy: {
          passThrough: true,
          mapUri: proxyMapUri
        }
      }
    })

    server.route({
      method: 'GET',
      path: '/license/v3/batch',
      options: {
        handler: async function (request, h) {
          const payload =
            {
              "requests": [
                { "method": "GET", "path": "/license/v3", "query": { "serialNumbers": "399-42672768", "includeRelatedAssets": "true" } },
                { "method": "GET", "path": "/license/v3", "query": { "serialNumbers": "399-42672867", "includeRelatedAssets": "true" } }
              ]
            };

          return (await server.inject({
            method: 'POST',
            url: '/batch',
            headers: request.headers,
            payload: payload
          })).result;
        }
      }
    }
    )
    await server.start()
    console.log('started')
  }
  catch (err) {
    console.log(err)
  }

}

startServer()

[spanditcaa]

@palmerjc I haven't nailed it down yet, but the problem has something to do with setting passThrough:true. I expect you need headers to pass through for your scenario for authentication.

[spanditcaa]

Got it - the content length header from the bassmaster post is getting passed through, and is greater than the content of the GET request. The proxy destination is waiting for all the content it expects on the GET - until we hit the timeout. fix coming..

[spanditcaa]

@palmerjc the problem with content-length is resolved with h2o2 8.1.1 -- let me know if your issue continues, there could be another header in your specific case that causes a conflict.

[palmerjc]

@spanditcaa - thanks for the update, the batch request seems to go through now but I get garbage back (almost like it isn't honoring my content-type of application/json). Thoughts as to why this would be?

[palmerjc]

Another view from the response object showing headers:

[palmerjc]

Turns out I need to specify the following request header:

Don't seem to have to do this for regular calls to the single (non-batch) endpoint.

Thanks again for the fix you put in, it is a big help to be able to run these requests as a batch.

[spanditcaa]

That is interesting @palmerjc.. take a look at the accept-encoding that is being sent by the bassmaster request. I'm curious if nothing is being passed so your final endpoint is defaulting to its native SOAP with some encoding, where the proxy route with passthrough is passing along a more specific browser accept-encoding which includes application/json.

[palmerjc]

The request coming into '/license/v3/batch' endpoint has gzip Accept-Encoding (which is odd) because calling the single endpoint the Accept-Encoding (inspected in the mapURI method) shows application/json.

[spanditcaa]

Hm.. I don't see 'gzip' in h2o2 or bassmaster.

@palmerjc cc:@cadecairos if you all want to dig further I'm always in for a puzzle, otherwise it looks like your use case is resolved and we can leave this closed.

[cadecairos]

I wonder if this has something to do with how Bassmaster injects requests. I don't have the time today to investigate, but if a bug is opened on the bassmaster repo I'll get to it in the future.

[ggoodman]

This is breaking non-injected requests. I'm seeing my static assets being truncated.

[spanditcaa]

@ggoodman can you provide a failing test ?

[spanditcaa]

#78

[lock]

This thread has been automatically locked due to inactivity. Please open a new issue for related bugs or questions following the new issue template instructions.