Endless redirects when behind a proxy/firewall

[sgranade]

I've run into an issue where request() never successfully redirects when I'm behind a work firewall/proxy. If I run the following:

@asyncio.coroutine
def fetch_page(link):
    response = yield from aiohttp.request('GET', link, allow_redirects=True)
    print(response.status)
    return (yield from response.text())

asyncio.get_event_loop().run_until_complete(fetch_page('http://google.com/'))

and step through it with the debugger, I see that the final response status is a 307 redirect, and that aiohttp is creating an endlessly-longer redirect url: first http://webproxy/<sessionID>/http://google/com/, then http://webproxy/<sessionID>/http://webproxy/<sessionID>/http://google.com/, then http://webproxy/<sessionID>/http://webproxy/<sessionID>/http://webproxy/<sessionID>/http://google.com/, and so on.

I don't see this behavior with urllib. urlopen('http://google.com/') handles the redirects and properly loads Google.

Any thoughts on what might be happening?

[fafhrd91]

could you print r_url value from this line https://github.com/KeepSafe/aiohttp/blob/master/aiohttp/client.py#L133

[sgranade]

Sure. I threw in a print(r_url) statement right after line 133. This is what I get (each line is from one redirect loop that goes nowhere):

http://WebProxy13/B0000D0000N0001F0000S0002R0004/http://google.com/
http://WebProxy13/B0000D0000N0001F0000S0002R0004/http://webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//google.com/
http://WebProxy13/B0000D0000N0001F0000S0002R0004/http://webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//google.com/
http://WebProxy13/B0000D0000N0001F0000S0002R0004/http://webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//google.com/
http://WebProxy13/B0000D0000N0001F0000S0002R0004/http://webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//google.com/
http://WebProxy13/B0000D0000N0001F0000S0002R0004/http://webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//google.com/
http://WebProxy13/B0000D0000N0001F0000S0002R0004/http://webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//google.com/
http://WebProxy13/B0000D0000N0001F0000S0002R0004/http://webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//google.com/
http://WebProxy13/B0000D0000N0001F0000S0002R0004/http://webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//webproxy13/B0000D0000N0001F0000S0002R0004/http%3A//google.com/

[fafhrd91]

should be fixed in master

[asvetlov]

@fafhrd91 please elaborate -- I don't follow.

[fafhrd91]

I think proxy expects http:// in path

On Thursday, January 15, 2015, Andrew Svetlov notifications@github.com
wrote:

@fafhrd91 https://github.com/fafhrd91 please elaborate -- I don't
follow.

—
Reply to this email directly or view it on GitHub
#246 (comment).

[sgranade]

Yeah, it looks like converting "http://" to "http%3A//" was what was causing problems with the redirect.

[asvetlov]

Got it, thanks.

[lock]

This thread has been automatically locked since there has not been
any recent activity after it was closed. Please open a new issue for
related bugs.

If you feel like there's important points made in this discussion,
please include those exceprts into that new issue.