How about mocking sockets with trio ?

[asmodehn]

Using mocket with trio, I get :

TypeError: descriptor 'send' requires a '_socket.socket' object but received a 'MocketSocket'

Is there any known way for this to work ?

[mindflayer]

Please add more information, maybe a few lines you are using for testing that case. I am not familiar with trio. From what I can infer, it seems they did not implement their module with duck typing in mind.

[asmodehn]

I think so, it's probably more following the modern python3 gradual typing mindset...

I am using https://github.com/theelous3/asks (asynchronous http client lib) to test it (I'm trying to mock sockets in there) and I made this quick script for checking:

import pytest
import json
import asks
from mocket import Mocketizer
from mocket.mockhttp import Entry


asks.init('trio')


@pytest.mark.trio
async def test_connection_httpbin():
    url_to_mock = 'http://httpbin.org/get'

    Entry.single_register(
        Entry.GET,
        url_to_mock,
        body=json.dumps({"args": {}}),
        headers={'content-type': 'application/json'}
    )

    with Mocketizer():
        session = asks.Session()
        resp = await session.get(url_to_mock,)

    assert "args" in resp.json() and resp["args"] == {}


if __name__ == '__main__':
    pytest.main(['-s', __file__])

It crashes, probably because :

sock = <socket.socket [closed] fd=-1, family=AddressFamily.AF_UNIX, type=SocketKind.SOCK_STREAM, proto=0>

    async def wait_socket_readable(sock):
        if type(sock) != _stdlib_socket.socket:
>           raise TypeError("need a socket")
E           TypeError: need a socket

../../../.local/share/virtualenvs/replies-YhL4Uusr/lib/python3.7/site-packages/trio/_core/__init__.py:44: TypeError

More investigation is needed here, but I guess my question becomes, couldn't 'MockerSocket' be a usual 'socket' to pass this kind of checks ?

[mindflayer]

Thanks for the hint. Actually, imho -of course-, there is no reason to check the type when dealing with socket objects, since there is one and only one implementation of that in the Python world, and the result is JUST breaking all the mocking libraries using the approach I used (httpretty to name another one, even more popular than mocket).
Out of curiosity, I've just tried to run the tests with MocketSocket inheriting from socket instead of object and I had a failure, so this is not an easy fix like you expect.

[mindflayer]

Just to add an information, mocket is compliant with popular asyncio libraries as aiohttp. You can verify it easily from the test suite, so the issue title is not accurate. Please rephrase it.

https://github.com/mindflayer/python-mocket/blob/master/tests/tests35/test_http_aiohttp.py

[asmodehn]

Oh thank I hadn't notice that, it's a good reference.
I'll have a look and compare trio and asyncio for this usecase...

[asmodehn]

I made a PR #75 adapting the aiohttp tests to asks with curio and trio. They can be used as reference when discussing what could or should be done.

[mindflayer]

Thank you very much for your contribution. I am not 100% sure I should do something to make MocketSocket inheriting from socket.socket. I see it as quite dangerous, complicating the debugging of possible issues due to the fact that MocketSocket acts as a socket following the duck typing principles. I would invest more time in advocating the removal of this check in those libraries, if they don't have very strong reasons for doing it.

[njsmith]

Hey, main author of Trio here. I wrote some more notes here: python-trio/trio#170 (comment)

Trio does check that the sockets you pass in are socket.socket objects – this is to avoid things like allowing a ssl.socket, which is a subclass of socket.socket but with different and incompatible behavior (from the point of view of an async library like trio). But, this isn't actually a problem at all, because mocket also patches socket.socket to refer to its special object, so that test passes :-). AFAICT there are two issues:

• Because of a strange quirk, trio tries calling the real socket.socket methods on the object that get passed in, and these get confused because they require a real-honest-to-goodness socket.socket object. (Hence the "descriptior 'send'" bit the original bug report.) There's more discussion of this in the thread linked above.

• Since Trio is an async library, it needs to be able to pass sockets to low-level OS primitives like epoll_ctl and select. It looks like mocket is not set up to support this, so even if we fixed the descriptor issue it still wouldn't work? I see what looks like an attempt to support this in MocketSocket.fileno() calling os.pipe, but I don't see how it can work (maybe it assumes that no-one ever calls fileno more than once? also it obviously only supports Unix, since on Windows pipe and sockets are totally incompatible – socket.socketpair might work better).

However, trio does have explicit support for socket mocking at a slightly higher level: if you have a class that implements the trio socket API (which is basically the regular socket API except the blocking functions are marked as async), then you can tell trio to use that instead of regular sockets. I wonder if there would be any way to implement a class like this, while reusing mocket's existing code for handling http requests etc.?