Fix tests which incorporate network requests

[AnshulMalik]

• bug report

Current behavior:
Currently, while testing, here, we actually fetch data from search engines.

This is not what we want, since network is unreliable. So our tests will also be unreliable, one test fails and second time another test fails.

Expected behavior:
We should provide custom input to the functions which parse the response, assuming response is available

[cclauss]

TL;DR Testing against real (not mock) backends is essential. We can change the xfails() if you want.

Perhaps @pytest.mark.xfails requires a bit of explanation...

• query_server can be running on Python 2 or Python 3
• tests can be run on Travis CI or on a contributor's local machine

Given that our code is not yet Python 3 compatible, we give query_server running on Python 3 an xfail on most test cases. We remove those xfails in #343

When we are running the tests on Travis CI we assume the network is working (or we never would have gotten to travis-ci.org in the first place). We test against the real search engines I believe is the right way to go. Mock payloads are no substitute for end-to-end tests that can alert the contributor to page misnumber issues, missing descriptions, Unicode characters, and captcha alerts.

Last but not least in the instance where we are running the tests on our local machines. I open two terminals in one I run query-server locally and in the other I run the tests. If they pass, pytest prints a capital X and if they fail, pytest prints a lowercase x. If I forget to start the local server then I get a lowercase x.

[starlord1311]

i would like to take up this issue. can i ? @cclauss @AnshulMalik

[cclauss]

@starlord1311 I think we still need to decide what the ToDo is before we start creating PRs. Let's first see how @AnshulMalik replies to my comment. Unless you already know what you want to do.

You might look at the tests in #297 and see of any of them should be a PR of their own.

[AnshulMalik]

I see a point in using data from actual network requests, since the parameters we are using to scrape the results might change, and we won't know it if we are testing on hard coded code.

Can we have both solution?
We can run tests fast locally while developing without costing network, so that one can to TDD
And travis can run network tests to be sure everytinng works fine.

[cclauss]

That sounds like a workable approach.

@pytest.mark.xfail(not TRAVIS_CI, reason="Avoid network costs for local testing") could be put above all tests that use the network so that they would run in Travis CI but would fail without halting the testing when doing local testing. Local users who have a query-server running will see these end-to-end tests pass. The new tests which are created around mock data should include data that includes Unicode characters so that we can ensure Python2 is doing the right thing.

@starlord1311 Do you want to create some test around mock data?

[starlord1311]

@cclauss can you explain what i am supposed to do?