Potential deadlock in get_registered_contract

[chipkent]

See #8.

I've seen cases where the code seems to deadlock in the cd = self._client.contract_registry.request_contract_details_blocking(contract) line below.

    def get_registered_contract(self, contract: Contract) -> RegisteredContract:
        """Gets a contract that has been registered in the framework.  The registered contract is confirmed to
        exist in the IB system and contains a complete description of the contract.

        Args:
            contract (Contract): contract to search for

        Returns:
            A contract that has been registered with deephaven-ib.

        Raises:
              Exception: problem executing action.
        """

        self._assert_connected()
        cd = self._client.contract_registry.request_contract_details_blocking(contract)
        return RegisteredContract(query_contract=contract, contract_details=cd)

Deadlock monitoring returns the following.

ERROR:ibapi.wrapper:ERROR 9 200 No security definition has been found for the request
r-Scheduler-Serial-1 | i.d.s.s.SessionState      | Internal Error '4936e794-be6c-49b5-8ca2-e4e51b937818' java.lang.RuntimeException: Error in Python interpreter:
Type: <class 'Exception'>
Value: ContractRegistry.request_contract_details_blocking() timed out after 120.0 sec.
Line: 181
Namespace: request_contract_details_blocking
File: /opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_tws/contract_registry.py
Traceback (most recent call last):
  File "<string>", line 232, in <module>
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/__init__.py", line 643, in get_registered_contract
  File "/opt/deephaven-venv/lib/python3.7/site-packages/deephaven_ib/_tws/contract_registry.py", line 181, in request_contract_details_blocking

        at org.jpy.PyLib.executeCode(PyLib.java:-2)
        at org.jpy.PyObject.executeCode(PyObject.java:138)
        at io.deephaven.engine.util.PythonEvaluatorJpy.evalScript(PythonEvaluatorJpy.java:56)
        at io.deephaven.engine.util.PythonDeephavenSession.lambda$evaluate$1(PythonDeephavenSession.java:188)
        at io.deephaven.util.locks.FunctionalLock.doLockedInterruptibly(FunctionalLock.java:46)
        at io.deephaven.engine.util.PythonDeephavenSession.evaluate(PythonDeephavenSession.java:187)
        at io.deephaven.engine.util.AbstractScriptSession.evaluateScript(AbstractScriptSession.java:145)
        at io.deephaven.engine.util.DelegatingScriptSession.evaluateScript(DelegatingScriptSession.java:71)
        at io.deephaven.engine.util.ScriptSession.evaluateScript(ScriptSession.java:84)
        at io.deephaven.server.console.ConsoleServiceGrpcImpl.lambda$executeCommand$8(ConsoleServiceGrpcImpl.java:187)
        at io.deephaven.server.session.SessionState$ExportBuilder.lambda$submit$2(SessionState.java:1299)
        at io.deephaven.server.session.SessionState$ExportObject.doExport(SessionState.java:847)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
        at java.util.concurrent.FutureTask.run(FutureTask.java:264)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
        at io.deephaven.server.runner.DeephavenApiServerModule$ThreadFactory.lambda$newThread$0(DeephavenApiServerModule.java:157)
        at java.lang.Thread.run(Thread.java:829)

It seems like TWS is not responding to some requests.

[chipkent]

Reported by Josh Shay in DHC Slack.

The triggering code is:

c = Contract()
c.symbol = "BZ:BZ_ALL"
c.secType = "NEWS"
c.exchange = "BZ"
rc = client.get_registered_contract(c)
print(rc)

The error is:

r-Scheduler-Serial-1 | .c.ConsoleServiceGrpcImpl | Error running script: java.lang.RuntimeException: Error in Python interpreter:
Type: <class 'Exception'>
Value: ContractRegistry.request_contract_details_blocking() timed out after 120.0 sec.  contract=0,BZ:BZ_ALL,NEWS,,0,,,BZ,,,,,False,,,,combo:
Line: 184
Namespace: request_contract_details_blocking
File: /usr/local/lib/python3.10/dist-packages/deephaven_ib/_tws/contract_registry.py
Traceback (most recent call last):
  File "<string>", line 6, in <module>
  File "/usr/local/lib/python3.10/dist-packages/deephaven_ib/_init_.py", line 695, in get_registered_contract
  File "/usr/local/lib/python3.10/dist-packages/deephaven_ib/_tws/contract_registry.py", line 184, in request_contract_details_blocking
	at org.jpy.PyLib.executeCode(Native Method)
	at org.jpy.PyObject.executeCode(PyObject.java:138)
	at io.deephaven.engine.util.PythonEvaluatorJpy.evalScript(PythonEvaluatorJpy.java:73)
	at io.deephaven.integrations.python.PythonDeephavenSession.lambda$evaluate$1(PythonDeephavenSession.java:194)
	at io.deephaven.util.locks.FunctionalLock.doLockedInterruptibly(FunctionalLock.java:51)
	at io.deephaven.integrations.python.PythonDeephavenSession.evaluate(PythonDeephavenSession.java:194)
	at io.deephaven.engine.util.AbstractScriptSession.lambda$evaluateScript$0(AbstractScriptSession.java:146)
	at io.deephaven.engine.context.ExecutionContext.lambda$apply$0(ExecutionContext.java:149)
	at io.deephaven.engine.context.ExecutionContext.apply(ExecutionContext.java:160)
	at io.deephaven.engine.context.ExecutionContext.apply(ExecutionContext.java:148)
	at io.deephaven.engine.util.AbstractScriptSession.evaluateScript(AbstractScriptSession.java:146)
	at io.deephaven.engine.util.DelegatingScriptSession.evaluateScript(DelegatingScriptSession.java:84)
	at io.deephaven.engine.util.ScriptSession.evaluateScript(ScriptSession.java:102)
	at io.deephaven.server.console.ConsoleServiceGrpcImpl.lambda$executeCommand$4(ConsoleServiceGrpcImpl.java:175)
	at io.deephaven.server.session.SessionState$ExportBuilder.lambda$submit$2(SessionState.java:1385)
	at io.deephaven.server.session.SessionState$ExportObject.doExport(SessionState.java:921)
	at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)
	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
	at io.deephaven.server.runner.scheduler.SchedulerModule$ThreadFactory.lambda$newThread$0(SchedulerModule.java:78)
	at org.jpy.PyLib.callAndReturnObject(Native Method)
	at org.jpy.PyObject.call(PyObject.java:449)
	at io.deephaven.server.console.python.DebuggingInitializer.lambda$createInitializer$0(DebuggingInitializer.java:30)
	at java.base/java.lang.Thread.run(Thread.java:829)