Log the time difference of create time and post time #240
Conversation
tests/client.py
Outdated
| #Get the current time in seconds | ||
| date= datetime.datetime.utcnow() - datetime.datetime(1970, 1, 1) | ||
| seconds =(date.total_seconds()) |
There was a problem hiding this comment.
| #Get the current time in seconds | |
| date= datetime.datetime.utcnow() - datetime.datetime(1970, 1, 1) | |
| seconds =(date.total_seconds()) | |
| # Get the current time in seconds | |
| date = datetime.datetime.utcnow() | |
| seconds = datetime.datetime.timestamp(date) |
This would only give you precision to the second. We probably want to go to millisecond or microsecond precision as this suggestion would do
There was a problem hiding this comment.
Changed the time to milliseconds as suggested
tests/test_hubspot_bookmarks.py
Outdated
| time_diff =0 | ||
| for _ in range(self.times): | ||
| record = self.test_client.create(stream) | ||
| self.expected_records[stream] += record | ||
| if stream in 'contacts': | ||
| if self.test_client.time_difference < self.minimum: | ||
| self.minimum = self.test_client.time_difference | ||
| if self.test_client.time_difference > self.maximum: | ||
| self.maximum = self.test_client.time_difference | ||
| time_diff += self.test_client.time_difference | ||
|
|
||
| if stream in 'contacts': | ||
| self.avg = time_diff/5 | ||
| self.test_client.record_create_times[stream] = time_diff |
There was a problem hiding this comment.
I don't think it is necessary to change the test to create more data.
It would be better to put all the data into your record_create_times in the client itself so that any test that creates this data by calling the client will have this data recorded.
Then once the test is finished you can use a tearDown method to analyze the data from the setup. We could add this teardown to any test that uses the contacts stream to get data from multiple tests instead of just this one.
There was a problem hiding this comment.
Moved the code to record the times to client. And, added teardown method to print the result at the end.
Description of change
https://jira.talendforge.org/browse/TDL-20939
The change is to add more logging to decide on further steps to a discrepancy in bookmark test that occurs occassionally in stream contacts.
Log the time difference between the post message and the create time when a record is created to decide if we want to have a lookback window.
Manual QA steps
Risks
Rollback steps