Existing traces can't be opened from the trace id lookup UI (404 error received) - ES backend #3270
Comments
cspwizard
changed the title
|
Could you please increase |
|
I feel like I'm suffering from the same issue. For a few days now, I've repeatedly looked up trace IDs in our Jaeger and they were not found: {"data":null,"total":0,"limit":0,"offset":0,"errors":[{"code":404,"msg":"trace not found"}]}I was just looking for trace When I click on the trace, I get the The best I could come up with as a reason would be that maybe I messed up the ES index rollover setup, but I don't see a direct connection. Please let me know if I should open a new issue for this and what I can provide to help. |
it's already 168h (7days), after 7days the index will be removed by ILM policy. And as I mentioned it is reproducible right after fresh deploy (no pre-existing indexes and so on). |
…Unix epoch Version 1.26 introduced an automatic configuration for the query lookback when using ElasticSearch with aliases enabled. When aliases are enabled, the ES plugin will look back 100 years. This pre-dates the Unix epoch, and while such dates can be modeled as negative timestamps, the model defined in `jaeger/model/time.go` only supports unsigned timestamps. As a result, the 100-year lookback ends up overflowing the time model, resulting in a distant-future lookback date, rather than a distant-past lookback date. While the time model could be updated to support negative timestamps, it seems unlikely that any Jaeger users would reasonably need to search for spans from the 1920s. This reduces the automatic lookback to 50 years to remove the overflow issue while still providing an extremely long search window that should serve even the most ambitious searches of historical trace data.
…Unix epoch Version 1.26 introduced an automatic configuration for the query lookback when using ElasticSearch with aliases enabled. When aliases are enabled, the ES plugin will look back 100 years. This pre-dates the Unix epoch, and while such dates can be modeled as negative timestamps, the model defined in `jaeger/model/time.go` only supports unsigned timestamps. As a result, the 100-year lookback ends up overflowing the time model, resulting in a distant-future lookback date, rather than a distant-past lookback date. While the time model could be updated to support negative timestamps, it seems unlikely that any Jaeger users would reasonably need to search for spans from the 1920s. This reduces the automatic lookback to 50 years to remove the overflow issue while still providing an extremely long search window that should serve even the most ambitious searches of historical trace data. Signed-off-by: Charles Treatman <charles_treatman@comcast.com>
…3299) * Close #3270: Prevent rollover lookback from passing the Unix epoch Version 1.26 introduced an automatic configuration for the query lookback when using ElasticSearch with aliases enabled. When aliases are enabled, the ES plugin will look back 100 years. This pre-dates the Unix epoch, and while such dates can be modeled as negative timestamps, the model defined in `jaeger/model/time.go` only supports unsigned timestamps. As a result, the 100-year lookback ends up overflowing the time model, resulting in a distant-future lookback date, rather than a distant-past lookback date. While the time model could be updated to support negative timestamps, it seems unlikely that any Jaeger users would reasonably need to search for spans from the 1920s. This reduces the automatic lookback to 50 years to remove the overflow issue while still providing an extremely long search window that should serve even the most ambitious searches of historical trace data. Signed-off-by: Charles Treatman <charles_treatman@comcast.com> * Update test for maxSpanAge when aliases are enabled Signed-off-by: Charles Treatman <charles_treatman@comcast.com>
Describe the bug
Trace id's cannot be opened through the trace lookup UI (the search bar at the top) the API 404 page is shown.
However I can find open the same trace from search UI.
To Reproduce
Steps to reproduce the behavior:
Expected behavior
Existing trace should be opened
Screenshots
Version (please complete the following information):
What troubleshooting steps did you try?
Verified the indexes and policies setup, Jaeger configuration, verified the presence of the trace in question
Additional context
Span and service indexes created with name jaeger-span-000001 and jaeger-service-000001 with aliases jaeger-span-read, jaeger-span-write and jaeger-service-read, jaeger-service-write. The issue is reproducible right away after fresh setup.
Both collector and UI are started with arguments:
I've take a look into the code, and I think the issue is in
func (s *SpanReader) multiRead(ctx context.Context, traceIDs []model.TraceID, startTime, endTime time.Time) ([]*model.Trace, error)which is called from api/trace/{id}. In case of using aliases it should make a query to the read aliasjaeger-span-readin the case, and not trying to construct the multi index search call which will fail due to different index naming not the expected date formatted but - 000001, 000002, etc. However I'm not a golang developer and I may miss something.In case if any other information will be required I'll try to provide it asap.
The text was updated successfully, but these errors were encountered: