Spec-breaking proposal to support N timestamps

[amacneil]

This is an incomplete draft proposal to add support for multiple (N) timestamps to mcap.

Today mcap supports recording exactly two timestamps per message: log_time and publish_time. Of these, only log_time is indexed, meaning it is impossible to seek to or play back records by publish_time without doing a full linear scan on the file.

While it is up to the user what to put in these timestamps, the original intention was that log_time would represent the time at which a given message was received and written by the logging node, and publish_time would represent when the message was originally sent by a publishing node.

However, this design has proven short-sighted, as:

• (a) efficient seek and playback by publish_time is desirable
• (b) some robot systems have more than two timestamps, for example they might want to record a "sensor time", "resimulation execution time", or others as metadata (rather than fields in the message data itself).
• (c) some robot systems only need one timestamp, in which case we are wasting 8 bytes per message which is fairly significant

This PR is an incomplete draft proposal to see what it would look like to add support for an arbitrary number of named timestamps. The idea is that one would specify an arbitrary-length list of timestamp names up front (with string names), then each message would supply exactly that list of timestamps.

I can see two ways to do this: configure a file-global list of timestamps which all records in the file support, or configure a list of supported per-channel timestamps (each channel could in theory have different named timestamps). In this PR I opted to try the per-channel approach, which would make mcap merge operations easier (if merging two files with differently named timestamps, we don't want to write a bunch of zero timestamps to every single message, so keeping the timestamps per-channel helps here).

However, when using the per-channel approach it leaves open a question about how to handle Attachment timestamps, and Summary record timestamps. I have left that question unanswered.

Another potential approach might be to introduce a file-global list of timestamps and give each timestamp a 4 bit identifier, then each message could provide a list of timestamp IDs and the corresponding timestamp. The downside of this approach is that in the Message record each timestamp would take up 12 instead of 8 bits.

As currently proposed this would be a backwards-incompatible change, so could only really be considered as part of an MCAPv2 spec. We could possibly solve only problem (a) in a backwards-compatible way (by introducing new Chunk Index records which index the publish_time, old readers would fall back to unindexed reading but would still be able to access all the file data). But I can't see any obvious way to retrofit N-timestamps without breaking existing readers, since timestamps are so core to how MCAP files work.

[defunctzombie]

• (b) some robot systems have more than two timestamps, for example they might want to record a "sensor time",

IMO these are better suited for the message data

• (c) some robot systems only need one timestamp, in which case we are wasting 8 bytes per message which is fairly significant

There's a lot of "waste" in other aspects of storing message data. The waste tradeoff comes with simplicity for parsing and handling of message record semantics.

There's definitely room to think about different ways to index and iterate through message data. A few other thoughts about the notion of "timestamps"

The reason we selected "log_time" as the index choice is because it "tends" to be monotonic which has some nice properties like knowing that a message you will encounter later occurs later. With publish time, you could encounter a message that says it was published 5 minutes ago.

• (a) efficient seek and playback by publish_time is desirable

This kind of exposes the disconnect in some pubsub systems because the time at which something is published is not necessarily the time at which it is consumed. One area that I find more fascinating and unexplored for debugging over time is what in web requests gets called "request tracing". You can follow how one request moves through other services in the system.

That aside, there's still debugging value in plotting and looking at some kinds of data from the POV of the publisher rather than the consumer.

My overall tl;dr; on this PR is that I remain unconvinced that we "need" more timestamps but am receptive to the idea that more efficient handling of publish time could be desired.

[amacneil]

Closing since this is not actively being worked on