end_time in task dataframes is not counted as an event (right open interval)

[juancq]

The end_time in a task dataframe is not counted as an event. I am not sure if this is a bug or if it's by design (if it's the latter, just a documentation update would do).

For example, in a task dataframe as listed below, for subject 1520408 let's assume two events before 2010-10-20 and one event on 2010-10-20, recording end_time as 2010-10-20 would be treated as a sequence of two events during the call to filter_to_min_seq_len in

EventStreamGPT/EventStream/data/pytorch_dataset.py

Line 322 in 2f433a6

def filter_to_min_seq_len(self):

.

subject_id	end_time	label	start_time
u32	datetime[μs]	u32	datetime[μs]
1520408	2010-10-20 00:00:00	1	null
1569956	2010-02-14 00:00:00	1	null
1230099	2010-06-27 00:00:00	2	null

If the endtime is not meant to be included (as it currently is), it would be helpful to have a note in the documentation stating this.

[mmcdermott]

@juancq I'm not sure, but is it possible the change here: #107 solves this issue? Or am I misunderstanding the scope of the issue here.