openCV.mat format/binarizing TTLs from ONIX

[emilyjanedennis]

hey folks,

I'm trying to do something fairly simple, but am coming up with only really complicated solutions, and am hoping for a more streamlined approach.

I have an external device sending TTL pulses on channel 0 via ONIX, and I want to just timestamp the 'up'/on states. Currently, I'm just saving out the matrices with write matrix with the following workflow:

AnalogIODevice > Data > Submatrix (to select the one channel) > MatrixWriter

and AnalogIODevice > Clock > MatrixWriter

I would prefer if I could have something like
AnalogIODevice > Data > Submatrix > smooth/threshold/binarize the stream > DistinctUntilChanged > TimeStamp > CsvWriter

If this were a stream of integers or floats, I get how to do this, but I don't think I understand the opencv matrix format enough to figure out which nodes to use. any/all help appreciated, thx!

[emilyjanedennis]

adding data snippet (.bin file compressed as .zip)
submatrix2024-05-08T09_38_50.zip

[bruno-f-cruz]

I need a few clues on how to open this file. What is the ChannelCount, Depth, and Layout?

[emilyjanedennis]

Sorry it's 1 channel, s16 depth. To reshape in python I use (-1,1)

…

On Wed, May 8, 2024, 20:20 brunocruz ***@***.***> wrote: I need a few clueson how to open this file. What is the ChannelCount, Depth, and Layout? — Reply to this email directly, view it on GitHub <#1783 (reply in thread)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABXL3DVFFZKZ55D4JWH3SSTZBK6MRAVCNFSM6AAAAABHNDH7G2VHI2DSMVQWIX3LMV43SRDJONRXK43TNFXW4Q3PNVWWK3TUHM4TGNRRGY2TI> . You are receiving this because you authored the thread.Message ID: ***@***.***>

[bruno-f-cruz]

Ok here's a 5min stab at it. Not sure if this is what you were looking for exactly but its what I think I would have used haha

This is the intresting bit. The rest is just making sure we offset the index of the rising edge given the current buffer.

Briefly,

1. Threshold your trace and cnovert it to an IplImage (as a side note, conversion between IplImages and Mat is very cheap, the only thing that changes is the header of the object, no data duplication takes place). That Tile is a bit of an hack since most image algorithms only work on 2d arrays (i.e. images).
2. Once you have an image, use FindContours to find find transitions in your image between 0-1 and 1-0 (i.e. segment your trace). As a bonus, you can use the Min and Max area properties to make sure you get rid of sporadic noise in your signal.
3. Find all binary regions in your image, these will correspond to periods where your signal changes. Then sort them by X position to make it easier to handle. Unwrap the collection and, for each region, calculate the extremes. These will correspond, in my example, to the index of the rising and falling edges, respectively.

Workflow (Copy+Paste me)

<?xml version="1.0" encoding="utf-8"?>
<WorkflowBuilder Version="2.8.1"
                 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                 xmlns:rx="clr-namespace:Bonsai.Reactive;assembly=Bonsai.Core"
                 xmlns:sys="clr-namespace:System;assembly=mscorlib"
                 xmlns:dsp="clr-namespace:Bonsai.Dsp;assembly=Bonsai.Dsp"
                 xmlns:cv="clr-namespace:Bonsai.Vision;assembly=Bonsai.Vision"
                 xmlns="https://bonsai-rx.org/2018/workflow">
  <Workflow>
    <Nodes>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="IntProperty">
          <Value>0</Value>
        </Combinator>
      </Expression>
      <Expression xsi:type="rx:BehaviorSubject">
        <Name>BufferSize</Name>
      </Expression>
      <Expression xsi:type="rx:BehaviorSubject" TypeArguments="sys:Single">
        <rx:Name>Indices</rx:Name>
      </Expression>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="dsp:MatrixReader">
          <dsp:Path>C:\Users\bruno.cruz\Desktop\submatrix2024-05-08T09_38_50.bin</dsp:Path>
          <dsp:Offset>0</dsp:Offset>
          <dsp:SampleRate>0</dsp:SampleRate>
          <dsp:ChannelCount>1</dsp:ChannelCount>
          <dsp:BufferLength>10000</dsp:BufferLength>
          <dsp:Depth>U16</dsp:Depth>
          <dsp:Layout>ColumnMajor</dsp:Layout>
        </Combinator>
      </Expression>
      <Expression xsi:type="rx:SelectMany">
        <Name>CalculateRisingEdges</Name>
        <Workflow>
          <Nodes>
            <Expression xsi:type="WorkflowInput">
              <Name>Source1</Name>
            </Expression>
            <Expression xsi:type="Combinator">
              <Combinator xsi:type="rx:Take">
                <rx:Count>1</rx:Count>
              </Combinator>
            </Expression>
            <Expression xsi:type="rx:AsyncSubject">
              <Name>Buffer</Name>
            </Expression>
            <Expression xsi:type="SubscribeSubject">
              <Name>Buffer</Name>
            </Expression>
            <Expression xsi:type="Combinator">
              <Combinator xsi:type="dsp:ConvertScale">
                <dsp:Depth>F32</dsp:Depth>
                <dsp:Scale>1</dsp:Scale>
                <dsp:Shift>0</dsp:Shift>
              </Combinator>
            </Expression>
            <Expression xsi:type="Combinator">
              <Combinator xsi:type="dsp:Threshold">
                <dsp:ThresholdValue>30000</dsp:ThresholdValue>
                <dsp:MaxValue>1</dsp:MaxValue>
                <dsp:ThresholdType>Binary</dsp:ThresholdType>
              </Combinator>
            </Expression>
            <Expression xsi:type="Combinator">
              <Combinator xsi:type="dsp:Tile">
                <dsp:RowTiles>10</dsp:RowTiles>
                <dsp:ColumnTiles>1</dsp:ColumnTiles>
              </Combinator>
            </Expression>
            <Expression xsi:type="Combinator">
              <Combinator xsi:type="dsp:ConvertScale">
                <dsp:Depth>U8</dsp:Depth>
                <dsp:Scale>1</dsp:Scale>
                <dsp:Shift>0</dsp:Shift>
              </Combinator>
            </Expression>
            <Expression xsi:type="Combinator">
              <Combinator xsi:type="cv:ConvertToImage" />
            </Expression>
            <Expression xsi:type="Combinator">
              <Combinator xsi:type="cv:Grayscale" />
            </Expression>
            <Expression xsi:type="Combinator">
              <Combinator xsi:type="cv:FindContours">
                <cv:Mode>External</cv:Mode>
                <cv:Method>ChainApproxNone</cv:Method>
                <cv:Offset>
                  <cv:X>0</cv:X>
                  <cv:Y>0</cv:Y>
                </cv:Offset>
                <cv:MinArea>10</cv:MinArea>
                <cv:MaxArea xsi:nil="true" />
              </Combinator>
            </Expression>
            <Expression xsi:type="Combinator">
              <Combinator xsi:type="cv:BinaryRegionAnalysis" />
            </Expression>
            <Expression xsi:type="rx:OrderBy">
              <rx:KeySelector>Centroid.X</rx:KeySelector>
            </Expression>
            <Expression xsi:type="Combinator">
              <Combinator xsi:type="rx:Merge" />
            </Expression>
            <Expression xsi:type="Combinator">
              <Combinator xsi:type="cv:BinaryRegionExtremes">
                <cv:Method>Horizontal</cv:Method>
              </Combinator>
            </Expression>
            <Expression xsi:type="MemberSelector">
              <Selector>Item1.X</Selector>
            </Expression>
            <Expression xsi:type="SubscribeSubject">
              <Name>BufferSize</Name>
            </Expression>
            <Expression xsi:type="PropertyMapping">
              <PropertyMappings>
                <Property Name="Value" />
              </PropertyMappings>
            </Expression>
            <Expression xsi:type="Add">
              <Operand xsi:type="FloatProperty">
                <Value>5940000</Value>
              </Operand>
            </Expression>
            <Expression xsi:type="MulticastSubject">
              <Name>Indices</Name>
            </Expression>
            <Expression xsi:type="Combinator">
              <Combinator xsi:type="rx:Last" />
            </Expression>
            <Expression xsi:type="rx:SelectMany">
              <Name>BufferSize++</Name>
              <Workflow>
                <Nodes>
                  <Expression xsi:type="SubscribeSubject">
                    <Name>BufferSize</Name>
                  </Expression>
                  <Expression xsi:type="Combinator">
                    <Combinator xsi:type="rx:Take">
                      <rx:Count>1</rx:Count>
                    </Combinator>
                  </Expression>
                  <Expression xsi:type="SubscribeSubject">
                    <Name>Buffer</Name>
                  </Expression>
                  <Expression xsi:type="MemberSelector">
                    <Selector>Cols</Selector>
                  </Expression>
                  <Expression xsi:type="PropertyMapping">
                    <PropertyMappings>
                      <Property Name="Value" />
                    </PropertyMappings>
                  </Expression>
                  <Expression xsi:type="Add">
                    <Operand xsi:type="IntProperty">
                      <Value>10000</Value>
                    </Operand>
                  </Expression>
                  <Expression xsi:type="MulticastSubject">
                    <Name>BufferSize</Name>
                  </Expression>
                  <Expression xsi:type="WorkflowOutput" />
                </Nodes>
                <Edges>
                  <Edge From="0" To="1" Label="Source1" />
                  <Edge From="1" To="5" Label="Source1" />
                  <Edge From="2" To="3" Label="Source1" />
                  <Edge From="3" To="4" Label="Source1" />
                  <Edge From="4" To="5" Label="Source2" />
                  <Edge From="5" To="6" Label="Source1" />
                  <Edge From="6" To="7" Label="Source1" />
                </Edges>
              </Workflow>
            </Expression>
            <Expression xsi:type="WorkflowOutput" />
          </Nodes>
          <Edges>
            <Edge From="0" To="1" Label="Source1" />
            <Edge From="1" To="2" Label="Source1" />
            <Edge From="3" To="4" Label="Source1" />
            <Edge From="4" To="5" Label="Source1" />
            <Edge From="5" To="6" Label="Source1" />
            <Edge From="6" To="7" Label="Source1" />
            <Edge From="7" To="8" Label="Source1" />
            <Edge From="8" To="9" Label="Source1" />
            <Edge From="9" To="10" Label="Source1" />
            <Edge From="10" To="11" Label="Source1" />
            <Edge From="11" To="12" Label="Source1" />
            <Edge From="12" To="13" Label="Source1" />
            <Edge From="13" To="14" Label="Source1" />
            <Edge From="14" To="15" Label="Source1" />
            <Edge From="15" To="18" Label="Source1" />
            <Edge From="16" To="17" Label="Source1" />
            <Edge From="17" To="18" Label="Source2" />
            <Edge From="18" To="19" Label="Source1" />
            <Edge From="19" To="20" Label="Source1" />
            <Edge From="20" To="21" Label="Source1" />
            <Edge From="21" To="22" Label="Source1" />
          </Edges>
        </Workflow>
      </Expression>
    </Nodes>
    <Edges>
      <Edge From="0" To="1" Label="Source1" />
      <Edge From="3" To="4" Label="Source1" />
    </Edges>
  </Workflow>
</WorkflowBuilder>

[bruno-f-cruz]

For completion sake you can also use arrays as I mentioned before:

Workflow (Copy+Paste me)

<?xml version="1.0" encoding="utf-8"?>
<WorkflowBuilder Version="2.8.1"
                 xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
                 xmlns:dsp="clr-namespace:Bonsai.Dsp;assembly=Bonsai.Dsp"
                 xmlns:rx="clr-namespace:Bonsai.Reactive;assembly=Bonsai.Core"
                 xmlns:sys="clr-namespace:System;assembly=mscorlib"
                 xmlns="https://bonsai-rx.org/2018/workflow">
  <Workflow>
    <Nodes>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="dsp:MatrixReader">
          <dsp:Path>C:\Users\bruno.cruz\Desktop\submatrix2024-05-08T09_38_50.bin</dsp:Path>
          <dsp:Offset>0</dsp:Offset>
          <dsp:SampleRate>0</dsp:SampleRate>
          <dsp:ChannelCount>1</dsp:ChannelCount>
          <dsp:BufferLength>10000</dsp:BufferLength>
          <dsp:Depth>U16</dsp:Depth>
          <dsp:Layout>ColumnMajor</dsp:Layout>
        </Combinator>
      </Expression>
      <Expression xsi:type="dsp:ConvertToArray">
        <dsp:Depth>S16</dsp:Depth>
      </Expression>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="rx:Merge" />
      </Expression>
      <Expression xsi:type="GreaterThan">
        <Operand xsi:type="WorkflowProperty" TypeArguments="sys:Int16">
          <Value>30000</Value>
        </Operand>
      </Expression>
      <Expression xsi:type="Combinator">
        <Combinator xsi:type="rx:ElementIndex" />
      </Expression>
      <Expression xsi:type="rx:DistinctUntilChangedBy">
        <rx:KeySelector>Value</rx:KeySelector>
      </Expression>
      <Expression xsi:type="MemberSelector">
        <Selector>Index</Selector>
      </Expression>
    </Nodes>
    <Edges>
      <Edge From="0" To="1" Label="Source1" />
      <Edge From="1" To="2" Label="Source1" />
      <Edge From="2" To="3" Label="Source1" />
      <Edge From="3" To="4" Label="Source1" />
      <Edge From="4" To="5" Label="Source1" />
      <Edge From="5" To="6" Label="Source1" />
    </Edges>
  </Workflow>
</WorkflowBuilder>

[emilyjanedennis]

thanks! these are both very helpful