D400 On-Chip Calibration #4658

dorodnic · 2019-08-18T14:32:26Z

This PR introduces handling of the new "lightweight" calibration stack in the RealSense Viewer and DQT:

On-chip calibration depends on firmware 5.13.x that is going to be released soon. Formal white-paper from intelrealsense.com is also expected to be published on the topic

On-Chip Calibration is a set of integrated calibration capabilities designed to make stereo camera calibration process more accessible. It is not a full replacement of Dynamic Calibration Toolchain but can help keep the camera in good health when used regularly. In particular, there may be severe aggregate calibration defects that can be only rectified by the full Dynamic Calibration tool.

On-Chip Calibration does not require any special target, but we recommend pointing the camera at a flat wall when calibrating (ceiling, floor, indoor any white wall will do, outdoor the wall needs to be sufficiently textured)

At the moment, this capability is only provided via the Viewer, but in the future it is likely to be also accessible from the API.

…re the family of devices using it

Introducing auto-calibration process Adding metadata window for stream

ev-mp · 2019-08-20T05:44:24Z

common/fw-update-helper.cpp

            else
+                ImGui::Text("Firmware updates is underway...\nPlease do not disconnect the device");


common/fw-update-helper.cpp

ev-mp · 2019-08-20T05:51:01Z

common/model-views.cpp

+                    to_string() << std::fixed << std::setprecision(1) << timestamp, 
+                    "Frame Timestamp is normalized represetation of when the frame was taken.\n"
+                    "It's a property of every frame, so when exact creation time is not provided by the hardware, an approximation will be used.\n"
+                    "Clock Domain feilds helps interpret the meaning of timestamp\n"


field helps to interpret

ev-mp · 2019-08-20T05:51:37Z

common/model-views.cpp

+                    "Timestamp is measured in milliseconds, and is allowed to roll-over (reset to zero) in some situations" });
+                stream_details.push_back({ "Clock Domain",
+                    to_string() << rs2_timestamp_domain_to_string(timestamp_domain), 
+                    "Clock Domain describes the format of Timestamp feild. It can be one of the following:\n"


ev-mp · 2019-08-20T05:55:57Z

common/model-views.cpp

+                    "1. System Time - When no hardware timestamp is available, system time of arrival will be used.\n"
+                    "                 System time benefits from being comparable between device, but suffers from not being able to approximate latency.\n"
+                    "2. Hardware Clock - Hardware timestamp is attached to the frame by the device, and is consistent accross device sensors.\n"
+                    "                    Hardware timestamp encodes percisely when frame was captured, but cannot be compared across devices\n"


~~cannot be compared across devices~~ cannot be used for inter-device synchronization.

ev-mp · 2019-08-20T06:00:51Z

common/model-views.cpp

+                    "2. Hardware Clock - Hardware timestamp is attached to the frame by the device, and is consistent accross device sensors.\n"
+                    "                    Hardware timestamp encodes percisely when frame was captured, but cannot be compared across devices\n"
+                    "3. Global Time - Global time is provided when the device can both offer hardware timestamp and implements Global Timestamp Protocol.\n"
+                    "                 Global timestamps encode exact time of capture and at the same time are comparable accross devices." });


I think of adding of disclamer - that the global time is an algorithm-augmented mechanism that may introduce interval fluctuations and therefore is best suited for synchronizing Video ( 100fps<) frames.
In addition it requires an initial settle tile

ev-mp · 2019-08-20T06:17:52Z

common/model-views.cpp

+                    "                 Global timestamps encode exact time of capture and at the same time are comparable accross devices." });
+                stream_details.push_back({ "Frame Number",
+                    to_string() << frame_number, "Frame Number is a rolling ID assigned to frames.\n"
+                    "Most device do not guaranty consequitive frames to have conseuquitive frame numbers\n"


Most devices do not guarantee consecutive frames to have sequential frame numbers
devices... guarantee

ev-mp · 2019-08-20T06:18:45Z

common/model-views.cpp

-                ImGui::SetTooltip("%s","Stream Resolution, Format");
+                stream_details.push_back({ "Hardware FPS",
+                    to_string() << std::setprecision(2) << std::fixed << fps.get_fps(), 
+                    "Hardware FPS captures number of frames per second produced by the device.\n"


the number of frames...

ev-mp · 2019-08-20T06:18:59Z

common/model-views.cpp

-            ImGui::SameLine();
+                    stream_details.push_back({ "Display Size",
+                        to_string() << size.x << " x " << size.y, 
+                        "When Post-Processing is enabled, actual display size of the frame may differ from original capture size" });


the actual display...

ev-mp · 2019-08-20T06:21:18Z

common/model-views.cpp

+
+            std::map<rs2_frame_metadata_value, std::string> descriptions = {
+                { RS2_FRAME_METADATA_FRAME_COUNTER                        , "A sequential index managed per-stream. Integer value" },
+                { RS2_FRAME_METADATA_FRAME_TIMESTAMP                      , "Timestamp set by device clock when data readout and transmit commence. usec" },


Disclamer - the units are mostly usec, but there are exceptions (SR300 with 100nsec units)

ev-mp · 2019-08-20T06:23:06Z

common/model-views.cpp

+
+            if (model->supports_on_chip_calib())
+            {
+                // Make sure we don't spam calibration remainers too often:


remainders...

ev-mp · 2019-08-20T06:25:38Z

common/model-views.cpp

+                    msg, manager, false);
+
+                // Recommend calibration once a week per device
+                if (rawtime - last_time < 60)


The weekly remainder could create a false feeling that the device shall be re-calibrated on a weekly basis.
Setting it to a monthly periods would, imho, mitigate this

Why monthly?

common/notifications.cpp

ev-mp · 2019-08-20T06:31:49Z

common/notifications.cpp

+        if (ImGui::IsItemHovered())
+        {
+            win.link_hovered();
+            ImGui::SetTooltip("Open release notes. Internet connection is required");


the Release Notes...

ev-mp · 2019-08-20T06:34:27Z

common/on-chip-calib.h

+
+        // Set calibration speed: 0 - slowest, 4 - fastest
+        // On-Chip Calib manager will reduce speed if needed
+        void set_speed(int speed) { _speed = speed; }


I would also check the input for validity

It is not user accessible

ev-mp · 2019-08-20T06:35:12Z

common/on-chip-calib.h

+        bool use_new_calib = true;
+        std::string _error_message = "";
+    };
+}


EOL is missing

ev-mp · 2019-08-20T06:36:49Z

src/ds5/ds5-private.cpp

-                intrinsics.fy = rect_params[1] * height / resolutions_list[resolution].y;
-                intrinsics.ppx = rect_params[2] * width / resolutions_list[resolution].x;
-                intrinsics.ppy = rect_params[3] * height / resolutions_list[resolution].y;
+                if (width == 256 && height == 144)


Pls put a comment that this shall be used in auto-calibration only

dorodnic added 3 commits August 18, 2019 06:32

Changing rolling-shutter base class to non-monochrome to better captu…

ec3f768

…re the family of devices using it

Handle special intrinsics of 256x144 (auto-calib) resolution

00510df

Refactoring of firmware-update UI base logic into process-manager class

6f2268b

Introducing auto-calibration process Adding metadata window for stream

dorodnic added the calibration label Aug 18, 2019

dorodnic requested a review from ev-mp August 18, 2019 14:34

dorodnic mentioned this pull request Aug 19, 2019

Where is the Intel.Realsense.DynamicCalibrator source file #4659

Closed

dorodnic force-pushed the auto_calib_rebased branch from 50aa36d to 23b9bc2 Compare August 19, 2019 10:51

Code review changes

9cad634

dorodnic force-pushed the auto_calib_rebased branch from 23b9bc2 to 9cad634 Compare August 19, 2019 12:31

Basic TARE calibration

ff485da

ev-mp reviewed Aug 20, 2019

View reviewed changes

common/fw-update-helper.cpp Show resolved Hide resolved

ev-mp reviewed Aug 20, 2019

View reviewed changes

common/notifications.cpp Show resolved Hide resolved

ev-mp reviewed Aug 20, 2019

View reviewed changes

common/on-chip-calib.h Outdated

bool use_new_calib = true;

std::string _error_message = "";

};

}

Copy link

Collaborator

ev-mp Aug 20, 2019

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

EOL is missing

ev-mp reviewed Aug 20, 2019

View reviewed changes

dorodnic added 2 commits August 20, 2019 00:58

More code review comments

f41123b

Ground truth in tare calibration is in uint32_t units (mm)

b6577da

ev-mp approved these changes Aug 20, 2019

View reviewed changes

ev-mp merged commit 033df88 into IntelRealSense:development Aug 20, 2019

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

D400 On-Chip Calibration #4658

D400 On-Chip Calibration #4658

dorodnic commented Aug 18, 2019

ev-mp Aug 20, 2019

ev-mp Aug 20, 2019

ev-mp Aug 20, 2019

ev-mp Aug 20, 2019

ev-mp Aug 20, 2019

ev-mp Aug 20, 2019

ev-mp Aug 20, 2019

ev-mp Aug 20, 2019

ev-mp Aug 20, 2019

ev-mp Aug 20, 2019

ev-mp Aug 20, 2019

ev-mp Aug 20, 2019

dorodnic Aug 20, 2019

ev-mp Aug 20, 2019

ev-mp Aug 20, 2019

dorodnic Aug 20, 2019

ev-mp Aug 20, 2019

ev-mp Aug 20, 2019

		else
		ImGui::Text("Firmware updates is underway...\nPlease do not disconnect the device");

D400 On-Chip Calibration #4658

D400 On-Chip Calibration #4658

Conversation

dorodnic commented Aug 18, 2019

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment