Demystifying Depth estimation with Intel RealSense D435i

Depth estimation is an essential task in computer vision. Estimating the distance of object from camera is a not so easy but necessary operations in applications in the field of robotics, autonomous driving ,3D reconstruction and Augmented reality. In robotics, depth is a key prerequisite to perform multiple tasks such as perception, navigation, and planning.

The purpose of this article is to explain depth estimation using stereo vision, Intel RealSense camera is used to explain the concept.

In computer vision, depth can be extracted using either single camera or two cameras called

• Monocular depth estimation (static or sequential)

• Stereo vision (Epipolar geometry)

There are many Challenges in the estimation process including

• Occlusion

• Moving objects in the background

• Imperfect stereo correspondence

Stereo imaging make use of two cameras to calculate depth which enable devices to view, understand, and in-tern interact with their environment

The Intel® RealSense™ D435i is member of D400 series of Intel’s latest depth‑sensing hardware. It works on stereoscopic vision principle. It is Perfect for developers, makers, and innovators working to bring depth sensing to intelligent vision‑equipped solutions like navigation, mapping ,scene understanding, object measurement ,Human machine interface ,Gesture and pose estimation

It can be characterized by

• Ease of setup

• small form factor/portability

• high depth resolution

• Active IR Stereo

• Wide Field of view

• Indoor as well as outdoor use

One of the additional feature of D435i is an inertial measurement unit (IMU) which helps in finding movement and orientation of device in 3D Space in six degrees of freedom.

We will study it in detail afterwards.

---

Getting Started with Intel RealSense

In order to start exploring Depth with Intel RealSense you need

• RealSense Camera

• Laptop/PC installed with

Intel® RealSense™ SDK 2.0

The camera has Baseline=50.2198mm i.e distance between left and right camera is 50.2198 mm

Intel® RealSense™ SDK 2.0 allows you to Start building your own depth application. It supports windows, linux,Mac as well as Android operating system. It can even be used with Raspberry Pi3 ,Jetson TX2 .

It supports many platforms and Languages including

• Python

• Unity

• MATLAB

• OpenCV

• Labview

We will be exploring Unity 3D and MATLAB.

You can download the sdk and Viewer depending on your device from

https://www.intelrealsense.com/sdk-2/

Depth viewer allows you to access to Real sense camera functionality through a simple, cross-platform User Interface.

---

After installing Intel® RealSense™ SDK 2.0 and Viewer and connecting RealSense to USB

Now ,you can Toggle Stereo and RGB Camera Module .You will be able to see screen with RGB image and depth map with 4 different depth levels as shown in figure.

Understanding the controls

Controls under Stereo Modules

Depth Visualization

You may change Visual Preset as well as color scheme

Preset available are Near,Far,Fixed and dynamic while the colors schemes like jet,black to white ,white to black and classic are available .The black color represents objects that are far while blue represents nearer objects.

The warm color scheme is given below

Number of meters represented by a single depth unit can be changed to change depth levels.

Under depth visualization

You may adjust min and maximum distance

Figure below shows 8 different depth ,In addition Some applications are intolerant of holes in the depth. For this, it becomes necessary to fill in the holes with best guesses based on neighboring values, or the RGB image.Output after applying hole filling filter.