Page 95 - Read Online

Shi et al. Art Int Surg 2024;4:247-57 https://dx.doi.org/10.20517/ais.2024.17 Page 249

METHODS
Preliminaries
Vanilla reprojection loss
Vanilla reprojection loss calculates the photometric errors between the target frame and two temporally
adjacent frames (first adjacent) and finally chooses the minimum error between them. For example, if I is
t
the target frame at time t, I and I the two source frames can be denoted as I for simplicity, D depth
t-1
t
s
t+1
estimation for target frame, p the homogeneous coordinates and K intrinsic matrix, and then view synthesis
t
from I to I can be formulated as:
s
t
(1)
Then view synthesis p can be used to obtain synthesized source frame of I :
s→t
s→t
(2)

Finally, the pixel-wise Vanilla reprojection loss Loss from a source frame I to target frame I can be
s
rl
t
calculated using synthesized target frame from source I :
s→t
(3)

Where RL is the reprojection loss with combined SSIM and L1 losses. For the two source frames of I , I ,
t-1
t+1
the Vanilla reprojection loss can be formulated as:
(4)

Figure 1 demonstrates the reprojection loss with source and target view synthesis in the point cloud and
then projection back to synthesize the target frame from a source. In the view synthesis approach, back-
projection and reprojection are crucial steps. First, back-projection converts 2D source image pixels into a
3D point cloud usingdepth information and camera intrinsics. This point cloud is then transformed into the
target point cloud using thepredicted camera pose. Next, reprojection projects the 3D target point cloud
back onto the 2D image plane of thetarget camera using its intrinsic parameters.

Smoothness loss
To mitigate the issue of smoothing over edges, we integrate an edge-aware smoothness loss, as employed
in , into our approach. This loss function effectively reduces the weight in regions with strong intensity
[8]
gradients, thereby promoting local smoothness in the predicted depth map. This enables our model to
preserve sharp edges and fine details in the depth estimation process. The smoothness loss aims to optimize
the predicted depth by considering image gradients. It encourages depth estimation to adhere to local
smoothness patterns based on the intensity gradients present in the input images. By incorporating this loss
function, the model can effectively capture and preserve the continuity and smoothness of depth variations
across the image, resulting in more visually coherent and accurate depth predictions, which has
[7]
demonstrated success in .
(5)