深度估计 双目深度估计+单目深度估计 ONNX运行程序
双目深度估计ONNX Practical Stereo Matching via Cascaded Recurrent Network with Adaptive Correlation https://github.com/ibaiGorordo/ONNX-CREStereo-Depth-Estimation
-
双目深度估计需要从dataset读取左右两个view的图片。
-
使用模型进行深度图的估计
model_path = f'models/crestereo_{version}_iter{iters}_{shape[0]}x{shape[1]}.onnx'
depth_estimator = CREStereo(model_path)
disparity_map = depth_estimator(left_img, right_img)
CREStereo model: https://github.com/megvii-research/CREStereo
CREStereo - Pytorch: https://github.com/ibaiGorordo/CREStereo-Pytorch
PINTO0309's model zoo: https://github.com/PINTO0309/PINTO_model_zoo
PINTO0309's model conversion tool: https://github.com/PINTO0309/openvino2tensorflow
Driving Stereo dataset: https://drivingstereo-dataset.github.io/
Depthai library: https://pypi.org/project/depthai/
Original paper: https://arxiv.org/abs/2203.11483
单目深度估计monocular
GLPDepth
-
下载github代码(稍微修改一下，用本地读取替换在线读取)
-
去PINTO0309’s model zoo: https://github.com/PINTO0309/PINTO_model_zoo下载模型
-
运行bash
monodepth2 https://github.com/nianticlabs/monodepth2
python test_simple.py --image_path assets/test_image.jpg --model_name mono+stereo_640x192
with torch.no_grad():
for idx, image_path in enumerate(paths):
if image_path.endswith("_disp.jpg"):
print('don t try to predict disparity for a disparity image!')
continue
# Load image and preprocess
input_image = pil.open(image_path).convert('RGB')
original_width, original_height = input_image.size
input_image = input_image.resize((feed_width, feed_height), pil.LANCZOS)
input_image = transforms.ToTensor()(input_image).unsqueeze(0)
# PREDICTION
input_image = input_image.to(device)
features = encoder(input_image)
outputs = depth_decoder(features)
disp = outputs[("disp", 0)]
# 还原大小 torch.Size([1, 1, 235, 638])
disp_resized = torch.nn.functional.interpolate(
disp, (original_height, original_width), mode="bilinear", align_corners=False)
# Saving numpy file
output_name = os.path.splitext(os.path.basename(image_path))[0]
scaled_disp, depth = disp_to_depth(disp, 0.1, 100)
if args.pred_metric_depth:
name_dest_npy = os.path.join(output_directory, "{}_depth.npy".format(output_name))
metric_depth = STEREO_SCALE_FACTOR * depth.cpu().numpy()
np.save(name_dest_npy, metric_depth)
else:
name_dest_npy = os.path.join(output_directory, "{}_disp.npy".format(output_name))
np.save(name_dest_npy, scaled_disp.cpu().numpy())
# Saving colormapped depth image
disp_resized_np = disp_resized.squeeze().cpu().numpy()
vmax = np.percentile(disp_resized_np, 95)
normalizer = mpl.colors.Normalize(vmin=disp_resized_np.min(), vmax=vmax)
mapper = cm.ScalarMappable(norm=normalizer, cmap='magma')
colormapped_im = (mapper.to_rgba(disp_resized_np)[:, :, :3] * 255).astype(np.uint8)
im = pil.fromarray(colormapped_im)
name_dest_im = os.path.join(output_directory, "{}_disp.jpeg".format(output_name))
im.save(name_dest_im)
print(" Processed {:d} of {:d} images - saved predictions to:".format(
idx + 1, len(paths)))
print(" - {}".format(name_dest_im))
print(" - {}".format(name_dest_npy))
print('-> Done!')
有时没有网络的情况下运行报错urllib.error.URLError: <urlopen error [Errno -2] Name or service not known>
# download_model_if_doesnt_exist(args.model_name)
参考与更多
单目深度估计总结
深入研究自监督单目深度估计：Monodepth2
consistent-video-deprh-estimation：github上的一个有趣的应用
效果惊艳！ONNX-CREStereo深度估计！
monodepth https://github.com/mrharicot/monodepth
https://github.com/nianticlabs/monodepth2
Digging Into Self-Supervised Monocular Depth Estimation
https://arxiv.org/abs/1806.01260
- https://www.bilibili.com/video/BV1HZ4y1c7nC
