How to do coordinate transform in BlenderProc

[hanyangclarence]

Thank you a lot for your great work!

I am now working with render_dataset_improved_mat.py. For each rendered image, I also need the coordinate of the furnitures in the scene in camera coordinate. So I have to transform the location, which is in world coordinate, to camera coordinate.

The transformation is simple, but I am not sure how to verify it. To verify, I further transformed the camera coordinate to pixel coordinate, but the results always cannot align with my observation in the rendered image. I guess this is due to some differenct axis convention in Blender, OpenGL, OpenCV, ...., but I am really confused about this.

So do you know whether this transformation is correct or not? Thank you a lot!

Here is my code:

cam_K = bproc.camera.get_intrinsics_as_K_matrix()  # line 122
cam_Ts = bproc.math.build_transformation_mat(...)  # line 216

# world to camera
world_coord = info['location']  # [x, y, z] coordinate loaded from 3D-FRONT
world_coord_h = np.append(world_coord, 1)
world_to_cam = np.linalg.inv(cam_Ts)
camera_coord_h = world_to_cam @ world_coord_h
camera_coord = camera_coord_h[:3]

# camera to pixel
image_coord_h = camera_coord / camera_coord[2]
image_coord_h = cam_K @ image_coord_h
image_coord = image_coord_h[:2] / image_coord_h[2]