Aaron/fix pin to 0 code #686
Conversation
|
The big question, does it work? It seems to! Here we have 3 plots - data with the original calibration (no pin to origin), with our current pin to origin code (the big skeleton guy in orange), and with the fixed pin to origin code in the PR. The fixed pin to origin and the original calibration now line up exactly |
|
Another indication that it works: Before this, we were seeing that the video chosen as But now the data is much more closely aligned regardless of which video was chosen as |
|
What this also means is that the true issue causing the bugs mentioned in this issue (#681) was at the core, never related to the OS or the video order. The reason that when we sorted the videos we saw alignment is just because the Thankfully, this also means that the video chosen as |
|
I've added an extra step to zero out the rotation as well before zeroing out the translation, which seems to work? |
After a good bit of digging, I believe I've found the issue the
pin_camera_0_to_origincode, which lies in how we're handling different coordinate systems.Currently we're trying to move each cameras
tvecby applying a transformation that would place camera 0's position at [0,0,0]. However, each camerastvecexists in its own local reference frame, not in the world coordinate system.So currently we calculate a translation in the context of camera 0's coordinate system and then try to apply this same translation to other cameras with different orientations. This means that this function hasn't been rigidly moving the cameras as a set - it's been changing the spatial relationships between cameras. This explains why different 'translations' that have been applied (for example, how which video was listed as 'camera 0' led to a different
translation to originvector) has led to differently scaled data.The solution that I've tried to implement here is to calculate the translation needed to get camera 0 to the origin in the world coordinate system, not in camera 0's specific coordinate system. Then, we take this world-space translation, transform it into each cameras local coordinate system, and apply it.