Aaron/fix pin to 0 code #686
Conversation
|
The big question, does it work? It seems to! Here we have 3 plots - data with the original calibration (no pin to origin), with our current pin to origin code (the big skeleton guy in orange), and with the fixed pin to origin code in the PR. The fixed pin to origin and the original calibration now line up exactly |
|
Another indication that it works: Before this, we were seeing that the video chosen as But now the data is much more closely aligned regardless of which video was chosen as |
|
What this also means is that the true issue causing the bugs mentioned in this issue (#681) was at the core, never related to the OS or the video order. The reason that when we sorted the videos we saw alignment is just because the Thankfully, this also means that the video chosen as |
|
I've added an extra step to zero out the rotation as well before zeroing out the translation, which seems to work? |
...processes/capture_volume_calibration/anipose_camera_calibration/anipose_camera_calibrator.py
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
This is really great work, and such a perfect example of good tech sleuthing to find this unknown silent bug in our reconstruction process. You really pulled the thread from the original discovery (and fixing) of the OS-dependent video ordering bug, to the discovery of that reconstruction depending on the choice of Camera0 into this fix with both gives us a fully 6DoF-zero'd Camera0 as well as more accurate scaling. I think this was also (one of?) the first times you properly schooled me on an issue of basic 3d geometry, which was honestly extremely gratifying. A lot of great work for a 1-file-change PR 😂
I'll merge this, and then figure out how to communicate changes in prep for a new minor release
|
Hey thanks for the kind words, I appreciate it. Glad that we were able to find an answer to some of the mystery scaling bugs that we had been seeing. |
After a good bit of digging, I believe I've found the issue the
pin_camera_0_to_origincode, which lies in how we're handling different coordinate systems.Currently we're trying to move each cameras
tvecby applying a transformation that would place camera 0's position at [0,0,0]. However, each camerastvecexists in its own local reference frame, not in the world coordinate system.So currently we calculate a translation in the context of camera 0's coordinate system and then try to apply this same translation to other cameras with different orientations. This means that this function hasn't been rigidly moving the cameras as a set - it's been changing the spatial relationships between cameras. This explains why different 'translations' that have been applied (for example, how which video was listed as 'camera 0' led to a different
translation to originvector) has led to differently scaled data.The solution that I've tried to implement here is to calculate the translation needed to get camera 0 to the origin in the world coordinate system, not in camera 0's specific coordinate system. Then, we take this world-space translation, transform it into each cameras local coordinate system, and apply it.