Probem of the result of test set that submitt on the KITTI benchmark #40

Feidashen1 · 2024-02-25T13:40:52Z

Hello, your work is very meaningful, but when I reproduced this work, I verified that the results in the paper could be achieved on the validation set. However, after submitting the results of the test set on the Kitti platform, there was a significant gap between your results and yours.

What do you think are the possible problems? How can I achieve results similar to yours on the test set?

Liu202209 · 2024-04-19T02:37:29Z

Hi, I would like to ask if you have reproduced the multi-modal results in the paper?Because my reproduction results were terrible:
bbox AP:98.9466, 90.4582, 89.9018
bev AP:90.1253, 88.8376, 80.2418
3d AP:89.7045, 87.3508, 79.3342
aos AP:98.53, 89.77, 88.96
Car AP_R40@0.70, 0.70, 0.70:
bbox AP:99.5549, 96.5883, 91.5515
bev AP:96.0639, 92.4806, 85.3564
3d AP:95.1403, 87.0011, 82.0037
aos AP:99.16, 95.74, 90.56
Car AP@0.70, 0.50, 0.50:
bbox AP:98.9466, 90.4582, 89.9018
bev AP:99.0606, 90.4330, 89.9194
3d AP:99.0323, 90.4258, 89.8929
aos AP:98.53, 89.77, 88.96
Car AP_R40@0.70, 0.50, 0.50:
bbox AP:99.5549, 96.5883, 91.5515
bev AP:99.5883, 96.5739, 91.5433
3d AP:99.5789, 96.5555, 91.5227
aos AP:99.16, 95.74, 90.56
which the score_threshold is 0.3. I have tried to set the score_threshold as 0.5, 0.7,which is worse.

ty-zeck · 2024-12-01T00:47:00Z

你好，想问一下你已经复现过论文中的多模态结果吗？因为我的复现结果很糟糕： bbox AP:98.9466, 90.4582, 89.9018 bev AP:90.1253, 88.8376, 80.2418 3d AP:89.7045, 87.3508, 79.3342 aos AP:98.53, 89.77, 88.96 汽车 AP_R40@0.70, 0.70, 0.70: bbox AP:99.5549, 96.5883, 91.5515 bev AP:96.0639, 92.4806, 85.3564 3d AP:95.1403, 87.0011, 82.0037 aos AP:99.16, 95.74, 90.56 汽车 AP@0.70, 0.50, 0.50: bbox AP:98.9466, 90.4582, 89.9018 bev AP:99.0606, 90.4330, 89.9194 3d AP:99.0323, 90.4258, 89.8929 aos AP:98.53, 89.77, 88.96 汽车 AP_R40@0.70, 0.50, 0.50: bbox AP:99.5549, 96.5883, 91.5515 bev AP:99.5883, 96.5739, 91.5433 3d AP:99.5789, 96.5555, 91.5227 aos AP:99.16, 95.74, 90.56 其中score_threshold 为0.3。我尝试将score_threshold设置为0.5、0.7，结果更糟糕。

我多模态的复现结果也很糟糕，而且epoch20-30评估的时候结果是很震荡的。

Liu202209 · 2024-12-01T09:05:47Z

ty-zeck · 2024-12-01T15:37:29Z

有一次的复现结果比较接近，但是还是差一些，差1%吧，主要是汽车中等类，当时的复现文件找不到了

同学，你是复现了很多次才得到一次接近的结果吗，我目前复现了两次，最高的也差好几个点，而且每个epoch的评估结果震荡很大。

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Probem of the result of test set that submitt on the KITTI benchmark #40

Probem of the result of test set that submitt on the KITTI benchmark #40

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Probem of the result of test set that submitt on the KITTI benchmark #40

Probem of the result of test set that submitt on the KITTI benchmark #40

Comments

Uh oh!

Uh oh!

Uh oh!

Uh oh!