Question: Are the middle-frame predictions (frames 1-5) expected to look point-cloud-like while frame 0 looks Gaussian-like? #5

Activity

1. primebo1 commented on Mar 20, 2026

   

   primebo1

   on Mar 20, 2026

   Collaborator

   More actions

   Hi Yukihiro, thank you very much for your question and illustration, which look pretty good.

   First, I would like to say that your understanding is completely correct:
   frame 0 is reconstructed directly, while frames 1–5 are temporally interpolated / predicted.

   Regarding your questions:

   1. Is it expected that the intermediate frames look significantly more point-cloud-like than the anchor frame?
   2. Is the strong visual gap between frame 0 and frames 1–5 the intended behavior of the current model?

   No. In fact, I do not think there is really a “point-cloud-like” rendering, since all the frames are rendered by Gaussians. The Gaussians in the intermediate frames are formed by moving and combining the Gaussians from the reference frames.

   The sparse-looking scenes you observed are a limitation of the current model, which is not an intended behavior. The prediction at every 6th frame is more accurate because it is reconstructed directly from seen images, while the interpolated frames may have defects due to factors such as inaccurate velocity field learning. In some regions, Gaussians may be missing, which makes the result look “point-cloud-like.” This is also one of our optimization targets.

   3. Is there a way to process every frame with Gaussian splatting, instead of having only the anchor frames look strongly Gaussian-like?

   Yes. We are currently working on a project to achieve this goal, but we cannot disclose more details for the moment. Please stay tuned.

   Sorry for not replying to your message in time. I hope this helps.

   👍React with 👍2Reacted by Haibao Yu and Yukihiro Saito
2. HCaelrs commented on Mar 24, 2026

   

   HCaelrs

   on Mar 24, 2026

   More actions

   It seems you have made many attempts on prob_all_render_frame_ids. Given prob_all_render_frame_ids = [0.7, 0.3, 0.2, 0.1, 0.1, 0.05, 0], will the model overfocus on the performance of frame 0 while neglecting that of input frame 6? Can adjusting prob_all_render_frame_ids alleviate this issue?

   👍React with 👍2Reacted by Yukihiro Saito and Qibin Hu
3. haibao-yu commented on Mar 24, 2026

   

   haibao-yu

   on Mar 24, 2026

   Contributor

   More actions

   @primebo1 Hi, thanks for releasing ReconDrive.

   I would like to confirm whether the current rendering behavior is expected.

   I reproduced the result with the official nuScenes mini sample and the released checkpoint, and I observed the following:

   • The prediction at every 6th frame (frame 0 of each window, e.g. 0, 6, 12, ...) looks relatively smooth and Gaussian-like.
   • The intermediate predictions (frames 1-5) look much more sparse / point-cloud-like.
   • This pattern is visible even on the official sample, so it does not seem to be caused only by custom data preprocessing.

   recondrive_nuscenes_mini_official_scene0103_compare_allframes.mp4
   My understanding is that:

   • frame 0 is reconstructed directly,
   • while frames 1-5 are temporally interpolated / predicted.

   So my questions are:

   1. Is it expected that the intermediate frames look significantly more point-cloud-like than the anchor frame?
   2. Is the strong visual gap between frame 0 and frames 1-5 the intended behavior of the current model?
   3. Is there a way to process every frame with Gaussian splatting, instead of having only the anchor frames look strongly Gaussian-like?

   I am attaching videos generated from the official nuScenes mini example:

   • GT / Pred / Left1m / Right1m comparison
   • all-frame version
   • stride-6 version

   The reason I am asking is that the anchor frames look closer to what I would expect from Gaussian splatting, but many of the middle frames look visually closer to sparse colored points.

   Any clarification would be very helpful.

   @yukkysaito @primebo1 Thank you for your interest in our work. To ensure the implementation is aligned with our framework, would you mind sharing your evaluation results for a quick cross-check?

   👍React with 👍1Reacted by Yukihiro Saito
4. yukkysaito commented on Mar 24, 2026

   

   yukkysaito

   on Mar 24, 2026

   Author

   More actions

   @primebo1 @haibao-yu
   Thank you for your reply. That clarified things.

   To ensure the implementation is aligned with our framework, would you mind sharing your evaluation results for a quick cross-check?

   Could you wait a moment?
5. 958207985 commented on Mar 27, 2026

   

   958207985

   on Mar 27, 2026

   More actions

   I encountered the same problem. I tested all 10 scenes in nuScenes mini, adjusted the frame skip to 1, and kept other parameters unchanged. The average PSNR was only 20.11. Is this abnormal? I found that the image distortion and quality were not normal.

   left_3.0m.mp4
6. primebo1 commented on Mar 27, 2026

   

   primebo1

   on Mar 27, 2026

   Collaborator

   More actions

   I encountered the same problem. I tested all 10 scenes in nuScenes mini, adjusted the frame skip to 1, and kept other parameters unchanged. The average PSNR was only 20.11. Is this abnormal? I found that the image distortion and quality were not normal.

   left_3.0m.mp4

   Hi,

   I’m afraid this is not expected. Could you first try reproducing our reported results using the exact scenes specified in our paper?

   official_scene_names = [
       'scene-0014', 'scene-0018', 'scene-0906', 'scene-0098',
       'scene-0100', 'scene-0103', 'scene-0270', 'scene-0271',
       'scene-0278', 'scene-0553', 'scene-0558',
       'scene-0802', 'scene-0968', 'scene-1065',
   ]

   The performance should exactly match what we reported. We have just verified the code in this repository, and it is functioning correctly.

   Also, could you confirm whether any modifications were made to the code or configuration?
7. hlnull commented on Mar 31, 2026

   

   hlnull

   on Mar 31, 2026

   Last edited by hlnull

   More actions

   Hi,

   Thanks for your great work.

   I encountered the same problem. The 0th and 6th images are correct, but the 1st through 5th images are incorrect.

   scene-0104_pred_grid_h264.mp4

   scene-0103_pred_grid_h264.mp4

   The entire environment was installed using Codex. There might be something misconfigured. I'll update this comment if I find the root cause.

   😕React with 😕1Reacted by Soverngity
8. 958207985 commented on Apr 1, 2026

   

   958207985

   on Apr 1, 2026

   More actions

   I encountered the same problem. I tested all 10 scenes in nuScenes mini, adjusted the frame skip to 1, and kept other parameters unchanged. The average PSNR was only 20.11. Is this abnormal? I found that the image distortion and quality were not normal.
   left_3.0m.mp4

   Hi,

   I’m afraid this is not expected. Could you first try reproducing our reported results using the exact scenes specified in our paper?

   official_scene_names = [
   'scene-0014', 'scene-0018', 'scene-0906', 'scene-0098',
   'scene-0100', 'scene-0103', 'scene-0270', 'scene-0271',
   'scene-0278', 'scene-0553', 'scene-0558',
   'scene-0802', 'scene-0968', 'scene-1065',
   ]
   The performance should exactly match what we reported. We have just verified the code in this repository, and it is functioning correctly.

   Also, could you confirm whether any modifications were made to the code or configuration?

   @primebo1 Thank you for your reply. I didn't change any of the relevant configurations. Could it be a problem with the dataset? You could also try the nuScenes mini dataset and see if the results are normal.

   👍React with 👍1Reacted by yhh