Project Page | Paper

Code for our paper: Latent Intuitive Physics: Learning to Transfer Hidden Physics from A 3D Video.

Xiangming Zhu, Huayu Deng, Haochen Yuan, Yunbo Wang^†, Xiaokang Yang

ICLR 2024

1. Create an environment

   conda create -n lip-env python=3.9
   conda activate lip-env

2. PyTorch and PyTorch3D

   Install PyTorch and PyTorch3D following the official guide.

   conda install pytorch=1.13.0 torchvision pytorch-cuda=11.6 -c pytorch -c nvidia
   conda install -c fvcore -c iopath -c conda-forge fvcore iopath
   conda install -c bottler nvidiacub
   conda install pytorch3d -c pytorch3d

3. Other dependencies

   pip install -r requirements.txt

Download the data and pretrained checkpoints from Google Drive and place them under ./data and ./pretrained_ckpts.

• Stage B: Visual posterior inference

python train_lip.py --expdir exps/cuboid_2000_0.065/stageB \
                            --expname latent1e-4 \
                            --config configs/stageB/cuboid.yaml \
                            --dataset cuboid_2000_0.065

• Stage C: Physical prior adaptation

python train_lip.py --expdir exps/cuboid_2000_0.065/stageC \
                            --expname encoder1e-4/ \
                            --config configs/stageC/cuboid.yaml \
                            --dataset cuboid_2000_0.065

If you find our work helps, please cite our paper.

@inproceedings{zhu2024lip,
  author={Zhu, Xiangming and Deng, Huayu and Yuan, Haochen and Wang, Yunbo and Yang, Xiaokang},
  title={Latent Intuitive Physics: Learning to Transfer Hidden Physics from a 3D Video},
  booktitle = {International Conference on Learning Representations},
  year={2024}
}

The implementation is based on the following repos:

https://github.com/syguan96/NeuroFluid

https://github.com/isl-org/DeepLagrangianFluids