Qi Zhang (张琦)

I am currently a Researcher in the Visual Computing Center@Tencent AI Lab. I do research in 3D computer vision and computational photography, where my interests focus on neural radiance fields (NeRF) and neural rendering, 3D reconstruction and modeling, AI generated content (AIGC), image rectifcation, and light field imaging.

Before joining Tencent AI Lab in Jun. 2021, I received my Ph.D. degree from the School of Computer Science of Northwestern Polytechnical University in 2021, I was supervised by Prof. Qing Wang. I was a visiting student at the Australian National University (ANU) between Jul. 2019 to Aug. 2020, which was supervised by Prof. Hongdong Li. I received the Outstanding Doctoral Dissertation Award Nominee from China Computer Federation (CCF) in 2021. I also won the 2021 ACM Xi'an Doctoral Dissertation Award and the 2023 NWPU Doctoral Dissertation Award.

At Tencent, I've worked on 4D Content Generation and 3D Chat on Tencetn Meeting If you are interested in the internship about neural rendering (e.g. NeRF, SDF, inverse rendering), digital avatars, and 3D generation, please feel free to contact me via e-mail and wechat.

Email  /  CV  /  Bio  /  Google Scholar  /  WeChat  /  Github

🎉🎉2024.03: 5 papers accetped to CVPR 2024!
🎉🎉2024.02: DINER extended to TPAMI!
🎉🎉2023.12: NeIF accetped to AAAI 2023
🎉🎉2023.08: LoD-NeuS accetped to SIGGRAPH Asia 2023
🎉🎉2023.07: Pyramid NeRF accetped to IJCV 2023
🎉🎉2023.03: 7 papers (with 1 highlight paper) accetped to CVPR 2023!
🎉🎉2022.08: 1 paper (journal track) accetped to SIGGRAPH Asia 2022
🎉🎉2022.03: 4 papers accetped to CVPR 2022
🎉🎉2021.12: CCF Outstanding Doctoral Dissertation Award Nominee

Please find below a complete list of my publications with representative papers highlighted. The IEEE TPAMI, IJCV are top journals in the field of computer vision and computational photography. The CVPR is the premier conference in Computer Vision research community.

We propose GS-IR, a novel inverse rendering approach based on 3D Gaussian Splatting (GS) that leverages forward mapping volume rendering to achieve photorealistic novel view synthesis and relighting results.

In this survey, we aim to introduce the fundamental methodologies of 3D generation methods and establish a structured roadmap, encompassing 3D representation, generation methods, datasets, and corresponding applications.

In this paper, we introduce a texture-consistent back view synthesis module that could transfer the reference image content to the back view through depth and text-guided attention injection with the help of stable diffusion model.

We propose GS-IR, a novel inverse rendering approach based on 3D Gaussian Splatting (GS) that leverages forward mapping volume rendering to achieve photorealistic novel view synthesis and relighting results.

We propose HumanNorm, a novel approach for high-quality and realistic 3D human generation by learning the normal diffusion model including a normal-adapted diffusion model and a normal-aligned diffusion model.

We have identified that this frequency-related problem can be greatly alleviated by introducing variable-periodic activation functions, for which we propose FINER.

HumanRef, a reference-guided 3D human generation framework, is capable of generating 3D clothed human with realistic, view-consistent texture and geometry from a single image input with the help of stable diffusion model.

In this paper, we introduce a texture-consistent back view synthesis module that could transfer the reference image content to the back view through depth and text-guided attention injection with the help of stable diffusion model.

We propose a fully differentiable framework named neural ambient illumination (NeAI) that uses Neural Radiance Fields (NeRF) as a lighting model to handle complex lighting in a physically based way.

In this paper, we find that such a frequency-related problem could be largely solved by re-arranging the coordinates of the input signal, for which we propose the disorder-invariant implicit neural representation (DINER) by augmenting a hash-table to a traditional INR backbone.

Our method, called LoD-NeuS, adaptively encodes Level of Detail (LoD) features derived from the multi-scale and multi-convoluted tri-plane representation. By optimizing a neural Signal Distance Field (SDF), our method is capable of reconstructing high-fidelity geometry

In this paper, we propose the Pyramid NeRF, which guides the NeRF training in a 'low-frequency first, high-frequency second' style using the image pyramids and could improve the training and inference speed at 15x and 805x, respectively.

We propose a new content-aware optimization framework to preserve both local conformal shape (e.g. face or salient regions) and global linear structures (straight lines).

this paper proposes a novel implicit camera model which represents the physical imaging process of a camera as a deep neural network. We demonstrate the power of this new implicit camera model on two inverse imaging tasks: i) generating all-in-focus photos, and ii) HDR imaging.

We propose LIRF (Local Implicit Ray Function), a generalizable neural rendering approach for novel view rendering. Given 3D positions within conical frustums, LIRF takes 3D coordinates and the features of conical frustums as inputs and predicts a local volumetric radiance field.

In this paper, we find that such a frequency-related problem could be largely solved by re-arranging the coordinates of the input signal, for which we propose the disorder-invariant implicit neural representation (DINER) by augmenting a hash-table to a traditional INR backbone.

We propose L2G-NeRF, a Local-to-Global registration method for bundle-adjusting Neural Radiance Fields, including the pixel-wise local alignment and the frame-wise global alignment.

We propose the UV Volumes, a new approach that can achieve real-time rendering, and editable NeRF, decomposing a dynamic human into 3D UV Volumes and a 2D appearance texture.

We present a novel paradigm for high-fidelity face swapping that faithfully preserves the desired subtle geometry and texture details.

We try to introduce explicit parameters into implicit dynamic NeRF representations to achieve editing of 3D human heads.

We propose a novel multiple homography image (MHI) representation, comprising of a set of scene planes with fixed normals and distances, for view synthesis from stereo images.

We present High Dynamic Range Neural Radiance Fields (HDR-NeRF) to recover an HDR radiance field from a set of low dynamic range (LDR) views with different exposures.

This paper studies the problem of hallucinated NeRF: i.e. recovering a realistic NeRF at a different time of day from a group of tourism images.

In this paper, we propose Deblur-NeRF, the first method that can recover a sharp NeRF from blurry input. A novel Deformable Sparse Kernel (DSK) module is presented for both camera motion blur and defocus blur.

A 3D-aware generator (FENeRF) is prorposed to produce view-consistent and locally-editable portrait images.

This paper fills in this gap by developing a novel ray-space epipolar geometry which intrinsically encapsulates the complete projective relationship between two light fields. Ray-space fundamental matrix and its properties are then derived to constrain ray-ray correspondences for general and special motions.

This paper is concerned with the problem of multi-view 3D reconstruction with an un-calibrated micro-lens array based light field camera..

By reinterpreting the usage of dark channels in estimating the amount of defocus, a novel region-based depth feature descriptor (RDFD) defined over the focal stack is proposed.

Our method employs the structure delivered by the four-dimensional light field over multiple views making use of superpixels for a full view optical flow estiamtion.

In the paper, we propose a novel ray-space projection model to transform sets of rays captured by multiple light field cameras in term of the Plucker coordinates.

The light field superpixel (LFSP) is first defined mathematically and then a refocus-invariant metric named LFSP self-similarity is proposed to evaluate the segmentation performance.

Instead of a planar checkerboard, we propose to calibrate light field camera using a concentric conics pattern based on the property and reconstruction of common self-polar triangle with respect to concentric circle and ellipse..

The MPC model can generally parameterize light field in different imaging formations, including conventional and focused light field cameras.

Qi Zhang, Qing Wang. Common self-polar triangle of separate circles for light field camera calibration[J]. Xibei Gongye Daxue Xuebao/Journal of Northwestern Polytechnical University, 2021.
Yaning Li, Qi Zhang, Xue Wang, Qing Wang. Light Field SLAM based on Ray-Space Projection Model[C]. Optoelectronic Imaging and Multimedia Technology VI, 2019.
Qi Zhang, Xue Wang, Qing Wang. Light Field Planar Homography and Its Application[C]. Optoelectronic Imaging and Multimedia Technology VI, 2019.
Zhao Ren, Qi Zhang, Hao Zhu, Qing Wang. Extending the FOV from disparity and color consistencies in multiview light fields[C]. IEEE International Conference on Image Processing (ICIP), 2017

This template is a modification to Jon Barron's website. Feel free to clone it for your own use while attributing the original author Jon Barron.