This paper may help a lot!
Given the surface reflectivity of an object, normal factor directions and other data, we are able to get the render results of various illumination conditions and observation views, based on some Surface Reflection Model(Lambertian).
With the development of 3D Rendering Technologies, more accurate models of reflection have been proposed, to make the render results as close to the real world images as possible. We consider the reverse process of the above process, that is, given the rendering results of different illumination and views, the surface reflectance and normal vector information of the object can be inferred through the surface reflection model, which can further obtain depth information for 3D reconstruction. The results of 3D reconstruction can be verified by the forward rendering effects.
However, in the general natural environment, illumination information is extremely complex and difficult to obtain, so it is not easy to realize the above prediction.
Under the condition of a single constant intensity light source in a fixed direction, the luminance of each point on the human face is closely related to the direction of the normal vector of the point.
Firstly, to simplify the problem, we consider a complete diffuse model. For each point of the forward face image
Let the direction of light be
Therefore, when the number of images in the training set exceeds
On the basis of the 3D reconstruction results, we can further render the face images under the new light angles.
For the images, the amplitude array of the 2D Fourier transform contains the gray information, while the phase array carries more information about the location of distinguishable objects in the images. In general, phases contain more visual information.
The two-dimensional FFT decomposes the image into several waves in the complex plane.
The translation in the time domain is equivalent to the phase shift in the frequency domain. The spatial position information of the image is contained in the phase, the amplitude in the space determines the signal strength after frequency domain transformation, and the phase determines the position information.
2D Fourier transform:
Analogous to a one-dimensional sine wave, a two-dimensional sine wave can be determined by frequency
To know more details, click here.
Perform a 2D Fourier transform on the partial derivative information obtained from
In the image used for training, for each pixel, if the illuminance is less than the average value or greater than the set threshold, we consider that point as a highlight point or an invalid point in the shadow, and mark it as invalid (not involved
Consider that if the angle between the incident direction of the light source and the X-Y plane is small enough, the image will be shaded by the convex part. Therefore, it is judged whether each point in the depth matrix will be occluded by the point in the incident direction of the light, and a shadow matrix is generated. Finally, blur the edges of the shadows.
For the rendered face images, based on the high degree of symmetry of the face in reality, we optimize the depth matrix to compensate for the asymmetry caused by the error in the calculation of the
The original rendered image is very sharp at the edges of the background. The dark background optimization is based on the pixels that are close to the non-shadows. The closer the distance, the greater the weight.