Publisher：Proceedings - 2023 IEEE Winter Conference on Applications of Computer Vision, WACV 2023  

Photometric stereo (PS) is a major technique to recover surface normal for each pixel. However, since it assumes Lambertian surface and directional light to estimate the value, a large number of images are usually required to avoid the effects of outliers and noise. In this paper, we propose a technique to reduce the number of images by using distributed light sources, where the patterns are optimized by a deep neural network (DNN). In addition, to efficiently realize the distributed light, we use an optical diffuser with a video projector, where the diffuser is illuminated by the projector from behind, the illuminated area on the diffuser works as if an arbitrary-shaped area light. To estimate the surface normal using the distributed light source, we propose a near-light photometric stereo (NLPS) using DNN. Since optimization of the pattern of distributed light is achieved by a differentiable renderer, it is connected with NLPS network, achieving end-to-end learning. The experiments are conducted to show the successful estimation of the surface normal by our method from a small number of images.

DOI： 10.1109/WACV56688.2023.00039

Web of Science

Scopus

Publisher：Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)  

Deep neural architecture plays an important role in underwater image enhancement in recent years. Although most approaches have successfully introduced different structures (e.g., U-Net, generative adversarial network (GAN) and attention mechanisms) and designed individual neural networks for this task, these networks usually rely on the designer’s knowledge, experience and intensive trials for validation. In this paper, we employ Neural Architecture Search (NAS) to automatically search the optimal U-Net architecture for underwater image enhancement, so that we can easily obtain an effective and lightweight deep network. Besides, to enhance the representation capability of the neural network, we propose a new search space including diverse operators, which is not limited to common operators, such as convolution or identity, but also transformers in our search space. Further, we apply the NAS mechanism to the transformer and propose a selectable transformer structure. In our transformer, the multi-head self-attention module is regarded as an optional unit and different self-attention modules can be used to replace the original one, thus deriving different transformer structures. This modification is able to further expand the search space and boost the learning capability of the deep model. The experiments on widely used underwater datasets are conducted to show the effectiveness of the proposed method. The code is released at https://github.com/piggy2009/autoEnhancer.

DOI： 10.1007/978-3-031-26313-2_8

Web of Science

Scopus

Publisher：Proceedings - International Conference on Pattern Recognition  

High-precision three-dimensional scanning systems have been intensively researched and developed. Recently, for acquisition of large scale scene with high density, simultaneous localisation and mapping (SLAM) technique is preferred because of its simplicity; a single sensor that is moved around freely during 3D scanning. However, to integrate multiple scans, captured data as well as position of each sensor must be highly accurate, making these systems difficult to use in environments not accessible by humans, such as underwater, internal body, or outer space. In this paper, we propose a new, flexible system with multiple line lasers that reconstructs dense and accurate 3D scenes. The advantages of our proposed system are (1) no need of synchronization nor precalibration between lasers and a camera, and (2) the system can reconstruct 3D scenes in extreme conditions, such as underwater. We propose a new self-calibration method leveraging coplanarity and Epipolar constraints is proposed. We also propose a new bundle adjustment (BA) technique that is tailored to the system for a dense integration of multiple line laser scans. Experimental evaluation in both air and underwater environments confirms the advantages of the proposed method.

DOI： 10.1109/ICPR56361.2022.9956128

Web of Science

Scopus

Publisher：Proceedings - International Conference on Image Processing, ICIP  

There are various demands for underwater 3D reconstruction, however, since most active stereo 3D reconstruction methods focus on the air environment, it is difficult to directly apply them to underwater due to the several critical reasons, such as refraction, water flow and severe attenuation. Typically, calibration-markers or laser-lines are strongly blurred and saturated by attenuation, which makes difficult to recover shape in the water. Another problem is that it is difficult to keep cameras, projectors and objects static in the water because of strong water flow, which prevents accurate calibration. In this paper, we propose a method to solve those problems by novel algorithm using deep neural network (DNN), epipolar constraint and specially designed devices. We also built a real system and tested it in the water, e.g., pool and sea. Experimental results confirmed the effectiveness of the proposed method. We also demonstrated real 3D scan in the sea.

DOI： 10.1109/ICIP46576.2022.9897733

Web of Science

Scopus

Publisher：Proceedings - International Conference on Pattern Recognition  

We propose a technique to estimate a high-resolution depth image from a sparse depth image captured by depth camera and a high-resolution shading image obtained by a RGB camera using deep neural network (DNN). In our technique, the network model is pretrained by synthetic images which are generated by rendering high-frequency shapes created by arithmetic model, such as sinusoidal wave of wide variation of parameters. Although the preparation of an appropriate synthetic dataset is critical for such tasks, it is not trivial to find a compact and optimal distribution of shape parameters. In this paper, we propose an auto augmentation technique to optimize hyperparameters for shapes achieving minimum number for training DNN. The proposed augmentation network directly optimizes the hyperparameters of a 3D scene including parameters of procedural shapes and their positions by gradient descent algorithm via a differentiable rendering technique. Unlike previous data augmentation techniques which only have basic image processing methods, such as affine and color transformations, the proposed method can generate optimal training dataset by changing the 3D shape and its position by using a differentiable renderer. In our experiments, we confirmed that our method improved the accuracy of high-resolution depth estimation as well as efficiency of training the network.

DOI： 10.1109/ICPR56361.2022.9956528

Web of Science

Scopus

Language：English   Publishing type：Research paper (scientific journal)  

We tackle an optical measurement of the internal structure of a diffuse surface object—we define as an object that has a diffuse surface and its interior is transparent, like grapes or hollow plastic bottles. Our approach is based on optical tomography that reconstructs the interior from observations of absorption of light rays from various views, under the projection of the light. The difficulty lies in the fact that a light ray that enters changes its direction at the interaction of the surface, unlike X-ray that travels straight through the object. We introduce a model of light path in the object called shortest-path model. We acquire the absorption of light rays through the object by the measurement upon the assumption of the model. Since this measurement acquires insufficient observation to reconstruct the interior by conventional reconstruction algorithms, we also introduce a reconstruction method based on a numerical optimization that a physical requirement of the absorption is taken into account. Our method is confirmed successful to measure the interior in a real-world experiment.

DOI： 10.1186/s41074-018-0051-x

Language：English   Publishing type：Research paper (other academic)  

Light interacts with a scene in various ways. For scene understanding, a light transport is useful because it describes a relationship between the incident light ray and the result of the interaction. Our goal is to acquire the 4D light transport between the projector and the camera, focusing on direct and short-range transport that include the effect of the diffuse reflections, subsurface scattering, and inter-reflections. The acquisition of the light transport is challenging since the acquisition of the full 4D light transport requires a large number of measurement. We propose an efficient method to acquire short range light transport, which is dominant in the general scene, using synchronized projector-camera system. We show the transport profile of various materials, including uniform or heterogeneous subsurface scattering.

DOI： 10.1145/3281505.3283377

Language：English   Publishing type：Research paper (other academic)  

Separation of light transport into direct and indirect paths has enabled new visualizations of light in everyday scenes. However, indirect light itself contains a variety of components from subsurface scattering to diffuse and specular interreflections, all of which contribute to complex visual appearance. In this paper, we present a new imaging technique that captures and analyzes these components of indirect light via light transport between epipolar planes of illumination and rays of received light. This plane-to-ray light transport is captured using a rectified projector-camera system where we vary the offset between projector and camera rows (implemented as synchronization delay) as well as the exposure of each camera row. The resulting delay-exposure stack of images can capture live short and long-range indirect light transport, disambiguate subsurface scattering, diffuse and specular interreflections, and distinguish materials according to their subsurface scattering properties.

DOI： 10.1109/ICCPHOT.2018.8368461

Language：English   Publishing type：Research paper (other academic)  

Acquisition of parameters for the Bidirectional Scattering Surface Reflectance Distribution Function (BSSRDF) has significant meanings in the study of computer graphics and vision research field. In this paper, we present an inverse rendering approach combined with a newly developed BSSRDF model, directional dipole model, for parameter estimation. To validate our algorithm, we estimate parameters from spheres with a wide range of radius in a simulated and real environment, respectively. According to the observations from both simulated and real experiments, we find that surface curvature significantly affects the estimation results.

DOI： 10.1109/NICOInt.2017.20

Language：English   Publishing type：Research paper (scientific journal)  

We aim to estimate internal slice of the scattering material by Computed Tomography (CT). In the scattering material, the light path is disturbed and is spread. Conventional CT cannot measure the scattering material because they rely on the assumption that rays are straight and parallel. We propose light path alignment to deal with scattering rays. Each path of disturbed scattering light is approximated with a straight line. Then the light path in the object corresponding to a single incident ray is modeled as straight paths spreading from incident point. These spreading paths are aligned to be parallel, so that they can be used directly by conventional reconstruction algorithm.

DOI： 10.1186/10.1186/s41074-016-0003-2

Event date： 2021.9

Language：English   Presentation type：Oral presentation (general)  

Venue：オンライン   Country：United States  

Presentation type：Oral presentation (general)  

Event date： 2023.7

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：浜松   Country：Japan  

Event date： 2023.7

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：浜松   Country：Japan  

Event date： 2023.7

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：浜松   Country：Japan  

Event date： 2023.7

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：浜松   Country：Japan  

Event date： 2023.7

Language：English   Presentation type：Oral presentation (general)  

Venue：浜松   Country：Japan  

Event date： 2022.12

Language：English   Presentation type：Oral presentation (general)  

Venue：静岡   Country：Japan  

Event date： 2022.7

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：姫路   Country：Japan  

Event date： 2022.7

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：姫路   Country：Japan  

Event date： 2022.7

Language：English   Presentation type：Oral presentation (general)  

Venue：姫路   Country：Japan  

Event date： 2022.7

Language：Japanese   Presentation type：Oral presentation (general)  

Country：Japan  

Event date： 2022.7

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：姫路   Country：Japan  

Event date： 2022.6

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：オンライン   Country：Japan  

Event date： 2022.6

Language：English   Presentation type：Oral presentation (general)  

Venue：オンライン   Country：Japan  

Event date： 2022.6

Language：English   Presentation type：Oral presentation (general)  

Venue：オンライン   Country：Japan  

Event date： 2022.6

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：オンライン   Country：Japan  

Event date： 2021.10

Language：English   Presentation type：Symposium, workshop panel (public)  

Venue：Onine   Country：Japan  

Event date： 2021.8

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：オンライン   Country：Japan  

Event date： 2021.7 - 2022.7

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：オンライン   Country：Japan  

Event date： 2020.6

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：大阪   Country：Japan  

Event date： 2020.6

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：オンライン   Country：Japan  

Event date： 2020.6

Language：Japanese   Presentation type：Oral presentation (general)  

Venue：Kusatsu   Country：Japan  

Event date： 2020.2

Language：English   Presentation type：Symposium, workshop panel (public)  

Venue：Ibusuki   Country：Japan