Introduction

Our research focuses on computer vision techniques for understanding the state of people and objects in a scene based on measurements from cameras and other sensors, as well as image and video synthesis techniques for generating more informative visual content. We also work on the underlying technologies of machine learning and sensing that support these topics. In particular, we are highly interested in scene understanding under challenging sensing conditions, such as when targets are partially or fully occluded, or in environments with low lighting or high noise. To address these issues, we explore sensing methods for occluded areas and estimation techniques using diverse sensor modalities. Our work also extends to broader areas including image recognition and medical image processing.

■ Estimation of Human Pose and Shape Using Diverse Sensors
We develop estimation techniques that leverage a variety of sensing modalities, including acoustic signals, wireless signals, and event-based cameras, to address scenarios where conventional RGB video struggles. These include scenes with low lighting, occlusion, fast motion, or heightened privacy concerns.
- Ryosuke Hori, Mariko Isogawa, Dan Mikami, Hideo Saito, "EventPointMesh: Human Mesh Recovery Solely from Event Point Clouds", IEEE TVCG, pp. 1-18, 2024.
- Kotaro Amaya, Mariko Isogawa. "Adaptive and Robust mmWave-based 3D Human Mesh Estimation for Diverse Poses", IEEE International Conference on Image Processing (ICIP), pp.455-459, 2023.
- Yuto Shibata, Yutaka Kawashima, Mariko Isogawa, Go Irie, Akisato Kimura, Yoshimitsu Aoki. "Listening Human Behavior: 3D Human Pose Estimation with Acoustic Signals", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp.13323-13332, 2023.

■ Imaging of Occluded Areas
We aim to enable the visualization and understanding of people and environments even when they are not directly visible to the sensor. This line of research has applications in safety monitoring, security, and situational awareness through technologies that allow posture estimation and imaging of occluded regions.
- Dixin Yang, Mariko Isogawa. "Efficient Circular and Confocal Non-Line-of-Sight Imaging with Transient Sinogram Super Resolution", IEEE International Conference on Image Processing (ICIP), pp. 2751-2757, 2024.
- Mariko Isogawa, Dorian Chan, Ye Yuan, Kris Kitani, and Matthew O'Toole, "Efficient Non-Line-of-Sight Imaging from Transient Sinograms", 16th European Conference on Computer Vision (ECCV), pp. 193-208, 2020.
- Mariko Isogawa, Ye Yuan, Matthew O'Toole, and Kris Kitani, "Optical Non-Line-of-Sight Physics-based 3D Human Pose Estimation", IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 7013-7022.

■ Image and Video Inpainting
Images and videos captured by cameras often include unwanted regions. We work on image and video inpainting techniques to remove such regions and restore the missing content, producing natural results.
- Mariko Isogawa, Dan Mikami, Kosuke Takahashi, Daisuke Iwai, Kosuke Sato, and Hideaki Kimata, "Which is the better inpainted image? Training data generation without any manual operations", International Journal of Computer Vision (IJCV), vol. 127, no. 11, pp. 1751-1766, 2019.

■ Medical Image Processing
In collaboration with Keio University School of Medicine, we also conduct research on medical video processing. For example, surgical videos are valuable for education and review, but important scenes are often occluded by the surgeon’s head or hands. We develop techniques to synthesize less-occluded viewpoint videos by combining footage captured simultaneously from multiple cameras.
- Yuna Kato, Mariko Isogawa, Shohei Mori, Hideo Saito, Hiroki Kajita, Yoshifumi Takatsume. "High-Quality Virtual Single-Viewpoint Surgical Video: Geometric Autocalibration of Multiple Cameras in Surgical Lights", International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI), pp.271–280, 2023.

Areas of Research

・Measurement Engineering
・Intelligent Information Science

Social Contributions

・Applications in elderly monitoring, crime prevention, and disaster rescue support
・Contributions to security, smart city development, and traffic safety systems
・Enhancing the quality of visual media through advanced image and video generation technologies
・Supporting medical education, postoperative review, and improvement of surgical quality