丹羽遼吾

University of Tsukuba, 2nd Year PhD Students

Digital Nature Group, JSPS DC1,

Part-time Researcher, Digital Nature Research and Development Center

Interests: Human Computer Interaction / Real-world oriented interface / Signal Processing / Computer Vision / High-speed Sensing / Event Camera

Research

Event-based Visual Microphone

We have developed a new sound recording system that does not require a microphone. Conventional methods such as LDV and high-speed cameras had issues with high cost, and the trade-off between shooting speed and resolution. Therefore, we propose a method using an event camera that only records changes in brightness, and we have successfully reconstructed the sound. This method does not require bright light, can record sound at a low cost, and offers new possibilities in the measurement of vibrations and sounds. It has the potential to record sounds from distances that are difficult with conventional microphones, and to clearly record only targeted sounds.

Air Talk-Starter

One of the biggest challenges that deaf and hard-of-hearing individuals face is not realizing when they are being spoken to. It's hard to notice the presence or degree of hearing impairment just by appearance or behavior, and not realizing when they're being spoken to often leads to significant losses in communication opportunities. Air Talk-Starter aims to mitigate this problem by directing a vortex of air at the person being spoken to, thereby indicating the direction from which the communication is coming and making it easier to initiate a conversation.

Drone Badminton

Drone Badminton makes it possible for visually impaired people to participate in a rally sport that involves recognizing a ball moving in the air, by using a drone as the ball. Players only need to prepare a specialized racket and a drone to participate. We have developed a system where the drone automatically moves in the opponent's direction once it passes through the frame, by attaching sensors in place of the racket strings. This product not only holds the potential to significantly broaden the variety of sports for the visually impaired, but also allows those with partial vision loss to re-experience sports where one used to look up and swing a racket.

EMS-Supported Throwing

In this study, we propose a learning support system that combines EMS (Electrical muscle stimulation) and prediction. The system is used to instruct the release point in order to improve underhand throw control ability, and we verified its learning effect. Experimental results showed that EMS was effective in instructing the release point of the ball, suggesting potential for supporting motor learning by applying EMS.

How See the Colorful Scenery?

Because existing description apps for visually impaired people often don't provide sufficient information about colors, researchers added extra descriptions to try and enrich the visual experience. In the experiments, while some participants expressed that they could imagine the colors, others found the descriptions hard to understand. Going forward, the aim is to build a corpus based on descriptions for people with disabilities, with a view to allowing for instantaneous and detailed comprehension of landscapes.

職歴

University of Tsukuba, Master's Program in Informatics, Ph.D. Candidate Apr 2024 - Present

Digital Nature Development Center, Part-time Researcher Dec 1, 2024 - Present

OMRON SINIC X Corporation, Research Intern Oct 2023 - Jun 2025

SONY Group R&D, Intern Sep 1 - Sep 31, 2022

University of Tsukuba, Graduate School of Informatics, Master's Program Apr 2022 - Mar 2024

AIST, Technical Staff Dec 2020 - Mar 2021

University of Tsukuba, College of Media Arts, Transfer Student Apr 2020 - Mar 2022

Nagoya University, School of Informatics, Department of Natural Informatics Apr 2017 - Mar 2020

Publication

Google Scholar

International Conference (peer reviewed)

Cooperative Design Optimization through Natural Language Interaction, Ryogo Niwa, Shigeo Yoshida, Yuki Koyama, Yoshitaka Ushiku, User Interface Software and Technology (UIST'25) (coming soon!)
Event-based Visual Microphone, Ryogo Niwa, Tatsuki Fushimi, Kenta Yamamoto, and Yoichi Ochai,
proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, pp. 4054-4055, 2023
Demonstrating SHITARA: Sending Haptic Induced Touchable Alarm by Ring-shaped Air vortex, Ryosei Kojima, Akihisa Shitara, Tatsuki Fushimi, Ryogo Niwa, Atsushi Shinoda, Ryo Iijima, Kengo Tanaka, Sayan Sarcar, and Yoichi Ochiai, Conference on Human Factors in Computing Systems
(CHI' 23) hot desk demo
A Preliminary Study on Eye Contact Framework toward Improving Gaze Awareness in Video Conferences, Kazuya Izumi, Shieru Suzuki, Ryogo Niwa, Atsushi Shinoda, Ryo Iijima, Ryosuke Hyakuta, and Yoichi Ochiai, Human Computer Interaction International 2023 (HCII' 23)
EMS-Supported Throwing:Preliminary investigation on EMS-supported training of movement form, Ryogo Niwa, Kazuya Izumi, Shieru Suzuki, and Yoichi Ochiai , Human Computer Interaction International 2022 (HCII' 22)
How See the Colorful Scenery?: The Color-centered Descriptive Text Generation for the Visually Impaired in Japan, Chieko Nishimura, Naruya Kondo, Takahito Murakami, Maya Torii, Ryogo Niwa, and Yoichi Ochiai , Human Computer Interaction International 2022, (HCII' 22)

Non-peer-reviewed

Event-based Visual Microphone: Non-contact Vibration Sensing using Event Camera, Ryogo Niwa, Tatsuki Fushimi, Kenta Yamamoto, and Yoichi Ochiai, 日本音響学会講演論文集, pp. 529-532, 2023
SHITARA: Sending Haptic Induced Touchable Alarm by Ring-shaped Air vortex, Ryosei Kojima, Akihisa Shitara, Fushimi Tatsuki, Ryogo Niwa, Atsushi Shinoda, Ryo Iijima, Kengo Tanaka, Sayan Sarcar, and Yoichi Ochiai,
arxiv https://arxiv.org/abs/2301.08107

Presentation

Event-based Visual Microphone: Non-contact Vibration Sensing using Event Camera, Ryogo Niwa, Tatsuki Fushimi, Kenta Yamamoto, and Yoichi Ochiai, 音響学会第150回(2023年秋季)研究発表会
EMS-Supported Throwing:Preliminary investigation on EMS-supported training of movement form, Ryogo Niwa, Kazuya Izumi, Shieru Suzuki, and Yoichi Ochiai, Human Computer Interaction International 2022

Grants and Awards

Tsukuba Alumni Association Esaki Award, 2023
Tsukuba Scholarship for Research Encouragement
Acoustical Society of Japan Student Presentation Award, 27th (2023 Autumn Meeting)
Leave a Nest Research Grant, incu・be Award, Encouragement Prize
CREST AIP Network Lab Director's Award, Top 5 of 41 participants
Drone Badminton, Accenture Interactive Award, 27th Student CG Contest
Yuru Gakki Hackathon, Excellent Award
Nile Hackathon, First Prize, ¥300,000 prize

Media

Page updated

Google Sites

Report abuse

Ryogo Niwa