DiffSound: Differentiable Modal Sound Simulation for Inverse Reasoning

TL;DR — We propose a differentiable sound simulation framework for physically based modal sound synthesis and inverse problems

Abstract

Accurately estimating and simulating the physical properties of objects from real-world audio observations is of great practical importance in the field of vision and embodied AI. However, previous differentiable rigid or soft body simulations cannot be directly applied to modal sound synthesis due to the high sampling rate of sound, and previous audio synthesizers do not fully model the physical properties of objects behind the modal analysis. We propose DiffSound, a differentiable sound simulation framework for physically based modal sound synthesis. Our framework is capable of solving a range of inverse problems, including object shape, material parameter, and impact position reasoning. Experimental results demonstrate the effectiveness of our approach, highlighting its ability to accurately estimate physical parameters and reproduce the target sound. Our DiffSound differentiable sound simulator serves as a valuable tool for applications requiring sound synthesis and analysis.