This project is a sub-project of Low-Cost, High-Quality Immersive Audio-Visual Telepresence. It is supervised by Professor Douglas L. Jones from university of illinois at urbana and champaign. In this sub-project, our vision is to create realistic 3D audio solutions for low-cost, high-quality telepresence.

One of the goals of telepresence is to provide face-to-face communication, overcoming the distance between persons. An ideal telepresence service should allow participants in distance places to communicate with each other in a natural environment, as if they were in the same room. Recently, most efforts are to advance the telecommunication technology for realistic teleconferencing service that produces a virtual environment where individuals can hear, see and feel a seemingly shared atmosphere. Therefore, it is driving the demand for better realistic 3D (three dimensional) audio technologies.

Our vision is to develop real-time realistic 3D audio techniques for low-cost, high-quality telepresence. Specifically, our research is focused on the following areas:

  • 3D audio direction-finding: find the sound source (who is speaking) in 3D space and real-time in the conference. The direction of sound source can be used to steer a high-resolution camera with a better focus, to help make a directional voice pickup, and to help create binaural 3D audio.
  • Directional voice acquisition and noise reduction: pick up the target voice only. It is important for good hearing with interference and noise reduced. In the case of playback by loudspeakers, echo cancellation systems may not be doing a good job. A directional voice acquisition system is able to avoid the echoes from different directions.
  • Binaural 3D audio reconstruction: create the audio effects which allow the users to feel as if the remote participants’ voices are coming from distinct directions in space and allow for better intelligibility and easier comprehension especially when multiple parties are speaking at once.

The research project is conducted in Advanced Digital Sciences Center (ADSC). A general project description can also be found here.