• Hey, guest user. Hope you're enjoying NeoGAF! Have you considered registering for an account? Come join us and add your take to the daily discourse.

Hardware Accelerated Audio - How Xbox Series X uses a dedicated audio chip to deliver next-gen audio

Bernkastel

Ask me about my fanboy energy!

With Xbox One, Microsoft used Spatial Sound to create "Full Audio Immersion"
Spatial sound is an enhanced audio experience where sounds can flow around, above, and below you in a three-dimensional virtual space that simulates a more realistic environment. Spatial sound provides an immersive atmosphere that traditional channel-based surround sound formats cannot replicate. Imagine playing Gears of War 4 and a Raven helicopter flies overhead. With surround sound, you’d hear the helicopter flying around you, but with spatial sound, you’ll hear it flying directly above you.
The Audio Processor Block(APB) present in Xbox One was updated to support Spatial Sound with the release of Xbox One X. But this takes up considerable CPU resources. So, to free up resources for constrained sound engineers, Xbox Series X will have a dedicated audio chip.
“It’s extremely exciting,” senior sound designer Daniele Galante said of the new console. “We’re going to have a dedicated chip to work with audio, which means we finally won’t have to fight with programmers and artists for memory and CPU power.”
The studio’s audio lead David Garcia added: “We take for granted that graphics are powered by their own video cards. But in audio, we haven’t had anything like that. Now we have some power dedicated to us.”
...
“I really like how Microsoft is giving us more tools to improve the sound and to be more creative,” he said. “Because at the end of the day it’s not a matter of, ‘we want more memory because we are more cool’, we just want to be able to do our jobs without thinking about limitations.
“Making games always has you thinking about technical limitations,” he added. “Eventually these limitations become less and less the more you evolve with new consoles, but at the same time it’s always a constant thing: These are the maximum number of voices we can have, because otherwise the game is going to lag.”
There are many aspects of audio in Xbox Series X.
Spatial Audio
Spatial Audio – Spatial Audio delivers deeply immersive audio which enables the player to more accurately pinpoint objects in a 3D play space. With full support for Dolby Atmos, DTS:X and Windows Sonic, Xbox Series X has custom audio hardware to offload audio processing from the CPU, dramatically improving the accessibility, quality and performance of these immersive experiences.
As a pure software implementation, Spatial Audio was present on Windows 10 and Hololens since 2015, and on Xbox One using the Audio Processor Block since Xbox One X launch.
On Xbox One X, it supported Windows Sonic, Dolby Atmos, DTS:X and WiSA(no word on WiSA support for Xbox Series X yet, they will probably add it later) and was used by some games like Gears 5.
Windows Sonic is Microsoft's own 3D Audio standard, by the Audio and Acoustics Research Group at Microsoft Research and provides improved HRTF-based spatialization.

Audio Ray Tracing
Similar to Image Ray Tracing, which traces the path of light waves, Audio Ray Tracing can trace the path of Sound Waves.
Jason R:
And now with the introduction of hardware accelerated Ray tracing with the Xbox series X, we're actually able to enable a whole new set of scenarios, whether that's more realistic lighting, better reflections, we can even use it for things like spatial audio and have Ray trace audio so that.
...
Larry Hryb:
Ray tracing audio is the first time I've heard that's interesting.

Jason R:
Absolutely. Absolutely. And that's the thing is what we're really focused on is really driving that next level of immersion.

Larry Hryb:
Yeah.

Jason R:
In your gaming experiences. And that imply, that applies to both the visuals as well as the audio experience that you have.

Larry Hryb:
Yeah, right. I mean Ray tracing is one of those things where people, it's tough to, I mean obviously we're in an audio format, here's a podcast. It's tough to describe but you need to really see it and the texture and just makes scenes and areas just come to life. Right,

Jason R:
Exactly, exactly. And with spatial audio a huge part of it as well as just really putting you in the Play space environment really understanding where the enemies are or being just that much more immersed.
Audio Ray Tracing is part of Xbox Series X's Hardware Accelerated DirectX Raytracing
Hardware Accelerated DirectX Raytracing (DXR) – From improved lighting, shadows and reflections as well as more realistic acoustics and spatial audio, raytracing enables developers to create more physically accurate worlds. For the very first time in a game console, Xbox Series X includes support for high performance, hardware accelerated raytracing. Xbox Series X uses a custom-designed GPU leveraging the latest innovation from our partners at AMD and built in collaboration with the same team who developed DirectX Raytracing. Developers will be able to deliver incredibly immersive visual and audio experiences using the same techniques on PC and beyond.
Project Acoustics
Project Acoustics is an acoustics engine that Microsoft has been working on since 2012.
Project Acoustics – Incubated over a decade by Microsoft Research, Project Acoustics accurately models sound propagation physics in mixed reality and games, employed by many AAA experiences today. It is unique in simulating wave effects like diffraction in complex scene geometries without straining CPU, enabling a much more immersive and lifelike auditory experience. Plug-in support for both the Unity and Unreal game engines empower the sound designer with expressive controls to mold reality. Developers will be able to easily leverage Project Acoustics with Xbox Series X through the addition of a new custom audio hardware block.
Project Acoustics is a wave acoustics engine for 3D interactive experiences. It models wave effects like occlusion, obstruction, portaling and reverberation effects in complex scenes without requiring manual zone markup or CPU intensive raytracing. It also includes game engine and audio middleware integration. Project Acoustics' philosophy is similar to static lighting: bake detailed physics offline to provide a physical baseline, and use a lightweight runtime with expressive design controls to meet your artistic goals for the acoustics of your virtual world.


Xbox Wireless Headset


Game loud and clear with the all-new Xbox Wireless Headset, featuring auto-mute and voice isolation for crystal-clear chat, intuitive design, and direct pairing to your console.

Best-in-class audio performance
Experience high quality audio with a low-latency, 100% wireless connection to your Xbox console, without the need for a dongle or a base station. Supports spatial sound technologies including Windows Sonic, Dolby Atmos, and DTS Headphone:X for realism and audio precision that fully surrounds you.
Comfortable & intuitive designBest-in-class audio performance
Play comfortably for hours with the ultra-soft earcups, lightweight design, and customizable fit. The rotating earcup dials of the new Xbox Wireless Headset provide an intuitive way to change volume and game/chat balance. Quickly adjust and get back to the game. Fully adjustable mic tucks up and out of the way when not in use.
Crystal clear chat
Voice isolation focuses the mic on your voice to reduce background noise. And when you aren’t talking, the auto-mute feature automatically mutes your microphone so you keep comms channels clear, while the manual mute button gives you a privacy option when you need it.
Tailored to you
Use the Xbox Accessories app to fine-tune the audio controls, including equalizer settings, bass boost, auto-mute sensitivity, and mic-monitoring levels. Pair directly to your console with Xbox Wireless radio without the need for a dongle, cable, or base station and connect to your mobile device with Bluetooth® to listen to music or chat while on the go.

Specifications

What’s in the boxXbox Wireless Headset
USB-C charging cable (14 inches long)
System RequirementsFor use with Xbox X|S, Xbox One, and Windows 10 devices. On supported consoles, some voice chat features may require Xbox Game Pass Ultimate or Xbox Live Gold (memberships sold separately). Windows 10 use requires Bluetooth 4.2+, Xbox Wireless adapter or compatible USB-C cable. Each sold separately.
MaterialsHeadband: inner steel metal band with foam cushioning
Ear cushions: oval design with polyurethane leather and foam cushioning
MicrophonesMic design: bendable boom with dual mics and Illuminated LED when mic is on
SpeakersSpeaker size: 40mm
Speaker material: Paper composite diaphragm and neodymium magnet
Speaker Impedance: 32 ohm
Speaker Response: 20Hz - 20kHz
ControlsButtons: power/pair, mute, game/chat audio balance dial (left earcup), volume dial (right earcup).
Bluetooth
Bluetooth version: 4.2 (A2DP, HFP, HSP)
Codec: SBC
Battery
Internal, rechargeable lithium-ion battery with up to 15 hours of battery life per charge. 30 minutes of charging provides about 4 hours of battery life and recharges to full battery life in about 3 hours when headset is not in use.
Weight
11 oz (312g)
Virtual Surround Sound
Supports Spatial Sound technologies like Windows Sonic, Dolby Atmos, and DTS Headphone:X on Xbox Series X|S and Xbox One
Xbox Accessories app
Adjust equalizer settings, bass boost, auto-mute, LED brightness, and mic monitoring



 
Last edited:

Bernkastel

Ask me about my fanboy energy!
More on Spatial Audio

More on Audio Ray Tracing
More on Project Acoustics
Hey guys thought I would shim in since my studio is currently implementing Project Acoustics in our game, here is a devlog where you can listen how it sound (use headphones)


Project acoustics is not raycast based in any way. At baketime they simulate how waves propagate the scene, this is much more complex than raycsting, a light ray survives a few bounces and can't not bend around corners.

Thanks to a physics phenomenon called diffraction a sound wave that hits a corner will create a new source for vibration to propagate around said corner.

This would be very expensive todo at runtime and also impossible todo with same fidelity, this is why raycsting solutions take shortcuts, here an exammple with Steam Audio, for example when i walk back and forth in that doorway the propagation is very uneven and the sound changes alot in character just by moving a little
Project acoustics simulates how a sound propagate from predefined points placed by the framework. this simulation takes days or weeks on a single machine (took 10 days on my AMD 3950x for a small/medium sized scene) so you offload it to the cloud. Costs about 50 to 200 USD depending on scene. Anyway this produces a dataset of a few hundred megabytes that is streamed into the memory depending on where the listener are,they then interpolate between points to get a perfect dataset for exactly the acoustics between the listener and audio source at a very little CPU cost

Will explore Project Acoustics in the future.

More from Microsoft's Audio and Acoustics group



From Hot Chips 2020

 
Last edited:
I'm not convinced.

Windows Sonic sounded like complete shit to me (on PC with headphones). Same with that Dolby processing, the Razer implementation and Sennheiser's. These all just turn the sound into an overprocessed mess.

Maybe every game needs to have exact support for this type of sound recording. I hope it works out.
 

darkinstinct

...lacks reading comprehension.
"Used by many games like Gears 5". 14. 14 games on Xbox One support it. Out of 1.744. And every Xbox to date has had a dedicated audio chip, the issue has always been shared RAM constraints and not fighting for CPU time (only secondary, because you have to move the sound data into RAM, for which you require CPU resources).

Dolby Atmos is great - if you like headphones. On a standard 5.1 setup (well, standard is actually 2.0 from the TV itself, followed by 2.1 soundbars) the effect is mild.
 

Bernkastel

Ask me about my fanboy energy!
"Used by many games like Gears 5". 14. 14 games on Xbox One support it. Out of 1.744. And every Xbox to date has had a dedicated audio chip, the issue has always been shared RAM constraints and not fighting for CPU time (only secondary, because you have to move the sound data into RAM, for which you require CPU resources).

Dolby Atmos is great - if you like headphones. On a standard 5.1 setup (well, standard is actually 2.0 from the TV itself, followed by 2.1 soundbars) the effect is mild.
And every Xbox to date has had a dedicated audio chip,
No they dont
We’re going to have a dedicated chip to work with audio, which means we finally won’t have to fight with programmers and artists for memory and CPU power.
Its more like a audio processing block in the cpu.
 
Last edited:

Bernkastel

Ask me about my fanboy energy!
I'm not convinced.

Windows Sonic sounded like complete shit to me (on PC with headphones). Same with that Dolby processing, the Razer implementation and Sennheiser's. These all just turn the sound into an overprocessed mess.

Maybe every game needs to have exact support for this type of sound recording. I hope it works out.
Enabling Windows Sonic on Windows 10, just resamples output audio to that format using whatever CPU/GPU you have. Xbox Series X will have dedicated hardware for it, with games built with Spatial Sound in mind.
 

Panajev2001a

GAF's Pleasant Genius
Last edited:

Bernkastel

Ask me about my fanboy energy!
This seems like the APU has dedicated audio processing blocks and for whatever reason cant find about it being present on Xbox One X
Still very different from Xbox Series X, where audio processing is handled separately by a dedicated hardware.
 
Last edited:

Area61

Member
So in a nutshell PS5 and XbSX both will have 3d audio? This is just like any other generation of console release scenario lol. Both almost identically same apart from their games.
 

Panajev2001a

GAF's Pleasant Genius
This seems like the APU has dedicated audio processing blocks and for whatever reason is absent in Xbox One X
Still very different from Xbox Series X, where audio processing is handled by a Sound Card equivalent.

Or maybe the documentation of the Xbox One X there is incomplete in that forum answer? It sounds a bit more likely to have incomplete forum answers than an entire HW block of not trivial performance being removed to software only when the CPU got only a small performance boost over the base Xbox One S CPU.
That would be way more reasonable IMHO.

I fail, beside acknowledging the new solution must be improved and dealing with more complex algorithms, to see the difference in your description of the HW block in the XSX vs what Durango/Xbox One had.
 
Last edited:

NXGamer

Member
No they dont

Atleast not Xbox One, I couldnt find anything about it having a dedicated audio chip.
They do , they both have. What is being offered here is enhanced with a DSP like processing but we really need to see and hear more to see what this offers and how much in empowers devs, the same for the PS5 Tempest engine but being a SPU in all but name with the DMA access it has been thought through with no CPU time required.

Does the Series X have DMA within its Audio chip design, I am still reading up on this at moment?

EDIT: Hotchips above post is it.
 
Last edited:

Panajev2001a

GAF's Pleasant Genius
So in a nutshell PS5 and XbSX both will have 3d audio? This is just like any other generation of console release scenario lol. Both almost identically same apart from their games.

Yes, as time goes on and we see more and more of the architecture explored in depth we can see how they compare. Sony really was beating its chest regarding the Tempest Engine and the modified CU they made available to audio and general processing and attached to the CPU as a dedicated vector DSP of old, they both seem to have spent a good amount of transistors on it :).
 
Last edited:

jimbojim

Banned
So in a nutshell PS5 and XbSX both will have 3d audio? This is just like any other generation of console release scenario lol. Both almost identically same apart from their games.

Similar maybe. But i think audio chip in PS5 is a bit more advanced.
 
Last edited:

Bernkastel

Ask me about my fanboy energy!
They do , they both have and PS4. What is being offered here is enhanced with a DSP like processing but we really need to see and hear more to see what this offers and how much in empowers devs, the same for the PS5 Tempest engine but being a SPU in all but name with the DMA access it has been thought through with no CPU time required.

Does the Series X have DMA within its Audio chip design, I am still reading up on this at moment?
Direct Memory Access access(DMA acess)? Considering they want less CPU intervention and that most Sound Cards have DMA, I dont know why you would assume it wont have DMA.
 
Last edited:

jimbojim

Banned
What makes you think so?

current X1 has more advanced audio capabilities than PS4, why would things be any different next gen? Microsoft has been investing heavily into spatial (i.e. 3D) and ray traced audio.

If X1 has more advanced audio, then PS5 can have it too. Same thing, PS4 has more TF than X1. Why things would be any different. /s
 

jimbojim

Banned
There is no mention of PS5 in my thread though. Nothing here is speculation either, I just made a thread on everything we know about audio in XSX.
No it does not. X1X has more advanced audio though.

Are you something like PR for MS? Discord member? What are you trying to prove?

You've opened very similar thread 2 months ago :


Btw. i wasn't the first who mentioned PS5 in this thread.
 
Last edited:

darkinstinct

...lacks reading comprehension.
I couldnt find any documentation on Xbox One S/X or PS4 having any dedicated audio processing hardware. Whatever the Xbox One had was removed with the S/X models.
Direct Memory Access access(DMA acess)? Considering they want less CPU intervention and that most Sound Cards have DMA, I dont know why you would assume it wont have DMA.

Are you serious right now? The same audio block that sits in Xbox One is in One S and One X. How do you suggest games designed for Xbox One would run on One S if you took away part of the audio hardware?
 

Bernkastel

Ask me about my fanboy energy!
Are you serious right now? The same audio block that sits in Xbox One is in One S and One X. How do you suggest games designed for Xbox One would run on One S if you took away part of the audio hardware?
Its not some hardware accelerated dedicated audio standard. It was just processing audio with separate blocks.
 
Last edited:

Bernkastel

Ask me about my fanboy energy!
Are you something like PR for MS? Discord member? What are you trying to prove?

You've opened very similar thread 2 months ago :


Btw. i wasn't the first who mentioned PS5 in this thread.
Who said I needed your permission to open threads? This is a complete thread on Xbox Series X audio capabilities.
 

NXGamer

Member
I couldnt find any documentation on Xbox One S/X or PS4 having any dedicated audio processing hardware. Whatever the Xbox One had was removed with the S/X models.
Direct Memory Access access(DMA acess)? Considering they want less CPU intervention and that most Sound Cards have DMA, I dont know why you would assume it wont have DMA.
It is above, they do.

Also I was not assuming it does not, I am learning how it is configured and more info is all. Based on you being the OP I was merely asking as you may know?
 

Rikkori

Member
I really hope the promise of audio doesn't end up being just another marketing point. I'm not so hopeful, because Sony seems to be focusing on headphones mostly, but at least Microsoft does deliver for us surround sound people with some of their first party titles. Right now, if you have more than a 5.1 setup it pretty much gets wasted, and even the surround speakers don't see as much use as you'd hope for. I really want to upgrade my main setup to 7.2.4 but need more titles to take advantage of it.
 

kuncol02

Member
I really hope the promise of audio doesn't end up being just another marketing point. I'm not so hopeful, because Sony seems to be focusing on headphones mostly, but at least Microsoft does deliver for us surround sound people with some of their first party titles. Right now, if you have more than a 5.1 setup it pretty much gets wasted, and even the surround speakers don't see as much use as you'd hope for. I really want to upgrade my main setup to 7.2.4 but need more titles to take advantage of it.
If you want REAL 3d audio, then you need to trow away your setup and invest in headphones. Old Barber Shop demo is miles above anything you can achieve with any surround system in terms of 3d audio and it works on cheapest headphones.
Check Steam Audio demo from while ago:

And thats software solution. Some people are waiting literally decades for that technology to come back into mainstream.
 
Last edited:

Rikkori

Member
If you want REAL 3d audio, then you need to trow away your setup and invest in headphones. Old Barber Shop demo is miles above anything you can achieve with any surround system in terms of 3d audio and it works on cheapest headphones.
Check Steam Audio demo from while ago:

And thats software solution. Some people are waiting literally decades for that technology to come back into mainstream.

Demos don't impress me, and also you're discounting the added benefit of being in a soundspace and not having speakers strapped to your head. HRTF demos sound convincing but I put them way below surround sound setups in terms of immersion. In particular, the problem with all the demos (in any domain) is they're very specific to what they demonstrate and especially when they select only the bits that sound good, i.e. you're not gonna get an effect as convincing when it comes to other sounds & scenarios. So yes, in the context of a barber shop and the scissors it's very convincing but if you try to do that with the sort of sound scenarios you'd have in a game, it will sound a lot less convincing.

That's why when you test out actual games that have incredibly good mixing for both headphones and surround (eg Division 2), surround still is above it by a long way.

 

fybyfyby

Member
If it will be binaural audio with hrtf profiling (like Sony promises) - then good job!
Another thing is hardware sound processing. Something, what is missing also on modern PCs.
 

ZywyPL

Banned
I think the sound will, as always, come down to the sound design by the the individual developers - back in the day we had CoD, KZ, and BFBC, all running on the exact same systems, with vastly different results, or Forza vs GT. Granted, back in those days LCD TVs were rather thick, with decently sized speaker up front, as oppose to today's tiny speakers that are located on the back of the TVs, so I guess dedicated sound setup, be it soundbar, surround sound, or headphones, will be more crucial than ever to experience good sound quality no matter how well it'll be implemented, but at the end of the day it good sound will be fully determined by the people who make it, those devs who will want to put a lot of effort into sound department will have the tools, the hardware, whereas the games where devs will want to just put some simple, generic sounds will sound exactly like that. hardware always mush go in pain with the software.
 
Last edited:

Dory16

Banned
Are you something like PR for MS? Discord member? What are you trying to prove?

You've opened very similar thread 2 months ago :


Btw. i wasn't the first who mentioned PS5 in this thread.
Let's ignore the content (all factual by the way) and investigate the poster shall we? Very mature.
 

phil_t98

Gold Member
I think the sound will, as always, come down to the sound design by the the individual developers - back in the day we had CoD, KZ, and BFBC, all running on the exact same systems, with vastly different results, or Forza vs GT. Granted, back in those days LCD TVs were rather thick, with decently sized speaker up front, as oppose to today's tiny speakers that are located on the back of the TVs, so I guess dedicated sound setup, be it soundbar, surround sound, or headphones, will be more crucial than ever to experience good sound quality no matter how well it'll be implemented, but at the end of the day it good sound will be fully determined by the people who make it, those devs who will want to put a lot of effort into sound department will have the tools, the hardware, whereas the games where devs will want to just put some simple, generic sounds will sound exactly like that. hardware always mush go in pain with the software.
I do think tv size is a big factor. When plasmas were a thing they were fair thick panels so decent speakers fitted into the tv. Now TVs are getting thinner and thinner and speaker tech hasn’t changed at all, the bigger the speaker cone the better the sounds. That’s were sound bars come into it but they are also smaller speaker woofers and dont produce great sound as well, yes they have subs with them to give bass but there is always that midrange missing.

The best way now is still to have a good amp and speaker setup still. Gonna be interesting to see what these audio chips do, will they just work with standard atmos or will they have to bring out a new audio codec .

3D sound using standard tv speakers will never be any good in my opinion once you have heard a full proper atmos setup
 

Bernkastel

Ask me about my fanboy energy!
I doubt that to be the case (what else took its place?!), but do you have a link to support that? Sorry, just curious...
Ok, I admit I was wrong. After some digging it seems like they removed the functionality of the audio processing block and replaced it with Spatial Sound.
The audio processor in Xbox One is fully transplanted across to Scorpio and gains new functionality - spatial surround, effectively adding a 'height' component to the existing 7.1 set-up. Scorpio is set to receive support for Dolby Atmos for gaming, Dolby Atmos for headphones plus a Microsoft proprietary format called HRTF, developed by the Hololens team. Because the APB (audio processor block) hardware is basically identical to that found in Xbox One, it means that all existing iterations of the console will get the spatial surround upgrade.
For whatever they stopped talking about and like it does not exist anymore(even Microsoft employees).
Update : I have updated the OP, the APB was updated to implement Spatial Sound when Xbox One X released.
 
Last edited:

skneogaf

Member
Dolby atmos in games is absolutely amazing on my 7.2.4 sound setup. I'm looking forward to hearing the next generation of sound from both systems.
 

Starhowl

Member
I hope it's gonna be possible to connect external audio processors like those from Creative Labs with Xbox Series X, since there won't be an optical output anymore.
 

M1chl

Currently Gif and Meme Champion
I hope it's gonna be possible to connect external audio processors like those from Creative Labs with Xbox Series X, since there won't be an optical output anymore.
Splitters exist and also does not TVs have also one?
 
It's awesome that the audio enthusiasts are in for a treat next gen. I almost always game with a nice headset I can't wait to hear the difference with these consoles.

I didn't know "audio ray tracing was a thing" before a month ago or so
 
Last edited:
Top Bottom