Original Notes (adapted)
Implementation and Integration of Oculus Audio SDK for VR Audio Spatialization
- Adding new audio extension which wraps the oculus audio sdk and the new XAPO plugin
- Adding new XAPO (Xaudio2 Audio Processing Object) effect for processing mono audio source streams inside the new module
- Added new enumeration which allows users to select which spatialization algorithm to use for spatialized mono sources
- Refactored the regular sound source spatialization/effect in XAudio2 device code to support the new HRTF mono-to-stereo effect
- Designed feature so that if the sound spatialization module isn't used, the spatialization will default to normal spatialization algorithm
Notes on implementation:
- Because the audio engine doesn't bifurcate spatialized vs. non-spatialized sound sources into separate source pools, I had to create up-front the effects for the full number of supported sounds 32). This is because the oculus sdk requires at initialization the total number of sound sources that will be used in the SDK.
- Because the oculus SDK refers to sound source instances by index (into the pre-allocated array of sources set at init), I had to save each sound source's index so that it can be used on the HRTF XAPO effect, and then piped to the oculus SDK inside the UE audio extension.
- The audio engine assumes that mono-sources will be treated a certain way (during send-routing and spatialization). Because this new HRTF effect is effectively turning mono-sources into stereo-sources, some code had to be changed to send/route these audio sources as if they were stereo.
- This implementation is slightly different than the original GDC implementation to better work with multiple audio devices. Each audio device creates an IAudioSpatializationAlgorithm object which contains the oculus HRTF processing contexts.
#codereview Nick.Whiting Marc.Audy
[CL 2488287 by Aaron McLeran in Main branch]
- Fixes looping issue with ADPCM samples
- Fixes Android not looping sound cues correctly
- Fixes one shot sound ADPCM problems where sound would get cut early
- Removed Android Audio Device update function - was never called and did nothing anyway so this will improve clarity
Changed ADPCM decompressor so that it uses the same code to decompress a full file as it does when streaming - old code was correct it was just silly have it duplicated.
[CL 2339877 by Robert Jones in Main branch]
Changed the routine which decides which format so that it only does the compare once and then returns that value for the whole cook.
#codereview Daniel.Lamb, Chris.Babcock
[CL 2262575 by Robert Jones in Main branch]
Added support for ADPCM decoding.
Added 'sound type' information to USoundWave
- Android uses this to detect which type of asset it needs to deal with
Added support for selected encoding type via project settings
#codereview Daniel.Lamb, Chris.Babcock
[CL 2260750 by Robert Jones in Main branch]
New files added to contain FVorbisAudioInfo and FOpusAudioInfo, added all necessary includes
Added a few more checks for whether the target platform supports audio streaming where appropriate
Moved #define to .h file as it was being used in two .cpp files (working because of unity build)
[CL 2137973 by Matthew Griffin in Main branch]
Moved most of the contents of FAudioDevice::FreeResource to USoundWave::FreeResources so that we can be sure that all memory is cleared up during teardown.
Removed FreeResource function from some subclasses of audio device, as they weren't doing anything different than the base one.
Also removed ListSounds function from these same classes as that seems to have fallen behind current base class version and isn't used anyway.
Removed platform specific audio buffer storage so that there isn't confusion about which is being used and ensured that casts are used when necessary.
Fixed compile issue with STAT_AudioResourceCreationTime for HTML5.
[CL 2109273 by Matthew Griffin in Main branch]
Moved functions to check for and select Compressed Audio Info class to the audio device as the audio formats are not accessible in cooked builds (and shouldn't really need to be either).
[CL 2057758 by Matthew Griffin in Main branch]
