Original Notes (adapted)
Implementation and Integration of Oculus Audio SDK for VR Audio Spatialization
- Adding new audio extension which wraps the oculus audio sdk and the new XAPO plugin
- Adding new XAPO (Xaudio2 Audio Processing Object) effect for processing mono audio source streams inside the new module
- Added new enumeration which allows users to select which spatialization algorithm to use for spatialized mono sources
- Refactored the regular sound source spatialization/effect in XAudio2 device code to support the new HRTF mono-to-stereo effect
- Designed feature so that if the sound spatialization module isn't used, the spatialization will default to normal spatialization algorithm
Notes on implementation:
- Because the audio engine doesn't bifurcate spatialized vs. non-spatialized sound sources into separate source pools, I had to create up-front the effects for the full number of supported sounds 32). This is because the oculus sdk requires at initialization the total number of sound sources that will be used in the SDK.
- Because the oculus SDK refers to sound source instances by index (into the pre-allocated array of sources set at init), I had to save each sound source's index so that it can be used on the HRTF XAPO effect, and then piped to the oculus SDK inside the UE audio extension.
- The audio engine assumes that mono-sources will be treated a certain way (during send-routing and spatialization). Because this new HRTF effect is effectively turning mono-sources into stereo-sources, some code had to be changed to send/route these audio sources as if they were stereo.
- This implementation is slightly different than the original GDC implementation to better work with multiple audio devices. Each audio device creates an IAudioSpatializationAlgorithm object which contains the oculus HRTF processing contexts.
#codereview Nick.Whiting Marc.Audy
[CL 2488287 by Aaron McLeran in Main branch]
- Fixes looping issue with ADPCM samples
- Fixes Android not looping sound cues correctly
- Fixes one shot sound ADPCM problems where sound would get cut early
- Removed Android Audio Device update function - was never called and did nothing anyway so this will improve clarity
Changed ADPCM decompressor so that it uses the same code to decompress a full file as it does when streaming - old code was correct it was just silly have it duplicated.
[CL 2339877 by Robert Jones in Main branch]
Updated android audio device/buffers/souce to support creation of procedural data from a sound source. Tested with MOD replay in QA test level.
#codereview daniel.lamb, chris.babcock, dmitry.rekman
[CL 2276233 by Robert Jones in Main branch]
Added support for ADPCM decoding.
Added 'sound type' information to USoundWave
- Android uses this to detect which type of asset it needs to deal with
Added support for selected encoding type via project settings
#codereview Daniel.Lamb, Chris.Babcock
[CL 2260750 by Robert Jones in Main branch]
Duplicate ResourceID would have caused the same problems in HTML5 and Linux as Android. Other duplications are errors waiting to happen, there are a couple of stats functions that could be using the base class versions and not displaying the correct information.
[CL 2119229 by Matthew Griffin in Main branch]
Moved most of the contents of FAudioDevice::FreeResource to USoundWave::FreeResources so that we can be sure that all memory is cleared up during teardown.
Removed FreeResource function from some subclasses of audio device, as they weren't doing anything different than the base one.
Also removed ListSounds function from these same classes as that seems to have fallen behind current base class version and isn't used anyway.
Removed platform specific audio buffer storage so that there isn't confusion about which is being used and ensured that casts are used when necessary.
Fixed compile issue with STAT_AudioResourceCreationTime for HTML5.
[CL 2109273 by Matthew Griffin in Main branch]
Followed pattern set out by texture streaming so that each chunk of audio data resides in its own bulkdata struct. It is currently possible to split audio into chunks when SoundWaves are marked for streaming but there is no way of doing this exposed at present.
Changed the parameters of FAudioDevice::GetRuntimeFormat so that the relevant SoundWave must be passed in, to allow for different formats for individual sounds/streaming options.
USoundWave::FreeResources no longer resets the NumChannels as it is unnecessary and causes sounds to be unable to play after the OGG data is flushed when attempting to switch to OPUS.
[CL 2099012 by Matthew Griffin in Main branch]
Fixes TTP #336826 where DTYPE_Realtime soundwaves were silently failing in OpenAL under Linux (and perhaps HTML5, too). May also help other platforms (IOS?) where DTYPE_Realtime is not implemented.
#codereview Matthew.Griffin, Marc.Audy
[CL 2088573 by Dmitry Rekman in Main branch]
Moved functions to check for and select Compressed Audio Info class to the audio device as the audio formats are not accessible in cooked builds (and shouldn't really need to be either).
[CL 2057758 by Matthew Griffin in Main branch]
Added an audio format for Opus, which also required a resampler to make sure that all imported sounds are converted to a compatible sample rate.
Added the speex resampler from the opus tools package to the third party source, including built libraries for windows and mac.
Changed FVorbisAudioInfo so that it inherits from an interface for any kind of compressed audio, which can be used everywhere instead of being wrapped in #WITH_VORBIS.
Added FOpusAudioInfo to decompress Opus data, not sure at this point whether it's only going to be used for streaming audio but works for non-streamed playback.
[CL 2056352 by Matthew Griffin in Main branch]
