Original Notes (adapted)
Implementation and Integration of Oculus Audio SDK for VR Audio Spatialization
- Adding new audio extension which wraps the oculus audio sdk and the new XAPO plugin
- Adding new XAPO (Xaudio2 Audio Processing Object) effect for processing mono audio source streams inside the new module
- Added new enumeration which allows users to select which spatialization algorithm to use for spatialized mono sources
- Refactored the regular sound source spatialization/effect in XAudio2 device code to support the new HRTF mono-to-stereo effect
- Designed feature so that if the sound spatialization module isn't used, the spatialization will default to normal spatialization algorithm
Notes on implementation:
- Because the audio engine doesn't bifurcate spatialized vs. non-spatialized sound sources into separate source pools, I had to create up-front the effects for the full number of supported sounds 32). This is because the oculus sdk requires at initialization the total number of sound sources that will be used in the SDK.
- Because the oculus SDK refers to sound source instances by index (into the pre-allocated array of sources set at init), I had to save each sound source's index so that it can be used on the HRTF XAPO effect, and then piped to the oculus SDK inside the UE audio extension.
- The audio engine assumes that mono-sources will be treated a certain way (during send-routing and spatialization). Because this new HRTF effect is effectively turning mono-sources into stereo-sources, some code had to be changed to send/route these audio sources as if they were stereo.
- This implementation is slightly different than the original GDC implementation to better work with multiple audio devices. Each audio device creates an IAudioSpatializationAlgorithm object which contains the oculus HRTF processing contexts.
#codereview Nick.Whiting Marc.Audy
[CL 2488287 by Aaron McLeran in Main branch]
- Created audio device manager which manages weak ref handles to audio devices
- Audio device handles are created with WorldContexts and accessed through UWorld objects
- All access to audio devices are now through audio device handles
- There is a main/default audio device for GEngine for use with the Editor and single worldcontext games
#codereview marc.audy, matthew.griffin
[CL 2477046 by Aaron McLeran in Main branch]
Removed GVolumeMultiplier and added an equivalent to FApp to contain the code to load the unfocused volume multiplier from config.
UE-4449 - Allow users to choose whether audio can be heard when focus is lost
#codereview Robert.Manuszewski
[CL 2341022 by Matthew Griffin in Main branch]
- Fixes looping issue with ADPCM samples
- Fixes Android not looping sound cues correctly
- Fixes one shot sound ADPCM problems where sound would get cut early
- Removed Android Audio Device update function - was never called and did nothing anyway so this will improve clarity
Changed ADPCM decompressor so that it uses the same code to decompress a full file as it does when streaming - old code was correct it was just silly have it duplicated.
[CL 2339877 by Robert Jones in Main branch]
Updated android audio device/buffers/souce to support creation of procedural data from a sound source. Tested with MOD replay in QA test level.
#codereview daniel.lamb, chris.babcock, dmitry.rekman
[CL 2276233 by Robert Jones in Main branch]
Changed the routine which decides which format so that it only does the compare once and then returns that value for the whole cook.
#codereview Daniel.Lamb, Chris.Babcock
[CL 2262575 by Robert Jones in Main branch]
Added support for ADPCM decoding.
Added 'sound type' information to USoundWave
- Android uses this to detect which type of asset it needs to deal with
Added support for selected encoding type via project settings
#codereview Daniel.Lamb, Chris.Babcock
[CL 2260750 by Robert Jones in Main branch]
New files added to contain FVorbisAudioInfo and FOpusAudioInfo, added all necessary includes
Added a few more checks for whether the target platform supports audio streaming where appropriate
Moved #define to .h file as it was being used in two .cpp files (working because of unity build)
[CL 2137973 by Matthew Griffin in Main branch]
Moved most of the contents of FAudioDevice::FreeResource to USoundWave::FreeResources so that we can be sure that all memory is cleared up during teardown.
Removed FreeResource function from some subclasses of audio device, as they weren't doing anything different than the base one.
Also removed ListSounds function from these same classes as that seems to have fallen behind current base class version and isn't used anyway.
Removed platform specific audio buffer storage so that there isn't confusion about which is being used and ensured that casts are used when necessary.
Fixed compile issue with STAT_AudioResourceCreationTime for HTML5.
[CL 2109273 by Matthew Griffin in Main branch]
Followed pattern set out by texture streaming so that each chunk of audio data resides in its own bulkdata struct. It is currently possible to split audio into chunks when SoundWaves are marked for streaming but there is no way of doing this exposed at present.
Changed the parameters of FAudioDevice::GetRuntimeFormat so that the relevant SoundWave must be passed in, to allow for different formats for individual sounds/streaming options.
USoundWave::FreeResources no longer resets the NumChannels as it is unnecessary and causes sounds to be unable to play after the OGG data is flushed when attempting to switch to OPUS.
[CL 2099012 by Matthew Griffin in Main branch]
Moved functions to check for and select Compressed Audio Info class to the audio device as the audio formats are not accessible in cooked builds (and shouldn't really need to be either).
[CL 2057758 by Matthew Griffin in Main branch]
Added an audio format for Opus, which also required a resampler to make sure that all imported sounds are converted to a compatible sample rate.
Added the speex resampler from the opus tools package to the third party source, including built libraries for windows and mac.
Changed FVorbisAudioInfo so that it inherits from an interface for any kind of compressed audio, which can be used everywhere instead of being wrapped in #WITH_VORBIS.
Added FOpusAudioInfo to decompress Opus data, not sure at this point whether it's only going to be used for streaming audio but works for non-streamed playback.
[CL 2056352 by Matthew Griffin in Main branch]
