Commit Graph

52 Commits

Author SHA1 Message Date
danny couture
d690175898 Fix game-thread stutter when compressing texture from worker threads
#rb Francis.Hurteau

[CL 13950969 by danny couture in ue5-main branch]
2020-07-27 19:55:21 -04:00
danny couture
ff158b6e95 Async Texture Compilation
- Feature can be activated in the Experimental section of the Editor Settings
 - Replace Texture2D/TextureCube resources by placeholders until their PlatformData is ready
 - Add a utility class allowing to encapsulate raw field pointers without breaking compatibility
 - Protect PlatformData from unsafe access through encapsulation.
 - Protect texture's resource from race conditions between game and render threads through encapsulation.
    - This allows to get rid of FlushRenderingCommands and long game-thread stutters when Updating a texture's resource.
    - UpdateResource was never safe to call without a FlushRenderingCommands and multiple call-site are doing exactly that, this will fix those cases.
    - Those were probably undetected due to their low occurence rate under normal conditions but can easily be reproed during async texture compilation on 32 cores.
 - Force wait on required texture compilations for MaterialBaking, ProxyMesh, Thumbnail generation for disk usage
 - Wait on all textures compilation whenever a wait for all shaders compilation is requested for safety (i.e. screenshot)
 - Compile UI and heightmap textures with higher priority to reduce visual artefacts
 - Increase priority of texture that have been rendered to improve time-to-usefulness of the editor under low core count
 - Async compilation is disabled for -game / non-editor mode as there is currently no support for async bulk data loading from external files
 - Properly cancel async tasks when UTexture is garbage collected before the compilation is finished
 - Show progress when explicitly waiting on compilation
 - Changing the mip settings in the texture editor (or any settings requiring the running platform data to be recomputed) will now be processed asynchronously.

DEBUGGING

 - Can be forcibly enabled/disabled through command-line via -asynctexturecompilation=[off, on, paused]
 - Can pause texture compilation using Editor.AsyncTextureCompilation = 2 or -asynctexturecompilation=paused
 - Can manually resume a specified amount of paused compilation using Editor.AsyncTextureCompilationResume [Num]
 - Can forcibly wait on all compilation using Editor.AsyncTextureCompilationFlushAll

BENCHMARKS

 - 3m15s to 1m20s when loading Apollo_Terrain with no textures in DDC (AMD TR 3970X)
 - 6m45s to 1m11s when loading Apollo_Terrain with no textures in DDC (-corelimit=8)
 - 3m10s to 1m54s when lauching PIE on Apollo_Terrain with no textures in DDC (AMD TR 3970X)
 - 7m43s to 1m36s when lauching PIE on Apollo_Terrain with no textures in DDC (-corelimit=8)
 - 0m57s to 0m42s when importing Attic_NVIDIA.usd with no textures in DDC (AMD TR 3970X)
 - 2m14s to 0m35s when importing Attic_NVIDIA.usd with no textures in DDC (-corelimit=4)

TESTS
 - Success on all material baking tests from EngineTests with -asynctexturecompilation=paused
 - Runned with -corelimit=1 all the way to unlimited
 - Cooking worked
 - Opening the texture editor/material editor will force the compilation to finish like expected.
 - Changing a setting in the texture editor will recompile async, even allowing to close the editor and continue doing other changes.
 - Unpausing the compilation will update the texture thumbnails properly.
 - Started with -asynctexturecompilation=paused, and then unpaused after a map loading, and then into a PIE session to stresstest UpdateResources.
 - Tested both dx11/dx12
 - Vulkan fails on Fortnite even with -asynctexturecompilation=off because of Landscape weigthmap, not this CL.
 - Compiled and tested FortniteGame / UE4 / ShooterGame projects

#rb Uriel.Doyon, Francis.Hurteau

[CL 13694814 by danny couture in ue5-main branch]
2020-06-16 22:16:25 -04:00
dmitriy dyomin
533509d73b Add an option to downscale textures without mips. Accepts values between 1.0 and 8.0.
Can be overriden per-platform, for a specifc texture or texture group.
Downscaling will keep texure size divisible by 4 so they can still be compressed
#rb chris.bunner


#ROBOMERGE-SOURCE: CL 12400453 via CL 12400465 via CL 12400467
#ROBOMERGE-BOT: (v671-12333473)

[CL 12402238 by dmitriy dyomin in Release-Engine-Staging branch]
2020-03-25 08:18:43 -04:00
JeanMichel Dignard
99b55fb883 Merged main @ cl 10981486
#rb none
#rnx

[CL 10985598 by JeanMichel Dignard in Dev-Tools-Staging branch]
2020-01-14 14:45:07 -05:00
julien stjean
6e4b0a168d Hot fix for a crash where we ended up creating more mip map level then we should had.
This issue was happening after using LeaveExistingMips for the MipGenSettings and setting a maximun texture size.
Added a validation when getting that data from the ddc as it could be invalid.

#jira UE-85228
#rb JeanMichel.Dignard, Mihnea.Balta
#lockdown nick.penwarden

#ROBOMERGE-SOURCE: CL 10950746 in //UE4/Release-4.24/...
#ROBOMERGE-BOT: RELEASE (Release-4.24 -> Main) (v632-10940481)

[CL 10950747 by julien stjean in Main branch]
2020-01-13 07:12:12 -05:00
JeanMichel Dignard
70d074639f Merging //UE4/Dev-Main @ 10886849 to Dev-Tools-Staging (//UE4/Dev-Tools-Staging)
#rb none
#rnx

[CL 10906274 by JeanMichel Dignard in Dev-Tools-Staging branch]
2020-01-08 13:26:18 -05:00
ryan durand
471d972e62 Updating copyright for Engine Developer.
#rnx
#rb none


#ROBOMERGE-SOURCE: CL 10869240 via CL 10869516 via CL 10869902
#ROBOMERGE-BOT: (v613-10869866)

[CL 10870584 by ryan durand in Main branch]
2019-12-26 15:32:37 -05:00
JeanLuc Corenthin
276d3af9b8 Copy up //UE4/Dev-Enterprise/... to //UE4/Dev-Tools-Staging/...
#rb none

[CL 10371791 by JeanLuc Corenthin in Dev-Tools-Staging branch]
2019-11-22 10:05:55 -05:00
rolando caloca
33e6e98b5c Add more info to failed composite texture
#rb trivial
#rnx


#ROBOMERGE-SOURCE: CL 10017753 via CL 10017763 via CL 10020182 via CL 10020254
#ROBOMERGE-BOT: (v562-10004402)

[CL 10020374 by rolando caloca in Main branch]
2019-11-05 14:56:48 -05:00
jeanmichel dignard
3ed35eb908 PR #6292: Engine changes for USD plugin (Contributed by niexuchina)
#jira UE-82046

#rb none

#ROBOMERGE-SOURCE: CL 9702186 in //UE4/Release-4.24/...
#ROBOMERGE-BOT: RELEASE (Release-4.24 -> Main) (v539-9700858)

[CL 9702189 by jeanmichel dignard in Main branch]
2019-10-18 17:21:11 -04:00
Rolando Caloca
1a30325d6f Copying //UE4/Dev-RenderPlat-Staging@9551419 to //UE4/Main
#rb none
#rnx

[CL 9551447 by Rolando Caloca in Main branch]
2019-10-11 15:33:31 -04:00
jeremy moore
560d7ebf39 Add YCoCg compression with per block scaling to the texture compression module
Runtime virtual texture uses streaming virtual texture to fill pages at distance
We recently moved to YCoCg encoding in the compressed base color texture to address visual quality issues
The runtime virtual texture used a per block scaling, but the offline generated streaming texture didn't
The offline texture looks much worse without the block scaling so that we see color artifacts in the distance before this change
[CODEREVIEW] Sebastien.Lussier


#ROBOMERGE-SOURCE: CL 9325494 via CL 9325668 via CL 9326152 via CL 9326166 via CL 9326179
#ROBOMERGE-BOT: (v455-9298202)

[CL 9326194 by jeremy moore in Main branch]
2019-10-02 00:04:28 -04:00
Marcus Wassmer
6517c68ef5 Copying //UE4/Dev-RenderPlat-Staging@8684824 to Dev-Main (//UE4/Dev-Main)
#rb none

[CL 8684840 by Marcus Wassmer in Main branch]
2019-09-14 09:45:25 -04:00
simon tourangeau
311a249228 GenerateMipChain() and AdjustImageColors() are made public and changed to ITextureCompressorModule::GenerateMipChain() and ITextureCompressorModule::AdjustImageColors(). This allows us to export the specific MipMap levels and the textures with adjustments to PNG.
Fix SRGB of 16bit cubemap

Import the existing texture asset without Dialog box
New parameter to SuppressImportOverwriteDialog which can suppress the dialog box that, when importing over an existing texture, force to overwrite its settings.

#jira UE-78899

PR #6123: Texture changes for NVIDIA USD plugin.

#rb jeanmichel.dignard
#lockdown nick.penwarden

#ROBOMERGE-SOURCE: CL 8127249 in //UE4/Release-4.23/...
#ROBOMERGE-BOT: RELEASE (Release-4.23 -> Main) (v401-8057353)

[CL 8127258 by simon tourangeau in Main branch]
2019-08-20 13:24:46 -04:00
Rolando Caloca
aa0d2303d6 Copying //UE4/Dev-Rendering to Dev-Main (//UE4/Dev-Main) @ 6944469
#rb none
#rnx

[CL 6944849 by Rolando Caloca in Main branch]
2019-06-11 18:27:07 -04:00
Marcus Wassmer
6995e20184 Merging //UE4/Dev-Main@4848702 to Dev-Rendering (//UE4/Dev-Rendering)
#rb none
#lockdown rolando.caloca

[CL 4854488 by Marcus Wassmer in Dev-Rendering branch]
2019-01-30 20:34:55 -05:00
Chris Gagnon
a48e767b94 Merging //UE4/Dev-Main to Dev-Editor (//UE4/Dev-Editor)
#rb none

[CL 4835066 by Chris Gagnon in Dev-Editor branch]
2019-01-29 16:15:19 -05:00
Brandon Schaefer
e3ac01dbbf Fix crash when taking a ref of a lambda function (Thanks Christina Tempelaarl!)
#jira UE-69040
#rb Arciel.Rekman, Christina.Tempelaarl
#lockdown matt.kuhlenschmidt

[CL 4815152 by Brandon Schaefer in Dev-Editor branch]
2019-01-25 15:16:23 -05:00
Rolando Caloca
732810a13b DR - Copying //UE4/Dev-Main@4785067 to //UE4-DevRendering
#rb none
#rnx
#lockdown Marcus.Wassmer

[CL 4790611 by Rolando Caloca in Dev-Rendering branch]
2019-01-23 17:01:56 -05:00
Chris Gagnon
1dd3e0189f Merging //UE4/Dev-Main to Dev-Editor (//UE4/Dev-Editor)
#rb none

[CL 4730305 by Chris Gagnon in Dev-Editor branch]
2019-01-15 18:47:22 -05:00
Marcus Wassmer
fee98d3ca5 Merging //UE4/Dev-Main@4729937 to Dev-Rendering (//UE4/Dev-Rendering)
#rb none

[CL 4730169 by Marcus Wassmer in Dev-Rendering branch]
2019-01-15 18:30:06 -05:00
Marcus Wassmer
cbfcbbb93b Merging //UE4/Dev-Main@4662404 to Dev-Rendering (//UE4/Dev-Rendering)
#rb none
Should be just copyright updates

[CL 4680440 by Marcus Wassmer in Dev-Rendering branch]
2019-01-03 19:16:26 -05:00
Christina TempelaarL
8c905859c2 Added comment re: when AdjustImageColors should be multithreaded.
#jira UE-62693
#rb none

[CL 4679068 by Christina TempelaarL in Dev-Editor branch]
2019-01-03 13:13:35 -05:00
Christina TempelaarL
5e71af3f97 Fixed Texture Editor HSV controls to work with HDR textures.
#jira UE-62693
#rb matt.kuhlenschmidt

[CL 4678963 by Christina TempelaarL in Dev-Editor branch]
2019-01-03 12:47:19 -05:00
Chris Gagnon
8fc25ea18e Merging //UE4/Dev-Main to Dev-Editor (//UE4/Dev-Editor)
#rb none

[CL 4676797 by Chris Gagnon in Dev-Editor branch]
2019-01-02 14:54:39 -05:00