Files
Michael Sartain c52bda1302 Clean up Linux inotify warnings and potential leaks
We still have some duplicate inotify watches, but this first pass will spew a lot more information when we hit inotify limits.

Adds a "DumpINotifyStats" command in non-release builds
  Spews global inotify & UE stats, along with physical count of directories, etc.

Canonicalize directory path in FDirectoryWatcherLinux::RegisterDirectoryChangedCallback_Handle
  Shootergame was adding 141 duplicate watches for Samples/Games/ShooterGame/Content w/o this. Was 1136, is now 995.

Change PathsToWatchDescriptors tmap to PathNameHashSet
  Don't need to store full paths for each watch directory twice

Fix bugs in TestPAL in addition to adding DumpStats() command, which looks ~ like this:

LogDirectoryWatcher: Warning: inotify limits
LogDirectoryWatcher: Warning:   max_queued_events: 16384
LogDirectoryWatcher: Warning:   max_user_instances: 128
LogDirectoryWatcher: Warning:   max_user_watches: 65536
LogDirectoryWatcher: Warning: inotify per-process stats
LogDirectoryWatcher: Warning:   systemd (pid 2239) watches:23 instances:3
...
LogDirectoryWatcher: Warning:   plugin_host-3.3 (pid 395041) watches:62 instances:1
LogDirectoryWatcher: Warning:   plugin_host-3.8 (pid 395044) watches:62 instances:1
LogDirectoryWatcher: Warning:   TestPAL (pid 396852) watches:2 instances:1
LogDirectoryWatcher: Warning: Total inotify Watches:392 Instances:28
LogDirectoryWatcher: Warning: Current watch requests
LogDirectoryWatcher: Warning:   /var/tmp/DirectoryWatcherTest396852: 2 watches
LogDirectoryWatcher: Warning:   Total count:2

The above is also dumped (once) when we fail to init or add a inotify watch.

Need to create better documentation and add a pointer to it, similar to what VSCode does: (hat tip Brandon)

    https://code.visualstudio.com/docs/setup/linux#_visual-studio-code-is-unable-to-watch-for-file-changes-in-this-large-workspace-error-enospc

Related bugs:

; FPS BP Cooking Content - errno = 28, Out of inotify watches
https://jira.it.epicgames.com/browse/UE-125210

; inotify Warnings when Cooking Content for Linux
https://jira.it.epicgames.com/browse/UE-119696

; Time Niagara Sequencer failed to play | Error: Couldn't find file for package
https://jira.it.epicgames.com/browse/UE-89750

; inotify warnings from Linux command line builds
https://jira.it.epicgames.com/browse/UE-76562

#review-17483609 @Brandon.Schaefer, @James.Singer
#jira UE-76562, UE-89750, UE-119696, UE-125210

[CL 17498916 by Michael Sartain in ue5-main branch]
2021-09-13 19:04:54 -04:00

131 lines
3.3 KiB
C++

// Copyright Epic Games, Inc. All Rights Reserved.
#include "Linux/DirectoryWatcherLinux.h"
#include "DirectoryWatcherPrivate.h"
FDirectoryWatcherLinux::FDirectoryWatcherLinux()
{
NumRequests = 0;
}
FDirectoryWatcherLinux::~FDirectoryWatcherLinux()
{
if (RequestMap.Num() != 0)
{
// Delete any remaining requests here. These requests are likely from modules which are still loaded at the time that this module unloads.
for (TMap<FString, FDirectoryWatchRequestLinux*>::TConstIterator RequestIt(RequestMap); RequestIt; ++RequestIt)
{
if (ensure(RequestIt.Value()))
{
delete RequestIt.Value();
NumRequests--;
}
}
RequestMap.Empty();
}
if (RequestsPendingDelete.Num() != 0)
{
for (int32 RequestIdx = 0; RequestIdx < RequestsPendingDelete.Num(); ++RequestIdx)
{
delete RequestsPendingDelete[RequestIdx];
NumRequests--;
}
}
// Make sure every request that was created is destroyed
ensure(NumRequests == 0);
}
bool FDirectoryWatcherLinux::RegisterDirectoryChangedCallback_Handle(const FString& InDirectory, const FDirectoryChanged& InDelegate, FDelegateHandle& OutHandle, uint32 Flags)
{
// Make sure the path is absolute. Without this, we were missing and duplicating some directories.
FString WatchDirectory = FPaths::ConvertRelativePathToFull(InDirectory);
FDirectoryWatchRequestLinux** RequestPtr = RequestMap.Find(WatchDirectory);
FDirectoryWatchRequestLinux* Request = NULL;
if (RequestPtr)
{
// There should be no NULL entries in the map
check (*RequestPtr);
Request = *RequestPtr;
}
else
{
Request = new FDirectoryWatchRequestLinux();
NumRequests++;
// Begin reading directory changes
if (!Request->Init(WatchDirectory, Flags))
{
UE_LOG(LogDirectoryWatcher, Warning, TEXT("Failed to begin reading directory changes for %s."), *WatchDirectory);
delete Request;
NumRequests--;
return false;
}
RequestMap.Add(WatchDirectory, Request);
}
OutHandle = Request->AddDelegate(InDelegate, Flags);
return true;
}
bool FDirectoryWatcherLinux::UnregisterDirectoryChangedCallback_Handle(const FString& InDirectory, FDelegateHandle InHandle)
{
// Make sure the path is absolute
FString WatchDirectory = FPaths::ConvertRelativePathToFull(InDirectory);
FDirectoryWatchRequestLinux** RequestPtr = RequestMap.Find(WatchDirectory);
if (RequestPtr)
{
// There should be no NULL entries in the map
check (*RequestPtr);
FDirectoryWatchRequestLinux* Request = *RequestPtr;
if (Request->RemoveDelegate(InHandle))
{
if (!Request->HasDelegates())
{
// Remove from the active map and add to the pending delete list
RequestMap.Remove(WatchDirectory);
RequestsPendingDelete.AddUnique(Request);
// Signal to end the watch which will mark this request for deletion
Request->EndWatchRequest();
}
return true;
}
}
return false;
}
void FDirectoryWatcherLinux::Tick(float DeltaSeconds)
{
// Delete unregistered requests
for (int32 RequestIdx = RequestsPendingDelete.Num() - 1; RequestIdx >= 0; --RequestIdx)
{
FDirectoryWatchRequestLinux* Request = RequestsPendingDelete[RequestIdx];
delete Request;
NumRequests--;
RequestsPendingDelete.RemoveAt(RequestIdx);
}
FDirectoryWatchRequestLinux::ProcessNotifications(RequestMap);
}
bool FDirectoryWatcherLinux::DumpStats()
{
FDirectoryWatchRequestLinux::DumpStats(RequestMap);
return true;
}