This is needed to fix the reason
0a2be46cfd (Modules: Invalidate out-of-date PCMs as they're
discovered) and 5b44a4b07fc1d ([modules] Do not cache invalid state for
modules that we attempted to load.) were reverted.
These patches changed Clang to use `isVolatile` when loading modules.
This had the side effect of not using mmap when loading modules, and
thus greatly increased memory usage.
The reason it wasn't using mmap is because `MemoryBuffer` plays some
games with file size when you request null termination, and it has to
disable these when `isVolatile` is set as the size may change by the
time it's mmapped. Clang by default passes
`RequiresNullTerminator = true`, and `shouldUseMmap` ignored if
`RequiresNullTerminator` was even requested.
This patch adds `RequiresNullTerminator` to the `FileManager` interface
so Clang can use it when loading modules, and changes `shouldUseMmap` to
only take volatility into account if `RequiresNullTerminator` is true.
This is fine as both `mmap` and a `read` loop are vulnerable to
modifying the file while reading, but are immune to the rename Clang
does when replacing a module file.
Differential Revision: https://reviews.llvm.org/D77772
This is how it should've been and brings it more in line with
std::string_view. There should be no functional change here.
This is mostly mechanical from a custom clang-tidy check, with a lot of
manual fixups. It uncovers a lot of minor inefficiencies.
This doesn't actually modify StringRef yet, I'll do that in a follow-up.
In the current implementation of clang the canonicalization of paths in
diagnostic messages (when using -fdiagnostics-absolute-paths) only works
if the symbolic link is in the directory part of the filename, not if
the file itself is a symbolic link to another file.
This patch adds support to canonicalize the complete path including the
file.
Reviewers: rsmith, hans, rnk, ikudrin
Reviewed By: rnk
Differential Revision: https://reviews.llvm.org/D70527
accessed name to the directory entry
This commit introduces a parallel API that returns a DirectoryEntryRef
to the FileManager, similar to the parallel FileEntryRef API. All
uses will have to be update in follow-up patches. The immediate use of the new API in this
patch fixes the issue where a file manager was reused in clang-scan-deps,
but reported an different file path whenever a framework lookup was done through a symlink.
Differential Revision: https://reviews.llvm.org/D67026
llvm-svn: 370562
If contents of a file that is part of a PCM are overridden when reading
it, but weren't overridden when the PCM was being built, the ASTReader
will emit an error. Now it creates a separate FileEntry for recovery,
bypassing the overridden content instead of discarding it. The
pre-existing testcase clang/test/PCH/remap-file-from-pch.cpp confirms
that the new recovery method works correctly.
This resolves a long-standing FIXME to avoid hypothetically invalidating
another precompiled module that's already using the overridden contents.
This also removes ContentCache-related API that would be unsafe to use
across `CompilerInstance`s in an implicit modules build. This helps to
unblock us sinking it from SourceManager into FileManager in the future,
which would allow us to delete `InMemoryModuleCache`.
https://reviews.llvm.org/D66710
llvm-svn: 370546
`FileManager::getFileRef` is a modern API which we expect to convert to
over time. We should modernize the error handling as well, using
`llvm::Expected` instead of `llvm::ErrorOr`, to help clients that care
about errors to ensure nothing is missed.
However, not all clients care. I've also added another path for those
that don't:
- `FileEntryRef` is now copy- and move-assignable (using a pointer
instead of a reference).
- `FileManager::getOptionalFileRef` returns an `llvm::Optional` instead
of `llvm::Expected`.
- Added an `llvm::expectedToOptional` utility in case this is useful
elsewhere.
https://reviews.llvm.org/D66705
llvm-svn: 369943
when the FileManager is reused across invocations
This commit introduces a parallel API to FileManager's getFile: getFileEntryRef, which returns
a reference to the FileEntry, and the name that was used to access the file. In the case of
a VFS with 'use-external-names', the FileEntyRef contains the external name of the file,
not the filename that was used to access it.
The new API is adopted only in the HeaderSearch and Preprocessor for include file lookup, so that the
accessed path can be propagated to SourceManager's FileInfo. SourceManager's FileInfo now can report this accessed path, using
the new getName method. This API is then adopted in the dependency collector, which now correctly reports dependencies when a file
is included both using a symlink and a real path in the case when the FileManager is reused across multiple Preprocessor invocations.
Note that this patch does not fix all dependency collector issues, as the same problem is still present in other cases when dependencies
are obtained using FileSkipped, InclusionDirective, and HasInclude. This will be fixed in follow-up commits.
Differential Revision: https://reviews.llvm.org/D65907
llvm-svn: 369680
Now that we've moved to C++14, we no longer need the llvm::make_unique
implementation from STLExtras.h. This patch is a mechanical replacement
of (hopefully) all the llvm::make_unique instances across the monorepo.
Differential revision: https://reviews.llvm.org/D66259
llvm-svn: 368942
Previously, the FileManager would use NULL returns to signify whether a file existed, but that doesn’t cover permissions issues or anything else that might occur while trying to stat or read a file. Instead, convert getFile and getDirectory into returning llvm::ErrorOr
Signed-off-by: Harlan Haskins <harlan@apple.com>
llvm-svn: 367615
Summary:
Previously, we would return true/false signifying if the cache/lookup
succeeded or failed. Instead, provide clients with the underlying error
that was thrown while attempting to look up in the cache.
Since clang::FileManager doesn't make use of this information, it discards the
error that's received and casts away to bool.
This change is NFC.
Reviewers: benlangmuir, arphaman
Subscribers: dexonsmith, cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D60735
llvm-svn: 358509
Summary:
FileData was only ever used as a container for the values in
llvm::vfs::Status, so they might as well be consolidated.
The `InPCH` member was also always set to false, and unused.
Subscribers: cfe-commits
Tags: #clang
Differential Revision: https://reviews.llvm.org/D58924
llvm-svn: 355368
The pathname wasn't previously filled when the getFile() method was called with openFile = false.
We are caching FileEntry-s in ParsedAST::Includes in clangd and this caused the problem.
This fixes an internal test failure in clangd - ClangdTests.GoToInclude.All
rdar://47536127
Differential Revision: https://reviews.llvm.org/D58213
llvm-svn: 354075
Summary:
r347205 fixed a bug in FileManager: first calling
getFile(shouldOpen=false) and then getFile(shouldOpen=true) results in
the file not being open.
Unfortunately, some code was (inadvertently?) relying on this bug: when
building with a PCH, the file entries are obtained first by passing
shouldOpen=false, and then later shouldOpen=true, without any intention
of reading them. After r347205, they do get unneccesarily opened.
Aside from extra operations, this means they need to be closed. Normally
files are closed when their contents are read. As these files are never
read, they stay open until clang exits. On platforms with a low
open-files limit (e.g. Mac), this can lead to spurious file-not-found
errors when building large projects with PCH enabled, e.g.
https://bugs.chromium.org/p/chromium/issues/detail?id=924225
Fixing the callsites to pass shouldOpen=false when the file won't be
read is not quite trivial (that info isn't available at the direct
callsite), and passing shouldOpen=false is a performance regression (it
results in open+fstat pairs being replaced by stat+open).
So an ideal fix is going to be a little risky and we need some fix soon
(especially for the llvm 8 branch).
The problem addressed by r347205 is rare and has only been observed in
clangd. It was present in llvm-7, so we can live with it for now.
Reviewers: bkramer, thakis
Subscribers: ilya-biryukov, ioeric, kadircet, cfe-commits
Differential Revision: https://reviews.llvm.org/D57165
llvm-svn: 352079
to reflect the new license.
We understand that people may be surprised that we're moving the header
entirely to discuss the new license. We checked this carefully with the
Foundation's lawyer and we believe this is the correct approach.
Essentially, all code in the project is now made available by the LLVM
project under our new license, so you will see that the license headers
include that license only. Some of our contributors have contributed
code under our old license, and accordingly, we have retained a copy of
our old license notice in the top-level files in each project and
repository.
llvm-svn: 351636
Michael Spencer
Jim Lin
Benjamin Kramer
Karl-Johan Karlsson
Alex Suhan
Volodymyr Sapsai
Alex Lorenz
Duncan P. N. Exon Smith
Richard Smith
Nico Weber
Jonas Devlieghere
Harlan Haskins
Jan Korous
Reid Kleckner
Sam McCall
Chandler Carruth