The VFS directory iterator and recursive directory iterator behave differently
from the LLVM counterparts. Once the VFS iterators hit a broken symlink they
immediately abort. The LLVM counterparts allow to recover from this issue by
clearing the error code and skipping to the next entry.
This change adds the same functionality to the VFS iterators. There should be
no change in current behavior in the current CLANG source base, because all
clients have loop exit conditions that also check the error code.
This fixes rdar://problem/30934619.
Differential Revision: https://reviews.llvm.org/D30768
llvm-svn: 297510
Merge all VFS mapped files inside -ivfsoverlay inputs into the vfs
overlay provided by the crash reproducer. This is the last missing piece
to allow crash reproducers to fully work with user frameworks; when
combined with headermaps, it allows clang to find additional frameworks.
rdar://problem/27913709
llvm-svn: 290326
Reapply r278457 with test fixed to not abouse fs case sensitivity.
When the VFS uses a YAML file, the real file path for a
virtual file is described in the "external-contents" field. Example:
...
{
'type': 'file',
'name': 'a.h',
'external-contents': '/a/b/c/a.h'
}
Currently, when parsing umbrella directories, we use
vfs::recursive_directory_iterator to gather the header files to generate the
equivalent modules for. If the external contents for a header does not exist,
we currently are unable to build a module, since the VFS
vfs::recursive_directory_iterator will fail when it finds an entry without a
reliable real path.
Since the YAML file could be prepared ahead of time and shared among
different compiler invocations, an entry might not yet have a reliable
path in 'external-contents', breaking the iteration.
Give the VFS the capability to skip such entries whenever
'ignore-non-existent-contents' property is set in the YAML file.
rdar://problem/27531549
llvm-svn: 278543
When the VFS uses a YAML file, the real file path for a
virtual file is described in the "external-contents" field. Example:
...
{
'type': 'file',
'name': 'a.h',
'external-contents': '/a/b/c/a.h'
}
Currently, when parsing umbrella directories, we use
vfs::recursive_directory_iterator to gather the header files to generate the
equivalent modules for. If the external contents for a header does not exist,
we currently are unable to build a module, since the VFS
vfs::recursive_directory_iterator will fail when it finds an entry without a
reliable real path.
Since the YAML file could be prepared ahead of time and shared among
different compiler invocations, an entry might not yet have a reliable
path in 'external-contents', breaking the iteration.
Give the VFS the capability to skip such entries whenever
'ignore-non-existent-contents' property is set in the YAML file.
rdar://problem/27531549
llvm-svn: 278457
Add 'ignore-non-existent-contents' to tell the VFS whether an invalid path
obtained via 'external-contents' should cause iteration on the VFS to stop.
If 'true', the VFS should ignore the entry and continue with the next. Allows
YAML files to be shared across multiple compiler invocations regardless of
prior existent paths in 'external-contents'. This global value is overridable
on a per-file basis.
This adds the parsing and write test part, but use by VFS comes next.
Differential Revision: https://reviews.llvm.org/D23422
rdar://problem/27531549
llvm-svn: 278456
Differential Revision: http://reviews.llvm.org/D19843
Corresponding LLVM change: http://reviews.llvm.org/D19842
Re-commit after addressing issues with of generating too many warnings for Windows and asan test failures.
Patch by Eric Niebler
llvm-svn: 272562
Reapply r269100 and r269270, reverted due to
https://llvm.org/bugs/show_bug.cgi?id=27725. Isolate the testcase that
corresponds to the new feature side of this commit and skip it on
windows hosts until we find why it does not work on these platforms.
Original commit message:
The way we currently build the internal VFS overlay representation leads
to inefficient path search and might yield wrong answers when asked for
recursive or regular directory iteration.
Currently, when reading an YAML file, each YAML root entry is placed
inside a new root in the filesystem overlay. In the crash reproducer, a
simple "@import Foundation" currently maps to 43 roots, and when looking
up paths, we traverse a directory tree for each of these different
roots, until we find a match (or don't). This has two consequences:
- It's slow.
- Directory iteration gives incomplete results since it only return
results within one root - since contents of the same directory can be
declared inside different roots, the result isn't accurate.
This is in part fault of the way we currently write out the YAML file
when emitting the crash reproducer - we could generate only one root and
that would make it fast and correct again. However, we should not rely
on how the client writes the YAML, but provide a good internal
representation regardless.
Build a proper virtual directory tree out of the YAML representation,
allowing faster search and proper iteration. Besides the crash
reproducer, this potentially benefits other VFS clients.
llvm-svn: 269327
The way we currently build the internal VFS overlay representation leads
to inefficient path search and might yield wrong answers when asked for
recursive or regular directory iteration.
Currently, when reading an YAML file, each YAML root entry is placed
inside a new root in the filesystem overlay. In the crash reproducer, a
simple "@import Foundation" currently maps to 43 roots, and when looking
up paths, we traverse a directory tree for each of these different
roots, until we find a match (or don't). This has two consequences:
- It's slow.
- Directory iteration gives incomplete results since it only return
results within one root - since contents of the same directory can be
declared inside different roots, the result isn't accurate.
This is in part fault of the way we currently write out the YAML file
when emitting the crash reproducer - we could generate only one root and
that would make it fast and correct again. However, we should not rely
on how the client writes the YAML, but provide a good internal
representation regardless.
This patch builds a proper virtual directory tree out of the YAML
representation, allowing faster search and proper iteration. Besides the
crash reproducer, this potentially benefits other VFS clients.
llvm-svn: 269270
Bruno made a couple valiant attempts but the bot is still red.
This reverts r269100 (primary commit), r269108 (fix attempt), r269133
(fix attempt).
llvm-svn: 269160
The way we currently build the internal VFS overlay representation leads
to inefficient path search and might yield wrong answers when asked for
recursive or regular directory iteration.
Currently, when reading an YAML file, each YAML root entry is placed
inside a new root in the filesystem overlay. In the crash reproducer, a
simple "@import Foundation" currently maps to 43 roots, and when looking
up paths, we traverse a directory tree for each of these different
roots, until we find a match (or don't). This has two consequences:
- It's slow.
- Directory iteration gives incomplete results since it only return
results within one root - since contents of the same directory can be
declared inside different roots, the result isn't accurate.
This is in part fault of the way we currently write out the YAML file
when emitting the crash reproducer - we could generate only one root and
that would make it fast and correct again. However, we should not rely
on how the client writes the YAML, but provide a good internal
representation regardless.
This patch builds a proper virtual directory tree out of the YAML
representation, allowing faster search and proper iteration. Besides the
crash reproducer, this potentially benefits other VFS clients.
llvm-svn: 269100
Hide the real paths when rebuilding from VFS by setting up the crash
reproducer to use 'use-external-names' = false. This way we avoid
module redifinition errors and consistently use the same paths against
all modules.
With this change on Darwin we are able to simulate a crash for a simple
application using "Foundation/Foundation.h" (which relies on a bunch of
different frameworks and headers) and successfully rebuild all the
modules by relying solely at the VFS overlay.
llvm-svn: 266234
The VFS YAML files contain empty directory entries to describe that it's
returning from a subdirectory before describing new files in the parent.
In the future, we should properly sort and write YAML files avoiding
such empty dirs and mitigate the extra recurson cost. However, since
this is used by previous existing YAMLs, make the traversal work in
their presence.
rdar://problem/24499339
llvm-svn: 264970
Juergen Ributzka
Pavel Labath
Bruno Cardoso Lopes
Taewook Oh
Benjamin Kramer
Sean Silva