Commit Graph

547 Commits

Author SHA1 Message Date
Mehdi Amini 5975742a11 ThinLTO: fix another non-determinism in bitcode writing
GlobalVars Refs are initialized from a DenseSet. We can sort them
using the value id to recover some determinism during serialization.

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269635 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-16 08:50:27 +00:00
Mehdi Amini 1fece96084 ThinLTO: fix non-determinism in bitcode writing
Calls are initialized from a DenseMap. We can sort them using the
value id to recover some determinism during serialization.

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269634 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-16 08:50:23 +00:00
Mehdi Amini efdbbbabac ThinLTO: fix non-determinism in bitcode writing
Refs are initialized from a DenseSet. We can sort them using the
value id to recover some determinism during serialization.

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269629 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-16 04:50:47 +00:00
Benjamin Kramer 95ce0c789b Move helper classes into anonymous namespaces. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269591 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-15 15:18:11 +00:00
Teresa Johnson 52af319151 [ThinLTO] Fix Windows debug failure in new iterator
This fixes a debug assert on Windows from the new iterator
implementation added in r269059. The Windows std::vector iterator
operator== checks in debug mode that the containers being iterated over
are the same, which they may not be.

Fixed by checking that we are iterating over the same container before
comparing the container iterators.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269232 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-11 20:46:22 +00:00
Teresa Johnson 27af205ce1 Restore "[ThinLTO] Emit individual index files for distributed backends"
This restores commit r268627:
    Summary:
    When launching ThinLTO backends in a distributed build (currently
    supported in gold via the thinlto-index-only plugin option), emit
    an individual index file for each backend process as described here:
    http://lists.llvm.org/pipermail/llvm-dev/2016-April/098272.html

    ...

    Differential Revision: http://reviews.llvm.org/D19556

Address msan failures by avoiding std::prev on map.end(), the
theory is that this is causing issues due to some known UB problems
in __tree.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269059 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-10 13:48:23 +00:00
Adrian Prantl e6b649c5df Implement a safer bitcode upgrade for DISubprogram.
The bitcode upgrade I added for DISubprogram in r266446 was based on the
assumption that the CU node for the subprogram was already materialized by the
time the DISubprogram is visited. This assumption may not hold true as future
versions of LLVM may decide to write out bitcode in a different order. This
patch corrects this by introducing a versioning bit next to the distinct flag to
unambiguously differentiate the new from the old record layouts.

Note for people stabilizing LLVM out-of-tree: This patch introduces a bitcode
incompatibility with llvm trunk revisions from r266446 — this commit. (But
D19987 will ensure that it degrades gracefully).

http://reviews.llvm.org/D20004
rdar://problem/26074194

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268816 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-06 22:53:06 +00:00
Peter Collingbourne 7a3e4d658c BitcodeWriter: Simplify. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268707 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-06 02:41:23 +00:00
Vitaly Buka eb80fa1253 Revert "[ThinLTO] Emit individual index files for distributed backends"
MemorySanitizer: use-of-uninitialized-value in lib/Bitcode/Writer/BitcodeWriter.cpp:364:70
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/12544/steps/check-llvm%20msan/logs/stdio

This reverts commit 0c4a898ea5.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268660 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-05 18:31:00 +00:00
Teresa Johnson 0c4a898ea5 [ThinLTO] Emit individual index files for distributed backends
Summary:
When launching ThinLTO backends in a distributed build (currently
supported in gold via the thinlto-index-only plugin option), emit
an individual index file for each backend process as described here:
http://lists.llvm.org/pipermail/llvm-dev/2016-April/098272.html

The individual index file encodes the summary and module information
required for implementing the importing/exporting decisions made
for a given module in the thin link step.
This is in place of the current mechanism that uses the combined index
to make importing decisions in each back end independently. It is an
enabler for doing global summary based optimizations in the thin link
step (which will be recorded in the individual index files), and reduces
the size of the index that must be sent to each backend process, and
the amount of work to scan it in the backends.

Rather than create entirely new ModuleSummaryIndex structures (and all
the included unique_ptrs) for each backend index file, a map is created
to record all of the GUID and summary pointers needed for a particular
index file. The IndexBitcodeWriter walks this map instead of the full
index (hiding the details of managing the appropriate summary iteration
in a new iterator subclass). This is more efficient than walking the
entire combined index and filtering out just the needed summaries during
each backend bitcode index write.

Depends on D19481.

Reviewers: joker.eph

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D19556

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268627 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-05 13:44:56 +00:00
Sjoerd Meijer 09aedbd06b Clean up to avoid compiler warnings for casting away const qualifiers.
Differential Revision: http://reviews.llvm.org/D19598


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267753 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-27 18:35:02 +00:00
Teresa Johnson 8777897f72 [ThinLTO] Use valueid instead of bitcode offsets in combined index file
Summary:
With the removal of support for lazy parsing of combined index summary
records (e.g. r267344), we no longer need to include the summary record
bitcode offset in the VST entries for definitions. Change the combined
index format to be similar to the per-module index format in using value
ids to cross-reference from the summary record to the VST entry (rather
than the summary record bitcode offset to cross-reference in the other
direction).

The visible changes are:
1) Add the value id to the combined summary records
2) Remove the summary offset from the combined VST records, which has
the following effects:
- No longer need the VST_CODE_COMBINED_GVDEFENTRY record, as all
  combined index VST entries now only contain the value id and
  corresponding GUID.
- No longer have duplicate VST entries in the case where there are
  multiple definitions of a symbol (e.g. weak/linkonce), as they all
  have the same value id and GUID.

An implication of #2 above is that in order to hook up an alias to the
correct aliasee based on the value id of the aliasee recorded in the
combined index alias record, we need to scan the entries in the index
for that GUID to find the one from the same module (i.e. the case where
there are multiple entries for the aliasee). But the reader no longer
has to maintain a special map to hook up the alias/aliasee.

Reviewers: joker.eph

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D19481

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267712 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-27 13:28:35 +00:00
Teresa Johnson ff1d72888b [ThinLTO] Remove GlobalValueInfo class from index
Summary:
Remove the GlobalValueInfo and change the ModuleSummaryIndex to directly
reference summary objects. The info structure was there to support lazy
parsing of the combined index summary objects, which is no longer
needed and not supported.

Reviewers: joker.eph

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D19462

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267344 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-24 14:57:11 +00:00
Aaron Ballman 37c06ef983 Silence two C4806 warnings ('|': unsafe operation: no value of type 'bool' promoted to type 'const unsigned int' can equal the given constant). The fact that they trigger with this code seems like it may be a bug, but the warning itself is still generally useful enough to retain it for now.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267337 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-24 13:03:20 +00:00
Mehdi Amini 822155bcb5 Add "hasSection" flag in the Summary
Reviewers: tejohnson

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D19405

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267329 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-24 05:31:43 +00:00
Mehdi Amini 65562d6474 Reorganize GlobalValueSummary with a "Flags" bitfield.
Right now it only contains the LinkageType, but will be extended
with "hasSection", "isOptSize", "hasInlineAssembly", etc.

Differential Revision: http://reviews.llvm.org/D19404

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267319 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-24 03:18:18 +00:00
Mehdi Amini 3f53abc0a5 Add a version field in the bitcode for the summary
Differential Revision: http://reviews.llvm.org/D19456

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267318 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-24 03:18:11 +00:00
Mehdi Amini 9ab3b37131 Store and emit original name in combined index
Summary:
As discussed in D18298, some local globals can't
be renamed/promoted (because they have a section, or because
they are referenced from inline assembly).
To be able to detect naming collision, we need to keep around
the "GUID" using their original name without taking the linkage
into account.

Reviewers: tejohnson

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D19454

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267304 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-23 23:38:17 +00:00
Duncan P. N. Exon Smith de7484036b DebugInfo: Remove MDString-based type references
Eliminate DITypeIdentifierMap and make DITypeRef a thin wrapper around
DIType*.  It is no longer legal to refer to a DICompositeType by its
'identifier:', and DIBuilder no longer retains all types with an
'identifier:' automatically.

Aside from the bitcode upgrade, this is mainly removing logic to resolve
an MDString-based reference to an actualy DIType.  The commits leading
up to this have made the implicit type map in DICompileUnit's
'retainedTypes:' field superfluous.

This does not remove DITypeRef, DIScopeRef, DINodeRef, and
DITypeRefArray, or stop using them in DI-related metadata.  Although as
of this commit they aren't serving a useful purpose, there are patchces
under review to reuse them for CodeView support.

The tests in LLVM were updated with deref-typerefs.sh, which is attached
to the thread "[RFC] Lazy-loading of debug info metadata":

  http://lists.llvm.org/pipermail/llvm-dev/2016-April/098318.html

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267296 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-23 21:08:00 +00:00
Teresa Johnson 26f6676ea8 Address comments.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267274 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-23 04:31:20 +00:00
Teresa Johnson 6eed7793be Refactor bitcode writer into classes (NFC)
Summary:
As discussed in on the mailing list yesterday, I have refactored
BitcodeWriter.cpp to use classes to manage the bitcode writing process,
instead of passing around long lists of parameters between static
functions. See:
  http://lists.llvm.org/pipermail/llvm-dev/2016-April/098610.html

I created a parent BitcodeWriter class to own the BitstreamWriter,
write the header, and contain the main entry point into the writing
process. There are two derived classes, one for writing a module and one
for writing a combined index file (for ThinLTO), which manage the
writing process specific to those bitcode file types.

I also changed the functions to conform to LLVM coding standards
(lowercase function name first letter). The only two routines that still
start with an uppercase letter are the two external interfaces, which
can be fixed as a follow-on (I wanted to keep this round just within
BitcodeWriter.cpp).

Reviewers: dexonsmith, joker.eph

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D19447

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267273 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-23 04:30:47 +00:00
Teresa Johnson 89c5c63c3c [ThinLTO] Prevent importing of "llvm.used" values
Summary:
This patch prevents importing from (and therefore exporting from) any
module with a "llvm.used" local value. Local values need to be promoted
and renamed when importing, and their presense on the llvm.used variable
indicates that there are opaque uses that won't see the rename. One such
example is a use in inline assembly.

See also the discussion at:
http://lists.llvm.org/pipermail/llvm-dev/2016-April/098047.html

As part of this, move collectUsedGlobalVariables out of Transforms/Utils
and into IR/Module so that it can be used more widely. There are several
other places in LLVM that used copies of this code that can be cleaned
up as a follow on NFC patch.

Reviewers: joker.eph

Subscribers: pcc, llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D18986

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266877 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-20 14:39:45 +00:00
Mehdi Amini f6071e14c5 [NFC] Header cleanup
Removed some unused headers, replaced some headers with forward class declarations.

Found using simple scripts like this one:
clear && ack --cpp -l '#include "llvm/ADT/IndexedMap.h"' | xargs grep -L 'IndexedMap[<]' | xargs grep -n --color=auto 'IndexedMap'

Patch by Eugene Kosov <claprix@yandex.ru>

Differential Revision: http://reviews.llvm.org/D19219

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266595 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-18 09:17:29 +00:00
Mehdi Amini eb79e6e85d ThinLTO: Make aliases explicit in the summary
To be able to work accurately on the reference graph when taking
decision about internalizing, promoting, renaming, etc. We need
to have the alias information explicit.

Differential Revision: http://reviews.llvm.org/D18836

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266517 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-16 06:56:44 +00:00
Adrian Prantl 4eeaa0da04 [PR27284] Reverse the ownership between DICompileUnit and DISubprogram.
Currently each Function points to a DISubprogram and DISubprogram has a
scope field. For member functions the scope is a DICompositeType. DIScopes
point to the DICompileUnit to facilitate type uniquing.

Distinct DISubprograms (with isDefinition: true) are not part of the type
hierarchy and cannot be uniqued. This change removes the subprograms
list from DICompileUnit and instead adds a pointer to the owning compile
unit to distinct DISubprograms. This would make it easy for ThinLTO to
strip unneeded DISubprograms and their transitively referenced debug info.

Motivation
----------

Materializing DISubprograms is currently the most expensive operation when
doing a ThinLTO build of clang.

We want the DISubprogram to be stored in a separate Bitcode block (or the
same block as the function body) so we can avoid having to expensively
deserialize all DISubprograms together with the global metadata. If a
function has been inlined into another subprogram we need to store a
reference the block containing the inlined subprogram.

Attached to https://llvm.org/bugs/show_bug.cgi?id=27284 is a python script
that updates LLVM IR testcases to the new format.

http://reviews.llvm.org/D19034
<rdar://problem/25256815>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@266446 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-15 15:57:41 +00:00