Commit Graph

701 Commits

Author SHA1 Message Date
Mehdi Amini 7b24ae662c Avoid temporary vector for sorting in BitcodeWriter
As suggested by Duncan, fixup for r269634 and r269635

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269715 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-16 22:47:15 +00:00
Mehdi Amini 4087d8ab8f ThinLTO: fix non-determinism in bitcode writing
Calls are initialized from a DenseMap. We can sort them using the
value id to recover some determinism during serialization.

From: mehdi_amini <mehdi_amini@91177308-0d34-0410-b5e6-96231b3b80d8>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269638 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-16 09:04:55 +00:00
Mehdi Amini 78132b5695 Revert "ThinLTO: fix non-determinism in bitcode writing"
This reverts commit r269634, bots are broken.

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269637 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-16 09:04:50 +00:00
Mehdi Amini 5975742a11 ThinLTO: fix another non-determinism in bitcode writing
GlobalVars Refs are initialized from a DenseSet. We can sort them
using the value id to recover some determinism during serialization.

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269635 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-16 08:50:27 +00:00
Mehdi Amini 1fece96084 ThinLTO: fix non-determinism in bitcode writing
Calls are initialized from a DenseMap. We can sort them using the
value id to recover some determinism during serialization.

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269634 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-16 08:50:23 +00:00
Mehdi Amini efdbbbabac ThinLTO: fix non-determinism in bitcode writing
Refs are initialized from a DenseSet. We can sort them using the
value id to recover some determinism during serialization.

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269629 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-16 04:50:47 +00:00
Benjamin Kramer 95ce0c789b Move helper classes into anonymous namespaces. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269591 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-15 15:18:11 +00:00
Teresa Johnson 52af319151 [ThinLTO] Fix Windows debug failure in new iterator
This fixes a debug assert on Windows from the new iterator
implementation added in r269059. The Windows std::vector iterator
operator== checks in debug mode that the containers being iterated over
are the same, which they may not be.

Fixed by checking that we are iterating over the same container before
comparing the container iterators.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269232 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-11 20:46:22 +00:00
Teresa Johnson 27af205ce1 Restore "[ThinLTO] Emit individual index files for distributed backends"
This restores commit r268627:
    Summary:
    When launching ThinLTO backends in a distributed build (currently
    supported in gold via the thinlto-index-only plugin option), emit
    an individual index file for each backend process as described here:
    http://lists.llvm.org/pipermail/llvm-dev/2016-April/098272.html

    ...

    Differential Revision: http://reviews.llvm.org/D19556

Address msan failures by avoiding std::prev on map.end(), the
theory is that this is causing issues due to some known UB problems
in __tree.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@269059 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-10 13:48:23 +00:00
Adrian Prantl e6b649c5df Implement a safer bitcode upgrade for DISubprogram.
The bitcode upgrade I added for DISubprogram in r266446 was based on the
assumption that the CU node for the subprogram was already materialized by the
time the DISubprogram is visited. This assumption may not hold true as future
versions of LLVM may decide to write out bitcode in a different order. This
patch corrects this by introducing a versioning bit next to the distinct flag to
unambiguously differentiate the new from the old record layouts.

Note for people stabilizing LLVM out-of-tree: This patch introduces a bitcode
incompatibility with llvm trunk revisions from r266446 — this commit. (But
D19987 will ensure that it degrades gracefully).

http://reviews.llvm.org/D20004
rdar://problem/26074194

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268816 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-06 22:53:06 +00:00
Peter Collingbourne 7a3e4d658c BitcodeWriter: Simplify. NFC.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268707 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-06 02:41:23 +00:00
Vitaly Buka eb80fa1253 Revert "[ThinLTO] Emit individual index files for distributed backends"
MemorySanitizer: use-of-uninitialized-value in lib/Bitcode/Writer/BitcodeWriter.cpp:364:70
http://lab.llvm.org:8011/builders/sanitizer-x86_64-linux-fast/builds/12544/steps/check-llvm%20msan/logs/stdio

This reverts commit 0c4a898ea5.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268660 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-05 18:31:00 +00:00
Teresa Johnson 0c4a898ea5 [ThinLTO] Emit individual index files for distributed backends
Summary:
When launching ThinLTO backends in a distributed build (currently
supported in gold via the thinlto-index-only plugin option), emit
an individual index file for each backend process as described here:
http://lists.llvm.org/pipermail/llvm-dev/2016-April/098272.html

The individual index file encodes the summary and module information
required for implementing the importing/exporting decisions made
for a given module in the thin link step.
This is in place of the current mechanism that uses the combined index
to make importing decisions in each back end independently. It is an
enabler for doing global summary based optimizations in the thin link
step (which will be recorded in the individual index files), and reduces
the size of the index that must be sent to each backend process, and
the amount of work to scan it in the backends.

Rather than create entirely new ModuleSummaryIndex structures (and all
the included unique_ptrs) for each backend index file, a map is created
to record all of the GUID and summary pointers needed for a particular
index file. The IndexBitcodeWriter walks this map instead of the full
index (hiding the details of managing the appropriate summary iteration
in a new iterator subclass). This is more efficient than walking the
entire combined index and filtering out just the needed summaries during
each backend bitcode index write.

Depends on D19481.

Reviewers: joker.eph

Subscribers: llvm-commits, joker.eph

Differential Revision: http://reviews.llvm.org/D19556

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@268627 91177308-0d34-0410-b5e6-96231b3b80d8
2016-05-05 13:44:56 +00:00
Sjoerd Meijer 09aedbd06b Clean up to avoid compiler warnings for casting away const qualifiers.
Differential Revision: http://reviews.llvm.org/D19598


git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267753 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-27 18:35:02 +00:00
Teresa Johnson 8777897f72 [ThinLTO] Use valueid instead of bitcode offsets in combined index file
Summary:
With the removal of support for lazy parsing of combined index summary
records (e.g. r267344), we no longer need to include the summary record
bitcode offset in the VST entries for definitions. Change the combined
index format to be similar to the per-module index format in using value
ids to cross-reference from the summary record to the VST entry (rather
than the summary record bitcode offset to cross-reference in the other
direction).

The visible changes are:
1) Add the value id to the combined summary records
2) Remove the summary offset from the combined VST records, which has
the following effects:
- No longer need the VST_CODE_COMBINED_GVDEFENTRY record, as all
  combined index VST entries now only contain the value id and
  corresponding GUID.
- No longer have duplicate VST entries in the case where there are
  multiple definitions of a symbol (e.g. weak/linkonce), as they all
  have the same value id and GUID.

An implication of #2 above is that in order to hook up an alias to the
correct aliasee based on the value id of the aliasee recorded in the
combined index alias record, we need to scan the entries in the index
for that GUID to find the one from the same module (i.e. the case where
there are multiple entries for the aliasee). But the reader no longer
has to maintain a special map to hook up the alias/aliasee.

Reviewers: joker.eph

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D19481

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267712 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-27 13:28:35 +00:00
Teresa Johnson ff1d72888b [ThinLTO] Remove GlobalValueInfo class from index
Summary:
Remove the GlobalValueInfo and change the ModuleSummaryIndex to directly
reference summary objects. The info structure was there to support lazy
parsing of the combined index summary objects, which is no longer
needed and not supported.

Reviewers: joker.eph

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D19462

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267344 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-24 14:57:11 +00:00
Aaron Ballman 37c06ef983 Silence two C4806 warnings ('|': unsafe operation: no value of type 'bool' promoted to type 'const unsigned int' can equal the given constant). The fact that they trigger with this code seems like it may be a bug, but the warning itself is still generally useful enough to retain it for now.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267337 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-24 13:03:20 +00:00
Mehdi Amini 822155bcb5 Add "hasSection" flag in the Summary
Reviewers: tejohnson

Subscribers: llvm-commits

Differential Revision: http://reviews.llvm.org/D19405

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267329 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-24 05:31:43 +00:00
Mehdi Amini 65562d6474 Reorganize GlobalValueSummary with a "Flags" bitfield.
Right now it only contains the LinkageType, but will be extended
with "hasSection", "isOptSize", "hasInlineAssembly", etc.

Differential Revision: http://reviews.llvm.org/D19404

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267319 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-24 03:18:18 +00:00
Mehdi Amini 3f53abc0a5 Add a version field in the bitcode for the summary
Differential Revision: http://reviews.llvm.org/D19456

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267318 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-24 03:18:11 +00:00
Mehdi Amini 9ab3b37131 Store and emit original name in combined index
Summary:
As discussed in D18298, some local globals can't
be renamed/promoted (because they have a section, or because
they are referenced from inline assembly).
To be able to detect naming collision, we need to keep around
the "GUID" using their original name without taking the linkage
into account.

Reviewers: tejohnson

Subscribers: joker.eph, llvm-commits

Differential Revision: http://reviews.llvm.org/D19454

From: Mehdi Amini <mehdi.amini@apple.com>

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267304 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-23 23:38:17 +00:00
Duncan P. N. Exon Smith de7484036b DebugInfo: Remove MDString-based type references
Eliminate DITypeIdentifierMap and make DITypeRef a thin wrapper around
DIType*.  It is no longer legal to refer to a DICompositeType by its
'identifier:', and DIBuilder no longer retains all types with an
'identifier:' automatically.

Aside from the bitcode upgrade, this is mainly removing logic to resolve
an MDString-based reference to an actualy DIType.  The commits leading
up to this have made the implicit type map in DICompileUnit's
'retainedTypes:' field superfluous.

This does not remove DITypeRef, DIScopeRef, DINodeRef, and
DITypeRefArray, or stop using them in DI-related metadata.  Although as
of this commit they aren't serving a useful purpose, there are patchces
under review to reuse them for CodeView support.

The tests in LLVM were updated with deref-typerefs.sh, which is attached
to the thread "[RFC] Lazy-loading of debug info metadata":

  http://lists.llvm.org/pipermail/llvm-dev/2016-April/098318.html

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267296 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-23 21:08:00 +00:00
Duncan P. N. Exon Smith 29fa04dd53 BitcodeWriter: Emit uniqued subgraphs after all distinct nodes
Since forward references for uniqued node operands are expensive (and
those for distinct node operands are cheap due to
DistinctMDOperandPlaceholder), minimize forward references in uniqued
node operands.

Moreover, guarantee that when a cycle is broken by a distinct node, none
of the uniqued nodes have any forward references.  In
ValueEnumerator::EnumerateMetadata, enumerate uniqued node subgraphs
first, delaying distinct nodes until all uniqued nodes have been
handled.  This guarantees that uniqued nodes only have forward
references when there is a uniquing cycle (since r267276 changed
ValueEnumerator::organizeMetadata to partition distinct nodes in front
of uniqued nodes as a post-pass).

Note that a single uniqued subgraph can hit multiple distinct nodes at
its leaves.  Ideally these would themselves be emitted in post-order,
but this commit doesn't attempt that; I think it requires an extra pass
through the edges, which I'm not convinced is worth it (since
DistinctMDOperandPlaceholder makes forward references quite cheap
between distinct nodes).

I've added two testcases:

  - test/Bitcode/mdnodes-distinct-in-post-order.ll is just like
    test/Bitcode/mdnodes-in-post-order.ll, except with distinct nodes
    instead of uniqued ones.  This confirms that, in the absence of
    uniqued nodes, distinct nodes are still emitted in post-order.

  - test/Bitcode/mdnodes-distinct-nodes-break-cycles.ll is the minimal
    example where a naive post-order traversal would cause one uniqued
    node to forward-reference another.  IOW, it's the motivating test.

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267278 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-23 04:59:22 +00:00
Duncan P. N. Exon Smith e7a2c97bc2 BitcodeWriter: Emit distinct nodes before uniqued nodes
When an operand of a distinct node hasn't been read yet, the reader can
use a DistinctMDOperandPlaceholder.  This is much cheaper than forward
referencing from a uniqued node.  Change
ValueEnumerator::organizeMetadata to partition distinct nodes and
uniqued nodes to reduce the overhead of cycles broken by distinct nodes.

Mehdi measured this for me; this removes most of the RAUW from the
importing step of -flto=thin, even after a WIP patch that removes
string-based DITypeRefs (introducing many more cycles to the metadata
graph).

git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267276 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-23 04:42:39 +00:00
Teresa Johnson 26f6676ea8 Address comments.
git-svn-id: https://llvm.org/svn/llvm-project/llvm/trunk@267274 91177308-0d34-0410-b5e6-96231b3b80d8
2016-04-23 04:31:20 +00:00