This commit introduces a pre-commit config, which checks all
Python files via black (a code foramtter) and flake8 (a style
checker).
This commit also updates all python files after reformatting
by black (obtained by running "pre-commit run --all").
One thing this commit does not do, however, is make any manual
change. In particular, this commit does not address the errors
reported by flake8.
Part of major reorganization of gnatcoll for better modularity and
maintainability.
Change-Id: Iad961b53634520081eb90c37aff2bfdbf58b2e06
TN: P909-033 no-precommit-check
This cleans up the dependency tree. The corresponding code
still exists and has been moved to the GPS repository, which
is probably the only place where this is needed.
For Q727-014
Change-Id: I2c9df154694fb3f1dcdc62c50be68d332a967f59
On 64 bits systems that support the mmap system call, we can now
read files up to one petabyte.
This change is backward compatible.
Add support for using madvise() system call when available. This can
provide some minor speed optimization depending on the system. For
instance, reading sequentially a 3Gb file has the following timing
info:
OSX Linux
use_advice: 14.98s 9.09s
use_sequential: 14.62s 8.74s
Q315-030
Change-Id: I85851b55f66db7f5d905923b8ddc0c007b6de1dd
A significant set of improvements were done to improve the performance,
namely:
When a trace is inactive, we no longer increase the counters, which are
atomic. This provides a minor speed up in multi-threaded applications.
Minor optimization to check whether the module has been finalized.
A single test is needed instead of two.
Change locking policy:
Instead of using GNAT.Task_Lock, we now use atomic counters and an
active loop to do the locking. This provides significant speed up in
multi threaded applications, and no slow down in mono-threaded.
Locks are now per-stream, instead of only using one global lock, which
will result in less contention when outputing to multiple streams.
Use C FILE* for the default file stream.
Bypasses Ada.Text_IO to use a lower-level directly.
This provides a slightly faster output, though the changes are not very
large.
Disable flushing after each message by default:
This commit also adds support for disabling the automatic flushing
after each message is printed, which also results in significant
speed up. A file stream can now be configured to force autoflush or not,
in the config file:
FOO >file.txt:buffer_size=0
File streams now have a setting (can be set in .gnatdebug) to specify
the size of the internal buffer at the system level.
This setting can only be set when the stream is declared in the config
file. This removes the need for dynamically checking the flag in Put,
and removes the need for fflush(), since we directly setup the
appropriate buffering in the FILE* handle.
Change design of decorators:
Decorators should now add to the buffered string, rather than output
directly on the stream. This is more efficient (a single operation to
the stream, and the critical section is much smaller, allowing better
parallelism).
These smaller critical sections allow better scaling on the
number of threads.
The use of a buffer also will make it possible to have asynchronous
streams (where the I/O is done in a separate thread to increase
throughput).
The buffer is implemented via GNATCOLL.Strings, since unbounded strings
systematically require a malloc and thus were slower. Typical slow
messages will require no malloc, although larger message will simply
allocate memory as needed.
This XString is also passed directly to the stream's put, so that it
can chose to store it in a buffer to implement async streams, without
a need for further memory allocation.
Make some parameters "not null" in gnatcoll-traces to match what
the code expects and avoid some checks from the compiler.
Make Trace_Handle_Record and Trace_Stream_Record limited types. These
can now be used to store limited types, like a File_Type or a protected
type.
Add Logger as an alias for Trace_Handle.
This is a more usual term in logging frameworks.
Store all active decorators in a separate array:
This makes traversing the list faster in Trace, which is the hotspot
in this package.
("DEBUG.MEMORY"): new decorator for gnatcoll.traces
Imported from GPS, since of general use
Fix memory barrier issue with the atomic operations.
Q212-003
P623-028
Comparing performance with master, when showing 200_000 messages
via 1, 2 or 100 threads. Main benefit is from not flushing
systematically, but the new implementation also scales better
with the number of threads. On OSX.
=======Old 1 threads ==========
Total time (auto flush) 0.315237000 100% 634443 ops/s
Total time (when inactive) 0.008458000 3% 23646252 ops/s
Total time (check active) 0.000443000 0% 451467264 ops/s
=======Old 2 threads ==========
Total time (auto flush) 1.066011000 338% 187615 ops/s
Total time (when inactive) 0.010407000 3% 19217834 ops/s
Total time (check active) 0.000297000 0% 673400704 ops/s
=======Old 100 threads ==========
Total time (auto flush) 1.192194000 378% 167758 ops/s
Total time (when inactive) 0.007754000 2% 25793140 ops/s
Total time (check active) 0.003973000 1% 50339792 ops/s
=======New 1 threads ==========
Total time 0.031278000 10% 6394271 ops/s
Total time (auto flush) 0.263455000 84% 759143 ops/s
Total time (when inactive) 0.004223000 1% 47359700 ops/s
Total time (check active) 0.000459000 0% 435729856 ops/s
=======New 2 threads ==========
Total time 0.050222000 16% 3982319 ops/s
Total time (auto flush) 0.398661000 126% 501679 ops/s
Total time (when inactive) 0.002369000 1% 84423800 ops/s
Total time (check active) 0.000330000 0% 606060608 ops/s
=======New 100 threads ==========
Total time 0.069576000 22% 2874554 ops/s
Total time (auto flush) 3.738483000 1186% 53498 ops/s
Total time (when inactive) 0.004329000 1% 46200044 ops/s
Total time (check active) 0.002613000 1% 76540368 ops/s
(cherry picked from commit 80b7a39ca5f7c35b394e8883e55103e54d8d0a94)
Change-Id: Id336f2420e2b3fc71de4c9c508a311dc8849576a
The incompatible API in Increase/Decrease_Indent is
breaking asistools, which breaks the GNAT package.
https://delphi.eu.adacore.com/#/e/19332542
(Also it looks like the GNATbench build failed)
This reverts commit 80b7a39ca5f7c35b394e8883e55103e54d8d0a94.
Change-Id: I2cc241487cbbddf4591ab6db57acbb0b9c039936
A significant set of improvements were done to improve the performance,
namely:
When a trace is inactive, we no longer increase the counters, which are
atomic. This provides a minor speed up in multi-threaded applications.
Minor optimization to check whether the module has been finalized.
A single test is needed instead of two.
Change locking policy:
Instead of using GNAT.Task_Lock, we now use atomic counters and an
active loop to do the locking. This provides significant speed up in
multi threaded applications, and no slow down in mono-threaded.
Locks are now per-stream, instead of only using one global lock, which
will result in less contention when outputing to multiple streams.
Use C FILE* for the default file stream.
Bypasses Ada.Text_IO to use a lower-level directly.
This provides a slightly faster output, though the changes are not very
large.
Disable flushing after each message by default:
This commit also adds support for disabling the automatic flushing
after each message is printed, which also results in significant
speed up. A file stream can now be configured to force autoflush or not,
in the config file:
FOO >file.txt:buffer_size=0
File streams now have a setting (can be set in .gnatdebug) to specify
the size of the internal buffer at the system level.
This setting can only be set when the stream is declared in the config
file. This removes the need for dynamically checking the flag in Put,
and removes the need for fflush(), since we directly setup the
appropriate buffering in the FILE* handle.
Change design of decorators:
Decorators should now add to the buffered string, rather than output
directly on the stream. This is more efficient (a single operation to
the stream, and the critical section is much smaller, allowing better
parallelism).
These smaller critical sections allow better scaling on the
number of threads.
The use of a buffer also will make it possible to have asynchronous
streams (where the I/O is done in a separate thread to increase
throughput).
The buffer is implemented via GNATCOLL.Strings, since unbounded strings
systematically require a malloc and thus were slower. Typical slow
messages will require no malloc, although larger message will simply
allocate memory as needed.
This XString is also passed directly to the stream's put, so that it
can chose to store it in a buffer to implement async streams, without
a need for further memory allocation.
Make some parameters "not null" in gnatcoll-traces to match what
the code expects and avoid some checks from the compiler.
Make Trace_Handle_Record and Trace_Stream_Record limited types. These
can now be used to store limited types, like a File_Type or a protected
type.
Add Logger as an alias for Trace_Handle.
This is a more usual term in logging frameworks.
Store all active decorators in a separate array:
This makes traversing the list faster in Trace, which is the hotspot
in this package.
("DEBUG.MEMORY"): new decorator for gnatcoll.traces
Imported from GPS, since of general use
Fix memory barrier issue with the atomic operations.
Q212-003
P623-028
Comparing performance with master, when showing 200_000 messages
via 1, 2 or 100 threads. Main benefit is from not flushing
systematically, but the new implementation also scales better
with the number of threads. On OSX.
=======Old 1 threads ==========
Total time (auto flush) 0.315237000 100% 634443 ops/s
Total time (when inactive) 0.008458000 3% 23646252 ops/s
Total time (check active) 0.000443000 0% 451467264 ops/s
=======Old 2 threads ==========
Total time (auto flush) 1.066011000 338% 187615 ops/s
Total time (when inactive) 0.010407000 3% 19217834 ops/s
Total time (check active) 0.000297000 0% 673400704 ops/s
=======Old 100 threads ==========
Total time (auto flush) 1.192194000 378% 167758 ops/s
Total time (when inactive) 0.007754000 2% 25793140 ops/s
Total time (check active) 0.003973000 1% 50339792 ops/s
=======New 1 threads ==========
Total time 0.031278000 10% 6394271 ops/s
Total time (auto flush) 0.263455000 84% 759143 ops/s
Total time (when inactive) 0.004223000 1% 47359700 ops/s
Total time (check active) 0.000459000 0% 435729856 ops/s
=======New 2 threads ==========
Total time 0.050222000 16% 3982319 ops/s
Total time (auto flush) 0.398661000 126% 501679 ops/s
Total time (when inactive) 0.002369000 1% 84423800 ops/s
Total time (check active) 0.000330000 0% 606060608 ops/s
=======New 100 threads ==========
Total time 0.069576000 22% 2874554 ops/s
Total time (auto flush) 3.738483000 1186% 53498 ops/s
Total time (when inactive) 0.004329000 1% 46200044 ops/s
Total time (check active) 0.002613000 1% 76540368 ops/s
Change-Id: If13722e1e1da29340f9a2a42d0b4e1eabdce9f98
We no longer use enumerated types to support the field types, which
makes it possible to add new types in third-party packages.
PB17-036
PB02-013
Change-Id: I520234e4e38401534a91632b781ef225aaee48d2
