mirror of
https://github.com/Dasharo/edk2.git
synced 2026-06-13 10:16:24 -07:00
darylm503
4710c53dca
These files are a subset of the python-2.7.2.tgz distribution from python.org. Changed files from PyMod-2.7.2 have been copied into the corresponding directories of this tree, replacing the original files in the distribution. Signed-off-by: daryl.mcdaniel@intel.com git-svn-id: https://edk2.svn.sourceforge.net/svnroot/edk2/trunk/edk2@13197 6f19259b-4bc3-4df7-8a09-765794883524
________________________________________________________________________ PYBENCH - A Python Benchmark Suite ________________________________________________________________________ Extendable suite of of low-level benchmarks for measuring the performance of the Python implementation (interpreter, compiler or VM). pybench is a collection of tests that provides a standardized way to measure the performance of Python implementations. It takes a very close look at different aspects of Python programs and let's you decide which factors are more important to you than others, rather than wrapping everything up in one number, like the other performance tests do (e.g. pystone which is included in the Python Standard Library). pybench has been used in the past by several Python developers to track down performance bottlenecks or to demonstrate the impact of optimizations and new features in Python. The command line interface for pybench is the file pybench.py. Run this script with option '--help' to get a listing of the possible options. Without options, pybench will simply execute the benchmark and then print out a report to stdout. Micro-Manual ------------ Run 'pybench.py -h' to see the help screen. Run 'pybench.py' to run the benchmark suite using default settings and 'pybench.py -f <file>' to have it store the results in a file too. It is usually a good idea to run pybench.py multiple times to see whether the environment, timers and benchmark run-times are suitable for doing benchmark tests. You can use the comparison feature of pybench.py ('pybench.py -c <file>') to check how well the system behaves in comparison to a reference run. If the differences are well below 10% for each test, then you have a system that is good for doing benchmark testings. Of you get random differences of more than 10% or significant differences between the values for minimum and average time, then you likely have some background processes running which cause the readings to become inconsistent. Examples include: web-browsers, email clients, RSS readers, music players, backup programs, etc. If you are only interested in a few tests of the whole suite, you can use the filtering option, e.g. 'pybench.py -t string' will only run/show the tests that have 'string' in their name. This is the current output of pybench.py --help: """ ------------------------------------------------------------------------ PYBENCH - a benchmark test suite for Python interpreters/compilers. ------------------------------------------------------------------------ Synopsis: pybench.py [option] files... Options and default settings: -n arg number of rounds (10) -f arg save benchmark to file arg () -c arg compare benchmark with the one in file arg () -s arg show benchmark in file arg, then exit () -w arg set warp factor to arg (10) -t arg run only tests with names matching arg () -C arg set the number of calibration runs to arg (20) -d hide noise in comparisons (0) -v verbose output (not recommended) (0) --with-gc enable garbage collection (0) --with-syscheck use default sys check interval (0) --timer arg use given timer (time.time) -h show this help text --help show this help text --debug enable debugging --copyright show copyright --examples show examples of usage Version: 2.0 The normal operation is to run the suite and display the results. Use -f to save them for later reuse or comparisons. Available timers: time.time time.clock systimes.processtime Examples: python2.1 pybench.py -f p21.pybench python2.5 pybench.py -f p25.pybench python pybench.py -s p25.pybench -c p21.pybench """ License ------- See LICENSE file. Sample output ------------- """ ------------------------------------------------------------------------------- PYBENCH 2.0 ------------------------------------------------------------------------------- * using Python 2.4.2 * disabled garbage collection * system check interval set to maximum: 2147483647 * using timer: time.time Calibrating tests. Please wait... Running 10 round(s) of the suite at warp factor 10: * Round 1 done in 6.388 seconds. * Round 2 done in 6.485 seconds. * Round 3 done in 6.786 seconds. ... * Round 10 done in 6.546 seconds. ------------------------------------------------------------------------------- Benchmark: 2006-06-12 12:09:25 ------------------------------------------------------------------------------- Rounds: 10 Warp: 10 Timer: time.time Machine Details: Platform ID: Linux-2.6.8-24.19-default-x86_64-with-SuSE-9.2-x86-64 Processor: x86_64 Python: Executable: /usr/local/bin/python Version: 2.4.2 Compiler: GCC 3.3.4 (pre 3.3.5 20040809) Bits: 64bit Build: Oct 1 2005 15:24:35 (#1) Unicode: UCS2 Test minimum average operation overhead ------------------------------------------------------------------------------- BuiltinFunctionCalls: 126ms 145ms 0.28us 0.274ms BuiltinMethodLookup: 124ms 130ms 0.12us 0.316ms CompareFloats: 109ms 110ms 0.09us 0.361ms CompareFloatsIntegers: 100ms 104ms 0.12us 0.271ms CompareIntegers: 137ms 138ms 0.08us 0.542ms CompareInternedStrings: 124ms 127ms 0.08us 1.367ms CompareLongs: 100ms 104ms 0.10us 0.316ms CompareStrings: 111ms 115ms 0.12us 0.929ms CompareUnicode: 108ms 128ms 0.17us 0.693ms ConcatStrings: 142ms 155ms 0.31us 0.562ms ConcatUnicode: 119ms 127ms 0.42us 0.384ms CreateInstances: 123ms 128ms 1.14us 0.367ms CreateNewInstances: 121ms 126ms 1.49us 0.335ms CreateStringsWithConcat: 130ms 135ms 0.14us 0.916ms CreateUnicodeWithConcat: 130ms 135ms 0.34us 0.361ms DictCreation: 108ms 109ms 0.27us 0.361ms DictWithFloatKeys: 149ms 153ms 0.17us 0.678ms DictWithIntegerKeys: 124ms 126ms 0.11us 0.915ms DictWithStringKeys: 114ms 117ms 0.10us 0.905ms ForLoops: 110ms 111ms 4.46us 0.063ms IfThenElse: 118ms 119ms 0.09us 0.685ms ListSlicing: 116ms 120ms 8.59us 0.103ms NestedForLoops: 125ms 137ms 0.09us 0.019ms NormalClassAttribute: 124ms 136ms 0.11us 0.457ms NormalInstanceAttribute: 110ms 117ms 0.10us 0.454ms PythonFunctionCalls: 107ms 113ms 0.34us 0.271ms PythonMethodCalls: 140ms 149ms 0.66us 0.141ms Recursion: 156ms 166ms 3.32us 0.452ms SecondImport: 112ms 118ms 1.18us 0.180ms SecondPackageImport: 118ms 127ms 1.27us 0.180ms SecondSubmoduleImport: 140ms 151ms 1.51us 0.180ms SimpleComplexArithmetic: 128ms 139ms 0.16us 0.361ms SimpleDictManipulation: 134ms 136ms 0.11us 0.452ms SimpleFloatArithmetic: 110ms 113ms 0.09us 0.571ms SimpleIntFloatArithmetic: 106ms 111ms 0.08us 0.548ms SimpleIntegerArithmetic: 106ms 109ms 0.08us 0.544ms SimpleListManipulation: 103ms 113ms 0.10us 0.587ms SimpleLongArithmetic: 112ms 118ms 0.18us 0.271ms SmallLists: 105ms 116ms 0.17us 0.366ms SmallTuples: 108ms 128ms 0.24us 0.406ms SpecialClassAttribute: 119ms 136ms 0.11us 0.453ms SpecialInstanceAttribute: 143ms 155ms 0.13us 0.454ms StringMappings: 115ms 121ms 0.48us 0.405ms StringPredicates: 120ms 129ms 0.18us 2.064ms StringSlicing: 111ms 127ms 0.23us 0.781ms TryExcept: 125ms 126ms 0.06us 0.681ms TryRaiseExcept: 133ms 137ms 2.14us 0.361ms TupleSlicing: 117ms 120ms 0.46us 0.066ms UnicodeMappings: 156ms 160ms 4.44us 0.429ms UnicodePredicates: 117ms 121ms 0.22us 2.487ms UnicodeProperties: 115ms 153ms 0.38us 2.070ms UnicodeSlicing: 126ms 129ms 0.26us 0.689ms ------------------------------------------------------------------------------- Totals: 6283ms 6673ms """ ________________________________________________________________________ Writing New Tests ________________________________________________________________________ pybench tests are simple modules defining one or more pybench.Test subclasses. Writing a test essentially boils down to providing two methods: .test() which runs .rounds number of .operations test operations each and .calibrate() which does the same except that it doesn't actually execute the operations. Here's an example: ------------------ from pybench import Test class IntegerCounting(Test): # Version number of the test as float (x.yy); this is important # for comparisons of benchmark runs - tests with unequal version # number will not get compared. version = 1.0 # The number of abstract operations done in each round of the # test. An operation is the basic unit of what you want to # measure. The benchmark will output the amount of run-time per # operation. Note that in order to raise the measured timings # significantly above noise level, it is often required to repeat # sets of operations more than once per test round. The measured # overhead per test round should be less than 1 second. operations = 20 # Number of rounds to execute per test run. This should be # adjusted to a figure that results in a test run-time of between # 1-2 seconds (at warp 1). rounds = 100000 def test(self): """ Run the test. The test needs to run self.rounds executing self.operations number of operations each. """ # Init the test a = 1 # Run test rounds # # NOTE: Use xrange() for all test loops unless you want to face # a 20MB process ! # for i in xrange(self.rounds): # Repeat the operations per round to raise the run-time # per operation significantly above the noise level of the # for-loop overhead. # Execute 20 operations (a += 1): a += 1 a += 1 a += 1 a += 1 a += 1 a += 1 a += 1 a += 1 a += 1 a += 1 a += 1 a += 1 a += 1 a += 1 a += 1 a += 1 a += 1 a += 1 a += 1 a += 1 def calibrate(self): """ Calibrate the test. This method should execute everything that is needed to setup and run the test - except for the actual operations that you intend to measure. pybench uses this method to measure the test implementation overhead. """ # Init the test a = 1 # Run test rounds (without actually doing any operation) for i in xrange(self.rounds): # Skip the actual execution of the operations, since we # only want to measure the test's administration overhead. pass Registering a new test module ----------------------------- To register a test module with pybench, the classes need to be imported into the pybench.Setup module. pybench will then scan all the symbols defined in that module for subclasses of pybench.Test and automatically add them to the benchmark suite. Breaking Comparability ---------------------- If a change is made to any individual test that means it is no longer strictly comparable with previous runs, the '.version' class variable should be updated. Therefafter, comparisons with previous versions of the test will list as "n/a" to reflect the change. Version History --------------- 2.0: rewrote parts of pybench which resulted in more repeatable timings: - made timer a parameter - changed the platform default timer to use high-resolution timers rather than process timers (which have a much lower resolution) - added option to select timer - added process time timer (using systimes.py) - changed to use min() as timing estimator (average is still taken as well to provide an idea of the difference) - garbage collection is turned off per default - sys check interval is set to the highest possible value - calibration is now a separate step and done using a different strategy that allows measuring the test overhead more accurately - modified the tests to each give a run-time of between 100-200ms using warp 10 - changed default warp factor to 10 (from 20) - compared results with timeit.py and confirmed measurements - bumped all test versions to 2.0 - updated platform.py to the latest version - changed the output format a bit to make it look nicer - refactored the APIs somewhat 1.3+: Steve Holden added the NewInstances test and the filtering option during the NeedForSpeed sprint; this also triggered a long discussion on how to improve benchmark timing and finally resulted in the release of 2.0 1.3: initial checkin into the Python SVN repository Have fun, -- Marc-Andre Lemburg mal@lemburg.com