diff --git a/testing/mozbase/README.md b/testing/mozbase/README.md index 7656598e9ea..4975039ff4d 100644 --- a/testing/mozbase/README.md +++ b/testing/mozbase/README.md @@ -12,7 +12,7 @@ the docs directory. Consult [open][] [bugs][] and feel free to file [new bugs][]. -[project page]: https://wiki.mozilla.org/Auto-tools/Projects/Mozbase +[project page]: https://wiki.mozilla.org/Auto-tools/Projects/MozBase [detailed docs]: http://mozbase.readthedocs.org/ [open]: https://bugzilla.mozilla.org/buglist.cgi?resolution=---&component=Mozbase&product=Testing [bugs]: https://bugzilla.mozilla.org/buglist.cgi?resolution=---&status_whiteboard_type=allwordssubstr&query_format=advanced&status_whiteboard=mozbase diff --git a/testing/mozbase/manifestdestiny/README.md b/testing/mozbase/manifestdestiny/README.md new file mode 100644 index 00000000000..b0527f0bdec --- /dev/null +++ b/testing/mozbase/manifestdestiny/README.md @@ -0,0 +1,414 @@ +Universal manifests for Mozilla test harnesses + +# What is ManifestDestiny? + +What ManifestDestiny gives you: + +* manifests are ordered lists of tests +* tests may have an arbitrary number of key, value pairs +* the parser returns an ordered list of test data structures, which + are just dicts with some keys. For example, a test with no + user-specified metadata looks like this: + + [{'expected': 'pass', + 'path': '/home/mozilla/mozmill/src/ManifestDestiny/manifestdestiny/tests/testToolbar/testBackForwardButtons.js', + 'relpath': 'testToolbar/testBackForwardButtons.js', + 'name': 'testBackForwardButtons.js', + 'here': '/home/mozilla/mozmill/src/ManifestDestiny/manifestdestiny/tests', + 'manifest': '/home/mozilla/mozmill/src/ManifestDestiny/manifestdestiny/tests/manifest.ini',}] + +The keys displayed here (path, relpath, name, here, and manifest) are reserved keys for ManifestDestiny and any consuming APIs. You can add additional key, value metadata to each test. + + +# Why have test manifests? + +It is desirable to have a unified format for test manifests for testing +[mozilla-central](http://hg.mozilla.org/mozilla-central), etc. + +* It is desirable to be able to selectively enable or disable tests based on platform or other conditions. This should be easy to do. Currently, since many of the harnesses just crawl directories, there is no effective way of disabling a test except for removal from mozilla-central +* It is desriable to do this in a universal way so that enabling and disabling tests as well as other tasks are easily accessible to a wider audience than just those intimately familiar with the specific test framework. +* It is desirable to have other metadata on top of the test. For instance, let's say a test is marked as skipped. It would be nice to give the reason why. + + +Most Mozilla test harnesses work by crawling a directory structure. +While this is straight-forward, manifests offer several practical +advantages:: + +* ability to turn a test off easily: if a test is broken on m-c + currently, the only way to turn it off, generally speaking, is just + removing the test. Often this is undesirable, as if the test should + be dismissed because other people want to land and it can't be + investigated in real time (is it a failure? is the test bad? is no + one around that knows the test?), then backing out a test is at best + problematic. With a manifest, a test may be disabled without + removing it from the tree and a bug filed with the appropriate + reason: + + [test_broken.js] + disabled = https://bugzilla.mozilla.org/show_bug.cgi?id=123456 + +* ability to run different (subsets of) tests on different + platforms. Traditionally, we've done a bit of magic or had the test + know what platform it would or would not run on. With manifests, you + can mark what platforms a test will or will not run on and change + these without changing the test. + + [test_works_on_windows_only.js] + run-if = os == 'win' + +* ability to markup tests with metadata. We have a large, complicated, + and always changing infrastructure. key, value metadata may be used + as an annotation to a test and appropriately curated and mined. For + instance, we could mark certain tests as randomorange with a bug + number, if it were desirable. + +* ability to have sane and well-defined test-runs. You can keep + different manifests for different test runs and ``[include:]`` + (sub)manifests as appropriate to your needs. + + +# Manifest Format + +Manifests are .ini file with the section names denoting the path +relative to the manifest: + + [foo.js] + [bar.js] + [fleem.js] + +The sections are read in order. In addition, tests may include +arbitrary key, value metadata to be used by the harness. You may also +have a `[DEFAULT]` section that will give key, value pairs that will +be inherited by each test unless overridden: + + [DEFAULT] + type = restart + + [lilies.js] + color = white + + [daffodils.js] + color = yellow + type = other + # override type from DEFAULT + + [roses.js] + color = red + +You can also include other manifests: + + [include:subdir/anothermanifest.ini] + +Manifests are included relative to the directory of the manifest with +the `[include:]` directive unless they are absolute paths. + + +# Data + +Manifest Destiny gives tests as a list of dictionaries (in python +terms). + +* path: full path to the test +* relpath: relative path starting from the root manifest location +* name: file name of the test +* here: the parent directory of the manifest +* manifest: the path to the manifest containing the test + +This data corresponds to a one-line manifest: + + [testToolbar/testBackForwardButtons.js] + +If additional key, values were specified, they would be in this dict +as well. + +Outside of the reserved keys, the remaining key, values +are up to convention to use. There is a (currently very minimal) +generic integration layer in ManifestDestiny for use of all harnesses, +`manifestparser.TestManifest`. +For instance, if the 'disabled' key is present, you can get the set of +tests without disabled (various other queries are doable as well). + +Since the system is convention-based, the harnesses may do whatever +they want with the data. They may ignore it completely, they may use +the provided integration layer, or they may provide their own +integration layer. This should allow whatever sort of logic is +desired. For instance, if in yourtestharness you wanted to run only on +mondays for a certain class of tests: + + tests = [] + for test in manifests.tests: + if 'runOnDay' in test: + if calendar.day_name[calendar.weekday(*datetime.datetime.now().timetuple()[:3])].lower() == test['runOnDay'].lower(): + tests.append(test) + else: + tests.append(test) + +To recap: +* the manifests allow you to specify test data +* the parser gives you this data +* you can use it however you want or process it further as you need + +Tests are denoted by sections in an .ini file (see +http://hg.mozilla.org/automation/ManifestDestiny/file/tip/manifestdestiny/tests/mozmill-example.ini). + +Additional manifest files may be included with an `[include:]` directive: + + [include:path-to-additional-file.manifest] + +The path to included files is relative to the current manifest. + +The `[DEFAULT]` section contains variables that all tests inherit from. + +Included files will inherit the top-level variables but may override +in their own `[DEFAULT]` section. + + +# ManifestDestiny Architecture + +There is a two- or three-layered approach to the ManifestDestiny +architecture, depending on your needs: + +1. ManifestParser: this is a generic parser for .ini manifests that +facilitates the `[include:]` logic and the inheritence of +metadata. Despite the internal variable being called `self.tests` +(an oversight), this layer has nothing in particular to do with tests. + +2. TestManifest: this is a harness-agnostic integration layer that is +test-specific. TestManifest faciliates `skip-if` and `run-if` logic. + +3. Optionally, a harness will have an integration layer than inherits +from TestManifest if more harness-specific customization is desired at +the manifest level. + +See the source code at https://github.com/mozilla/mozbase/tree/master/manifestdestiny +and +https://github.com/mozilla/mozbase/blob/master/manifestdestiny/manifestparser.py +in particular. + + +# Using Manifests + +A test harness will normally call `TestManifest.active_tests`: + + def active_tests(self, exists=True, disabled=True, **tags): + +The manifests are passed to the `__init__` or `read` methods with +appropriate arguments. `active_tests` then allows you to select the +tests you want: + +- exists : return only existing tests +- disabled : whether to return disabled tests; if not these will be + filtered out; if True (the default), the `disabled` key of a + test's metadata will be present and will be set to the reason that a + test is disabled +- tags : keys and values to filter on (e.g. `os='linux'`) + +`active_tests` looks for tests with `skip-if` +`run-if`. If the condition is or is not fulfilled, +respectively, the test is marked as disabled. For instance, if you +pass `**dict(os='linux')` as `**tags`, if a test contains a line +`skip-if = os == 'linux'` this test will be disabled, or +`run-if = os = 'win'` in which case the test will also be disabled. It +is up to the harness to pass in tags appropriate to its usage. + + +# Creating Manifests + +ManifestDestiny comes with a console script, `manifestparser create`, that +may be used to create a seed manifest structure from a directory of +files. Run `manifestparser help create` for usage information. + + +# Copying Manifests + +To copy tests and manifests from a source: + + manifestparser [options] copy from_manifest to_directory -tag1 -tag2 --key1=value1 key2=value2 ... + + +# Upating Tests + +To update the tests associated with with a manifest from a source +directory: + + manifestparser [options] update manifest from_directory -tag1 -tag2 --key1=value1 --key2=value2 ... + + +# Usage example + +Here is an example of how to create manifests for a directory tree and +update the tests listed in the manifests from an external source. + +## Creating Manifests + +Let's say you want to make a series of manifests for a given directory structure containing `.js` test files: + + testing/mozmill/tests/firefox/ + testing/mozmill/tests/firefox/testAwesomeBar/ + testing/mozmill/tests/firefox/testPreferences/ + testing/mozmill/tests/firefox/testPrivateBrowsing/ + testing/mozmill/tests/firefox/testSessionStore/ + testing/mozmill/tests/firefox/testTechnicalTools/ + testing/mozmill/tests/firefox/testToolbar/ + testing/mozmill/tests/firefox/restartTests + +You can use `manifestparser create` to do this: + + $ manifestparser help create + Usage: manifestparser.py [options] create directory <...> + + create a manifest from a list of directories + + Options: + -p PATTERN, --pattern=PATTERN + glob pattern for files + -i IGNORE, --ignore=IGNORE + directories to ignore + -w IN_PLACE, --in-place=IN_PLACE + Write .ini files in place; filename to write to + +We only want `.js` files and we want to skip the `restartTests` directory. +We also want to write a manifest per directory, so I use the `--in-place` +option to write the manifests: + + manifestparser create . -i restartTests -p '*.js' -w manifest.ini + +This creates a manifest.ini per directory that we care about with the JS test files: + + testing/mozmill/tests/firefox/manifest.ini + testing/mozmill/tests/firefox/testAwesomeBar/manifest.ini + testing/mozmill/tests/firefox/testPreferences/manifest.ini + testing/mozmill/tests/firefox/testPrivateBrowsing/manifest.ini + testing/mozmill/tests/firefox/testSessionStore/manifest.ini + testing/mozmill/tests/firefox/testTechnicalTools/manifest.ini + testing/mozmill/tests/firefox/testToolbar/manifest.ini + +The top-level `manifest.ini` merely has `[include:]` references to the sub manifests: + + [include:testAwesomeBar/manifest.ini] + [include:testPreferences/manifest.ini] + [include:testPrivateBrowsing/manifest.ini] + [include:testSessionStore/manifest.ini] + [include:testTechnicalTools/manifest.ini] + [include:testToolbar/manifest.ini] + +Each sub-level manifest contains the (`.js`) test files relative to it. + +## Updating the tests from manifests + +You may need to update tests as given in manifests from a different source directory. +`manifestparser update` was made for just this purpose: + + Usage: manifestparser [options] update manifest directory -tag1 -tag2 --key1=value1 --key2=value2 ... + + update the tests as listed in a manifest from a directory + +To update from a directory of tests in `~/mozmill/src/mozmill-tests/firefox/` run: + + manifestparser update manifest.ini ~/mozmill/src/mozmill-tests/firefox/ + + +# Tests + +ManifestDestiny includes a suite of tests: + +https://github.com/mozilla/mozbase/tree/master/manifestdestiny/tests + +`test_manifest.txt` is a doctest that may be helpful in figuring out +how to use the API. Tests are run via `python test.py`. + + +# Bugs + +Please file any bugs or feature requests at + +https://bugzilla.mozilla.org/enter_bug.cgi?product=Testing&component=ManifestParser + +Or contact jhammel @mozilla.org or in #ateam on irc.mozilla.org + + +# CLI + +Run `manifestparser help` for usage information. + +To create a manifest from a set of directories: + + manifestparser [options] create directory <...> [create-options] + +To output a manifest of tests: + + manifestparser [options] write manifest <...> -tag1 -tag2 --key1=value1 --key2=value2 ... + +To copy tests and manifests from a source: + + manifestparser [options] copy from_manifest to_manifest -tag1 -tag2 --key1=value1 key2=value2 ... + +To update the tests associated with with a manifest from a source +directory: + + manifestparser [options] update manifest from_directory -tag1 -tag2 --key1=value1 --key2=value2 ... + + +# Design Considerations + +Contrary to some opinion, manifestparser.py and the associated .ini +format were not magically plucked from the sky but were descended upon +through several design considerations. + +* test manifests should be ordered. While python 2.6 and greater has + a ConfigParser that can use an ordered dictionary, it is a + requirement that we support python 2.4 for the build + testing + environment. To that end, a `read_ini` function was implemented + in manifestparser.py that should be the equivalent of the .ini + dialect used by ConfigParser. + +* the manifest format should be easily human readable/writable. While + there was initially some thought of using JSON, there was pushback + that JSON was not easily editable. An ideal manifest format would + degenerate to a line-separated list of files. While .ini format + requires an additional `[]` per line, and while there have been + complaints about this, hopefully this is good enough. + +* python does not have an in-built YAML parser. Since it was + undesirable for manifestparser.py to have any dependencies, YAML was + dismissed as a format. + +* we could have used a proprietary format but decided against it. + Everyone knows .ini and there are good tools to deal with it. + However, since read_ini is the only function that transforms a + manifest to a list of key, value pairs, while the implications for + changing the format impacts downstream code, doing so should be + programmatically simple. + +* there should be a single file that may easily be + transported. Traditionally, test harnesses have lived in + mozilla-central. This is less true these days and it is increasingly + likely that more tests will not live in mozilla-central going + forward. So `manifestparser.py` should be highly consumable. To + this end, it is a single file, as appropriate to mozilla-central, + which is also a working python package deployed to PyPI for easy + installation. + + +# Developing ManifestDestiny + +ManifestDestiny is developed and maintained by Mozilla's +[Automation and Testing Team](https://wiki.mozilla.org/Auto-tools). +The project page is located at +https://wiki.mozilla.org/Auto-tools/Projects/ManifestDestiny . + + +# Historical Reference + +Date-ordered list of links about how manifests came to be where they are today:: + +* https://wiki.mozilla.org/Auto-tools/Projects/UniversalManifest +* http://alice.nodelman.net/blog/post/2010/05/ +* http://alice.nodelman.net/blog/post/universal-manifest-for-unit-tests-a-proposal/ +* https://elvis314.wordpress.com/2010/07/05/improving-personal-hygiene-by-adjusting-mochitests/ +* https://elvis314.wordpress.com/2010/07/27/types-of-data-we-care-about-in-a-manifest/ +* https://bugzilla.mozilla.org/show_bug.cgi?id=585106 +* http://elvis314.wordpress.com/2011/05/20/converting-xpcshell-from-listing-directories-to-a-manifest/ +* https://bugzilla.mozilla.org/show_bug.cgi?id=616999 +* https://wiki.mozilla.org/Auto-tools/Projects/ManifestDestiny +* https://developer.mozilla.org/en/Writing_xpcshell-based_unit_tests#Adding_your_tests_to_the_xpcshell_manifest diff --git a/testing/mozbase/manifestdestiny/manifestparser/manifestparser.py b/testing/mozbase/manifestdestiny/manifestparser/manifestparser.py index 7afc3c4d997..5d2613c9e14 100755 --- a/testing/mozbase/manifestdestiny/manifestparser/manifestparser.py +++ b/testing/mozbase/manifestdestiny/manifestparser/manifestparser.py @@ -19,7 +19,30 @@ import sys from fnmatch import fnmatch from optparse import OptionParser -relpath = os.path.relpath +# we need relpath, but it is introduced in python 2.6 +# http://docs.python.org/library/os.path.html +try: + relpath = os.path.relpath +except AttributeError: + def relpath(path, start): + """ + Return a relative version of a path + from /usr/lib/python2.6/posixpath.py + """ + + if not path: + raise ValueError("no path specified") + + start_list = os.path.abspath(start).split(os.path.sep) + path_list = os.path.abspath(path).split(os.path.sep) + + # Work out how much of the filepath is shared by start and path. + i = len(os.path.commonprefix([start_list, path_list])) + + rel_list = [os.path.pardir] * (len(start_list)-i) + path_list[i:] + if not rel_list: + return os.curdir + return os.path.join(*rel_list) # expr.py # from: diff --git a/testing/mozbase/manifestdestiny/setup.py b/testing/mozbase/manifestdestiny/setup.py index 8e5a2396edb..e58039d5127 100644 --- a/testing/mozbase/manifestdestiny/setup.py +++ b/testing/mozbase/manifestdestiny/setup.py @@ -3,19 +3,28 @@ # You can obtain one at http://mozilla.org/MPL/2.0/. from setuptools import setup +import sys +import os + +here = os.path.dirname(os.path.abspath(__file__)) +try: + filename = os.path.join(here, 'README.md') + description = file(filename).read() +except: + description = '' PACKAGE_NAME = "ManifestDestiny" -PACKAGE_VERSION = '0.5.7' +PACKAGE_VERSION = '0.5.6' setup(name=PACKAGE_NAME, version=PACKAGE_VERSION, - description="Library to create and manage test manifests", - long_description="see http://mozbase.readthedocs.org/", + description="Universal manifests for Mozilla test harnesses", + long_description=description, classifiers=[], # Get strings from http://pypi.python.org/pypi?%3Aaction=list_classifiers keywords='mozilla manifests', author='Mozilla Automation and Testing Team', author_email='tools@lists.mozilla.org', - url='https://wiki.mozilla.org/Auto-tools/Projects/Mozbase', + url='https://wiki.mozilla.org/Auto-tools/Projects/MozBase', license='MPL', zip_safe=False, packages=['manifestparser'], diff --git a/testing/mozbase/manifestdestiny/tests/comment-example.ini b/testing/mozbase/manifestdestiny/tests/comment-example.ini deleted file mode 100644 index 030ceffdb3b..00000000000 --- a/testing/mozbase/manifestdestiny/tests/comment-example.ini +++ /dev/null @@ -1,11 +0,0 @@ -; See https://bugzilla.mozilla.org/show_bug.cgi?id=813674 - -[test_0180_fileInUse_xp_win_complete.js] -[test_0181_fileInUse_xp_win_partial.js] -[test_0182_rmrfdirFileInUse_xp_win_complete.js] -[test_0183_rmrfdirFileInUse_xp_win_partial.js] -[test_0184_fileInUse_xp_win_complete.js] -[test_0185_fileInUse_xp_win_partial.js] -[test_0186_rmrfdirFileInUse_xp_win_complete.js] -[test_0187_rmrfdirFileInUse_xp_win_partial.js] -; [test_0202_app_launch_apply_update_dirlocked.js] # Test disabled, bug 757632 \ No newline at end of file diff --git a/testing/mozbase/manifestdestiny/tests/manifest.ini b/testing/mozbase/manifestdestiny/tests/manifest.ini index 6dad6e95935..b06b951ecc5 100644 --- a/testing/mozbase/manifestdestiny/tests/manifest.ini +++ b/testing/mozbase/manifestdestiny/tests/manifest.ini @@ -2,4 +2,3 @@ [test_expressionparser.py] [test_manifestparser.py] [test_testmanifest.py] -[test_read_ini.py] diff --git a/testing/mozbase/manifestdestiny/tests/test_manifestparser.py b/testing/mozbase/manifestdestiny/tests/test_manifestparser.py index ef58590e05e..cb87fff1dfb 100755 --- a/testing/mozbase/manifestdestiny/tests/test_manifestparser.py +++ b/testing/mozbase/manifestdestiny/tests/test_manifestparser.py @@ -215,16 +215,5 @@ class TestManifestparser(unittest.TestCase): os.path.join(here, 'fleem')) - def test_comments(self): - """ - ensure comments work, see - https://bugzilla.mozilla.org/show_bug.cgi?id=813674 - """ - comment_example = os.path.join(here, 'comment-example.ini') - manifest = ManifestParser(manifests=(comment_example,)) - self.assertEqual(len(manifest.tests), 8) - names = [i['name'] for i in manifest.tests] - self.assertFalse('test_0202_app_launch_apply_update_dirlocked.js' in names) - if __name__ == '__main__': unittest.main() diff --git a/testing/mozbase/manifestdestiny/tests/test_read_ini.py b/testing/mozbase/manifestdestiny/tests/test_read_ini.py deleted file mode 100755 index fb018fa8613..00000000000 --- a/testing/mozbase/manifestdestiny/tests/test_read_ini.py +++ /dev/null @@ -1,69 +0,0 @@ -#!/usr/bin/env python - -""" -test .ini parsing - -ensure our .ini parser is doing what we want; to be deprecated for -python's standard ConfigParser when 2.7 is reality so OrderedDict -is the default: - -http://docs.python.org/2/library/configparser.html -""" - -import unittest -from manifestparser import read_ini -from ConfigParser import ConfigParser -from StringIO import StringIO - -class IniParserTest(unittest.TestCase): - - def test_inline_comments(self): - """ - We have no inline comments; so we're testing to ensure we don't: - https://bugzilla.mozilla.org/show_bug.cgi?id=855288 - """ - - # test '#' inline comments (really, the lack thereof) - string = """[test_felinicity.py] -kittens = true # This test requires kittens -""" - buffer = StringIO() - buffer.write(string) - buffer.seek(0) - result = read_ini(buffer)[0][1]['kittens'] - self.assertEqual(result, "true # This test requires kittens") - - # compare this to ConfigParser - # python 2.7 ConfigParser does not support '#' as an - # inline comment delimeter (for "backwards compatability"): - # http://docs.python.org/2/library/configparser.html - buffer.seek(0) - parser = ConfigParser() - parser.readfp(buffer) - control = parser.get('test_felinicity.py', 'kittens') - self.assertEqual(result, control) - - # test ';' inline comments (really, the lack thereof) - string = string.replace('#', ';') - buffer = StringIO() - buffer.write(string) - buffer.seek(0) - result = read_ini(buffer)[0][1]['kittens'] - self.assertEqual(result, "true ; This test requires kittens") - - # compare this to ConfigParser - # python 2.7 ConfigParser *does* support ';' as an - # inline comment delimeter (ibid). - # Python 3.x configparser, OTOH, does not support - # inline-comments by default. It does support their specification, - # though they are weakly discouraged: - # http://docs.python.org/dev/library/configparser.html - buffer.seek(0) - parser = ConfigParser() - parser.readfp(buffer) - control = parser.get('test_felinicity.py', 'kittens') - self.assertNotEqual(result, control) - - -if __name__ == '__main__': - unittest.main() diff --git a/testing/mozbase/manifestdestiny/tests/test_testmanifest.py b/testing/mozbase/manifestdestiny/tests/test_testmanifest.py index 898706f4d04..1a93df81ee0 100644 --- a/testing/mozbase/manifestdestiny/tests/test_testmanifest.py +++ b/testing/mozbase/manifestdestiny/tests/test_testmanifest.py @@ -29,17 +29,6 @@ class TestTestManifest(unittest.TestCase): last_test = manifest.active_tests(exists=False, toolkit='cocoa')[-1] self.assertEqual(last_test['expected'], 'fail') - def test_comments(self): - """ - ensure comments work, see - https://bugzilla.mozilla.org/show_bug.cgi?id=813674 - """ - comment_example = os.path.join(here, 'comment-example.ini') - manifest = TestManifest(manifests=(comment_example,)) - self.assertEqual(len(manifest.tests), 8) - names = [i['name'] for i in manifest.tests] - self.assertFalse('test_0202_app_launch_apply_update_dirlocked.js' in names) - if __name__ == '__main__': unittest.main() diff --git a/testing/mozbase/mozb2g/mozb2g/b2gmixin.py b/testing/mozbase/mozb2g/mozb2g/b2gmixin.py index e0f5c974f40..b17a320e91f 100644 --- a/testing/mozbase/mozb2g/mozb2g/b2gmixin.py +++ b/testing/mozbase/mozb2g/mozb2g/b2gmixin.py @@ -19,30 +19,18 @@ class B2GMixin(object): userJS = "/data/local/user.js" marionette = None - def __init__(self, host=None, marionetteHost=None, marionettePort=2828, - **kwargs): - - # (allowing marionneteHost to be specified seems a bit - # counter-intuitive since we normally get it below from the ip - # address, however we currently need it to be able to connect - # via adb port forwarding and localhost) - if marionetteHost: - self.marionetteHost = marionetteHost - elif host: - self.marionetteHost = host - self.marionettePort = marionettePort + def __init__(self, host=None, marionette_port=2828, **kwargs): + self.marionetteHost = host + self.marionettePort = marionette_port def cleanup(self): - """ - If a user profile was setup on the device, restore it to the original. - """ if self.profileDir: self.restoreProfile() def waitForPort(self, timeout): - """Waits for the marionette server to respond, until the timeout specified. - - :param timeout: Timeout parameter in seconds. + """ + Wait for the marionette server to respond. + Timeout parameter is in seconds """ print "waiting for port" starttime = datetime.datetime.now() @@ -62,11 +50,11 @@ class B2GMixin(object): time.sleep(1) raise DMError("Could not communicate with Marionette port") - def setupMarionette(self, scriptTimeout=60000): + def setupMarionette(self): """ - Starts a marionette session. - If no host was given at init, the ip of the device will be retrieved - and networking will be established. + Start a marionette session. + If no host is given, then this will get the ip + of the device, and set up networking if needed. """ if not self.marionetteHost: self.setupDHCP() @@ -77,11 +65,9 @@ class B2GMixin(object): self.waitForPort(30) self.marionette.start_session() - self.marionette.set_script_timeout(scriptTimeout) - def restartB2G(self): """ - Restarts the b2g process on the device. + Restarts the b2g process on the device """ #restart b2g so we start with a clean slate if self.marionette and self.marionette.session: @@ -97,16 +83,16 @@ class B2GMixin(object): self.shellCheckOutput(['start', 'b2g']) def setupProfile(self, prefs=None): - """Sets up the user profile on the device. - - :param prefs: String of user_prefs to add to the profile. Defaults to a standard b2g testing profile. """ - # currently we have no custom prefs to set (when bug 800138 is fixed, - # we will probably want to enable marionette on an external ip by - # default) + Sets up the user profile on the device, + The 'prefs' is a string of user_prefs to add to the profile. + If it is not set, it will default to a standard b2g testing profile. + """ if not prefs: - prefs = "" - + prefs = """ +user_pref("power.screen.timeout", 999999); +user_pref("devtools.debugger.force-local", false); + """ #remove previous user.js if there is one if not self.profileDir: self.profileDir = tempfile.mkdtemp() @@ -115,7 +101,7 @@ class B2GMixin(object): os.remove(our_userJS) #copy profile try: - self.getFile(self.userJS, our_userJS) + output = self.getFile(self.userJS, our_userJS) except subprocess.CalledProcessError: pass #if we successfully copied the profile, make a backup of the file @@ -123,39 +109,31 @@ class B2GMixin(object): self.shellCheckOutput(['dd', 'if=%s' % self.userJS, 'of=%s.orig' % self.userJS]) with open(our_userJS, 'a') as user_file: user_file.write("%s" % prefs) - self.pushFile(our_userJS, self.userJS) self.restartB2G() self.setupMarionette() - def setupDHCP(self, interfaces=['eth0', 'wlan0']): - """Sets up networking. - - :param interfaces: Network connection types to try. Defaults to eth0 and wlan0. + def setupDHCP(self, conn_type='eth0'): """ - all_interfaces = [line.split()[0] for line in \ - self.shellCheckOutput(['netcfg']).splitlines()[1:]] - interfaces_to_try = filter(lambda i: i in interfaces, all_interfaces) + Sets up networking. + If conn_type is not set, it will assume eth0. + """ tries = 5 - print "Setting up DHCP..." while tries > 0: print "attempts left: %d" % tries try: - for interface in interfaces_to_try: - self.shellCheckOutput(['netcfg', interface, 'dhcp'], - timeout=10) - if self.getIP(interfaces=[interface]): - return + self.shellCheckOutput(['netcfg', conn_type, 'dhcp'], timeout=10) + if self.getIP(): + return except DMError: pass - time.sleep(1) - tries -= 1 + tries = tries - 1 raise DMError("Could not set up network connection") def restoreProfile(self): """ - Restores the original user profile on the device. + Restores the original profile """ if not self.profileDir: raise DMError("There is no profile to restore") @@ -169,8 +147,6 @@ class B2GMixin(object): def getAppInfo(self): """ Returns the appinfo, with an additional "date" key. - - :rtype: dictionary """ if not self.marionette or not self.marionette.session: self.setupMarionette() diff --git a/testing/mozbase/mozb2g/setup.py b/testing/mozbase/mozb2g/setup.py index bf646199a0b..80155312cb5 100644 --- a/testing/mozbase/mozb2g/setup.py +++ b/testing/mozbase/mozb2g/setup.py @@ -2,21 +2,29 @@ # License, v. 2.0. If a copy of the MPL was not distributed with this file, # You can obtain one at http://mozilla.org/MPL/2.0/. +import os from setuptools import setup -PACKAGE_VERSION = '0.3' +PACKAGE_VERSION = '0.1' -deps = ['mozdevice >= 0.16', 'marionette_client >= 0.5.2'] +# take description from README +here = os.path.dirname(os.path.abspath(__file__)) +try: + description = file(os.path.join(here, 'README.md')).read() +except (OSError, IOError): + description = '' + +deps = ['mozdevice', 'marionette_client'] setup(name='mozb2g', version=PACKAGE_VERSION, description="B2G specific code for device automation", - long_description="see http://mozbase.readthedocs.org/", + long_description=description, classifiers=[], # Get strings from http://pypi.python.org/pypi?%3Aaction=list_classifiers keywords='', author='Mozilla Automation and Testing Team', author_email='tools@lists.mozilla.org', - url='https://wiki.mozilla.org/Auto-tools/Projects/Mozbase', + url='https://wiki.mozilla.org/Auto-tools/Projects/MozBase', license='MPL', packages=['mozb2g'], include_package_data=True, diff --git a/testing/mozbase/mozcrash/setup.py b/testing/mozbase/mozcrash/setup.py index f40c4b3f657..0b20106d0e5 100644 --- a/testing/mozbase/mozcrash/setup.py +++ b/testing/mozbase/mozcrash/setup.py @@ -4,7 +4,7 @@ from setuptools import setup -PACKAGE_VERSION = '0.8' +PACKAGE_VERSION = '0.6' # dependencies deps = ['mozfile >= 0.3', @@ -18,7 +18,7 @@ setup(name='mozcrash', keywords='mozilla', author='Mozilla Automation and Tools team', author_email='tools@lists.mozilla.org', - url='https://wiki.mozilla.org/Auto-tools/Projects/Mozbase', + url='https://wiki.mozilla.org/Auto-tools/Projects/MozBase', license='MPL', packages=['mozcrash'], include_package_data=True, diff --git a/testing/mozbase/mozcrash/tests/test.py b/testing/mozbase/mozcrash/tests/test.py index d14ebb938d9..40d9c7e8f8b 100644 --- a/testing/mozbase/mozcrash/tests/test.py +++ b/testing/mozbase/mozcrash/tests/test.py @@ -8,7 +8,7 @@ import os, unittest, subprocess, tempfile, shutil, urlparse, zipfile, StringIO import mozcrash, mozlog, mozhttpd # Make logs go away -log = mozlog.getLogger("mozcrash", handler=mozlog.FileHandler(os.devnull)) +log = mozlog.getLogger("mozcrash", os.devnull) def popen_factory(stdouts): """ diff --git a/testing/mozbase/mozdevice/mozdevice/__init__.py b/testing/mozbase/mozdevice/mozdevice/__init__.py index ea11ab985b7..5753225413d 100644 --- a/testing/mozbase/mozdevice/mozdevice/__init__.py +++ b/testing/mozbase/mozdevice/mozdevice/__init__.py @@ -2,7 +2,7 @@ # License, v. 2.0. If a copy of the MPL was not distributed with this file, # You can obtain one at http://mozilla.org/MPL/2.0/. -from devicemanager import DeviceManager, DMError, ZeroconfListener +from devicemanager import DeviceManager, DMError from devicemanagerADB import DeviceManagerADB from devicemanagerSUT import DeviceManagerSUT from droid import DroidADB, DroidSUT, DroidConnectByHWID diff --git a/testing/mozbase/mozdevice/mozdevice/devicemanager.py b/testing/mozbase/mozdevice/mozdevice/devicemanager.py index 6e38f386f05..b6be6476669 100644 --- a/testing/mozbase/mozdevice/mozdevice/devicemanager.py +++ b/testing/mozbase/mozdevice/mozdevice/devicemanager.py @@ -6,7 +6,6 @@ import hashlib import mozlog import socket import os -import posixpath import re import struct import StringIO @@ -239,14 +238,15 @@ class DeviceManager(object): WARNING: does not create last part of the path. For example, if asked to create `/mnt/sdcard/foo/bar/baz`, it will only create `/mnt/sdcard/foo/bar` """ - filename = posixpath.normpath(filename) - containing = posixpath.dirname(filename) - if not self.dirExists(containing): + dirParts = filename.rsplit('/', 1) + if not self.dirExists(dirParts[0]): parts = filename.split('/') - name = "/" - for part in parts[:-1]: + name = "" + for part in parts: + if part is parts[-1]: + break if part != "": - name = posixpath.join(name, part) + name += '/' + part self.mkDir(name) # mkDir will check previous existence @abstractmethod @@ -258,8 +258,7 @@ class DeviceManager(object): @abstractmethod def fileExists(self, filepath): """ - Return whether filepath exists on the device file system, - regardless of file type. + Return whether filepath exists and is a file on the device file system. """ @abstractmethod diff --git a/testing/mozbase/mozdevice/mozdevice/devicemanagerADB.py b/testing/mozbase/mozdevice/mozdevice/devicemanagerADB.py index 94e12c4de93..afcf8ad8d45 100644 --- a/testing/mozbase/mozdevice/mozdevice/devicemanagerADB.py +++ b/testing/mozbase/mozdevice/mozdevice/devicemanagerADB.py @@ -156,35 +156,6 @@ class DeviceManagerADB(DeviceManager): return None - def forward(self, local, remote): - """ - Forward socket connections. - - Forward specs are one of: - tcp: - localabstract: - localreserved: - localfilesystem: - dev: - jdwp: (remote only) - """ - return self._checkCmd(['forward', local, remote]) - - def remount(self): - "Remounts the /system partition on the device read-write." - return self._checkCmd(['remount']) - - def devices(self): - "Return a list of connected devices as (serial, status) tuples." - proc = self._runCmd(['devices']) - proc.stdout.readline() # ignore first line of output - devices = [] - for line in iter(proc.stdout.readline, ''): - result = re.match('(.*?)\t(.*)', line) - if result: - devices.append((result.group(1), result.group(2))) - return devices - def _connectRemoteADB(self): self._checkCmd(["connect", self.host + ":" + str(self.port)]) @@ -430,7 +401,7 @@ class DeviceManagerADB(DeviceManager): elif offset is not None: f.seek(offset) ret = f.read() - else: + else: ret = f.read() f.close() @@ -521,10 +492,12 @@ class DeviceManagerADB(DeviceManager): raise DMError("Failed to get application root for: %s" % packageName) def reboot(self, wait = False, **kwargs): - self._checkCmd(["reboot"]) - if not wait: + self._runCmd(["reboot"]) + if (not wait): return - self._checkCmd(["wait-for-device", "shell", "ls", "/sbin"]) + countdown = 40 + while (countdown > 0): + self._checkCmd(["wait-for-device", "shell", "ls", "/sbin"]) def updateApp(self, appBundlePath, **kwargs): return self._runCmd(["install", "-r", appBundlePath]).stdout.read() @@ -533,7 +506,7 @@ class DeviceManagerADB(DeviceManager): timestr = self._runCmd(["shell", "date", "+%s"]).stdout.read().strip() if (not timestr or not timestr.isdigit()): raise DMError("Unable to get current time using date (got: '%s')" % timestr) - return int(timestr)*1000 + return str(int(timestr)*1000) def getInfo(self, directive=None): ret = {} diff --git a/testing/mozbase/mozdevice/mozdevice/devicemanagerSUT.py b/testing/mozbase/mozdevice/mozdevice/devicemanagerSUT.py index 0e03115fbc9..acf4c8262c6 100644 --- a/testing/mozbase/mozdevice/mozdevice/devicemanagerSUT.py +++ b/testing/mozbase/mozdevice/mozdevice/devicemanagerSUT.py @@ -368,13 +368,15 @@ class DeviceManagerSUT(DeviceManager): existentDirectories = [] for root, dirs, files in os.walk(localDir, followlinks=True): - _, subpath = root.split(localDir) - subpath = subpath.lstrip('/') - remoteRoot = posixpath.join(remoteDir, subpath) + parts = root.split(localDir) for f in files: - remoteName = posixpath.join(remoteRoot, f) + remoteRoot = remoteDir + '/' + parts[1] + if (remoteRoot.endswith('/')): + remoteName = remoteRoot + f + else: + remoteName = remoteRoot + '/' + f - if subpath == "": + if (parts[1] == ""): remoteRoot = remoteDir parent = os.path.dirname(remoteName) @@ -395,24 +397,19 @@ class DeviceManagerSUT(DeviceManager): def fileExists(self, filepath): # Because we always have / style paths we make this a lot easier with some # assumptions - filepath = posixpath.normpath(filepath) - # / should always exist but we can use this to check for things like - # having access to the filesystem - if filepath == '/': - return self.dirExists(filepath) - (containingpath, filename) = posixpath.split(filepath) - return filename in self.listFiles(containingpath) + s = filepath.split('/') + containingpath = '/'.join(s[:-1]) + return s[-1] in self.listFiles(containingpath) def listFiles(self, rootdir): - rootdir = posixpath.normpath(rootdir) - if not self.dirExists(rootdir): + rootdir = rootdir.rstrip('/') + if (self.dirExists(rootdir) == False): return [] data = self._runCmds([{ 'cmd': 'cd ' + rootdir }, { 'cmd': 'ls' }]) files = filter(lambda x: x, data.splitlines()) if len(files) == 1 and files[0] == '': - # special case on the agent: empty directories return just the - # string "" + # special case on the agent: empty directories return just the string "" return [] return files @@ -877,7 +874,7 @@ class DeviceManagerSUT(DeviceManager): self._logger.debug("updateApp: got status back: %s" % status) def getCurrentTime(self): - return int(self._runCmds([{ 'cmd': 'clok' }]).strip()) + return self._runCmds([{ 'cmd': 'clok' }]).strip() def _getCallbackIpAndPort(self, aIp, aPort): """ diff --git a/testing/mozbase/mozdevice/setup.py b/testing/mozbase/mozdevice/setup.py index 4d9386bc036..4b7ada51e81 100644 --- a/testing/mozbase/mozdevice/setup.py +++ b/testing/mozbase/mozdevice/setup.py @@ -4,7 +4,7 @@ from setuptools import setup -PACKAGE_VERSION = '0.28' +PACKAGE_VERSION = '0.27' setup(name='mozdevice', version=PACKAGE_VERSION, diff --git a/testing/mozbase/mozdevice/sut_tests/test_fileExists.py b/testing/mozbase/mozdevice/sut_tests/test_fileExists.py deleted file mode 100644 index d888306a255..00000000000 --- a/testing/mozbase/mozdevice/sut_tests/test_fileExists.py +++ /dev/null @@ -1,37 +0,0 @@ -# This Source Code Form is subject to the terms of the Mozilla Public -# License, v. 2.0. If a copy of the MPL was not distributed with this -# file, You can obtain one at http://mozilla.org/MPL/2.0/. - -import tempfile -import posixpath - -from dmunit import DeviceManagerTestCase - -class FileExistsTestCase(DeviceManagerTestCase): - """This tests the "fileExists" command. - """ - - def testOnRoot(self): - self.assertTrue(self.dm.fileExists('/')) - - def testOnNonexistent(self): - self.assertFalse(self.dm.fileExists('/doesNotExist')) - - def testOnRegularFile(self): - remote_path = posixpath.join(self.dm.getDeviceRoot(), 'testFile') - self.assertFalse(self.dm.fileExists(remote_path)) - with tempfile.NamedTemporaryFile() as f: - self.dm.pushFile(f.name, remote_path) - self.assertTrue(self.dm.fileExists(remote_path)) - self.dm.removeFile(remote_path) - - def testOnDirectory(self): - remote_path = posixpath.join(self.dm.getDeviceRoot(), 'testDir') - remote_path_file = posixpath.join(remote_path, 'testFile') - self.assertFalse(self.dm.fileExists(remote_path)) - with tempfile.NamedTemporaryFile() as f: - self.dm.pushFile(f.name, remote_path_file) - self.assertTrue(self.dm.fileExists(remote_path)) - self.dm.removeFile(remote_path_file) - self.dm.removeDir(remote_path) - diff --git a/testing/mozbase/mozdevice/tests/manifest.ini b/testing/mozbase/mozdevice/tests/manifest.ini index 8e67f66a433..d9682256b9c 100644 --- a/testing/mozbase/mozdevice/tests/manifest.ini +++ b/testing/mozbase/mozdevice/tests/manifest.ini @@ -1,21 +1,9 @@ [DEFAULT] skip-if = os == 'win' -[sut_app.py] [sut_basic.py] -[sut_chmod.py] -[sut_fileExists.py] -[sut_fileMethods.py] -[sut_info.py] -[sut_ip.py] -[sut_kill.py] -[sut_list.py] -[sut_logcat.py] [sut_mkdir.py] -[sut_ps.py] [sut_push.py] [sut_pull.py] -[sut_remove.py] -[sut_time.py] -[sut_unpackfile.py] -[droidsut_launch.py] +[sut_ps.py] +[droidsut_launch.py] \ No newline at end of file diff --git a/testing/mozbase/mozdevice/tests/sut_app.py b/testing/mozbase/mozdevice/tests/sut_app.py deleted file mode 100644 index a28f753d0f9..00000000000 --- a/testing/mozbase/mozdevice/tests/sut_app.py +++ /dev/null @@ -1,20 +0,0 @@ -#/usr/bin/env python -import mozdevice -import mozlog -import unittest -from sut import MockAgent - - -class TestApp(unittest.TestCase): - - def test_getAppRoot(self): - command = [("getapproot org.mozilla.firefox", - "/data/data/org.mozilla.firefox")] - - m = MockAgent(self, commands=command) - d = mozdevice.DroidSUT("127.0.0.1", port=m.port, logLevel=mozlog.DEBUG) - - self.assertEqual(command[0][1], d.getAppRoot('org.mozilla.firefox')) - -if __name__ == '__main__': - unittest.main() diff --git a/testing/mozbase/mozdevice/tests/sut_chmod.py b/testing/mozbase/mozdevice/tests/sut_chmod.py deleted file mode 100644 index d0d6694d370..00000000000 --- a/testing/mozbase/mozdevice/tests/sut_chmod.py +++ /dev/null @@ -1,21 +0,0 @@ -#/usr/bin/env python -import mozdevice -import mozlog -import unittest -from sut import MockAgent - - -class TestChmod(unittest.TestCase): - - def test_chmod(self): - - command = [('chmod /mnt/sdcard/test', 'Changing permissions for /storage/emulated/legacy/Test\n' - ' \n' - 'chmod /storage/emulated/legacy/Test ok\n')] - m = MockAgent(self, commands=command) - d = mozdevice.DroidSUT('127.0.0.1', port=m.port, logLevel=mozlog.DEBUG) - - self.assertEqual(None, d.chmodDir('/mnt/sdcard/test')) - -if __name__ == '__main__': - unittest.main() diff --git a/testing/mozbase/mozdevice/tests/sut_fileExists.py b/testing/mozbase/mozdevice/tests/sut_fileExists.py deleted file mode 100644 index aa7157b394c..00000000000 --- a/testing/mozbase/mozdevice/tests/sut_fileExists.py +++ /dev/null @@ -1,29 +0,0 @@ -from sut import MockAgent -import mozdevice -import unittest - -class FileExistsTest(unittest.TestCase): - - commands = [('isdir /', 'TRUE'), - ('cd /', ''), - ('ls', 'init')] - - def test_onRoot(self): - root_commands = [('isdir /', 'TRUE')] - a = MockAgent(self, commands=root_commands) - d = mozdevice.DroidSUT("127.0.0.1", port=a.port) - self.assertTrue(d.fileExists('/')) - - def test_onNonexistent(self): - a = MockAgent(self, commands=self.commands) - d = mozdevice.DroidSUT("127.0.0.1", port=a.port) - self.assertFalse(d.fileExists('/doesNotExist')) - - def test_onRegularFile(self): - a = MockAgent(self, commands=self.commands) - d = mozdevice.DroidSUT("127.0.0.1", port=a.port) - self.assertTrue(d.fileExists('/init')) - -if __name__ == '__main__': - unittest.main() - diff --git a/testing/mozbase/mozdevice/tests/sut_fileMethods.py b/testing/mozbase/mozdevice/tests/sut_fileMethods.py deleted file mode 100644 index cef24d91fc6..00000000000 --- a/testing/mozbase/mozdevice/tests/sut_fileMethods.py +++ /dev/null @@ -1,74 +0,0 @@ -#!/usr/bin/env python - -import hashlib -import mock -import mozdevice -import mozlog -import shutil -import tempfile -import os -import unittest -from sut import MockAgent - - -class TestFileMethods(unittest.TestCase): - """ Class to test misc file methods """ - - content = "What is the answer to the life, universe and everything? 42" - h = hashlib.md5() - h.update(content) - temp_hash = h.hexdigest() - - def test_validateFile(self): - - with tempfile.NamedTemporaryFile() as f: - f.write(self.content) - f.flush() - - # Test Valid Hashes - commands_valid = [("hash /sdcard/test/file", self.temp_hash)] - - m = MockAgent(self, commands=commands_valid) - d = mozdevice.DroidSUT("127.0.0.1", port=m.port, logLevel=mozlog.DEBUG) - self.assertTrue(d.validateFile('/sdcard/test/file', f.name)) - - # Test invalid hashes - commands_invalid = [("hash /sdcard/test/file", "0this0hash0is0completely0invalid")] - - m = MockAgent(self, commands=commands_invalid) - d = mozdevice.DroidSUT("127.0.0.1", port=m.port, logLevel=mozlog.DEBUG) - self.assertFalse(d.validateFile('/sdcard/test/file', f.name)) - - def test_getFile(self): - - fname = "/mnt/sdcard/file" - commands = [("pull %s" % fname, "%s,%s\n%s" % (fname, len(self.content), self.content)), - ("hash %s" % fname, self.temp_hash)] - - with tempfile.NamedTemporaryFile() as f: - m = MockAgent(self, commands=commands) - d = mozdevice.DroidSUT("127.0.0.1", port=m.port, logLevel=mozlog.DEBUG) - # No error means success - self.assertEqual(None, d.getFile(fname, f.name)) - - def test_getDirectory(self): - - fname = "/mnt/sdcard/file" - commands = [("isdir /mnt/sdcard", "TRUE"), - ("isdir /mnt/sdcard", "TRUE"), - ("cd /mnt/sdcard", ""), - ("ls", "file"), - ("isdir %s" % fname, "FALSE"), - ("pull %s" % fname, "%s,%s\n%s" % (fname, len(self.content), self.content)), - ("hash %s" % fname, self.temp_hash)] - - tmpdir = tempfile.mkdtemp() - m = MockAgent(self, commands=commands) - d = mozdevice.DroidSUT("127.0.0.1", port=m.port, logLevel=mozlog.DEBUG) - self.assertEqual(None, d.getDirectory("/mnt/sdcard", tmpdir)) - - # Cleanup - shutil.rmtree(tmpdir) - -if __name__ == '__main__': - unittest.main() diff --git a/testing/mozbase/mozdevice/tests/sut_info.py b/testing/mozbase/mozdevice/tests/sut_info.py deleted file mode 100644 index bb13be9721a..00000000000 --- a/testing/mozbase/mozdevice/tests/sut_info.py +++ /dev/null @@ -1,49 +0,0 @@ -#/usr/bin/env python -import mozdevice -import mozlog -import re -import unittest -from sut import MockAgent - - -class TestGetInfo(unittest.TestCase): - - commands = {'os': ('info os', 'JDQ39'), - 'id': ('info id', '11:22:33:44:55:66'), - 'uptime': ('info uptime', '0 days 0 hours 7 minutes 0 seconds 0 ms'), - 'uptimemillis': ('info uptimemillis', '666'), - 'systime': ('info systime', '2013/04/2 12:42:00:007'), - 'screen': ('info screen', 'X:768 Y:1184'), - 'rotation': ('info rotation', 'ROTATION:0'), - 'memory': ('info memory', 'PA:1351032832, FREE: 878645248'), - 'process': ('info process', '1000 527 system\n' - '10091 3443 org.mozilla.firefox\n' - '10112 3137 com.mozilla.SUTAgentAndroid\n' - '10035 807 com.android.launcher'), - 'disk': ('info disk', '/data: 6084923392 total, 980922368 available\n' - '/system: 867999744 total, 332333056 available\n' - '/mnt/sdcard: 6084923392 total, 980922368 available'), - 'power': ('info power', 'Power status:\n' - ' AC power OFFLINE\n' - ' Battery charge LOW DISCHARGING\n' - ' Remaining charge: 20%\n' - ' Battery Temperature: 25.2 (c)'), - 'sutuserinfo': ('info sutuserinfo', 'User Serial:0'), - 'temperature': ('info temperature', 'Temperature: unknown') - } - - def test_getInfo(self): - - for directive in self.commands.keys(): - m = MockAgent(self, commands=[self.commands[directive]]) - d = mozdevice.DroidSUT('127.0.0.1', port=m.port, logLevel=mozlog.DEBUG) - - expected = re.sub(r'\ +', ' ', self.commands[directive][1]).split('\n') - # Account for slightly different return format for 'process' - if directive is 'process': - expected = [[x] for x in expected] - - self.assertEqual(d.getInfo(directive=directive)[directive], expected) - -if __name__ == '__main__': - unittest.main() diff --git a/testing/mozbase/mozdevice/tests/sut_ip.py b/testing/mozbase/mozdevice/tests/sut_ip.py deleted file mode 100644 index d8dafb953b7..00000000000 --- a/testing/mozbase/mozdevice/tests/sut_ip.py +++ /dev/null @@ -1,37 +0,0 @@ -#/usr/bin/env python -import mozdevice -import mozlog -import unittest -from sut import MockAgent - - -class TestGetIP(unittest.TestCase): - """ class to test IP methods """ - - commands = [('exec ifconfig eth0', 'eth0: ip 192.168.0.1 ' - 'mask 255.255.255.0 flags [up broadcast running multicast]\n' - 'return code [0]'), - ('exec ifconfig wlan0', 'wlan0: ip 10.1.39.126\n' - 'mask 255.255.0.0 flags [up broadcast running multicast]\n' - 'return code [0]'), - ('exec ifconfig fake0', '##AGENT-WARNING## [ifconfig] ' - 'command with arg(s) = [fake0] is currently not implemented.') - ] - - def test_getIP_eth0(self): - m = MockAgent(self, commands=[self.commands[0]]) - d = mozdevice.DroidSUT("127.0.0.1", port=m.port, logLevel=mozlog.DEBUG) - self.assertEqual('192.168.0.1', d.getIP(interfaces=['eth0'])) - - def test_getIP_wlan0(self): - m = MockAgent(self, commands=[self.commands[1]]) - d = mozdevice.DroidSUT("127.0.0.1", port=m.port, logLevel=mozlog.DEBUG) - self.assertEqual('10.1.39.126', d.getIP(interfaces=['wlan0'])) - - def test_getIP_error(self): - m = MockAgent(self, commands=[self.commands[2]]) - d = mozdevice.DroidSUT("127.0.0.1", port=m.port, logLevel=mozlog.DEBUG) - self.assertRaises(mozdevice.DMError, d.getIP, interfaces=['fake0']) - -if __name__ == '__main__': - unittest.main() diff --git a/testing/mozbase/mozdevice/tests/sut_kill.py b/testing/mozbase/mozdevice/tests/sut_kill.py deleted file mode 100644 index c5cd44a8949..00000000000 --- a/testing/mozbase/mozdevice/tests/sut_kill.py +++ /dev/null @@ -1,24 +0,0 @@ -#!/usr/bin/env python - -import mozdevice -import mozlog -import unittest -from sut import MockAgent - - -class TestKill(unittest.TestCase): - - def test_killprocess(self): - commands = [("ps", "1000 1486 com.android.settings\n" - "10016 420 com.android.location.fused\n" - "10023 335 com.android.systemui\n"), - ("kill com.android.settings", - "Successfully killed com.android.settings\n")] - m = MockAgent(self, commands=commands) - d = mozdevice.DroidSUT("127.0.0.1", port=m.port, logLevel=mozlog.DEBUG) - # No error raised means success - self.assertEqual(None, d.killProcess("com.android.settings")) - - -if __name__ == '__main__': - unittest.main() diff --git a/testing/mozbase/mozdevice/tests/sut_list.py b/testing/mozbase/mozdevice/tests/sut_list.py deleted file mode 100644 index 6f76ea1c979..00000000000 --- a/testing/mozbase/mozdevice/tests/sut_list.py +++ /dev/null @@ -1,22 +0,0 @@ -#/usr/bin/env python -import mozdevice -import mozlog -import unittest -from sut import MockAgent - - -class TestListFiles(unittest.TestCase): - commands = [("isdir /mnt/sdcard", "TRUE"), - ("cd /mnt/sdcard", ""), - ("ls", "Android\nMusic\nPodcasts\nRingtones\nAlarms\n" - "Notifications\nPictures\nMovies\nDownload\nDCIM\n")] - - def test_listFiles(self): - m = MockAgent(self, commands=self.commands) - d = mozdevice.DroidSUT("127.0.0.1", port=m.port, logLevel=mozlog.DEBUG) - - expected = (self.commands[2][1].strip()).split("\n") - self.assertEqual(expected, d.listFiles("/mnt/sdcard")) - -if __name__ == '__main__': - unittest.main() diff --git a/testing/mozbase/mozdevice/tests/sut_logcat.py b/testing/mozbase/mozdevice/tests/sut_logcat.py deleted file mode 100644 index 9cbb80cf178..00000000000 --- a/testing/mozbase/mozdevice/tests/sut_logcat.py +++ /dev/null @@ -1,51 +0,0 @@ -#!/usr/bin/env python - -import mozdevice -import mozlog -import unittest -from sut import MockAgent - - -class TestLogCat(unittest.TestCase): - """ Class to test methods assosiated with logcat """ - - def test_getLogcat(self): - - logcat_output = ("07-17 00:51:10.377 I/SUTAgentAndroid( 2933): onCreate\n\r" - "07-17 00:51:10.457 D/dalvikvm( 2933): GC_CONCURRENT freed 351K, 17% free 2523K/3008K, paused 5ms+2ms, total 38ms\n\r" - "07-17 00:51:10.497 I/SUTAgentAndroid( 2933): Caught exception creating file in /data/local/tmp: open failed: EACCES (Permission denied)\n\r" - "07-17 00:51:10.507 E/SUTAgentAndroid( 2933): ERROR: Cannot access world writeable test root\n\r" - "07-17 00:51:10.547 D/GeckoHealthRec( 3253): Initializing profile cache.\n\r" - "07-17 00:51:10.607 D/GeckoHealthRec( 3253): Looking for /data/data/org.mozilla.fennec/files/mozilla/c09kfhne.default/times.json\n\r" - "07-17 00:51:10.637 D/GeckoHealthRec( 3253): Using times.json for profile creation time.\n\r" - "07-17 00:51:10.707 D/GeckoHealthRec( 3253): Incorporating environment: times.json profile creation = 1374026758604\n\r" - "07-17 00:51:10.507 D/GeckoHealthRec( 3253): Requested prefs.\n\r" - "07-17 06:50:54.907 I/SUTAgentAndroid( 3876): \n\r" - "07-17 06:50:54.907 I/SUTAgentAndroid( 3876): Total Private Dirty Memory 3176 kb\n\r" - "07-17 06:50:54.907 I/SUTAgentAndroid( 3876): Total Proportional Set Size Memory 5679 kb\n\r" - "07-17 06:50:54.907 I/SUTAgentAndroid( 3876): Total Shared Dirty Memory 9216 kb\n\r" - "07-17 06:55:21.627 I/SUTAgentAndroid( 3876): 127.0.0.1 : execsu /system/bin/logcat -v time -d dalvikvm:I " - "ConnectivityService:S WifiMonitor:S WifiStateTracker:S wpa_supplicant:S NetworkStateTracker:S\n\r" - "07-17 06:55:21.827 I/dalvikvm-heap( 3876): Grow heap (frag case) to 3.019MB for 102496-byte allocation\n\r" - "return code [0]") - - inp = ("execsu /system/bin/logcat -v time -d " - "dalvikvm:I ConnectivityService:S WifiMonitor:S " - "WifiStateTracker:S wpa_supplicant:S NetworkStateTracker:S") - - commands = [(inp, logcat_output)] - m = MockAgent(self, commands=commands) - d = mozdevice.DroidSUT("127.0.0.1", port=m.port, logLevel=mozlog.DEBUG) - self.assertEqual(logcat_output[:-17].split('\r'), d.getLogcat()) - - def test_recordLogcat(self): - - commands = [("execsu /system/bin/logcat -c", "return code [0]")] - - m = MockAgent(self, commands=commands) - d = mozdevice.DroidSUT("127.0.0.1", port=m.port, logLevel=mozlog.DEBUG) - # No error raised means success - self.assertEqual(None, d.recordLogcat()) - -if __name__ == '__main__': - unittest.main() diff --git a/testing/mozbase/mozdevice/tests/sut_mkdir.py b/testing/mozbase/mozdevice/tests/sut_mkdir.py index 43ab0d251d8..14cc658474f 100644 --- a/testing/mozbase/mozdevice/tests/sut_mkdir.py +++ b/testing/mozbase/mozdevice/tests/sut_mkdir.py @@ -59,15 +59,6 @@ class MkDirsTest(unittest.TestCase): d.mkDirs('/mnt/sdcard/foo/foo') a.wait() - def test_mkdirs_on_root(self): - cmds = [('isdir /', 'TRUE')] - a = MockAgent(self, commands=cmds) - d = mozdevice.DroidSUT('127.0.0.1', port=a.port, - logLevel=mozlog.DEBUG) - d.mkDirs('/foo') - - a.wait() - if __name__ == '__main__': unittest.main() diff --git a/testing/mozbase/mozdevice/tests/sut_push.py b/testing/mozbase/mozdevice/tests/sut_push.py index 8abd5a7c787..8f2e7b7269c 100644 --- a/testing/mozbase/mozdevice/tests/sut_push.py +++ b/testing/mozbase/mozdevice/tests/sut_push.py @@ -44,21 +44,21 @@ class PushTest(unittest.TestCase): f.write(pushfile) f.flush() - subTests = [ { 'cmds': [ ("isdir /mnt/sdcard/baz", "TRUE"), - ("isdir /mnt/sdcard/baz", "TRUE"), - ("push /mnt/sdcard/baz/%s %s\r\n%s" % + subTests = [ { 'cmds': [ ("isdir /mnt/sdcard//baz", "TRUE"), + ("isdir /mnt/sdcard//baz", "TRUE"), + ("push /mnt/sdcard//baz/%s %s\r\n%s" % (os.path.basename(f.name), len(pushfile), pushfile), expectedFileResponse) ], 'expectException': False }, - { 'cmds': [ ("isdir /mnt/sdcard/baz", "TRUE"), - ("isdir /mnt/sdcard/baz", "TRUE"), - ("push /mnt/sdcard/baz/%s %s\r\n%s" % + { 'cmds': [ ("isdir /mnt/sdcard//baz", "TRUE"), + ("isdir /mnt/sdcard//baz", "TRUE"), + ("push /mnt/sdcard//baz/%s %s\r\n%s" % (os.path.basename(f.name), len(pushfile), pushfile), "BADHASH") ], 'expectException': True }, - { 'cmds': [ ("isdir /mnt/sdcard/baz", "FALSE"), + { 'cmds': [ ("isdir /mnt/sdcard//baz", "FALSE"), ("isdir /mnt", "FALSE"), ("mkdr /mnt", "##AGENT-WARNING## Could not create the directory /mnt") ], diff --git a/testing/mozbase/mozdevice/tests/sut_remove.py b/testing/mozbase/mozdevice/tests/sut_remove.py deleted file mode 100644 index 65d289d35f5..00000000000 --- a/testing/mozbase/mozdevice/tests/sut_remove.py +++ /dev/null @@ -1,24 +0,0 @@ -#/usr/bin/env python -import mozdevice -import mozlog -import unittest -from sut import MockAgent - - -class TestRemove(unittest.TestCase): - - def test_removeDir(self): - commands = [("isdir /mnt/sdcard/test", "TRUE"), - ("rmdr /mnt/sdcard/test", "Deleting file(s) from " - "/storage/emulated/legacy/Moztest\n" - " \n" - "Deleting directory " - "/storage/emulated/legacy/Moztest\n")] - - m = MockAgent(self, commands=commands) - d = mozdevice.DroidSUT("127.0.0.1", port=m.port, logLevel=mozlog.DEBUG) - # No error implies we're all good - self.assertEqual(None, d.removeDir("/mnt/sdcard/test")) - -if __name__ == '__main__': - unittest.main() diff --git a/testing/mozbase/mozdevice/tests/sut_time.py b/testing/mozbase/mozdevice/tests/sut_time.py deleted file mode 100644 index 0136d0e47bf..00000000000 --- a/testing/mozbase/mozdevice/tests/sut_time.py +++ /dev/null @@ -1,18 +0,0 @@ -#/usr/bin/env python -import mozdevice -import mozlog -import unittest -from sut import MockAgent - - -class TestGetCurrentTime(unittest.TestCase): - - def test_getCurrentTime(self): - command = [('clok', '1349980200')] - - m = MockAgent(self, commands=command) - d = mozdevice.DroidSUT("127.0.0.1", port=m.port, logLevel=mozlog.DEBUG) - self.assertEqual(d.getCurrentTime(), int(command[0][1])) - -if __name__ == '__main__': - unittest.main() diff --git a/testing/mozbase/mozdevice/tests/sut_unpackfile.py b/testing/mozbase/mozdevice/tests/sut_unpackfile.py deleted file mode 100644 index 2235b597da4..00000000000 --- a/testing/mozbase/mozdevice/tests/sut_unpackfile.py +++ /dev/null @@ -1,24 +0,0 @@ -#!/usr/bin/env python - -import mozdevice -import mozlog -import unittest -from sut import MockAgent - - -class TestUnpack(unittest.TestCase): - - def test_unpackFile(self): - - commands = [("isdir /mnt/sdcard/tests", "TRUE"), - ("unzp /data/test/sample.zip /data/test/", - "Checksum: 653400271\n" - "1 of 1 successfully extracted\n")] - m = MockAgent(self, commands=commands) - d = mozdevice.DroidSUT("127.0.0.1", port=m.port, logLevel=mozlog.DEBUG) - # No error being thrown imples all is well - self.assertEqual(None, d.unpackFile("/data/test/sample.zip", - "/data/test/")) - -if __name__ == '__main__': - unittest.main() diff --git a/testing/mozbase/mozfile/mozfile/mozfile.py b/testing/mozbase/mozfile/mozfile/mozfile.py index 3405c3f432a..582db221865 100644 --- a/testing/mozbase/mozfile/mozfile/mozfile.py +++ b/testing/mozbase/mozfile/mozfile/mozfile.py @@ -2,9 +2,7 @@ # License, v. 2.0. If a copy of the MPL was not distributed with this # file, You can obtain one at http://mozilla.org/MPL/2.0/. -from contextlib import contextmanager import os -import shutil import tarfile import tempfile import urlparse @@ -17,8 +15,7 @@ __all__ = ['extract_tarball', 'is_url', 'load', 'rmtree', - 'NamedTemporaryFile', - 'TemporaryDirectory'] + 'NamedTemporaryFile'] ### utilities for extracting archives @@ -52,8 +49,7 @@ def extract_zip(src, dest): for name in namelist: filename = os.path.realpath(os.path.join(dest, name)) if name.endswith('/'): - if not os.path.isdir(filename): - os.makedirs(filename) + os.makedirs(filename) else: path = os.path.dirname(filename) if not os.path.isdir(path): @@ -93,16 +89,14 @@ def extract(src, dest=None): src) # namelist returns paths with forward slashes even in windows - top_level_files = [os.path.join(dest, name.rstrip('/')) for name in namelist + top_level_files = [os.path.join(dest, name) for name in namelist if len(name.rstrip('/').split('/')) == 1] # namelist doesn't include folders, append these to the list for name in namelist: - index = name.find('/') - if index != -1: - root = os.path.join(dest, name[:index]) - if root not in top_level_files: - top_level_files.append(root) + root = os.path.join(dest, name[:name.find('/')]) + if root not in top_level_files: + top_level_files.append(root) return top_level_files @@ -238,19 +232,3 @@ def load(resource): return file(resource) return urllib2.urlopen(resource) - -@contextmanager -def TemporaryDirectory(): - """ - create a temporary directory using tempfile.mkdtemp, and then clean it up. - - Example usage: - with TemporaryDirectory() as tmp: - open(os.path.join(tmp, "a_temp_file"), "w").write("data") - - """ - tempdir = tempfile.mkdtemp() - try: - yield tempdir - finally: - shutil.rmtree(tempdir) diff --git a/testing/mozbase/mozfile/setup.py b/testing/mozbase/mozfile/setup.py index bcd082a1132..6689995452c 100644 --- a/testing/mozbase/mozfile/setup.py +++ b/testing/mozbase/mozfile/setup.py @@ -4,7 +4,7 @@ from setuptools import setup -PACKAGE_VERSION = '0.10' +PACKAGE_VERSION = '0.7' setup(name='mozfile', version=PACKAGE_VERSION, @@ -14,7 +14,7 @@ setup(name='mozfile', keywords='mozilla', author='Mozilla Automation and Tools team', author_email='tools@lists.mozilla.org', - url='https://wiki.mozilla.org/Auto-tools/Projects/Mozbase', + url='https://wiki.mozilla.org/Auto-tools/Projects/MozBase', license='MPL', packages=['mozfile'], include_package_data=True, diff --git a/testing/mozbase/mozfile/tests/manifest.ini b/testing/mozbase/mozfile/tests/manifest.ini index ce337e48fc8..008b38bfb1a 100644 --- a/testing/mozbase/mozfile/tests/manifest.ini +++ b/testing/mozbase/mozfile/tests/manifest.ini @@ -1,6 +1,4 @@ -[test_extract.py] +[test.py] [test_load.py] -[test_rmtree.py] -[test_tempdir.py] [test_tempfile.py] [test_url.py] diff --git a/testing/mozbase/mozfile/tests/stubs.py b/testing/mozbase/mozfile/tests/stubs.py deleted file mode 100644 index ee7c1168743..00000000000 --- a/testing/mozbase/mozfile/tests/stubs.py +++ /dev/null @@ -1,34 +0,0 @@ -import os -import shutil -import tempfile - - -# stub file paths -files = [('foo.txt',), - ('foo', 'bar.txt'), - ('foo', 'bar', 'fleem.txt'), - ('foobar', 'fleem.txt'), - ('bar.txt')] - - -def create_stub(): - """create a stub directory""" - - tempdir = tempfile.mkdtemp() - try: - for path in files: - fullpath = os.path.join(tempdir, *path) - dirname = os.path.dirname(fullpath) - if not os.path.exists(dirname): - os.makedirs(dirname) - contents = path[-1] - f = file(fullpath, 'w') - f.write(contents) - f.close() - return tempdir - except Exception, e: - try: - shutil.rmtree(tempdir) - except: - pass - raise e diff --git a/testing/mozbase/mozfile/tests/test_extract.py b/testing/mozbase/mozfile/tests/test.py old mode 100644 new mode 100755 similarity index 70% rename from testing/mozbase/mozfile/tests/test_extract.py rename to testing/mozbase/mozfile/tests/test.py index d9e6e0750b6..e2e344637d7 --- a/testing/mozbase/mozfile/tests/test_extract.py +++ b/testing/mozbase/mozfile/tests/test.py @@ -1,21 +1,53 @@ #!/usr/bin/env python +""" +tests for mozfile +""" + import mozfile import os import shutil import tarfile import tempfile -import stubs import unittest import zipfile +# stub file paths +files = [('foo.txt',), + ('foo', 'bar.txt'), + ('foo', 'bar', 'fleem.txt'), + ('foobar', 'fleem.txt'), + ('bar.txt')] + +def create_stub(): + """create a stub directory""" + + tempdir = tempfile.mkdtemp() + try: + for path in files: + fullpath = os.path.join(tempdir, *path) + dirname = os.path.dirname(fullpath) + if not os.path.exists(dirname): + os.makedirs(dirname) + contents = path[-1] + f = file(fullpath, 'w') + f.write(contents) + f.close() + return tempdir + except Exception, e: + try: + shutil.rmtree(tempdir) + except: + pass + raise e + class TestExtract(unittest.TestCase): """test extracting archives""" def ensure_directory_contents(self, directory): """ensure the directory contents match""" - for f in stubs.files: + for f in files: path = os.path.join(directory, *f) exists = os.path.exists(path) if not exists: @@ -101,11 +133,11 @@ class TestExtract(unittest.TestCase): def create_tarball(self): """create a stub tarball for testing""" - tempdir = stubs.create_stub() + tempdir = create_stub() filename = tempfile.mktemp(suffix='.tar') archive = tarfile.TarFile(filename, mode='w') try: - for path in stubs.files: + for path in files: archive.add(os.path.join(tempdir, *path), arcname=os.path.join(*path)) except: os.remove(archive) @@ -118,11 +150,11 @@ class TestExtract(unittest.TestCase): def create_zip(self): """create a stub zipfile for testing""" - tempdir = stubs.create_stub() + tempdir = create_stub() filename = tempfile.mktemp(suffix='.zip') archive = zipfile.ZipFile(filename, mode='w') try: - for path in stubs.files: + for path in files: archive.write(os.path.join(tempdir, *path), arcname=os.path.join(*path)) except: os.remove(filename) @@ -131,3 +163,28 @@ class TestExtract(unittest.TestCase): shutil.rmtree(tempdir) archive.close() return filename + +class TestRemoveTree(unittest.TestCase): + """test our ability to remove a directory tree""" + + def test_remove_directory(self): + tempdir = create_stub() + self.assertTrue(os.path.exists(tempdir)) + self.assertTrue(os.path.isdir(tempdir)) + try: + mozfile.rmtree(tempdir) + except: + shutil.rmtree(tempdir) + raise + self.assertFalse(os.path.exists(tempdir)) + +class TestNamedTemporaryFile(unittest.TestCase): + """test our fix for NamedTemporaryFile""" + + def test_named_temporary_file(self): + temp = mozfile.NamedTemporaryFile() + temp.write("A simple test") + del temp + +if __name__ == '__main__': + unittest.main() diff --git a/testing/mozbase/mozfile/tests/test_load.py b/testing/mozbase/mozfile/tests/test_load.py index 13a5b519c13..d1d6d62d691 100755 --- a/testing/mozbase/mozfile/tests/test_load.py +++ b/testing/mozbase/mozfile/tests/test_load.py @@ -10,7 +10,6 @@ import tempfile import unittest from mozfile import load - class TestLoad(unittest.TestCase): """test the load function""" @@ -26,16 +25,18 @@ class TestLoad(unittest.TestCase): host = '127.0.0.1' httpd = mozhttpd.MozHttpd(host=host, + port=8888, urlhandlers=[{'method': 'GET', 'path': '.*', 'function': example}]) try: httpd.start(block=False) - content = load(httpd.get_url()).read() + content = load('http://127.0.0.1:8888/foo').read() self.assertEqual(content, 'example') finally: httpd.stop() + def test_file_path(self): """test loading from file path""" try: diff --git a/testing/mozbase/mozfile/tests/test_rmtree.py b/testing/mozbase/mozfile/tests/test_rmtree.py deleted file mode 100644 index 689467afe5d..00000000000 --- a/testing/mozbase/mozfile/tests/test_rmtree.py +++ /dev/null @@ -1,59 +0,0 @@ -#!/usr/bin/env python - -import mozfile -import mozinfo -import os -import shutil -import tempfile -import unittest -import stubs - - -class TestRemoveTree(unittest.TestCase): - """test our ability to remove a directory tree""" - - def setUp(self): - # Generate a stub - self.tempdir = stubs.create_stub() - - def tearDown(self): - # Cleanup the stub if it sill exists - if os.path.isdir(self.tempdir): - mozfile.rmtree(self.tempdir) - - def test_remove_directory(self): - self.assertTrue(os.path.isdir(self.tempdir)) - try: - mozfile.rmtree(self.tempdir) - except: - shutil.rmtree(self.tempdir) - raise - self.assertFalse(os.path.exists(self.tempdir)) - - def test_remove_directory_with_open_file(self): - """ Tests handling when removing a directory tree - which has a file in it is still open """ - # Open a file in the generated stub - filepath = os.path.join(self.tempdir, *stubs.files[1]) - f = file(filepath, 'w') - f.write('foo-bar') - # keep file open and then try removing the dir-tree - if mozinfo.isWin: - # On the Windows family WindowsError should be raised. - self.assertRaises(WindowsError, mozfile.rmtree, self.tempdir) - else: - # Folder should be deleted on all other platforms - mozfile.rmtree(self.tempdir) - self.assertFalse(os.path.exists(self.tempdir)) - - def test_remove_directory_after_closing_file(self): - """ Test that the call to mozfile.rmtree succeeds on - all platforms after file is closed """ - - filepath = os.path.join(self.tempdir, *stubs.files[1]) - with open(filepath, 'w') as f: - f.write('foo-bar') - # Delete directory tree - mozfile.rmtree(self.tempdir) - # Check deletion is successful - self.assertFalse(os.path.exists(self.tempdir)) diff --git a/testing/mozbase/mozfile/tests/test_tempdir.py b/testing/mozbase/mozfile/tests/test_tempdir.py deleted file mode 100644 index 81f03d095b5..00000000000 --- a/testing/mozbase/mozfile/tests/test_tempdir.py +++ /dev/null @@ -1,42 +0,0 @@ -#!/usr/bin/env python - -# This Source Code Form is subject to the terms of the Mozilla Public -# License, v. 2.0. If a copy of the MPL was not distributed with this -# file, You can obtain one at http://mozilla.org/MPL/2.0/. - -""" -tests for mozfile.TemporaryDirectory -""" - -from mozfile import TemporaryDirectory -import os -import unittest - - -class TestTemporaryDirectory(unittest.TestCase): - - def test_removed(self): - """ensure that a TemporaryDirectory gets removed""" - path = None - with TemporaryDirectory() as tmp: - path = tmp - self.assertTrue(os.path.isdir(tmp)) - tmpfile = os.path.join(tmp, "a_temp_file") - open(tmpfile, "w").write("data") - self.assertTrue(os.path.isfile(tmpfile)) - self.assertFalse(os.path.isdir(path)) - self.assertFalse(os.path.exists(path)) - - def test_exception(self): - """ensure that TemporaryDirectory handles exceptions""" - path = None - with self.assertRaises(Exception): - with TemporaryDirectory() as tmp: - path = tmp - self.assertTrue(os.path.isdir(tmp)) - raise Exception("oops") - self.assertFalse(os.path.isdir(path)) - self.assertFalse(os.path.exists(path)) - -if __name__ == '__main__': - unittest.main() diff --git a/testing/mozbase/mozfile/tests/test_tempfile.py b/testing/mozbase/mozfile/tests/test_tempfile.py index 3c3d26d5d6f..5cabd89041f 100644 --- a/testing/mozbase/mozfile/tests/test_tempfile.py +++ b/testing/mozbase/mozfile/tests/test_tempfile.py @@ -12,26 +12,7 @@ import mozfile import os import unittest - class TestNamedTemporaryFile(unittest.TestCase): - """test our fix for NamedTemporaryFile""" - - def test_named_temporary_file(self): - """ Ensure the fix for re-opening a NamedTemporaryFile works - - Refer to https://bugzilla.mozilla.org/show_bug.cgi?id=818777 - and https://bugzilla.mozilla.org/show_bug.cgi?id=821362 - """ - - test_string = "A simple test" - with mozfile.NamedTemporaryFile() as temp: - # Test we can write to file - temp.write(test_string) - # Forced flush, so that we can read later - temp.flush() - - # Test we can open the file again on all platforms - self.assertEqual(open(temp.name).read(), test_string) def test_iteration(self): """ensure the line iterator works""" @@ -52,7 +33,7 @@ class TestNamedTemporaryFile(unittest.TestCase): lines = [] for line in tf: lines.append(line.strip()) - self.assertEqual(lines, []) # because we did not seek(0) + self.assertEqual(lines, []) # because we did not seek(0) tf.seek(0) lines = [] for line in tf: diff --git a/testing/mozbase/mozfile/tests/test_url.py b/testing/mozbase/mozfile/tests/test_url.py index 7d2b12b3997..fa2af3206b7 100755 --- a/testing/mozbase/mozfile/tests/test_url.py +++ b/testing/mozbase/mozfile/tests/test_url.py @@ -7,7 +7,6 @@ tests for is_url import unittest from mozfile import is_url - class TestIsUrl(unittest.TestCase): """test the is_url function""" diff --git a/testing/mozbase/mozhttpd/README.md b/testing/mozbase/mozhttpd/README.md new file mode 100644 index 00000000000..c2f69ddb176 --- /dev/null +++ b/testing/mozbase/mozhttpd/README.md @@ -0,0 +1 @@ +basic python webserver, tested with talos diff --git a/testing/mozbase/mozhttpd/mozhttpd/__init__.py b/testing/mozbase/mozhttpd/mozhttpd/__init__.py index a9c0b06e50e..c9b5d5ef3a2 100644 --- a/testing/mozbase/mozhttpd/mozhttpd/__init__.py +++ b/testing/mozbase/mozhttpd/mozhttpd/__init__.py @@ -2,45 +2,6 @@ # License, v. 2.0. If a copy of the MPL was not distributed with this file, # You can obtain one at http://mozilla.org/MPL/2.0/. -""" -Mozhttpd is a simple http webserver written in python, designed expressly -for use in automated testing scenarios. It is designed to both serve static -content and provide simple web services. - -The server is based on python standard library modules such as -SimpleHttpServer, urlparse, etc. The ThreadingMixIn is used to -serve each request on a discrete thread. - -Some existing uses of mozhttpd include Peptest_, Eideticker_, and Talos_. - -.. _Peptest: https://github.com/mozilla/peptest/ - -.. _Eideticker: https://github.com/mozilla/eideticker/ - -.. _Talos: http://hg.mozilla.org/build/ - -The following simple example creates a basic HTTP server which serves -content from the current directory, defines a single API endpoint -`/api/resource/` and then serves requests indefinitely: - -:: - - import mozhttpd - - @mozhttpd.handlers.json_response - def resource_get(request, objid): - return (200, { 'id': objid, - 'query': request.query }) - - - httpd = mozhttpd.MozHttpd(port=8080, docroot='.', - urlhandlers = [ { 'method': 'GET', - 'path': '/api/resources/([^/]+)/?', - 'function': resource_get } ]) - print "Serving '%s' at %s:%s" % (httpd.docroot, httpd.host, httpd.port) - httpd.start(block=True) - -""" - from mozhttpd import MozHttpd, Request, RequestHandler, main from handlers import json_response +import iface diff --git a/testing/mozbase/mozhttpd/mozhttpd/iface.py b/testing/mozbase/mozhttpd/mozhttpd/iface.py new file mode 100644 index 00000000000..44f6d133abd --- /dev/null +++ b/testing/mozbase/mozhttpd/mozhttpd/iface.py @@ -0,0 +1,33 @@ +# This Source Code Form is subject to the terms of the Mozilla Public +# License, v. 2.0. If a copy of the MPL was not distributed with this file, +# You can obtain one at http://mozilla.org/MPL/2.0/. + +import os +import socket +if os.name != 'nt': + import fcntl + import struct + +def _get_interface_ip(ifname): + s = socket.socket(socket.AF_INET, socket.SOCK_DGRAM) + return socket.inet_ntoa(fcntl.ioctl( + s.fileno(), + 0x8915, # SIOCGIFADDR + struct.pack('256s', ifname[:15]) + )[20:24]) + +def get_lan_ip(): + try: + ip = socket.gethostbyname(socket.gethostname()) + except socket.gaierror: # for Mac OS X + ip = socket.gethostbyname(socket.gethostname() + ".local") + + if ip.startswith("127.") and os.name != "nt": + interfaces = ["eth0", "eth1", "eth2", "wlan0", "wlan1", "wifi0", "ath0", "ath1", "ppp0"] + for ifname in interfaces: + try: + ip = _get_interface_ip(ifname) + break; + except IOError: + pass + return ip diff --git a/testing/mozbase/mozhttpd/mozhttpd/mozhttpd.py b/testing/mozbase/mozhttpd/mozhttpd/mozhttpd.py index c82adf8836a..b7ea21b0c08 100755 --- a/testing/mozbase/mozhttpd/mozhttpd/mozhttpd.py +++ b/testing/mozbase/mozhttpd/mozhttpd/mozhttpd.py @@ -16,7 +16,7 @@ import os import urllib import urlparse import re -import moznetwork +import iface import time from SocketServer import ThreadingMixIn @@ -163,23 +163,14 @@ class RequestHandler(SimpleHTTPServer.SimpleHTTPRequestHandler): class MozHttpd(object): """ - :param host: Host from which to serve (default 127.0.0.1) - :param port: Port from which to serve (default 8888) - :param docroot: Server root (default os.getcwd()) - :param urlhandlers: Handlers to specify behavior against method and path match (default None) - :param proxy_host_dirs: Toggle proxy behavior (default False) - :param log_requests: Toggle logging behavior (default False) - Very basic HTTP server class. Takes a docroot (path on the filesystem) and a set of urlhandler dictionaries of the form: - :: - - { - 'method': HTTP method (string): GET, POST, or DEL, - 'path': PATH_INFO (regular expression string), - 'function': function of form fn(arg1, arg2, arg3, ..., request) - } + { + 'method': HTTP method (string): GET, POST, or DEL, + 'path': PATH_INFO (regular expression string), + 'function': function of form fn(arg1, arg2, arg3, ..., request) + } and serves HTTP. For each request, MozHttpd will either return a file off the docroot, or dispatch to a handler function (if both path and @@ -201,7 +192,7 @@ class MozHttpd(object): True. """ - def __init__(self, host="127.0.0.1", port=0, docroot=None, + def __init__(self, host="127.0.0.1", port=8888, docroot=None, urlhandlers=None, proxy_host_dirs=False, log_requests=False): self.host = host self.port = int(port) @@ -225,11 +216,9 @@ class MozHttpd(object): def start(self, block=False): """ - Starts the server. - - If `block` is True, the call will not return. If `block` is False, the - server will be started on a separate thread that can be terminated by - a call to stop(). + Start the server. If block is True, the call will not return. + If block is False, the server will be started on a separate thread that + can be terminated by a call to .stop() """ self.httpd = EasyServer((self.host, self.port), self.handler_class) if block: @@ -240,11 +229,6 @@ class MozHttpd(object): self.server.start() def stop(self): - """ - Stops the server. - - If the server is not running, this method has no effect. - """ if self.httpd: ### FIXME: There is no shutdown() method in Python 2.4... try: @@ -253,18 +237,6 @@ class MozHttpd(object): pass self.httpd = None - def get_url(self, path="/"): - """ - Returns a URL that can be used for accessing the server (e.g. http://192.168.1.3:4321/) - - :param path: Path to append to URL (e.g. if path were /foobar.html you would get a URL like - http://192.168.1.3:4321/foobar.html). Default is `/`. - """ - if not self.httpd: - return None - - return "http://%s:%s%s" % (self.host, self.httpd.server_port, path) - __del__ = stop @@ -290,7 +262,7 @@ def main(args=sys.argv[1:]): parser.error("mozhttpd does not take any arguments") if options.external_ip: - host = moznetwork.get_lan_ip() + host = iface.get_lan_ip() else: host = options.host diff --git a/testing/mozbase/mozhttpd/setup.py b/testing/mozbase/mozhttpd/setup.py index db8bb8f5e01..7f027d0c616 100644 --- a/testing/mozbase/mozhttpd/setup.py +++ b/testing/mozbase/mozhttpd/setup.py @@ -2,20 +2,28 @@ # License, v. 2.0. If a copy of the MPL was not distributed with this file, # You can obtain one at http://mozilla.org/MPL/2.0/. +import os from setuptools import setup -PACKAGE_VERSION = '0.6' -deps = ['moznetwork >= 0.1'] +try: + here = os.path.dirname(os.path.abspath(__file__)) + description = file(os.path.join(here, 'README.md')).read() +except IOError: + description = None + +PACKAGE_VERSION = '0.5' + +deps = [] setup(name='mozhttpd', version=PACKAGE_VERSION, - description="Python webserver intended for use with Mozilla testing", - long_description="see http://mozbase.readthedocs.org/", + description="basic python webserver, tested with talos", + long_description=description, classifiers=[], # Get strings from http://pypi.python.org/pypi?%3Aaction=list_classifiers keywords='mozilla', author='Mozilla Automation and Testing Team', author_email='tools@lists.mozilla.org', - url='https://wiki.mozilla.org/Auto-tools/Projects/Mozbase', + url='https://wiki.mozilla.org/Auto-tools/Projects/MozBase', license='MPL', packages=['mozhttpd'], include_package_data=True, diff --git a/testing/mozbase/mozhttpd/tests/baseurl.py b/testing/mozbase/mozhttpd/tests/baseurl.py deleted file mode 100644 index 048a79b7550..00000000000 --- a/testing/mozbase/mozhttpd/tests/baseurl.py +++ /dev/null @@ -1,18 +0,0 @@ -import mozhttpd -import unittest - -class BaseUrlTest(unittest.TestCase): - - def test_base_url(self): - httpd = mozhttpd.MozHttpd(port=0) - self.assertEqual(httpd.get_url(), None) - httpd.start(block=False) - self.assertEqual("http://127.0.0.1:%s/" % httpd.httpd.server_port, - httpd.get_url()) - self.assertEqual("http://127.0.0.1:%s/cheezburgers.html" % \ - httpd.httpd.server_port, - httpd.get_url(path="/cheezburgers.html")) - httpd.stop() - -if __name__ == '__main__': - unittest.main() diff --git a/testing/mozbase/mozhttpd/tests/basic.py b/testing/mozbase/mozhttpd/tests/basic.py deleted file mode 100644 index 8d64b4332e0..00000000000 --- a/testing/mozbase/mozhttpd/tests/basic.py +++ /dev/null @@ -1,46 +0,0 @@ -#!/usr/bin/env python - -import mozhttpd -import mozfile -import os -import tempfile -import unittest - - -class TestBasic(unittest.TestCase): - """ Test basic Mozhttpd capabilites """ - - def test_basic(self): - """ Test mozhttpd can serve files """ - - tempdir = tempfile.mkdtemp() - - # sizes is a dict of the form: name -> [size, binary_string, filepath] - sizes = {'small': [128], 'large': [16384]} - - for k in sizes.keys(): - # Generate random binary string - sizes[k].append(os.urandom(sizes[k][0])) - - # Add path of file with binary string to list - fpath = os.path.join(tempdir, k) - sizes[k].append(fpath) - - # Write binary string to file - with open(fpath, 'wb') as f: - f.write(sizes[k][1]) - - server = mozhttpd.MozHttpd(docroot=tempdir) - server.start() - server_url = server.get_url() - - # Retrieve file and check contents matchup - for k in sizes.keys(): - retrieved_content = mozfile.load(server_url + k).read() - self.assertEqual(retrieved_content, sizes[k][1]) - - # Cleanup tempdir and related files - mozfile.rmtree(tempdir) - -if __name__ == '__main__': - unittest.main() diff --git a/testing/mozbase/mozhttpd/tests/manifest.ini b/testing/mozbase/mozhttpd/tests/manifest.ini index 4d1eac07196..370125380b9 100644 --- a/testing/mozbase/mozhttpd/tests/manifest.ini +++ b/testing/mozbase/mozhttpd/tests/manifest.ini @@ -1,5 +1,3 @@ -[api.py] -[baseurl.py] -[basic.py] [filelisting.py] +[api.py] [requestlog.py] diff --git a/testing/mozbase/mozinfo/mozinfo/__init__.py b/testing/mozbase/mozinfo/mozinfo/__init__.py index 904dfef71a7..8d293c128ee 100644 --- a/testing/mozbase/mozinfo/mozinfo/__init__.py +++ b/testing/mozbase/mozinfo/mozinfo/__init__.py @@ -51,6 +51,4 @@ Module variables: """ -import mozinfo from mozinfo import * -__all__ = mozinfo.__all__ diff --git a/testing/mozbase/mozinfo/mozinfo/mozinfo.py b/testing/mozbase/mozinfo/mozinfo/mozinfo.py index d54f2bcf5ee..e77cebfb652 100755 --- a/testing/mozbase/mozinfo/mozinfo/mozinfo.py +++ b/testing/mozbase/mozinfo/mozinfo/mozinfo.py @@ -108,12 +108,9 @@ def sanitize(info): # method for updating information def update(new_info): - """ - Update the info. - - :param new_info: Either a dict containing the new info or a path/url - to a json file containing the new info. - """ + """Update the info. + new_info can either be a dict or a path/url + to a json file containing a dict.""" if isinstance(new_info, basestring): f = mozfile.load(new_info) @@ -131,50 +128,13 @@ def update(new_info): if isLinux or isBsd: globals()['isUnix'] = True -def find_and_update_from_json(*dirs): - """ - Find a mozinfo.json file, load it, and update the info with the - contents. - - :param dirs: Directories in which to look for the file. They will be - searched after first looking in the root of the objdir - if the current script is being run from a Mozilla objdir. - - Returns the full path to mozinfo.json if it was found, or None otherwise. - """ - # First, see if we're in an objdir - try: - from mozbuild.base import MozbuildObject - build = MozbuildObject.from_environment() - json_path = _os.path.join(build.topobjdir, "mozinfo.json") - if _os.path.isfile(json_path): - update(json_path) - return json_path - except ImportError: - pass - - for d in dirs: - d = _os.path.abspath(d) - json_path = _os.path.join(d, "mozinfo.json") - if _os.path.isfile(json_path): - update(json_path) - return json_path - - return None - update({}) # exports __all__ = info.keys() __all__ += ['is' + os_name.title() for os_name in choices['os']] -__all__ += [ - 'info', - 'unknown', - 'main', - 'choices', - 'update', - 'find_and_update_from_json', - ] +__all__ += ['info', 'unknown', 'main', 'choices', 'update'] + def main(args=None): diff --git a/testing/mozbase/mozinfo/setup.py b/testing/mozbase/mozinfo/setup.py index 582b0673d4a..0080ee059a7 100644 --- a/testing/mozbase/mozinfo/setup.py +++ b/testing/mozbase/mozinfo/setup.py @@ -4,7 +4,7 @@ from setuptools import setup -PACKAGE_VERSION = '0.6' +PACKAGE_VERSION = '0.5' # dependencies deps = ['mozfile >= 0.6'] @@ -21,7 +21,7 @@ setup(name='mozinfo', keywords='mozilla', author='Mozilla Automation and Testing Team', author_email='tools@lists.mozilla.org', - url='https://wiki.mozilla.org/Auto-tools/Projects/Mozbase', + url='https://wiki.mozilla.org/Auto-tools/Projects/MozBase', license='MPL', packages=['mozinfo'], include_package_data=True, diff --git a/testing/mozbase/mozinfo/tests/manifest.ini b/testing/mozbase/mozinfo/tests/manifest.ini deleted file mode 100644 index 528fdea7b65..00000000000 --- a/testing/mozbase/mozinfo/tests/manifest.ini +++ /dev/null @@ -1 +0,0 @@ -[test.py] diff --git a/testing/mozbase/mozinfo/tests/test.py b/testing/mozbase/mozinfo/tests/test.py deleted file mode 100644 index a106f5d8329..00000000000 --- a/testing/mozbase/mozinfo/tests/test.py +++ /dev/null @@ -1,88 +0,0 @@ -#!/usr/bin/env python -# -# This Source Code Form is subject to the terms of the Mozilla Public -# License, v. 2.0. If a copy of the MPL was not distributed with this file, -# You can obtain one at http://mozilla.org/MPL/2.0/. - -import json -import mock -import os -import shutil -import sys -import tempfile -import unittest -import mozinfo - -class TestMozinfo(unittest.TestCase): - def setUp(self): - reload(mozinfo) - self.tempdir = os.path.abspath(tempfile.mkdtemp()) - - # When running from an objdir mozinfo will use a build generated json file - # instead of the ones created for testing. Prevent that from happening. - # See bug 896038 for details. - sys.modules['mozbuild'] = None - - def tearDown(self): - shutil.rmtree(self.tempdir) - del sys.modules['mozbuild'] - - def test_basic(self): - """Test that mozinfo has a few attributes.""" - self.assertNotEqual(mozinfo.os, None) - # should have isFoo == True where os == "foo" - self.assertTrue(getattr(mozinfo, "is" + mozinfo.os[0].upper() + mozinfo.os[1:])) - - def test_update(self): - """Test that mozinfo.update works.""" - mozinfo.update({"foo": 123}) - self.assertEqual(mozinfo.info["foo"], 123) - - def test_update_file(self): - """Test that mozinfo.update can load a JSON file.""" - j = os.path.join(self.tempdir, "mozinfo.json") - with open(j, "w") as f: - f.write(json.dumps({"foo": "xyz"})) - mozinfo.update(j) - self.assertEqual(mozinfo.info["foo"], "xyz") - - def test_update_file_invalid_json(self): - """Test that mozinfo.update handles invalid JSON correctly""" - j = os.path.join(self.tempdir,'test.json') - with open(j, 'w') as f: - f.write('invalid{"json":') - self.assertRaises(ValueError,mozinfo.update,[j]) - - def test_find_and_update_file(self): - """Test that mozinfo.find_and_update_from_json can - find mozinfo.json in a directory passed to it.""" - j = os.path.join(self.tempdir, "mozinfo.json") - with open(j, "w") as f: - f.write(json.dumps({"foo": "abcdefg"})) - self.assertEqual(mozinfo.find_and_update_from_json(self.tempdir), j) - self.assertEqual(mozinfo.info["foo"], "abcdefg") - - def test_find_and_update_file_invalid_json(self): - """Test that mozinfo.find_and_update_from_json can - handle invalid JSON""" - j = os.path.join(self.tempdir, "mozinfo.json") - with open(j, 'w') as f: - f.write('invalid{"json":') - self.assertRaises(ValueError, mozinfo.find_and_update_from_json, self.tempdir) - - - def test_find_and_update_file_mozbuild(self): - """Test that mozinfo.find_and_update_from_json can - find mozinfo.json using the mozbuild module.""" - j = os.path.join(self.tempdir, "mozinfo.json") - with open(j, "w") as f: - f.write(json.dumps({"foo": "123456"})) - m = mock.MagicMock() - # Mock the value of MozbuildObject.from_environment().topobjdir. - m.MozbuildObject.from_environment.return_value.topobjdir = self.tempdir - with mock.patch.dict(sys.modules, {"mozbuild": m, "mozbuild.base": m}): - self.assertEqual(mozinfo.find_and_update_from_json(), j) - self.assertEqual(mozinfo.info["foo"], "123456") - -if __name__ == '__main__': - unittest.main() diff --git a/testing/mozbase/mozinstall/mozinstall/mozinstall.py b/testing/mozbase/mozinstall/mozinstall/mozinstall.py index 4ff258e5e55..cbbeeeea997 100755 --- a/testing/mozbase/mozinstall/mozinstall/mozinstall.py +++ b/testing/mozbase/mozinstall/mozinstall/mozinstall.py @@ -61,8 +61,7 @@ def get_binary(path, app_name): # On OS X we can get the real binary from the app bundle if mozinfo.isMac: plist = '%s/Contents/Info.plist' % path - if not os.path.isfile(plist): - raise InvalidBinary('%s/Contents/Info.plist not found' % path) + assert os.path.isfile(plist), '"%s" has not been found.' % plist binary = os.path.join(path, 'Contents/MacOS/', readPlist(plist)['CFBundleExecutable']) @@ -206,7 +205,7 @@ def uninstall(install_folder): # Ensure that we remove any trace of the installation. Even the uninstaller # on Windows leaves files behind we have to explicitely remove. - mozfile.rmtree(install_folder) + shutil.rmtree(install_folder) def _install_dmg(src, dest): diff --git a/testing/mozbase/mozinstall/setup.py b/testing/mozbase/mozinstall/setup.py index da5e244d405..9c14f033c7f 100644 --- a/testing/mozbase/mozinstall/setup.py +++ b/testing/mozbase/mozinstall/setup.py @@ -11,7 +11,7 @@ try: except IOError: description = None -PACKAGE_VERSION = '1.7' +PACKAGE_VERSION = '1.6' deps = ['mozinfo >= 0.4', 'mozfile' @@ -33,7 +33,7 @@ setup(name='mozInstall', keywords='mozilla', author='Mozilla Automation and Tools team', author_email='tools@lists.mozilla.org', - url='https://wiki.mozilla.org/Auto-tools/Projects/Mozbase', + url='https://wiki.mozilla.org/Auto-tools/Projects/MozBase', license='MPL 2.0', packages=['mozinstall'], include_package_data=True, diff --git a/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg b/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg deleted file mode 100644 index f7f36f6318a..00000000000 Binary files a/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.dmg and /dev/null differ diff --git a/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.exe b/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.exe deleted file mode 100644 index c1b0564fde7..00000000000 Binary files a/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.exe and /dev/null differ diff --git a/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.tar.bz2 b/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.tar.bz2 deleted file mode 100644 index cb046a0e7ff..00000000000 Binary files a/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.tar.bz2 and /dev/null differ diff --git a/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.zip b/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.zip deleted file mode 100644 index 7c3f61a5e97..00000000000 Binary files a/testing/mozbase/mozinstall/tests/Installer-Stubs/firefox.zip and /dev/null differ diff --git a/testing/mozbase/mozinstall/tests/manifest.ini b/testing/mozbase/mozinstall/tests/manifest.ini deleted file mode 100644 index 528fdea7b65..00000000000 --- a/testing/mozbase/mozinstall/tests/manifest.ini +++ /dev/null @@ -1 +0,0 @@ -[test.py] diff --git a/testing/mozbase/mozinstall/tests/test.py b/testing/mozbase/mozinstall/tests/test.py deleted file mode 100644 index 53e36d10f32..00000000000 --- a/testing/mozbase/mozinstall/tests/test.py +++ /dev/null @@ -1,151 +0,0 @@ -#!/usr/bin/env python - -# This Source Code Form is subject to the terms of the Mozilla Public -# License, v. 2.0. If a copy of the MPL was not distributed with this file, -# You can obtain one at http://mozilla.org/MPL/2.0/. - -import mozinfo -import mozinstall -import mozfile -import os -import tarfile -import tempfile -import unittest -import zipfile - -# Store file location at load time -here = os.path.dirname(os.path.abspath(__file__)) - -class TestMozInstall(unittest.TestCase): - - @classmethod - def setUpClass(cls): - """ Setting up stub installers """ - cls.dmg = os.path.join(here, 'Installer-Stubs', 'firefox.dmg') - cls.exe = os.path.join(here, 'Installer-Stubs', 'firefox.exe') - cls.zipfile = os.path.join(here, 'Installer-Stubs', 'firefox.zip') - cls.bz2 = os.path.join(here, 'Installer-Stubs', 'firefox.tar.bz2') - - def setUp(self): - self.tempdir = tempfile.mkdtemp() - - def tearDown(self): - mozfile.rmtree(self.tempdir) - - def test_get_binary(self): - """ Test mozinstall's get_binary method """ - - if mozinfo.isLinux: - installdir = mozinstall.install(self.bz2, self.tempdir) - binary = os.path.join(installdir, 'firefox') - self.assertEqual(binary, mozinstall.get_binary(installdir, 'firefox')) - - elif mozinfo.isWin: - installdir_exe = mozinstall.install(self.exe, - os.path.join(self.tempdir, 'exe')) - binary_exe = os.path.join(installdir_exe, 'firefox', 'firefox', - 'firefox.exe') - self.assertEqual(binary_exe, mozinstall.get_binary(installdir_exe, - 'firefox')) - - installdir_zip = mozinstall.install(self.zipfile, - os.path.join(self.tempdir, 'zip')) - binary_zip = os.path.join(installdir_zip, 'firefox.exe') - self.assertEqual(binary_zip, mozinstall.get_binary(installdir_zip, - 'firefox')) - - elif mozinfo.isMac: - installdir = mozinstall.install(self.dmg, self.tempdir) - binary = os.path.join(installdir, 'Contents', 'MacOS', 'firefox') - self.assertEqual(binary, mozinstall.get_binary(installdir, 'firefox')) - - def test_get_binary_error(self): - """ Test an InvalidBinary error is raised """ - - tempdir_empty = tempfile.mkdtemp() - self.assertRaises(mozinstall.InvalidBinary, mozinstall.get_binary, - tempdir_empty, 'firefox') - mozfile.rmtree(tempdir_empty) - - def test_is_installer(self): - """ Test we can identify a correct installer """ - - if mozinfo.isLinux: - self.assertTrue(mozinstall.is_installer(self.bz2)) - - if mozinfo.isWin: - # test zip installer - self.assertTrue(mozinstall.is_installer(self.zipfile)) - # test exe installer - self.assertTrue(mozinstall.is_installer(self.exe)) - - if mozinfo.isMac: - self.assertTrue(mozinstall.is_installer(self.dmg)) - - def test_invalid_source_error(self): - """ Test InvalidSource error is raised with an incorrect installer """ - - if mozinfo.isLinux: - self.assertRaises(mozinstall.InvalidSource, mozinstall.install, - self.dmg, 'firefox') - - elif mozinfo.isWin: - self.assertRaises(mozinstall.InvalidSource, mozinstall.install, - self.bz2, 'firefox') - - elif mozinfo.isMac: - self.assertRaises(mozinstall.InvalidSource, mozinstall.install, - self.exe, 'firefox') - - def test_install(self): - """ Test mozinstall's install capability """ - - if mozinfo.isLinux: - installdir = mozinstall.install(self.bz2, self.tempdir) - self.assertEqual(os.path.join(self.tempdir, 'firefox'), installdir) - - elif mozinfo.isWin: - installdir_exe = mozinstall.install(self.exe, - os.path.join(self.tempdir, 'exe')) - self.assertEqual(os.path.join(self.tempdir, 'exe', 'firefox'), - installdir_exe) - - installdir_zip = mozinstall.install(self.zipfile, - os.path.join(self.tempdir, 'zip')) - self.assertEqual(os.path.join(self.tempdir, 'zip', 'firefox'), - installdir_zip) - - elif mozinfo.isMac: - installdir = mozinstall.install(self.dmg, self.tempdir) - self.assertEqual(os.path.join(os.path.realpath(self.tempdir), - 'FirefoxStub.app'), installdir) - - def test_uninstall(self): - """ Test mozinstall's uninstall capabilites """ - # Uninstall after installing - - if mozinfo.isLinux: - installdir = mozinstall.install(self.bz2, self.tempdir) - mozinstall.uninstall(installdir) - self.assertFalse(os.path.exists(installdir)) - - elif mozinfo.isWin: - # Exe installer for Windows - installdir_exe = mozinstall.install(self.exe, - os.path.join(self.tempdir, 'exe')) - mozinstall.uninstall(installdir_exe) - self.assertFalse(os.path.exists(installdir_exe)) - - # Zip installer for Windows - installdir_zip = mozinstall.install(self.zipfile, - os.path.join(self.tempdir, 'zip')) - mozinstall.uninstall(installdir_zip) - self.assertFalse(os.path.exists(installdir_zip)) - - elif mozinfo.isMac: - installdir = mozinstall.install(self.dmg, self.tempdir) - mozinstall.uninstall(installdir) - self.assertFalse(os.path.exists(installdir)) - -if __name__ == '__main__': - unittest.main() diff --git a/testing/mozbase/mozlog/README.md b/testing/mozbase/mozlog/README.md new file mode 100644 index 00000000000..b13bc6b7a4b --- /dev/null +++ b/testing/mozbase/mozlog/README.md @@ -0,0 +1,18 @@ +[Mozlog](https://github.com/mozilla/mozbase/tree/master/mozlog) +is a python package intended to simplify and standardize logs in the Mozilla universe. +It wraps around python's [logging](http://docs.python.org/library/logging.html) +module and adds some additional functionality. + +# Usage + +Import mozlog instead of [logging](http://docs.python.org/library/logging.html) +(all functionality in the logging module is also available from the mozlog module). +To get a logger, call mozlog.getLogger passing in a name and the path to a log file. +If no log file is specified, the logger will log to stdout. + + import mozlog + logger = mozlog.getLogger('LOG_NAME', 'log_file_path') + logger.setLevel(mozlog.DEBUG) + logger.info('foo') + logger.testPass('bar') + mozlog.shutdown() diff --git a/testing/mozbase/mozlog/mozlog/__init__.py b/testing/mozbase/mozlog/mozlog/__init__.py index 040eb89194c..06bbdf025fb 100644 --- a/testing/mozbase/mozlog/mozlog/__init__.py +++ b/testing/mozbase/mozlog/mozlog/__init__.py @@ -2,9 +2,4 @@ # License, v. 2.0. If a copy of the MPL was not distributed with this file, # You can obtain one at http://mozilla.org/MPL/2.0/. -"""Mozlog aims to standardize log formatting within Mozilla. -It simply wraps Python's logging_ module and adds a few convenience methods for logging test results and events. -""" - from logger import * -from loglistener import LogMessageServer diff --git a/testing/mozbase/mozlog/mozlog/logger.py b/testing/mozbase/mozlog/mozlog/logger.py index c139fbde490..6396409e2bb 100644 --- a/testing/mozbase/mozlog/mozlog/logger.py +++ b/testing/mozbase/mozlog/mozlog/logger.py @@ -7,11 +7,7 @@ from logging import * # Some of the build slave environments don't see the following when doing # 'from logging import *' # see https://bugzilla.mozilla.org/show_bug.cgi?id=700415#c35 -from logging import getLoggerClass, addLevelName, setLoggerClass, shutdown, debug, info, basicConfig -try: - import json -except ImportError: - import simplejson as json +from logging import getLoggerClass, addLevelName, setLoggerClass, shutdown _default_level = INFO _LoggerClass = getLoggerClass() @@ -22,83 +18,34 @@ END = _default_level + 2 PASS = _default_level + 3 KNOWN_FAIL = _default_level + 4 FAIL = _default_level + 5 -CRASH = _default_level + 6 # Define associated text of log levels addLevelName(START, 'TEST-START') addLevelName(END, 'TEST-END') addLevelName(PASS, 'TEST-PASS') addLevelName(KNOWN_FAIL, 'TEST-KNOWN-FAIL') addLevelName(FAIL, 'TEST-UNEXPECTED-FAIL') -addLevelName(CRASH, 'PROCESS-CRASH') -class MozLogger(_LoggerClass): +class _MozLogger(_LoggerClass): """ - MozLogger class which adds some convenience log levels - related to automated testing in Mozilla and ability to - output structured log messages. + MozLogger class which adds three convenience log levels + related to automated testing in Mozilla """ def testStart(self, message, *args, **kwargs): - """Logs a test start message""" self.log(START, message, *args, **kwargs) def testEnd(self, message, *args, **kwargs): - """Logs a test end message""" self.log(END, message, *args, **kwargs) def testPass(self, message, *args, **kwargs): - """Logs a test pass message""" self.log(PASS, message, *args, **kwargs) def testFail(self, message, *args, **kwargs): - """Logs a test fail message""" self.log(FAIL, message, *args, **kwargs) def testKnownFail(self, message, *args, **kwargs): - """Logs a test known fail message""" self.log(KNOWN_FAIL, message, *args, **kwargs) - def processCrash(self, message, *args, **kwargs): - """Logs a process crash message""" - self.log(CRASH, message, *args, **kwargs) - - def log_structured(self, action, params=None): - """Logs a structured message object.""" - if (params is None): - params = {} - - level = params.get('_level', _default_level) - if isinstance(level, int): - params['_level'] = getLevelName(level) - else: - params['_level'] = level - level = getLevelName(level.upper()) - - # If the logger is fed a level number unknown to the logging - # module, getLevelName will return a string. Unfortunately, - # the logging module will raise a type error elsewhere if - # the level is not an integer. - if not isinstance(level, int): - level = _default_level - - params['_namespace'] = self.name - params['action'] = action - - message = params.get('message', 'UNKNOWN') - self.log(level, message, extra={'params': params}) - -class JSONFormatter(Formatter): - """Log formatter for emitting structured JSON entries.""" - - def format(self, record): - params = getattr(record, 'params') - params['_time'] = int(round(record.created * 1000, 0)) - - if params.get('indent') is not None: - return json.dumps(params, indent=params['indent']) - - return json.dumps(params) - -class MozFormatter(Formatter): +class _MozFormatter(Formatter): """ MozFormatter class used to standardize formatting If a different format is desired, this can be explicitly @@ -127,36 +74,29 @@ class MozFormatter(Formatter): fmt = '%(name)s %(levelname)s ' + sep + ' %(message)s' return fmt % record.__dict__ -def getLogger(name, handler=None): +def getLogger(name, logfile=None): """ Returns the logger with the specified name. If the logger doesn't exist, it is created. - If handler is specified, adds it to the logger. Otherwise a default handler - that logs to standard output will be used. - :param name: The name of the logger to retrieve - :param handler: A handler to add to the logger. If the logger already exists, - and a handler is specified, an exception will be raised. To - add a handler to an existing logger, call that logger's - addHandler method. + name - The name of the logger to retrieve + [filePath] - If specified, the logger will log to the specified filePath + Otherwise, the logger logs to stdout + This parameter only has an effect if the logger doesn't exist """ - setLoggerClass(MozLogger) + setLoggerClass(_MozLogger) if name in Logger.manager.loggerDict: - if (handler): - raise ValueError('The handler parameter requires ' + \ - 'that a logger by this name does ' + \ - 'not already exist') - return Logger.manager.loggerDict[name] + return getSysLogger(name) logger = getSysLogger(name) logger.setLevel(_default_level) - if handler is None: + if logfile: + handler = FileHandler(logfile) + else: handler = StreamHandler() - handler.setFormatter(MozFormatter()) - + handler.setFormatter(_MozFormatter()) logger.addHandler(handler) - logger.propagate = False return logger diff --git a/testing/mozbase/mozlog/mozlog/loglistener.py b/testing/mozbase/mozlog/mozlog/loglistener.py deleted file mode 100644 index ffb5c084fd7..00000000000 --- a/testing/mozbase/mozlog/mozlog/loglistener.py +++ /dev/null @@ -1,50 +0,0 @@ -# This Source Code Form is subject to the terms of the Mozilla Public -# License, v. 2.0. If a copy of the MPL was not distributed with this file, -# You can obtain one at http://mozilla.org/MPL/2.0/. - -import SocketServer -import socket -try: - import json -except ImportError: - import simplejson as json - -class LogMessageServer(SocketServer.TCPServer): - def __init__(self, server_address, logger, message_callback=None, timeout=3): - SocketServer.TCPServer.__init__(self, server_address, LogMessageHandler) - self._logger = logger - self._message_callback = message_callback - self.timeout = timeout - -class LogMessageHandler(SocketServer.BaseRequestHandler): - """Processes output from a connected log source, logging to an - existing logger upon receipt of a well-formed log messsage.""" - - def handle(self): - """Continually listens for log messages.""" - self._partial_message = '' - self.request.settimeout(self.server.timeout) - - while True: - try: - data = self.request.recv(1024) - if not data: - return - self.process_message(data) - except socket.timeout: - return - - def process_message(self, data): - """Processes data from a connected log source. Messages are assumed - to be newline delimited, and generally well-formed JSON.""" - for part in data.split('\n'): - msg_string = self._partial_message + part - try: - msg = json.loads(msg_string) - self._partial_message = '' - self.server._logger.log_structured(msg.get('action', 'UNKNOWN'), msg) - if self.server._message_callback: - self.server._message_callback() - - except ValueError: - self._partial_message = msg_string diff --git a/testing/mozbase/mozlog/setup.py b/testing/mozbase/mozlog/setup.py index 2719203dea1..f88f51cc621 100644 --- a/testing/mozbase/mozlog/setup.py +++ b/testing/mozbase/mozlog/setup.py @@ -2,28 +2,37 @@ # License, v. 2.0. If a copy of the MPL was not distributed with this file, # You can obtain one at http://mozilla.org/MPL/2.0/. +import os +import sys from setuptools import setup PACKAGE_NAME = "mozlog" -PACKAGE_VERSION = '1.3' +PACKAGE_VERSION = "1.1" + +desc = """Robust log handling specialized for logging in the Mozilla universe""" +# take description from README +here = os.path.dirname(os.path.abspath(__file__)) +try: + description = file(os.path.join(here, 'README.md')).read() +except IOError, OSError: + description = '' setup(name=PACKAGE_NAME, version=PACKAGE_VERSION, - description="Robust log handling specialized for logging in the Mozilla universe", - long_description="see http://mozbase.readthedocs.org/", + description=desc, + long_description=description, author='Mozilla Automation and Testing Team', author_email='tools@lists.mozilla.org', - url='https://wiki.mozilla.org/Auto-tools/Projects/Mozbase', - license='MPL 1.1/GPL 2.0/LGPL 2.1', + url='https://wiki.mozilla.org/Auto-tools/Projects/MozBase', + license='MPL 2', packages=['mozlog'], zip_safe=False, - tests_require=['mozfile'], platforms =['Any'], classifiers=['Development Status :: 4 - Beta', 'Environment :: Console', 'Intended Audience :: Developers', - 'License :: OSI Approved :: Mozilla Public License 1.1 (MPL 1.1)', - 'Operating System :: OS Independent', + 'License :: OSI Approved :: Mozilla Public License 2.0 (MPL 2.0)', + ' Operating System :: OS Independent', 'Topic :: Software Development :: Libraries :: Python Modules', ] ) diff --git a/testing/mozbase/mozlog/tests/manifest.ini b/testing/mozbase/mozlog/tests/manifest.ini deleted file mode 100644 index 6f55f3c9898..00000000000 --- a/testing/mozbase/mozlog/tests/manifest.ini +++ /dev/null @@ -1 +0,0 @@ -[test_logger.py] diff --git a/testing/mozbase/mozlog/tests/test_logger.py b/testing/mozbase/mozlog/tests/test_logger.py deleted file mode 100644 index c62887a649b..00000000000 --- a/testing/mozbase/mozlog/tests/test_logger.py +++ /dev/null @@ -1,170 +0,0 @@ -# This Source Code Form is subject to the terms of the Mozilla Public -# License, v. 2.0. If a copy of the MPL was not distributed with this file, -# You can obtain one at http://mozilla.org/MPL/2.0/. - -import mozlog -import mozfile -import unittest -import socket -import time -import threading -import json - -class ListHandler(mozlog.Handler): - """Mock handler appends messages to a list for later inspection.""" - - def __init__(self): - mozlog.Handler.__init__(self) - self.messages = [] - - def emit(self, record): - self.messages.append(self.format(record)) - -class TestLogging(unittest.TestCase): - """Tests behavior of basic mozlog api.""" - - def test_logger_defaults(self): - """Tests the default logging format and behavior.""" - - default_logger = mozlog.getLogger('default.logger') - self.assertEqual(default_logger.name, 'default.logger') - self.assertEqual(len(default_logger.handlers), 1) - self.assertTrue(isinstance(default_logger.handlers[0], - mozlog.StreamHandler)) - - f = mozfile.NamedTemporaryFile() - list_logger = mozlog.getLogger('file.logger', - handler=mozlog.FileHandler(f.name)) - self.assertEqual(len(list_logger.handlers), 1) - self.assertTrue(isinstance(list_logger.handlers[0], - mozlog.FileHandler)) - f.close() - - self.assertRaises(ValueError, mozlog.getLogger, - 'file.logger', handler=ListHandler()) - -class TestStructuredLogging(unittest.TestCase): - """Tests structured output in mozlog.""" - - def setUp(self): - self.handler = ListHandler() - self.handler.setFormatter(mozlog.JSONFormatter()) - self.logger = mozlog.MozLogger('test.Logger') - self.logger.addHandler(self.handler) - self.logger.setLevel(mozlog.DEBUG) - - def check_messages(self, expected, actual): - """Checks actual for equality with corresponding fields in actual. - The actual message should contain all fields in expected, and - should be identical, with the exception of the timestamp field. - The actual message should contain no fields other than the timestamp - field and those present in expected.""" - - self.assertTrue(isinstance(actual['_time'], (int, long))) - - for k, v in expected.items(): - self.assertEqual(v, actual[k]) - - for k in actual.keys(): - if k != '_time': - self.assertTrue(expected.get(k) is not None) - - def test_structured_output(self): - self.logger.log_structured('test_message', - {'_level': mozlog.INFO, - 'message': 'message one'}) - self.logger.log_structured('test_message', - {'_level': mozlog.INFO, - 'message': 'message two'}) - - message_one_expected = {'_namespace': 'test.Logger', - '_level': 'INFO', - 'message': 'message one', - 'action': 'test_message'} - message_two_expected = {'_namespace': 'test.Logger', - '_level': 'INFO', - 'message': 'message two', - 'action': 'test_message'} - - message_one_actual = json.loads(self.handler.messages[0]) - message_two_actual = json.loads(self.handler.messages[1]) - - self.check_messages(message_one_expected, message_one_actual) - self.check_messages(message_two_expected, message_two_actual) - - def message_callback(self): - if len(self.handler.messages) == 3: - message_one_expected = {'_namespace': 'test.Logger', - '_level': 'DEBUG', - 'message': 'socket message one', - 'action': 'test_message'} - message_two_expected = {'_namespace': 'test.Logger', - '_level': 'DEBUG', - 'message': 'socket message two', - 'action': 'test_message'} - message_three_expected = {'_namespace': 'test.Logger', - '_level': 'DEBUG', - 'message': 'socket message three', - 'action': 'test_message'} - - message_one_actual = json.loads(self.handler.messages[0]) - - message_two_actual = json.loads(self.handler.messages[1]) - - message_three_actual = json.loads(self.handler.messages[2]) - - self.check_messages(message_one_expected, message_one_actual) - self.check_messages(message_two_expected, message_two_actual) - self.check_messages(message_three_expected, message_three_actual) - - def test_log_listener(self): - connection = '127.0.0.1', 0 - self.log_server = mozlog.LogMessageServer(connection, - self.logger, - message_callback=self.message_callback, - timeout=0.5) - - # The namespace fields of these messages will be overwritten. - message_string_one = json.dumps({'message': 'socket message one', - 'action': 'test_message', - '_level': 'DEBUG', - '_namespace': 'foo.logger'}) - - message_string_two = json.dumps({'message': 'socket message two', - 'action': 'test_message', - '_level': 'DEBUG', - '_namespace': 'foo.logger'}) - - message_string_three = json.dumps({'message': 'socket message three', - 'action': 'test_message', - '_level': 'DEBUG', - '_namespace': 'foo.logger'}) - - message_string = message_string_one + '\n' + \ - message_string_two + '\n' + \ - message_string_three + '\n' - - server_thread = threading.Thread(target=self.log_server.handle_request) - server_thread.start() - - host, port = self.log_server.server_address - - sock = socket.socket() - sock.connect((host, port)) - - # Sleeps prevent listener from receiving entire message in a single call - # to recv in order to test reconstruction of partial messages. - sock.sendall(message_string[:8]) - time.sleep(.01) - sock.sendall(message_string[8:32]) - time.sleep(.01) - sock.sendall(message_string[32:64]) - time.sleep(.01) - sock.sendall(message_string[64:128]) - time.sleep(.01) - sock.sendall(message_string[128:]) - - server_thread.join() - -if __name__ == '__main__': - unittest.main() diff --git a/testing/mozbase/moznetwork/moznetwork/__init__.py b/testing/mozbase/moznetwork/moznetwork/__init__.py index 493ad63543c..332c9385cec 100644 --- a/testing/mozbase/moznetwork/moznetwork/__init__.py +++ b/testing/mozbase/moznetwork/moznetwork/__init__.py @@ -2,23 +2,4 @@ # License, v. 2.0. If a copy of the MPL was not distributed with this file, # You can obtain one at http://mozilla.org/MPL/2.0/. -""" -moznetwork is a very simple module designed for one task: getting the -network address of the current machine. - -Example usage: - -:: - - import moznetwork - - try: - ip = moznetwork.get_ip() - print "The external IP of your machine is '%s'" % ip - except moznetwork.NetworkError: - print "Unable to determine IP address of machine" - raise - -""" - from moznetwork import * diff --git a/testing/mozbase/moznetwork/moznetwork/moznetwork.py b/testing/mozbase/moznetwork/moznetwork/moznetwork.py index 5afa4c3a04e..ae5344e5d97 100644 --- a/testing/mozbase/moznetwork/moznetwork/moznetwork.py +++ b/testing/mozbase/moznetwork/moznetwork/moznetwork.py @@ -2,16 +2,16 @@ # License, v. 2.0. If a copy of the MPL was not distributed with this file, # You can obtain one at http://mozilla.org/MPL/2.0/. +import os import socket import array import struct -import mozinfo - -if mozinfo.isLinux: +if os.name != 'nt': import fcntl + class NetworkError(Exception): - """Exception thrown when unable to obtain interface or IP.""" + """Unable to obtain interface or IP""" def _get_interface_list(): @@ -40,10 +40,8 @@ def _get_interface_list(): def get_ip(): - """Provides an available network interface address, for example - "192.168.1.3". - - A `NetworkError` exception is raised in case of failure.""" + """Provides an available network interface address. A + NetworkError exception is raised in case of failure.""" try: try: ip = socket.gethostbyname(socket.gethostname()) @@ -54,7 +52,7 @@ def get_ip(): # case this will always fail ip = None - if (ip is None or ip.startswith("127.")) and mozinfo.isLinux: + if (ip is None or ip.startswith("127.")) and os.name != "nt": interfaces = _get_interface_list() for ifconfig in interfaces: if ifconfig[0] == 'lo': diff --git a/testing/mozbase/moznetwork/setup.py b/testing/mozbase/moznetwork/setup.py index 327b63eccb8..c0891c2e81f 100644 --- a/testing/mozbase/moznetwork/setup.py +++ b/testing/mozbase/moznetwork/setup.py @@ -4,9 +4,9 @@ from setuptools import setup -PACKAGE_VERSION = '0.22' +PACKAGE_VERSION = '0.21' -deps=[ 'mozinfo' ] +deps=[] setup(name='moznetwork', version=PACKAGE_VERSION, @@ -16,7 +16,7 @@ setup(name='moznetwork', keywords='mozilla', author='Mozilla Automation and Tools team', author_email='tools@lists.mozilla.org', - url='https://wiki.mozilla.org/Auto-tools/Projects/Mozbase', + url='https://wiki.mozilla.org/Auto-tools/Projects/MozBase', license='MPL', packages=['moznetwork'], include_package_data=True, diff --git a/testing/mozbase/moznetwork/tests/manifest.ini b/testing/mozbase/moznetwork/tests/manifest.ini deleted file mode 100644 index 490e534fe9a..00000000000 --- a/testing/mozbase/moznetwork/tests/manifest.ini +++ /dev/null @@ -1,3 +0,0 @@ -[test.py] -# Bug 892087 - Doesn't work reliably on osx -skip-if = os == 'mac' diff --git a/testing/mozbase/moznetwork/tests/test.py b/testing/mozbase/moznetwork/tests/test.py deleted file mode 100644 index 81be0b16c54..00000000000 --- a/testing/mozbase/moznetwork/tests/test.py +++ /dev/null @@ -1,78 +0,0 @@ -#!/usr/bin/env python -""" -Unit-Tests for moznetwork -""" - -import mock -import mozinfo -import moznetwork -import re -import subprocess -import unittest - - -def verify_ip_in_list(ip): - """ - Helper Method to check if `ip` is listed in Network Adresses - returned by ipconfig/ifconfig, depending on the platform in use - - :param ip: IPv4 address in the xxx.xxx.xxx.xxx format as a string - Example Usage: - verify_ip_in_list('192.168.0.1') - - returns True if the `ip` is in the list of IPs in ipconfig/ifconfig - """ - - # Regex to match IPv4 addresses. - # 0-255.0-255.0-255.0-255, note order is important here. - regexip = re.compile("((25[0-5]|2[0-4]\d|1\d\d|[1-9]\d|\d)\.){3}" - "(25[0-5]|2[0-4]\d|1\d\d|[1-9]\d|\d)") - - if mozinfo.isLinux or mozinfo.isMac or mozinfo.isBsd: - args = ["ifconfig"] - - if mozinfo.isWin: - args = ["ipconfig"] - - ps = subprocess.Popen(args, stdout=subprocess.PIPE) - standardoutput, standarderror = ps.communicate() - - # Generate a list of IPs by parsing the output of ip/ifconfig - ip_list = [x.group() for x in re.finditer(regexip, standardoutput)] - - # Check if ip is in list - if ip in ip_list: - return True - else: - return False - - -class TestGetIP(unittest.TestCase): - - def test_get_ip(self): - """ Attempt to test the IP address returned by - moznetwork.get_ip() is valid """ - - ip = moznetwork.get_ip() - - # Check the IP returned by moznetwork is in the list - self.assertTrue(verify_ip_in_list(ip)) - - def test_get_ip_using_get_interface(self): - """ Test that the control flow path for get_ip() using - _get_interface_list() is works """ - - if mozinfo.isLinux: - - with mock.patch('socket.gethostbyname') as byname: - # Force socket.gethostbyname to return None - byname.return_value = None - - ip = moznetwork.get_ip() - - # Check the IP returned by moznetwork is in the list - self.assertTrue(verify_ip_in_list(ip)) - - -if __name__ == '__main__': - unittest.main() diff --git a/testing/mozbase/mozprocess/setup.py b/testing/mozbase/mozprocess/setup.py index cc748490a15..e284fa154ad 100644 --- a/testing/mozbase/mozprocess/setup.py +++ b/testing/mozbase/mozprocess/setup.py @@ -21,7 +21,7 @@ setup(name='mozprocess', keywords='mozilla', author='Mozilla Automation and Tools team', author_email='tools@lists.mozilla.org', - url='https://wiki.mozilla.org/Auto-tools/Projects/Mozbase', + url='https://wiki.mozilla.org/Auto-tools/Projects/MozBase', license='MPL 2.0', packages=['mozprocess'], include_package_data=True, diff --git a/testing/mozbase/mozprocess/tests/Makefile b/testing/mozbase/mozprocess/tests/Makefile index ea7163b0066..9f3a0c357b5 100644 --- a/testing/mozbase/mozprocess/tests/Makefile +++ b/testing/mozbase/mozprocess/tests/Makefile @@ -26,7 +26,7 @@ proclaunch: proclaunch.obj $(LINK) $(LFLAGS) proclaunch.obj clean: - $(RM) proclaunch.exe proclaunch.obj + $(RM) proclaunch.exe projloaunch.obj else # *nix/Mac LFLAGS = -L.. -liniparser diff --git a/testing/mozbase/mozprofile/mozprofile/__init__.py b/testing/mozbase/mozprofile/mozprofile/__init__.py index 18570e53377..f0ebf3f0df3 100644 --- a/testing/mozbase/mozprofile/mozprofile/__init__.py +++ b/testing/mozbase/mozprofile/mozprofile/__init__.py @@ -10,9 +10,8 @@ subclasses ``FirefoxProfile`` and ``ThundebirdProfile`` are available with preset preferences for those applications. """ +from profile import * from addons import * from cli import * -from permissions import * from prefs import * -from profile import * from webapps import * diff --git a/testing/mozbase/mozprofile/mozprofile/permissions.py b/testing/mozbase/mozprofile/mozprofile/permissions.py index a6a5d814a41..197314aff2d 100644 --- a/testing/mozbase/mozprofile/mozprofile/permissions.py +++ b/testing/mozbase/mozprofile/mozprofile/permissions.py @@ -8,7 +8,7 @@ add permissions to the profile """ __all__ = ['MissingPrimaryLocationError', 'MultiplePrimaryLocationsError', - 'DEFAULT_PORTS', 'DuplicateLocationError', 'BadPortLocationError', + 'DuplicateLocationError', 'BadPortLocationError', 'LocationsSyntaxError', 'Location', 'ServerLocations', 'Permissions'] @@ -22,10 +22,10 @@ except ImportError: import urlparse # http://hg.mozilla.org/mozilla-central/file/b871dfb2186f/build/automation.py.in#l28 -DEFAULT_PORTS = { 'http': '8888', - 'https': '4443', - 'ws': '4443', - 'wss': '4443' } +_DEFAULT_PORTS = { 'http': '8888', + 'https': '4443', + 'ws': '4443', + 'wss': '4443' } class LocationError(Exception): """Signifies an improperly formed location.""" @@ -187,7 +187,7 @@ class ServerLocations(object): host, port = netloc.rsplit(':', 1) except ValueError: host = netloc - port = DEFAULT_PORTS.get(scheme, '80') + port = _DEFAULT_PORTS.get(scheme, '80') try: location = Location(scheme, host, port, options) @@ -297,7 +297,7 @@ class Permissions(object): return preferences for Proxy Auto Config. originally taken from http://mxr.mozilla.org/mozilla-central/source/build/automation.py.in """ - proxy = DEFAULT_PORTS.copy() + proxy = _DEFAULT_PORTS # We need to proxy every server but the primary one. origins = ["'%s'" % l.url() diff --git a/testing/mozbase/mozprofile/mozprofile/prefs.py b/testing/mozbase/mozprofile/mozprofile/prefs.py index fc4b07ec246..e026f58baf3 100644 --- a/testing/mozbase/mozprofile/mozprofile/prefs.py +++ b/testing/mozbase/mozprofile/mozprofile/prefs.py @@ -156,16 +156,8 @@ class Preferences(object): return prefs @classmethod - def read_prefs(cls, path, pref_setter='user_pref', interpolation=None): - """ - Read preferences from (e.g.) prefs.js - - :param path: The path to the preference file to read. - :param pref_setter: The name of the function used to set preferences - in the preference file. - :param interpolation: If provided, a dict that will be passed - to str.format to interpolate preference values. - """ + def read_prefs(cls, path, pref_setter='user_pref'): + """read preferences from (e.g.) prefs.js""" comment = re.compile('/\*([^*]|[\r\n]|(\*+([^*/]|[\r\n])))*\*+/', re.MULTILINE) @@ -192,8 +184,6 @@ class Preferences(object): retval = [] def pref(a, b): - if interpolation and isinstance(b, basestring): - b = b.format(**interpolation) retval.append((a, b)) lines = [i.strip().rstrip(';') for i in string.split('\n') if i.strip()] diff --git a/testing/mozbase/mozprofile/mozprofile/profile.py b/testing/mozbase/mozprofile/mozprofile/profile.py index c170a8a7565..95f0aaa9f22 100644 --- a/testing/mozbase/mozprofile/mozprofile/profile.py +++ b/testing/mozbase/mozprofile/mozprofile/profile.py @@ -2,10 +2,7 @@ # License, v. 2.0. If a copy of the MPL was not distributed with this file, # You can obtain one at http://mozilla.org/MPL/2.0/. -__all__ = ['Profile', - 'FirefoxProfile', - 'MetroFirefoxProfile', - 'ThunderbirdProfile'] +__all__ = ['Profile', 'FirefoxProfile', 'ThunderbirdProfile'] import os import time @@ -257,12 +254,11 @@ class Profile(object): class FirefoxProfile(Profile): """Specialized Profile subclass for Firefox""" - preferences = {# Don't automatically update the application 'app.update.enabled' : False, # Don't restore the last open set of tabs if the browser has crashed 'browser.sessionstore.resume_from_crash': False, - # Don't check for the default web browser during startup + # Don't check for the default web browser 'browser.shell.checkDefaultBrowser' : False, # Don't warn on exit when multiple tabs are open 'browser.tabs.warnOnClose' : False, @@ -294,47 +290,8 @@ class FirefoxProfile(Profile): 'toolkit.telemetry.enabledPreRelease' : False, } -class MetroFirefoxProfile(Profile): - """Specialized Profile subclass for Firefox Metro""" - - preferences = {# Don't automatically update the application for desktop and metro build - 'app.update.enabled' : False, - 'app.update.metro.enabled' : False, - # Don't restore the last open set of tabs if the browser has crashed - 'browser.sessionstore.resume_from_crash': False, - # Don't check for the default web browser during startup - 'browser.shell.checkDefaultBrowser' : False, - # Don't send Firefox health reports to the production server - 'datareporting.healthreport.documentServerURI' : 'http://%(server)s/healthreport/', - # Only install add-ons from the profile and the application scope - # Also ensure that those are not getting disabled. - # see: https://developer.mozilla.org/en/Installing_extensions - 'extensions.enabledScopes' : 5, - 'extensions.autoDisableScopes' : 10, - # Don't install distribution add-ons from the app folder - 'extensions.installDistroAddons' : False, - # Dont' run the add-on compatibility check during start-up - 'extensions.showMismatchUI' : False, - # Disable strict compatibility checks to allow add-ons enabled by default - 'extensions.strictCompatibility' : False, - # Don't automatically update add-ons - 'extensions.update.enabled' : False, - # Don't open a dialog to show available add-on updates - 'extensions.update.notifyUser' : False, - # Enable test mode to run multiple tests in parallel - 'focusmanager.testmode' : True, - # Suppress delay for main action in popup notifications - 'security.notification_enable_delay' : 0, - # Suppress automatic safe mode after crashes - 'toolkit.startup.max_resumed_crashes' : -1, - # Don't report telemetry information - 'toolkit.telemetry.enabled' : False, - 'toolkit.telemetry.enabledPreRelease' : False, - } - class ThunderbirdProfile(Profile): """Specialized Profile subclass for Thunderbird""" - preferences = {'extensions.update.enabled' : False, 'extensions.update.notifyUser' : False, 'browser.shell.checkDefaultBrowser' : False, diff --git a/testing/mozbase/mozprofile/setup.py b/testing/mozbase/mozprofile/setup.py index aabe68b32d4..1fa41651a3c 100644 --- a/testing/mozbase/mozprofile/setup.py +++ b/testing/mozbase/mozprofile/setup.py @@ -5,7 +5,7 @@ import sys from setuptools import setup -PACKAGE_VERSION = '0.12' +PACKAGE_VERSION = '0.9' # we only support python 2 right now assert sys.version_info[0] == 2 diff --git a/testing/mozbase/mozprofile/tests/files/prefs_with_interpolation.js b/testing/mozbase/mozprofile/tests/files/prefs_with_interpolation.js deleted file mode 100644 index d0b30bf7bba..00000000000 --- a/testing/mozbase/mozprofile/tests/files/prefs_with_interpolation.js +++ /dev/null @@ -1,4 +0,0 @@ -user_pref("browser.foo", "http://{server}"); -user_pref("zoom.minPercent", 30); -user_pref("webgl.verbose", "false"); -user_pref("browser.bar", "{abc}xyz"); diff --git a/testing/mozbase/mozprofile/tests/test_preferences.py b/testing/mozbase/mozprofile/tests/test_preferences.py index 54a8123c602..cfc8cbe94db 100755 --- a/testing/mozbase/mozprofile/tests/test_preferences.py +++ b/testing/mozbase/mozprofile/tests/test_preferences.py @@ -261,24 +261,6 @@ user_pref("webgl.force-enabled", true); path = os.path.join(here, 'files', 'prefs_with_comments.js') self.assertEqual(dict(Preferences.read_prefs(path)), self._prefs_with_comments) - def test_read_prefs_with_interpolation(self): - """test reading preferences from a prefs.js file whose values - require interpolation""" - - expected_prefs = { - "browser.foo": "http://server-name", - "zoom.minPercent": 30, - "webgl.verbose": "false", - "browser.bar": "somethingxyz" - } - values = { - "server": "server-name", - "abc": "something" - } - path = os.path.join(here, 'files', 'prefs_with_interpolation.js') - read_prefs = Preferences.read_prefs(path, interpolation=values) - self.assertEqual(dict(read_prefs), expected_prefs) - def test_read_prefs_ttw(self): """test reading preferences through the web via mozhttpd""" diff --git a/testing/mozbase/mozrunner/mozrunner/__init__.py b/testing/mozbase/mozrunner/mozrunner/__init__.py index a6d51942178..8cd69343e30 100644 --- a/testing/mozbase/mozrunner/mozrunner/__init__.py +++ b/testing/mozbase/mozrunner/mozrunner/__init__.py @@ -1,9 +1,5 @@ # This Source Code Form is subject to the terms of the Mozilla Public # License, v. 2.0. If a copy of the MPL was not distributed with this file, # You can obtain one at http://mozilla.org/MPL/2.0/. -from local import * -from local import LocalRunner as Runner -from remote import * -runners = local_runners -runners.update(remote_runners) +from runner import * diff --git a/testing/mozbase/mozrunner/mozrunner/local.py b/testing/mozbase/mozrunner/mozrunner/local.py deleted file mode 100644 index ee0333f0db9..00000000000 --- a/testing/mozbase/mozrunner/mozrunner/local.py +++ /dev/null @@ -1,386 +0,0 @@ -#!/usr/bin/env python - -# This Source Code Form is subject to the terms of the Mozilla Public -# License, v. 2.0. If a copy of the MPL was not distributed with this file, -# You can obtain one at http://mozilla.org/MPL/2.0/. - -__all__ = ['CLI', - 'cli', - 'package_metadata', - 'Runner', - 'local_runners', - 'FirefoxRunner', - 'MetroFirefoxRunner', - 'ThunderbirdRunner'] - -import mozinfo -import optparse -import os -import platform -import subprocess -import sys -import ConfigParser - -from utils import get_metadata_from_egg -from utils import findInPath -from mozprofile import Profile, FirefoxProfile, MetroFirefoxProfile, ThunderbirdProfile, MozProfileCLI -from runner import Runner - -if mozinfo.isMac: - from plistlib import readPlist - -package_metadata = get_metadata_from_egg('mozrunner') - -# Map of debugging programs to information about them -# from http://mxr.mozilla.org/mozilla-central/source/build/automationutils.py#59 -debuggers = {'gdb': {'interactive': True, - 'args': ['-q', '--args'],}, - 'valgrind': {'interactive': False, - 'args': ['--leak-check=full']} - } - -def debugger_arguments(debugger, arguments=None, interactive=None): - """ - finds debugger arguments from debugger given and defaults - * debugger : debugger name or path to debugger - * arguments : arguments to the debugger, or None to use defaults - * interactive : whether the debugger should be run in interactive mode, or None to use default - """ - - # find debugger executable if not a file - executable = debugger - if not os.path.exists(executable): - executable = findInPath(debugger) - if executable is None: - raise Exception("Path to '%s' not found" % debugger) - - # if debugger not in dictionary of knowns return defaults - dirname, debugger = os.path.split(debugger) - if debugger not in debuggers: - return ([executable] + (arguments or []), bool(interactive)) - - # otherwise use the dictionary values for arguments unless specified - if arguments is None: - arguments = debuggers[debugger].get('args', []) - if interactive is None: - interactive = debuggers[debugger].get('interactive', False) - return ([executable] + arguments, interactive) - -class LocalRunner(Runner): - """Handles all running operations. Finds bins, runs and kills the process.""" - - profile_class = Profile # profile class to use by default - - @classmethod - def create(cls, binary=None, cmdargs=None, env=None, kp_kwargs=None, profile_args=None, - clean_profile=True, process_class=None): - profile = cls.profile_class(**(profile_args or {})) - return cls(profile, binary=binary, cmdargs=cmdargs, env=env, kp_kwargs=kp_kwargs, - clean_profile=clean_profile, process_class=process_class) - - def __init__(self, profile, binary, cmdargs=None, env=None, - kp_kwargs=None, clean_profile=None, process_class=None): - - super(LocalRunner, self).__init__(profile, clean_profile=clean_profile, kp_kwargs=None, - process_class=process_class, env=None) - - # find the binary - self.binary = binary - if not self.binary: - raise Exception("Binary not specified") - if not os.path.exists(self.binary): - raise OSError("Binary path does not exist: %s" % self.binary) - - # To be safe the absolute path of the binary should be used - self.binary = os.path.abspath(self.binary) - - # allow Mac binaries to be specified as an app bundle - plist = '%s/Contents/Info.plist' % self.binary - if mozinfo.isMac and os.path.exists(plist): - info = readPlist(plist) - self.binary = os.path.join(self.binary, "Contents/MacOS/", - info['CFBundleExecutable']) - - self.cmdargs = cmdargs or [] - _cmdargs = [i for i in self.cmdargs - if i != '-foreground'] - if len(_cmdargs) != len(self.cmdargs): - # foreground should be last; see - # - https://bugzilla.mozilla.org/show_bug.cgi?id=625614 - # - https://bugzilla.mozilla.org/show_bug.cgi?id=626826 - self.cmdargs = _cmdargs - self.cmdargs.append('-foreground') - - # process environment - if env is None: - self.env = os.environ.copy() - else: - self.env = env.copy() - # allows you to run an instance of Firefox separately from any other instances - self.env['MOZ_NO_REMOTE'] = '1' - # keeps Firefox attached to the terminal window after it starts - self.env['NO_EM_RESTART'] = '1' - - # set the library path if needed on linux - if sys.platform == 'linux2' and self.binary.endswith('-bin'): - dirname = os.path.dirname(self.binary) - if os.environ.get('LD_LIBRARY_PATH', None): - self.env['LD_LIBRARY_PATH'] = '%s:%s' % (os.environ['LD_LIBRARY_PATH'], dirname) - else: - self.env['LD_LIBRARY_PATH'] = dirname - - @property - def command(self): - """Returns the command list to run.""" - commands = [self.binary, '-profile', self.profile.profile] - # Bug 775416 - Ensure that binary options are passed in first - commands[1:1] = self.cmdargs - return commands - - def get_repositoryInfo(self): - """Read repository information from application.ini and platform.ini.""" - - config = ConfigParser.RawConfigParser() - dirname = os.path.dirname(self.binary) - repository = { } - - for file, section in [('application', 'App'), ('platform', 'Build')]: - config.read(os.path.join(dirname, '%s.ini' % file)) - - for key, id in [('SourceRepository', 'repository'), - ('SourceStamp', 'changeset')]: - try: - repository['%s_%s' % (file, id)] = config.get(section, key); - except: - repository['%s_%s' % (file, id)] = None - - return repository - - - def start(self, debug_args=None, interactive=False, timeout=None, outputTimeout=None): - """ - Run self.command in the proper environment. - - debug_args: arguments for the debugger - - interactive: uses subprocess.Popen directly - - read_output: sends program output to stdout [default=False] - - timeout: see process_handler.waitForFinish - - outputTimeout: see process_handler.waitForFinish - """ - - # ensure you are stopped - self.stop() - - # ensure the profile exists - if not self.profile.exists(): - self.profile.reset() - assert self.profile.exists(), "%s : failure to reset profile" % self.__class__.__name__ - - cmd = self._wrap_command(self.command) - - # attach a debugger, if specified - if debug_args: - cmd = list(debug_args) + cmd - - if interactive: - self.process_handler = subprocess.Popen(cmd, env=self.env) - # TODO: other arguments - else: - # this run uses the managed processhandler - self.process_handler = self.process_class(cmd, env=self.env, **self.kp_kwargs) - self.process_handler.run(timeout, outputTimeout) - - - def _wrap_command(self, cmd): - """ - If running on OS X 10.5 or older, wrap |cmd| so that it will - be executed as an i386 binary, in case it's a 32-bit/64-bit universal - binary. - """ - if mozinfo.isMac and hasattr(platform, 'mac_ver') and \ - platform.mac_ver()[0][:4] < '10.6': - return ["arch", "-arch", "i386"] + cmd - return cmd - - -class FirefoxRunner(Runner): - """Specialized Runner subclass for running Firefox.""" - - profile_class = FirefoxProfile - - def __init__(self, profile, binary=None, **kwargs): - - # take the binary from BROWSER_PATH environment variable - binary = binary or os.environ.get('BROWSER_PATH') - Runner.__init__(self, profile, binary, **kwargs) - - -class MetroFirefoxRunner(Runner): - """Specialized Runner subclass for running Firefox.Metro""" - - profile_class = MetroFirefoxProfile - - # helper application to launch Firefox in Metro mode - here = os.path.dirname(os.path.abspath(__file__)) - immersiveHelperPath = os.path.sep.join([here, - 'resources', - 'metrotestharness.exe']) - - def __init__(self, profile, binary=None, **kwargs): - - # take the binary from BROWSER_PATH environment variable - binary = binary or os.environ.get('BROWSER_PATH') - Runner.__init__(self, profile, binary, **kwargs) - - if not os.path.exists(self.immersiveHelperPath): - raise OSError('Can not find Metro launcher: %s' % self.immersiveHelperPath) - - if not mozinfo.isWin: - raise Exception('Firefox Metro mode is only supported on Windows 8 and onwards') - - @property - def command(self): - command = Runner.command.fget(self) - command[:0] = [self.immersiveHelperPath, '-firefoxpath'] - - return command - - -class ThunderbirdRunner(Runner): - """Specialized Runner subclass for running Thunderbird""" - profile_class = ThunderbirdProfile - -local_runners = {'firefox': FirefoxRunner, - 'metrofirefox' : MetroFirefoxRunner, - 'thunderbird': ThunderbirdRunner} - -class CLI(MozProfileCLI): - """Command line interface.""" - - module = "mozrunner" - - def __init__(self, args=sys.argv[1:]): - """ - Setup command line parser and parse arguments - - args : command line arguments - """ - - self.metadata = getattr(sys.modules[self.module], - 'package_metadata', - {}) - version = self.metadata.get('Version') - parser_args = {'description': self.metadata.get('Summary')} - if version: - parser_args['version'] = "%prog " + version - self.parser = optparse.OptionParser(**parser_args) - self.add_options(self.parser) - (self.options, self.args) = self.parser.parse_args(args) - - if getattr(self.options, 'info', None): - self.print_metadata() - sys.exit(0) - - # choose appropriate runner and profile classes - try: - self.runner_class = local_runners[self.options.app] - except KeyError: - self.parser.error('Application "%s" unknown (should be one of "%s")' % - (self.options.app, ', '.join(local_runners.keys()))) - - def add_options(self, parser): - """add options to the parser""" - - # add profile options - MozProfileCLI.add_options(self, parser) - - # add runner options - parser.add_option('-b', "--binary", - dest="binary", help="Binary path.", - metavar=None, default=None) - parser.add_option('--app', dest='app', default='firefox', - help="Application to use [DEFAULT: %default]") - parser.add_option('--app-arg', dest='appArgs', - default=[], action='append', - help="provides an argument to the test application") - parser.add_option('--debugger', dest='debugger', - help="run under a debugger, e.g. gdb or valgrind") - parser.add_option('--debugger-args', dest='debugger_args', - action='append', default=None, - help="arguments to the debugger") - parser.add_option('--interactive', dest='interactive', - action='store_true', - help="run the program interactively") - if self.metadata: - parser.add_option("--info", dest="info", default=False, - action="store_true", - help="Print module information") - - ### methods for introspecting data - - def get_metadata_from_egg(self): - import pkg_resources - ret = {} - dist = pkg_resources.get_distribution(self.module) - if dist.has_metadata("PKG-INFO"): - for line in dist.get_metadata_lines("PKG-INFO"): - key, value = line.split(':', 1) - ret[key] = value - if dist.has_metadata("requires.txt"): - ret["Dependencies"] = "\n" + dist.get_metadata("requires.txt") - return ret - - def print_metadata(self, data=("Name", "Version", "Summary", "Home-page", - "Author", "Author-email", "License", "Platform", "Dependencies")): - for key in data: - if key in self.metadata: - print key + ": " + self.metadata[key] - - ### methods for running - - def command_args(self): - """additional arguments for the mozilla application""" - return self.options.appArgs - - def runner_args(self): - """arguments to instantiate the runner class""" - return dict(cmdargs=self.command_args(), - binary=self.options.binary, - profile_args=self.profile_args()) - - def create_runner(self): - return self.runner_class.create(**self.runner_args()) - - def run(self): - runner = self.create_runner() - self.start(runner) - runner.cleanup() - - def debugger_arguments(self): - """ - returns a 2-tuple of debugger arguments: - (debugger_arguments, interactive) - """ - debug_args = self.options.debugger_args - interactive = self.options.interactive - if self.options.debugger: - debug_args, interactive = debugger_arguments(self.options.debugger) - return debug_args, interactive - - def start(self, runner): - """Starts the runner and waits for Firefox to exit or Keyboard Interrupt. - Shoule be overwritten to provide custom running of the runner instance.""" - - # attach a debugger if specified - debug_args, interactive = self.debugger_arguments() - runner.start(debug_args=debug_args, interactive=interactive) - print 'Starting:', ' '.join(runner.command) - try: - runner.wait() - except KeyboardInterrupt: - runner.stop() - - -def cli(args=sys.argv[1:]): - CLI(args).run() - -if __name__ == '__main__': - cli() diff --git a/testing/mozbase/mozrunner/mozrunner/remote.py b/testing/mozbase/mozrunner/mozrunner/remote.py deleted file mode 100644 index f0da61d7976..00000000000 --- a/testing/mozbase/mozrunner/mozrunner/remote.py +++ /dev/null @@ -1,335 +0,0 @@ -import ConfigParser -import os -import posixpath -import re -import shutil -import subprocess -import tempfile -import time -import traceback - -from runner import Runner -from mozdevice import DMError -import mozcrash -import mozlog - -__all__ = ['RemoteRunner', 'B2GRunner', 'remote_runners'] - -class RemoteRunner(Runner): - - def __init__(self, profile, - devicemanager, - clean_profile=None, - process_class=None, - env=None, - remote_test_root=None, - restore=True): - - super(RemoteRunner, self).__init__(profile, clean_profile=clean_profile, - process_class=process_class, env=env) - self.log = mozlog.getLogger('RemoteRunner') - - self.dm = devicemanager - self.remote_test_root = remote_test_root or self.dm.getDeviceRoot() - self.remote_profile = posixpath.join(self.remote_test_root, 'profile') - self.restore = restore - self.backup_files = set([]) - - def backup_file(self, remote_path): - if not self.restore: - return - - if self.dm.fileExists(remote_path): - self.dm.shellCheckOutput(['dd', 'if=%s' % remote_path, 'of=%s.orig' % remote_path]) - self.backup_files.add(remote_path) - - - def check_for_crashes(self, symbols_path, last_test=None): - crashed = False - remote_dump_dir = posixpath.join(self.remote_profile, 'minidumps') - self.log.info("checking for crashes in '%s'" % remote_dump_dir) - if self.dm.dirExists(remote_dump_dir): - local_dump_dir = tempfile.mkdtemp() - self.dm.getDirectory(remote_dump_dir, local_dump_dir) - try: - crashed = mozcrash.check_for_crashes(local_dump_dir, symbols_path, test_name=last_test) - except: - traceback.print_exc() - finally: - shutil.rmtree(local_dump_dir) - self.dm.removeDir(remote_dump_dir) - return crashed - - def cleanup(self): - if not self.restore: - return - - super(RemoteRunner, self).cleanup() - - for backup_file in self.backup_files: - # Restore the original profiles.ini - self.dm.shellCheckOutput(['dd', 'if=%s.orig' % backup_file, 'of=%s' % backup_file]) - self.dm.removeFile("%s.orig" % backup_file) - - # Delete any bundled extensions - extension_dir = posixpath.join(self.remote_profile, 'extensions', 'staged') - if self.dm.dirExists(extension_dir): - for filename in self.dm.listFiles(extension_dir): - try: - self.dm.removeDir(posixpath.join(self.bundles_dir, filename)) - except DMError: - pass - # Remove the test profile - self.dm.removeDir(self.remote_profile) - -class B2GRunner(RemoteRunner): - - def __init__(self, profile, devicemanager, marionette, context_chrome=True, - test_script=None, test_script_args=None, **kwargs): - - super(B2GRunner, self).__init__(profile, devicemanager, **kwargs) - self.log = mozlog.getLogger('B2GRunner') - - tmpfd, processLog = tempfile.mkstemp(suffix='pidlog') - os.close(tmpfd) - tmp_env = self.env or {} - self.env = { 'MOZ_CRASHREPORTER': '1', - 'MOZ_CRASHREPORTER_NO_REPORT': '1', - 'MOZ_HIDE_RESULTS_TABLE': '1', - 'MOZ_PROCESS_LOG': processLog, - 'NO_EM_RESTART': '1', } - self.env.update(tmp_env) - self.last_test = "automation" - self.marionette = marionette - self.context_chrome = context_chrome - self.test_script = test_script - self.test_script_args = test_script_args - self.remote_profiles_ini = '/data/b2g/mozilla/profiles.ini' - self.bundles_dir = '/system/b2g/distribution/bundles' - self.user_js = '/data/local/user.js' - - @property - def command(self): - cmd = [self.dm._adbPath] - if self.dm._deviceSerial: - cmd.extend(['-s', self.dm._deviceSerial]) - cmd.append('shell') - for k, v in self.env.iteritems(): - cmd.append("%s=%s" % (k, v)) - cmd.append('/system/bin/b2g.sh') - return cmd - - def start(self, timeout=None, outputTimeout=None): - self.timeout = timeout - self.outputTimeout = outputTimeout - self._setup_remote_profile() - # reboot device so it starts up with the proper profile - if not self.marionette.emulator: - self._reboot_device() - #wait for wlan to come up - if not self._wait_for_net(): - raise Exception("network did not come up, please configure the network" + - " prior to running before running the automation framework") - - self.dm.shellCheckOutput(['stop', 'b2g']) - - self.kp_kwargs['processOutputLine'] = [self.on_output] - self.kp_kwargs['onTimeout'] = [self.on_timeout] - self.process_handler = self.process_class(self.command, **self.kp_kwargs) - self.process_handler.run(timeout=timeout, outputTimeout=outputTimeout) - - # Set up port forwarding again for Marionette, since any that - # existed previously got wiped out by the reboot. - if not self.marionette.emulator: - subprocess.Popen([self.dm._adbPath, - 'forward', - 'tcp:%s' % self.marionette.port, - 'tcp:%s' % self.marionette.port]).communicate() - self.marionette.wait_for_port() - - # start a marionette session - session = self.marionette.start_session() - if 'b2g' not in session: - raise Exception("bad session value %s returned by start_session" % session) - - if self.marionette.emulator: - # Disable offline status management (bug 777145), otherwise the network - # will be 'offline' when the mochitests start. Presumably, the network - # won't be offline on a real device, so we only do this for emulators. - self.marionette.set_context(self.marionette.CONTEXT_CHROME) - self.marionette.execute_script(""" - Components.utils.import("resource://gre/modules/Services.jsm"); - Services.io.manageOfflineStatus = false; - Services.io.offline = false; - """) - - if self.context_chrome: - self.marionette.set_context(self.marionette.CONTEXT_CHROME) - else: - self.marionette.set_context(self.marionette.CONTEXT_CONTENT) - - # run the script that starts the tests - - if self.test_script: - if os.path.isfile(self.test_script): - script = open(self.test_script, 'r') - self.marionette.execute_script(script.read(), script_args=self.test_script_args) - script.close() - elif isinstance(self.test_script, basestring): - self.marionette.execute_script(self.test_script, script_args=self.test_script_args) - else: - # assumes the tests are started on startup automatically - pass - - def on_output(self, line): - print line - match = re.findall(r"TEST-START \| ([^\s]*)", line) - if match: - self.last_test = match[-1] - - def on_timeout(self): - self.log.testFail("%s | application timed " - "out after %s seconds with no output", - self.last_test, self.timeout) - - def _reboot_device(self): - serial, status = self._get_device_status() - self.dm.shellCheckOutput(['/system/bin/reboot']) - - # The reboot command can return while adb still thinks the device is - # connected, so wait a little bit for it to disconnect from adb. - time.sleep(10) - - # wait for device to come back to previous status - self.log.info('waiting for device to come back online after reboot') - start = time.time() - rserial, rstatus = self._get_device_status(serial) - while rstatus != 'device': - if time.time() - start > 120: - # device hasn't come back online in 2 minutes, something's wrong - raise Exception("Device %s (status: %s) not back online after reboot" % (serial, rstatus)) - time.sleep(5) - rserial, rstatus = self.getDeviceStatus(serial) - self.log.info('device:', serial, 'status:', rstatus) - - def _get_device_status(self, serial=None): - # If we know the device serial number, we look for that, - # otherwise we use the (presumably only) device shown in 'adb devices'. - serial = serial or self.dm._deviceSerial - status = 'unknown' - - proc = subprocess.Popen([self.dm._adbPath, 'devices'], stdout=subprocess.PIPE) - line = proc.stdout.readline() - while line != '': - result = re.match('(.*?)\t(.*)', line) - if result: - thisSerial = result.group(1) - if not serial or thisSerial == serial: - serial = thisSerial - status = result.group(2) - break - line = proc.stdout.readline() - return (serial, status) - - def _wait_for_net(self): - active = False - time_out = 0 - while not active and time_out < 40: - proc = subprocess.Popen([self.dm._adbPath, 'shell', '/system/bin/netcfg']) - proc.stdout.readline() # ignore first line - line = proc.stdout.readline() - while line != "": - if (re.search(r'UP\s+(?:[0-9]{1,3}\.){3}[0-9]{1,3}', line)): - active = True - break - line = proc.stdout.readline() - time_out += 1 - time.sleep(1) - return active - - def _setup_remote_profile(self): - """ - Copy profile and update the remote profiles ini file to point to the new profile - """ - - # copy the profile to the device. - if self.dm.dirExists(self.remote_profile): - self.dm.shellCheckOutput(['rm', '-r', self.remote_profile]) - - try: - self.dm.pushDir(self.profile.profile, self.remote_profile) - except DMError: - self.log.error("Automation Error: Unable to copy profile to device.") - raise - - extension_dir = os.path.join(self.profile.profile, 'extensions', 'staged') - if os.path.isdir(extension_dir): - # Copy the extensions to the B2G bundles dir. - # need to write to read-only dir - subprocess.Popen([self.dm._adbPath, 'remount']).communicate() - for filename in os.listdir(extension_dir): - self.dm.shellCheckOutput(['rm', '-rf', - os.path.join(self.bundles_dir, filename)]) - try: - self.dm.pushDir(extension_dir, self.bundles_dir) - except DMError: - self.log.error("Automation Error: Unable to copy extensions to device.") - raise - - if not self.dm.fileExists(self.remote_profiles_ini): - raise DMError("The profiles.ini file '%s' does not exist on the device" % self.remote_profiles_ini) - - local_profiles_ini = tempfile.NamedTemporaryFile() - self.dm.getFile(self.remote_profiles_ini, local_profiles_ini.name) - - config = ProfileConfigParser() - config.read(local_profiles_ini.name) - for section in config.sections(): - if 'Profile' in section: - config.set(section, 'IsRelative', 0) - config.set(section, 'Path', self.remote_profile) - - new_profiles_ini = tempfile.NamedTemporaryFile() - config.write(open(new_profiles_ini.name, 'w')) - - self.backup_file(self.remote_profiles_ini) - self.dm.pushFile(new_profiles_ini.name, self.remote_profiles_ini) - - # In B2G, user.js is always read from /data/local, not the profile - # directory. Backup the original user.js first so we can restore it. - self.backup_file(self.user_js) - self.dm.pushFile(os.path.join(self.profile.profile, "user.js"), self.user_js) - - def cleanup(self): - super(B2GRunner, self).cleanup() - if getattr(self.marionette, 'instance', False): - self.marionette.instance.close() - del self.marionette - -class ProfileConfigParser(ConfigParser.RawConfigParser): - """Subclass of RawConfigParser that outputs .ini files in the exact - format expected for profiles.ini, which is slightly different - than the default format. - """ - - def optionxform(self, optionstr): - return optionstr - - def write(self, fp): - if self._defaults: - fp.write("[%s]\n" % ConfigParser.DEFAULTSECT) - for (key, value) in self._defaults.items(): - fp.write("%s=%s\n" % (key, str(value).replace('\n', '\n\t'))) - fp.write("\n") - for section in self._sections: - fp.write("[%s]\n" % section) - for (key, value) in self._sections[section].items(): - if key == "__name__": - continue - if (value is not None) or (self._optcre == self.OPTCRE): - key = "=".join((key, str(value).replace('\n', '\n\t'))) - fp.write("%s\n" % (key)) - fp.write("\n") - -remote_runners = {'b2g': 'B2GRunner', - 'fennec': 'FennecRunner'} diff --git a/testing/mozbase/mozrunner/mozrunner/resources/metrotestharness.exe b/testing/mozbase/mozrunner/mozrunner/resources/metrotestharness.exe deleted file mode 100644 index d3bcbfbee9a..00000000000 Binary files a/testing/mozbase/mozrunner/mozrunner/resources/metrotestharness.exe and /dev/null differ diff --git a/testing/mozbase/mozrunner/mozrunner/runner.py b/testing/mozbase/mozrunner/mozrunner/runner.py index 872895b9786..245fd8e1a35 100755 --- a/testing/mozbase/mozrunner/mozrunner/runner.py +++ b/testing/mozbase/mozrunner/mozrunner/runner.py @@ -3,37 +3,161 @@ # This Source Code Form is subject to the terms of the Mozilla Public # License, v. 2.0. If a copy of the MPL was not distributed with this file, # You can obtain one at http://mozilla.org/MPL/2.0/. + +__all__ = ['Runner', 'ThunderbirdRunner', 'FirefoxRunner', 'runners', 'CLI', 'cli', 'package_metadata'] + +import mozinfo +import optparse +import os +import platform import subprocess +import sys +import ConfigParser +from utils import get_metadata_from_egg +from utils import findInPath +from mozprofile import * from mozprocess.processhandler import ProcessHandler -import mozlog -# we can replace this method with 'abc' -# (http://docs.python.org/library/abc.html) when we require Python 2.6+ -def abstractmethod(method): - line = method.func_code.co_firstlineno - filename = method.func_code.co_filename - def not_implemented(*args, **kwargs): - raise NotImplementedError('Abstract method %s at File "%s", line %s ' - 'should be implemented by a concrete class' % - (repr(method), filename, line)) - return not_implemented +if mozinfo.isMac: + from plistlib import readPlist + +package_metadata = get_metadata_from_egg('mozrunner') + +# Map of debugging programs to information about them +# from http://mxr.mozilla.org/mozilla-central/source/build/automationutils.py#59 +debuggers = {'gdb': {'interactive': True, + 'args': ['-q', '--args'],}, + 'valgrind': {'interactive': False, + 'args': ['--leak-check=full']} + } + +def debugger_arguments(debugger, arguments=None, interactive=None): + """ + finds debugger arguments from debugger given and defaults + * debugger : debugger name or path to debugger + * arguments : arguments to the debugger, or None to use defaults + * interactive : whether the debugger should be run in interactive mode, or None to use default + """ + + # find debugger executable if not a file + executable = debugger + if not os.path.exists(executable): + executable = findInPath(debugger) + if executable is None: + raise Exception("Path to '%s' not found" % debugger) + + # if debugger not in dictionary of knowns return defaults + dirname, debugger = os.path.split(debugger) + if debugger not in debuggers: + return ([executable] + (arguments or []), bool(interactive)) + + # otherwise use the dictionary values for arguments unless specified + if arguments is None: + arguments = debuggers[debugger].get('args', []) + if interactive is None: + interactive = debuggers[debugger].get('interactive', False) + return ([executable] + arguments, interactive) class Runner(object): + """Handles all running operations. Finds bins, runs and kills the process.""" - def __init__(self, profile, clean_profile=True, process_class=None, kp_kwargs=None, env=None): - self.clean_profile = clean_profile - self.env = env or {} - self.kp_kwargs = kp_kwargs or {} - self.process_class = process_class or ProcessHandler + profile_class = Profile # profile class to use by default + + @classmethod + def create(cls, binary=None, cmdargs=None, env=None, kp_kwargs=None, profile_args=None, + clean_profile=True, process_class=ProcessHandler): + profile = cls.profile_class(**(profile_args or {})) + return cls(profile, binary=binary, cmdargs=cmdargs, env=env, kp_kwargs=kp_kwargs, + clean_profile=clean_profile, process_class=process_class) + + def __init__(self, profile, binary, cmdargs=None, env=None, + kp_kwargs=None, clean_profile=True, process_class=ProcessHandler): self.process_handler = None + self.process_class = process_class self.profile = profile - self.log = mozlog.getLogger('MozRunner') + self.clean_profile = clean_profile - @abstractmethod - def start(self, *args, **kwargs): + # find the binary + self.binary = binary + if not self.binary: + raise Exception("Binary not specified") + if not os.path.exists(self.binary): + raise OSError("Binary path does not exist: %s" % self.binary) + + # allow Mac binaries to be specified as an app bundle + plist = '%s/Contents/Info.plist' % self.binary + if mozinfo.isMac and os.path.exists(plist): + info = readPlist(plist) + self.binary = os.path.join(self.binary, "Contents/MacOS/", + info['CFBundleExecutable']) + + self.cmdargs = cmdargs or [] + _cmdargs = [i for i in self.cmdargs + if i != '-foreground'] + if len(_cmdargs) != len(self.cmdargs): + # foreground should be last; see + # - https://bugzilla.mozilla.org/show_bug.cgi?id=625614 + # - https://bugzilla.mozilla.org/show_bug.cgi?id=626826 + self.cmdargs = _cmdargs + self.cmdargs.append('-foreground') + + # process environment + if env is None: + self.env = os.environ.copy() + else: + self.env = env.copy() + # allows you to run an instance of Firefox separately from any other instances + self.env['MOZ_NO_REMOTE'] = '1' + # keeps Firefox attached to the terminal window after it starts + self.env['NO_EM_RESTART'] = '1' + + # set the library path if needed on linux + if sys.platform == 'linux2' and self.binary.endswith('-bin'): + dirname = os.path.dirname(self.binary) + if os.environ.get('LD_LIBRARY_PATH', None): + self.env['LD_LIBRARY_PATH'] = '%s:%s' % (os.environ['LD_LIBRARY_PATH'], dirname) + else: + self.env['LD_LIBRARY_PATH'] = dirname + + # arguments for ProfessHandler.Process + self.kp_kwargs = kp_kwargs or {} + + @property + def command(self): + """Returns the command list to run.""" + return [self.binary, '-profile', self.profile.profile] + + def get_repositoryInfo(self): + """Read repository information from application.ini and platform.ini.""" + + config = ConfigParser.RawConfigParser() + dirname = os.path.dirname(self.binary) + repository = { } + + for file, section in [('application', 'App'), ('platform', 'Build')]: + config.read(os.path.join(dirname, '%s.ini' % file)) + + for key, id in [('SourceRepository', 'repository'), + ('SourceStamp', 'changeset')]: + try: + repository['%s_%s' % (file, id)] = config.get(section, key); + except: + repository['%s_%s' % (file, id)] = None + + return repository + + def is_running(self): + return self.process_handler is not None + + def start(self, debug_args=None, interactive=False, timeout=None, outputTimeout=None): """ - Run the process + Run self.command in the proper environment. + - debug_args: arguments for the debugger + - interactive: uses subprocess.Popen directly + - read_output: sends program output to stdout [default=False] + - timeout: see process_handler.waitForFinish + - outputTimeout: see process_handler.waitForFinish """ # ensure you are stopped @@ -44,7 +168,7 @@ class Runner(object): self.profile.reset() assert self.profile.exists(), "%s : failure to reset profile" % self.__class__.__name__ - cmd = self._wrap_command(self.command) + cmd = self._wrap_command(self.command+self.cmdargs) # attach a debugger, if specified if debug_args: @@ -60,7 +184,7 @@ class Runner(object): def wait(self, timeout=None): """ - Wait for the process to exit. + Wait for the app to exit. If timeout is not None, will return after timeout seconds. Use is_running() to determine whether or not a timeout occured. @@ -74,21 +198,13 @@ class Runner(object): else: self.process_handler.wait(timeout) if self.process_handler.proc.poll() is None: - # wait timed out + # waitForFinish timed out return + self.process_handler = None - def is_running(self): - """ - Returns True if the process is still running, False otherwise - """ - return self.process_handler is not None - - def stop(self): - """ - Kill the process - """ + """Kill the app""" if self.process_handler is None: return self.process_handler.kill() @@ -96,18 +212,177 @@ class Runner(object): def reset(self): """ - Reset the runner to its default state + reset the runner between runs + currently, only resets the profile, but probably should do more """ - if getattr(self, 'profile', False): - self.profile.reset() + self.profile.reset() def cleanup(self): - """ - Cleanup all runner state - """ - if self.is_running(): - self.stop() - if getattr(self, 'profile', False) and self.clean_profile: + self.stop() + if self.clean_profile: self.profile.cleanup() + def _wrap_command(self, cmd): + """ + If running on OS X 10.5 or older, wrap |cmd| so that it will + be executed as an i386 binary, in case it's a 32-bit/64-bit universal + binary. + """ + if mozinfo.isMac and hasattr(platform, 'mac_ver') and \ + platform.mac_ver()[0][:4] < '10.6': + return ["arch", "-arch", "i386"] + cmd + return cmd + __del__ = cleanup + + +class FirefoxRunner(Runner): + """Specialized Runner subclass for running Firefox.""" + + profile_class = FirefoxProfile + + def __init__(self, profile, binary=None, **kwargs): + + # take the binary from BROWSER_PATH environment variable + if (not binary) and 'BROWSER_PATH' in os.environ: + binary = os.environ['BROWSER_PATH'] + + Runner.__init__(self, profile, binary, **kwargs) + +class ThunderbirdRunner(Runner): + """Specialized Runner subclass for running Thunderbird""" + profile_class = ThunderbirdProfile + +runners = {'firefox': FirefoxRunner, + 'thunderbird': ThunderbirdRunner} + +class CLI(MozProfileCLI): + """Command line interface.""" + + module = "mozrunner" + + def __init__(self, args=sys.argv[1:]): + """ + Setup command line parser and parse arguments + - args : command line arguments + """ + + self.metadata = getattr(sys.modules[self.module], + 'package_metadata', + {}) + version = self.metadata.get('Version') + parser_args = {'description': self.metadata.get('Summary')} + if version: + parser_args['version'] = "%prog " + version + self.parser = optparse.OptionParser(**parser_args) + self.add_options(self.parser) + (self.options, self.args) = self.parser.parse_args(args) + + if getattr(self.options, 'info', None): + self.print_metadata() + sys.exit(0) + + # choose appropriate runner and profile classes + try: + self.runner_class = runners[self.options.app] + except KeyError: + self.parser.error('Application "%s" unknown (should be one of "firefox" or "thunderbird")' % self.options.app) + + def add_options(self, parser): + """add options to the parser""" + + # add profile options + MozProfileCLI.add_options(self, parser) + + # add runner options + parser.add_option('-b', "--binary", + dest="binary", help="Binary path.", + metavar=None, default=None) + parser.add_option('--app', dest='app', default='firefox', + help="Application to use [DEFAULT: %default]") + parser.add_option('--app-arg', dest='appArgs', + default=[], action='append', + help="provides an argument to the test application") + parser.add_option('--debugger', dest='debugger', + help="run under a debugger, e.g. gdb or valgrind") + parser.add_option('--debugger-args', dest='debugger_args', + action='append', default=None, + help="arguments to the debugger") + parser.add_option('--interactive', dest='interactive', + action='store_true', + help="run the program interactively") + if self.metadata: + parser.add_option("--info", dest="info", default=False, + action="store_true", + help="Print module information") + + ### methods for introspecting data + + def get_metadata_from_egg(self): + import pkg_resources + ret = {} + dist = pkg_resources.get_distribution(self.module) + if dist.has_metadata("PKG-INFO"): + for line in dist.get_metadata_lines("PKG-INFO"): + key, value = line.split(':', 1) + ret[key] = value + if dist.has_metadata("requires.txt"): + ret["Dependencies"] = "\n" + dist.get_metadata("requires.txt") + return ret + + def print_metadata(self, data=("Name", "Version", "Summary", "Home-page", + "Author", "Author-email", "License", "Platform", "Dependencies")): + for key in data: + if key in self.metadata: + print key + ": " + self.metadata[key] + + ### methods for running + + def command_args(self): + """additional arguments for the mozilla application""" + return self.options.appArgs + + def runner_args(self): + """arguments to instantiate the runner class""" + return dict(cmdargs=self.command_args(), + binary=self.options.binary, + profile_args=self.profile_args()) + + def create_runner(self): + return self.runner_class.create(**self.runner_args()) + + def run(self): + runner = self.create_runner() + self.start(runner) + runner.cleanup() + + def debugger_arguments(self): + """ + returns a 2-tuple of debugger arguments: + (debugger_arguments, interactive) + """ + debug_args = self.options.debugger_args + interactive = self.options.interactive + if self.options.debugger: + debug_args, interactive = debugger_arguments(self.options.debugger) + return debug_args, interactive + + def start(self, runner): + """Starts the runner and waits for Firefox to exit or Keyboard Interrupt. + Shoule be overwritten to provide custom running of the runner instance.""" + + # attach a debugger if specified + debug_args, interactive = self.debugger_arguments() + runner.start(debug_args=debug_args, interactive=interactive) + print 'Starting:', ' '.join(runner.command) + try: + runner.wait() + except KeyboardInterrupt: + runner.stop() + + +def cli(args=sys.argv[1:]): + CLI(args).run() + +if __name__ == '__main__': + cli() diff --git a/testing/mozbase/mozrunner/setup.py b/testing/mozbase/mozrunner/setup.py index 6de698a9239..20a58e585fb 100644 --- a/testing/mozbase/mozrunner/setup.py +++ b/testing/mozbase/mozrunner/setup.py @@ -6,16 +6,13 @@ import sys from setuptools import setup PACKAGE_NAME = "mozrunner" -PACKAGE_VERSION = '5.21' +PACKAGE_VERSION = '5.15' desc = """Reliable start/stop/configuration of Mozilla Applications (Firefox, Thunderbird, etc.)""" -deps = ['mozcrash >= 0.3', - 'mozdevice >= 0.28', - 'mozinfo >= 0.4', - 'mozlog >= 1.3', +deps = ['mozinfo >= 0.4', 'mozprocess >= 0.8', - 'mozprofile >= 0.11', + 'mozprofile >= 0.4', ] # we only support python 2 right now @@ -39,9 +36,6 @@ setup(name=PACKAGE_NAME, url='https://wiki.mozilla.org/Auto-tools/Projects/Mozbase', license='MPL 2.0', packages=['mozrunner'], - package_data={'mozrunner': [ - 'resources/metrotestharness.exe' - ]}, zip_safe=False, install_requires = deps, entry_points=""" diff --git a/testing/mozbase/moztest/README.md b/testing/mozbase/moztest/README.md new file mode 100644 index 00000000000..4fed84b57ea --- /dev/null +++ b/testing/mozbase/moztest/README.md @@ -0,0 +1,16 @@ +# Moztest + +Package for handling Mozilla test results. + + +## Usage example + +This shows how you can create an xUnit representation of python unittest results. + + from results import TestResultCollection + from output import XUnitOutput + + collection = TestResultCollection.from_unittest_results(results) + out = XUnitOutput() + with open('out.xml', 'w') as f: + out.serialize(collection, f) diff --git a/testing/mozbase/moztest/setup.py b/testing/mozbase/moztest/setup.py index 28ef8e63c9f..8336dad9456 100644 --- a/testing/mozbase/moztest/setup.py +++ b/testing/mozbase/moztest/setup.py @@ -2,9 +2,18 @@ # License, v. 2.0. If a copy of the MPL was not distributed with this file, # You can obtain one at http://mozilla.org/MPL/2.0/. + +import os from setuptools import setup -PACKAGE_VERSION = '0.2' +PACKAGE_VERSION = '0.1' + +# get documentation from the README +try: + here = os.path.dirname(os.path.abspath(__file__)) + description = file(os.path.join(here, 'README.md')).read() +except (OSError, IOError): + description = '' # dependencies deps = ['mozinfo'] @@ -16,12 +25,12 @@ except ImportError: setup(name='moztest', version=PACKAGE_VERSION, description="Package for storing and outputting Mozilla test results", - long_description="see http://mozbase.readthedocs.org/", + long_description=description, classifiers=[], # Get strings from http://pypi.python.org/pypi?%3Aaction=list_classifiers keywords='mozilla', author='Mozilla Automation and Tools team', author_email='tools@lists.mozilla.org', - url='https://wiki.mozilla.org/Auto-tools/Projects/Mozbase', + url='https://wiki.mozilla.org/Auto-tools/Projects/MozBase', license='MPL', packages=['moztest'], include_package_data=True, diff --git a/testing/mozbase/setup_development.py b/testing/mozbase/setup_development.py index 6d934f04bc7..a6951d87cc2 100755 --- a/testing/mozbase/setup_development.py +++ b/testing/mozbase/setup_development.py @@ -10,9 +10,10 @@ Setup mozbase packages for development. Packages may be specified as command line arguments. If no arguments are given, install all packages. -See https://wiki.mozilla.org/Auto-tools/Projects/Mozbase +See https://wiki.mozilla.org/Auto-tools/Projects/MozBase """ +import pkg_resources import os import subprocess import sys @@ -30,9 +31,7 @@ here = os.path.dirname(os.path.abspath(__file__)) # all python packages mozbase_packages = [i for i in os.listdir(here) if os.path.exists(os.path.join(here, i, 'setup.py'))] -extra_packages = ["sphinx", # documentation: https://wiki.mozilla.org/Auto-tools/Projects/Mozbase#Documentation - "mock", # testing: https://wiki.mozilla.org/Auto-tools/Projects/Mozbase#Tests - ] +extra_packages = ["sphinx"] def cycle_check(order, dependencies): """ensure no cyclic dependencies""" @@ -229,13 +228,6 @@ def main(args=sys.argv[1:]): call([sys.executable, 'setup.py', 'develop', '--no-deps'], cwd=os.path.join(here, reverse_mapping[package])) - # add the directory of sys.executable to path to aid the correct - # `easy_install` getting called - # https://bugzilla.mozilla.org/show_bug.cgi?id=893878 - os.environ['PATH'] = '%s%s%s' % (os.path.dirname(os.path.abspath(sys.executable)), - os.path.pathsep, - os.environ.get('PATH', '').strip(os.path.pathsep)) - # install non-mozbase dependencies # these need to be installed separately and the --no-deps flag # subsequently used due to a bug in setuptools; see diff --git a/testing/mozbase/test-manifest.ini b/testing/mozbase/test-manifest.ini index 6d9fe680acf..30ac9f65246 100644 --- a/testing/mozbase/test-manifest.ini +++ b/testing/mozbase/test-manifest.ini @@ -13,10 +13,6 @@ [include:mozdevice/tests/manifest.ini] [include:mozfile/tests/manifest.ini] [include:mozhttpd/tests/manifest.ini] -[include:mozinfo/tests/manifest.ini] -[include:mozinstall/tests/manifest.ini] -[include:mozlog/tests/manifest.ini] [include:mozprocess/tests/manifest.ini] [include:mozprofile/tests/manifest.ini] [include:moztest/tests/manifest.ini] -[include:moznetwork/tests/manifest.ini]