Bug 1247758 - README needs to updated with information on running with ./mach - r=maja_zf,SingingTree

MozReview-Commit-ID: K30pgZQMQYo
This commit is contained in:
Syd Polk 2016-02-15 13:31:02 -06:00
parent d8b434d3d6
commit 2800ae8755

View File

@ -6,7 +6,23 @@ external-media-tests
Setup
-----
Normally, you get this source by cloning a firefox repo such as mozilla-central. The path to these tests would be in <mozilla-central>/dom/media/test/external, and these instuctions refer to this path as '$PROJECT_HOME'.
Normally, you get this source by cloning a firefox repo such as mozilla-central. The path to these tests would be in <mozilla-central>/dom/media/test/external, and these instructions refer to this path as '$PROJECT_HOME'.
Running from a build
--------------------
If you have built Firefox using ./mach build from a source tree such as mozilla-central, you can run the following command:
'''sh
$ ./mach external-media-tests
'''
You can pass any of the test options on this command line. They are listed below.
Running with an installer and a tests payload
---------------------------------------------
If you are testing a version of Firefox that you have not built, you must setup a virtualenv to run tests from. You will need a path to the installer or binary of Firefox.
* Create a virtualenv called `foo`.
@ -27,30 +43,55 @@ Now `external-media-tests` should be a recognized command. Try `external-media-t
Running the Tests
-----------------
In the examples below, `$FF_PATH` is a path to a recent Firefox binary.
In the examples below, `$FF_PATH` is a path to a recent Firefox binary. If you are running from a source build, the commands below should be invoked with:
'''sh
./mach external-media-tests
'''
If you are running with a virtualenv, you will need to run like this:
'''sh
external-media-tests --binary $FF_PATH
'''
or
'''sh
external-media-tests --installer $FF_INSTALLER_PATH
'''
or
'''sh
external-media-tests --installer-url <url to installer package>
'''
The following examples assume that you will use of these command lines instead of $EXTERNAL-MEDIA-TESTS.
This runs all the tests listed in `$PROJECT_HOME/external_media_tests/manifest.ini`:
```sh
$ external-media-tests --binary $FF_PATH
$ $EXTERNAL-MEDIA-TESTS
```
You can also run all the tests at a particular path:
```sh
$ external-media-tests --binary $FF_PATH some/path/foo
$ $EXTERNAL-MEDIA-TESTS some/path/foo
```
Or you can run the tests that are listed in a manifest file of your choice.
```sh
$ external-media-tests --binary $FF_PATH some/other/path/manifest.ini
$ $EXTERNAL-MEDIA-TESTS some/other/path/manifest.ini
```
By default, the urls listed in `external_media_tests/urls/default.ini` are used for the tests, but you can also supply your own ini file of urls:
```sh
$ external-media-tests --binary $FF_PATH --urls some/other/path/my_urls.ini
$ $EXTERNAL-MEDIA-TESTS --urls some/other/path/my_urls.ini
```
### Running EME tests
@ -58,7 +99,7 @@ By default, the urls listed in `external_media_tests/urls/default.ini` are used
In order to run EME tests, you must use a Firefox profile that has a signed plugin-container.exe and voucher.bin. With Netflix, this will be created when you log in and save the credentials. You must also use a custom .ini file for urls to the provider's content and indicate which test to run, like above. Ex:
```sh
$ external-media-tests --binary $FF_PATH some/path/tests.ini --profile custom_profile --urls some/path/provider-urls.ini
$ $EXTERNAL-MEDIA-TESTS some/path/tests.ini --profile custom_profile --urls some/path/provider-urls.ini
```
@ -66,7 +107,7 @@ In order to run EME tests, you must use a Firefox profile that has a signed plug
What if Firefox crashes during a test run? You want to know why! To report useful crash data, the test runner needs access to a "minidump_stackwalk" binary and a "symbols.zip" file.
1. Download a `minidump_stackwalk` binary for your platform (save it whereever). Get it from http://hg.mozilla.org/build/tools/file/tip/breakpad/.
1. Download a `minidump_stackwalk` binary for your platform (save it wherever). Get it from http://hg.mozilla.org/build/tools/file/tip/breakpad/.
2. Make `minidump_stackwalk` executable
```sh
@ -84,7 +125,7 @@ What if Firefox crashes during a test run? You want to know why! To report usefu
5. Run the tests with a `--symbols-path` flag
```sh
$ external-media-tests --binary $FF_PATH --symbols-path path/to/example/firefox-38.0a2.en-US.win32.crashreporter-symbols
$ $EXTERNAL-MEDIA-TESTS --symbols-path path/to/example/firefox-38.0a2.en-US.win32.crashreporter-symbols
```
To check whether the above setup is working for you, trigger a (silly) Firefox crash while the tests are running. One way to do this is with the [crashme add-on](https://github.com/luser/crashme) -- you can add it to Firefox even while the tests are running. Another way on Linux and Mac OS systems:
@ -124,17 +165,15 @@ Crash address: 0x104616900
1. Download the browsermob proxy zip file from http://bmp.lightbody.net/. The most current version as of this writing is browsermob-proxy-2.1.0-beta-2-bin.zip.
2. Unpack the .zip file.
3. Verify that you can launch browsermobproxy on your machine by running \<browsermob\>/bin/browsermob-proxy on your machine. I had to do a lot of work to install and use a java that browsermobproxy would like.
3. Verify that you can launch browsermobproxy on your machine by running \<browsermob\>/bin/browsermob-proxy (or browsermob-proxy.bat on Windows) on your machine. I had to do a lot of work to install and use a java that browsermobproxy would like.
4. Import the certificate into your Firefox profile. Select Preferences->Advanced->Certificates->View Certificates->Import... Navigate to <browsermob>/ssl-support and select cybervilliansCA.cer. Select all of the checkboxes.
5. Tell marionette where browsermobproxy is and what port to start it on. Add the following command-line parameters to your firefox-media-tests command line:
5. Tell marionette where browsermobproxy is and what port to start it on. Add the following command-line parameters to your external-media-tests command line:
<pre><code>
--browsermob-script <browsermob>/bin/browsermob-proxy --browsermob-port 999 --profile <your saved profile>
</code></pre>
On Windows, use browsermob-proxy.bat.
You can then call browsermob to shape the network. You can find an example in firefox_media_tests/playback/test_playback_limiting_bandwidth.py. Another example can be found at https://dxr.mozilla.org/mozilla-central/source/testing/marionette/client/marionette/tests/unit/test_browsermobproxy.py.
You can then call browsermob to shape the network. You can find an example in external_media_tests/playback/test_playback_limiting_bandwidth.py. Another example can be found at https://dxr.mozilla.org/mozilla-central/source/testing/marionette/client/marionette/tests/unit/test_browsermobproxy.py.
### A warning about video URLs
The ini files in `external_media_tests/urls` may contain URLs pulled from Firefox crash or bug data. Automated tests don't care about video content, but you might: visit these at your own risk and be aware that they may be NSFW. We do not intend to ever moderate or filter these URLs.