Adding support for passing a subset of packages to run_unit_tests.

Also

- Adding instruction to `CONTRIBUTING` about how to run
  a subset of tests for packages of your choosing
- Moving the "Adding Features" section to the top of the
  `CONTRIBUTING` doc
- Making tiny edits to `CONTRIBUTING` doc, e.g. using
  a "repo" hyperlink for the repository and updating the
  link to the `tox` docs

Author: dhermes

CONTRIBUTING.rst

@@ -8,16 +8,29 @@ Contributing
 
 Here are some guidelines for hacking on ``google-cloud-python``.
 
+Adding Features
+---------------
+
+In order to add a feature to ``google-cloud-python``:
+
+- The feature must be documented in both the API and narrative
+  documentation (in ``docs/``).
+
+- The feature must work fully on the following CPython versions:  2.7,
+  3.4, and 3.5 on both UNIX and Windows.
+
+- The feature must not add unnecessary dependencies (where
+  "unnecessary" is of course subjective, but new dependencies should
+  be discussed).
+
 Using a Development Checkout
 ----------------------------
 
-You'll have to create a development environment to hack on ``google-cloud-python``,
-using a Git checkout:
+You'll have to create a development environment to hack on
+``google-cloud-python``, using a Git checkout:
 
-- While logged into your GitHub account, navigate to the ``google-cloud-python`` repo
-  on GitHub.
-
-  https://github.com/GoogleCloudPlatform/google-cloud-python
+- While logged into your GitHub account, navigate to the
+  ``google-cloud-python`` `repo`_ on GitHub.
 
 - Fork and clone the ``google-cloud-python`` repository to your GitHub account by
   clicking the "Fork" button.
@@ -42,6 +55,8 @@ repo, from which you can submit a pull request.
 To work on the codebase and run the tests, we recommend using ``tox``,
 but you can also use a ``virtualenv`` of your own creation.
 
+.. _repo: https://github.com/GoogleCloudPlatform/google-cloud-python
+
 Using a custom ``virtualenv``
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -96,6 +111,19 @@ Using ``tox``
   by the ``tox`` environment, so if you make changes, you'll need
   to again ``--recreate`` the environment.
 
+- To run unit tests on a restricted set of packages::
+
+    $ tox -e py27 -- core datastore
+
+  Alternatively, you can just navigate directly to the package you are
+  currently developing and run tests there::
+
+    $ export GIT_ROOT=$(pwd)
+    $ cd ${GIT_ROOT}/core/
+    $ tox -e py27
+    $ cd ${GIT_ROOT}/datastore/
+    $ tox -e py27
+
 Note on Editable Installs / Develop Mode
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -124,21 +152,6 @@ On Debian/Ubuntu::
 
   $ sudo apt-get install python-dev
 
-Adding Features
----------------
-
-In order to add a feature to ``google-cloud-python``:
-
-- The feature must be documented in both the API and narrative
-  documentation (in ``docs/``).
-
-- The feature must work fully on the following CPython versions:  2.7,
-  3.4, and 3.5 on both UNIX and Windows.
-
-- The feature must not add unnecessary dependencies (where
-  "unnecessary" is of course subjective, but new dependencies should
-  be discussed).
-
 Coding Style
 ------------
 
@@ -170,21 +183,25 @@ Running Tests
 
 - To run all tests for ``google-cloud-python`` on a single Python version, run
   ``py.test`` from your development virtualenv (See
-  *Using a Development Checkout* above).
+  `Using a Development Checkout`_ above).
+
+.. _Using a Development Checkout: #using-a-development-checkout
 
 - To run the full set of ``google-cloud-python`` tests on all platforms, install
-  ``tox`` (https://testrun.org/tox/) into a system Python.  The ``tox`` console
-  script will be installed into the scripts location for that Python.  While
-  ``cd``'ed to the ``google-cloud-python`` checkout root directory (it contains
-  ``tox.ini``), invoke the ``tox`` console script.  This will read the
-  ``tox.ini`` file and execute the tests on multiple Python versions and
-  platforms; while it runs, it creates a virtualenv for each version/platform
-  combination.  For example::
+  ``tox`` (https://tox.readthedocs.io/en/latest/) into a system Python.  The
+  ``tox`` console script will be installed into the scripts location for that
+  Python.  While ``cd``'-ed to the ``google-cloud-python`` checkout root
+  directory (it contains ``tox.ini``), invoke the ``tox`` console script.
+  This will read the ``tox.ini`` file and execute the tests on multiple
+  Python versions and platforms; while it runs, it creates a ``virtualenv`` for
+  each version/platform combination.  For example::
 
    $ sudo --set-home /usr/bin/pip install tox
    $ cd ${HOME}/hack-on-google-cloud-python/
    $ /usr/bin/tox
 
+.. _Using a Development Checkout: #using-a-development-checkout
+
 Running System Tests
 --------------------
 

scripts/run_unit_tests.py

@@ -41,6 +41,7 @@
     (3, 4): 'py34',
     (3, 5): 'py35',
 }
+UNSET_SENTINEL = object()  # Sentinel for argparser
 
 
 def check_output(*args):
@@ -82,6 +83,46 @@ def get_package_directories():
     return result
 
 
+def verify_packages(subset, all_packages):
+    """Verify that a subset of packages are among all packages.
+
+    :type subset: list
+    :param subset: List of a subset of package names.
+
+    :type all_packages: list
+    :param all_packages: List of all package names.
+
+    :raises: :class:`~exceptions.ValueError` if there are unknown packages
+             in ``subset``
+    """
+    left_out = set(subset) - set(all_packages)
+    if left_out:
+        raise ValueError('Unknown packages',
+                         sorted(left_out))
+
+def get_test_packages():
+    """Get a list of packages which need tests run.
+
+    Filters the package list in the following order:
+
+    * Check command line for packages passed in as positional arguments
+    * Just use all packages
+
+    :rtype: list
+    :returns: A list of all package directories where tests
+              need be run.
+    """
+    all_packages = get_package_directories()
+
+    parser = get_parser()
+    args = parser.parse_args()
+    if args.packages is not UNSET_SENTINEL:
+        verify_packages(args.packages, all_packages)
+        return sorted(args.packages)
+    else:
+        return all_packages
+
+
 def run_package(package, tox_env):
     """Run tox environment for a given package.
 
@@ -116,6 +157,9 @@ def get_parser():
     parser.add_argument(
         '--tox-env', dest='tox_env',
         help='The tox environment(s) to run in sub-packages.')
+    packages_help = 'Optional list of sub-packages to be tested.'
+    parser.add_argument('packages', nargs='*',
+                        default=UNSET_SENTINEL, help=packages_help)
     return parser
 
 
@@ -164,7 +208,7 @@ def get_tox_env():
 
 def main():
     """Run all the unit tests that need to be run."""
-    packages_to_run = get_package_directories()
+    packages_to_run = get_test_packages()
     if not packages_to_run:
         print('No tests to run.')
         return

tox.ini

@@ -114,7 +114,7 @@ covercmd =
 
 [testenv]
 commands =
-    python {toxinidir}/scripts/run_unit_tests.py
+    python {toxinidir}/scripts/run_unit_tests.py {posargs}
 deps =
 skip_install =
     py27: True
@@ -123,7 +123,8 @@ skip_install =
     isolated-cover: True
 
 [testenv:isolated-cover]
-commands = {[testenv]commands} --tox-env cover
+commands =
+    python {toxinidir}/scripts/run_unit_tests.py {posargs} --tox-env cover
 
 [testenv:py27-pandas]
 basepython =