Merge pull request #2475 from dhermes/collapse-travis-config-2

Adding flexibility to how run_unit_tests determines what to run.

Author: dhermes

.travis.yml

@@ -5,11 +5,11 @@ install:
   - pip install --upgrade pip tox
 
 script:
-  - tox -e py27
-  - tox -e py34
+  - python2.7 scripts/run_unit_tests.py
+  - python3.4 scripts/run_unit_tests.py
   - tox -e lint
   - tox -e cover
-  - tox -e isolated-cover
+  - python scripts/run_unit_tests.py --tox-env cover
   - tox -e system-tests
   - tox -e system-tests3
   - scripts/update_docs.sh

CONTRIBUTING.rst

@@ -8,16 +8,29 @@ Contributing
 
 Here are some guidelines for hacking on ``google-cloud-python``.
 
+Adding Features
+---------------
+
+In order to add a feature to ``google-cloud-python``:
+
+- The feature must be documented in both the API and narrative
+  documentation (in ``docs/``).
+
+- The feature must work fully on the following CPython versions:  2.7,
+  3.4, and 3.5 on both UNIX and Windows.
+
+- The feature must not add unnecessary dependencies (where
+  "unnecessary" is of course subjective, but new dependencies should
+  be discussed).
+
 Using a Development Checkout
 ----------------------------
 
-You'll have to create a development environment to hack on ``google-cloud-python``,
-using a Git checkout:
+You'll have to create a development environment to hack on
+``google-cloud-python``, using a Git checkout:
 
-- While logged into your GitHub account, navigate to the ``google-cloud-python`` repo
-  on GitHub.
-
-  https://github.com/GoogleCloudPlatform/google-cloud-python
+- While logged into your GitHub account, navigate to the
+  ``google-cloud-python`` `repo`_ on GitHub.
 
 - Fork and clone the ``google-cloud-python`` repository to your GitHub account by
   clicking the "Fork" button.
@@ -42,6 +55,8 @@ repo, from which you can submit a pull request.
 To work on the codebase and run the tests, we recommend using ``tox``,
 but you can also use a ``virtualenv`` of your own creation.
 
+.. _repo: https://github.com/GoogleCloudPlatform/google-cloud-python
+
 Using a custom ``virtualenv``
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -96,6 +111,19 @@ Using ``tox``
   by the ``tox`` environment, so if you make changes, you'll need
   to again ``--recreate`` the environment.
 
+- To run unit tests on a restricted set of packages::
+
+    $ tox -e py27 -- core datastore
+
+  Alternatively, you can just navigate directly to the package you are
+  currently developing and run tests there::
+
+    $ export GIT_ROOT=$(pwd)
+    $ cd ${GIT_ROOT}/core/
+    $ tox -e py27
+    $ cd ${GIT_ROOT}/datastore/
+    $ tox -e py27
+
 Note on Editable Installs / Develop Mode
 ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
 
@@ -124,21 +152,6 @@ On Debian/Ubuntu::
 
   $ sudo apt-get install python-dev
 
-Adding Features
----------------
-
-In order to add a feature to ``google-cloud-python``:
-
-- The feature must be documented in both the API and narrative
-  documentation (in ``docs/``).
-
-- The feature must work fully on the following CPython versions:  2.7,
-  3.4, and 3.5 on both UNIX and Windows.
-
-- The feature must not add unnecessary dependencies (where
-  "unnecessary" is of course subjective, but new dependencies should
-  be discussed).
-
 Coding Style
 ------------
 
@@ -170,21 +183,25 @@ Running Tests
 
 - To run all tests for ``google-cloud-python`` on a single Python version, run
   ``py.test`` from your development virtualenv (See
-  *Using a Development Checkout* above).
+  `Using a Development Checkout`_ above).
+
+.. _Using a Development Checkout: #using-a-development-checkout
 
 - To run the full set of ``google-cloud-python`` tests on all platforms, install
-  ``tox`` (https://testrun.org/tox/) into a system Python.  The ``tox`` console
-  script will be installed into the scripts location for that Python.  While
-  ``cd``'ed to the ``google-cloud-python`` checkout root directory (it contains
-  ``tox.ini``), invoke the ``tox`` console script.  This will read the
-  ``tox.ini`` file and execute the tests on multiple Python versions and
-  platforms; while it runs, it creates a virtualenv for each version/platform
-  combination.  For example::
+  ``tox`` (https://tox.readthedocs.io/en/latest/) into a system Python.  The
+  ``tox`` console script will be installed into the scripts location for that
+  Python.  While ``cd``'-ed to the ``google-cloud-python`` checkout root
+  directory (it contains ``tox.ini``), invoke the ``tox`` console script.
+  This will read the ``tox.ini`` file and execute the tests on multiple
+  Python versions and platforms; while it runs, it creates a ``virtualenv`` for
+  each version/platform combination.  For example::
 
    $ sudo --set-home /usr/bin/pip install tox
    $ cd ${HOME}/hack-on-google-cloud-python/
    $ /usr/bin/tox
 
+.. _Using a Development Checkout: #using-a-development-checkout
+
 Running System Tests
 --------------------
 

scripts/run_unit_tests.py

@@ -35,9 +35,13 @@
     'scripts',
     'system_tests',
 )
-ENV_REMAP = {
-    'isolated-cover': 'cover',
+TOX_ENV_VAR = 'TOXENV'
+ACCEPTED_VERSIONS = {
+    (2, 7): 'py27',
+    (3, 4): 'py34',
+    (3, 5): 'py35',
 }
+UNSET_SENTINEL = object()  # Sentinel for argparser
 
 
 def check_output(*args):
@@ -79,6 +83,47 @@ def get_package_directories():
     return result
 
 
+def verify_packages(subset, all_packages):
+    """Verify that a subset of packages are among all packages.
+
+    :type subset: list
+    :param subset: List of a subset of package names.
+
+    :type all_packages: list
+    :param all_packages: List of all package names.
+
+    :raises: :class:`~exceptions.ValueError` if there are unknown packages
+             in ``subset``
+    """
+    left_out = set(subset) - set(all_packages)
+    if left_out:
+        raise ValueError('Unknown packages',
+                         sorted(left_out))
+
+
+def get_test_packages():
+    """Get a list of packages which need tests run.
+
+    Filters the package list in the following order:
+
+    * Check command line for packages passed in as positional arguments
+    * Just use all packages
+
+    :rtype: list
+    :returns: A list of all package directories where tests
+              need be run.
+    """
+    all_packages = get_package_directories()
+
+    parser = get_parser()
+    args = parser.parse_args()
+    if args.packages is not UNSET_SENTINEL:
+        verify_packages(args.packages, all_packages)
+        return sorted(args.packages)
+    else:
+        return all_packages
+
+
 def run_package(package, tox_env):
     """Run tox environment for a given package.
 
@@ -111,28 +156,60 @@ def get_parser():
     description = 'Run tox environment(s) in all sub-packages.'
     parser = argparse.ArgumentParser(description=description)
     parser.add_argument(
-        '--tox-env-dir', dest='tox_env_dir',
-        help='The current tox environment directory.')
+        '--tox-env', dest='tox_env',
+        help='The tox environment(s) to run in sub-packages.')
+    packages_help = 'Optional list of sub-packages to be tested.'
+    parser.add_argument('packages', nargs='*',
+                        default=UNSET_SENTINEL, help=packages_help)
     return parser
 
 
+def get_tox_env_from_version():
+    """Get ``tox`` environment from the current Python version.
+
+    :rtype: str
+    :returns: The current ``tox`` environment to be used, e.g. ``"py27"``.
+    :raises: :class:`EnvironmentError` if the first two options
+             don't yield any value and the current version of
+             Python is not in ``ACCEPTED_VERSIONS``.
+    """
+    version_info = sys.version_info[:2]
+    try:
+        return ACCEPTED_VERSIONS[version_info]
+    except KeyError:
+        raise EnvironmentError(
+            'Invalid Python version', version_info,
+            'Accepted versions are',
+            sorted(ACCEPTED_VERSIONS.keys()))
+
+
 def get_tox_env():
     """Get the environment to be used with ``tox``.
 
+    Tries to infer the ``tox`` environment in the following order
+
+    * From the ``--tox-env`` command line flag
+    * From the ``TOXENV`` environment variable
+    * From the version of the current running Python
+
     :rtype: str
     :returns: The current ``tox`` environment to be used, e.g. ``"py27"``.
     """
     parser = get_parser()
     args = parser.parse_args()
-    env_dir = args.tox_env_dir
-    _, tox_env = os.path.split(env_dir)
-    tox_env = ENV_REMAP.get(tox_env, tox_env)
+    if args.tox_env is not None:
+        tox_env = args.tox_env
+    elif TOX_ENV_VAR in os.environ:
+        tox_env = os.environ[TOX_ENV_VAR]
+    else:
+        tox_env = get_tox_env_from_version()
+
     return tox_env
 
 
 def main():
     """Run all the unit tests that need to be run."""
-    packages_to_run = get_package_directories()
+    packages_to_run = get_test_packages()
     if not packages_to_run:
         print('No tests to run.')
         return
@@ -150,7 +227,7 @@ def main():
             msg_parts.append('- ' + package)
         msg = '\n'.join(msg_parts)
         print(msg, file=sys.stderr)
-        sys.exit(len(failed_packages))
+        sys.exit(1)
 
 
 if __name__ == '__main__':

tox.ini

@@ -1,6 +1,6 @@
 [tox]
 envlist =
-    cover,docs,lint
+    py27,py34,py35,cover,docs,lint
 
 [testing]
 deps =
@@ -114,7 +114,7 @@ covercmd =
 
 [testenv]
 commands =
-    python {toxinidir}/scripts/run_unit_tests.py --tox-env-dir {envdir}
+    python {toxinidir}/scripts/run_unit_tests.py {posargs}
 deps =
 skip_install =
     py27: True
@@ -123,7 +123,8 @@ skip_install =
     isolated-cover: True
 
 [testenv:isolated-cover]
-commands = {[testenv]commands}
+commands =
+    python {toxinidir}/scripts/run_unit_tests.py {posargs} --tox-env cover
 
 [testenv:cover]
 basepython =