Tests fail on master locally due to an error un subprocess #244
Description
Hello,
I cannot manage to run the tests for pytest-django successfully on my system: I get an error related to python's subprocess library every time. I am probably not the only person affected as I could reproduce it in a brand new vanilla Linux container on Ubuntu 14.04 LTS.
Steps to reproduce :
- setup a new computer/LXC running Ubuntu 14.04 or 15.05
apt-get install build-essential git virtualenv python2.7 command-not-found python-minimalgit clone [email protected]:pytest-dev/pytest-django.gitcd pytest-djangomake test
Result
1 test fails, with the exception OSError: [Errno 2] No such file or directory
Full error log
okso@pytestdjango:/shared/contrib/pytest-django-master$ make test
virtualenv .
Running virtualenv with interpreter /usr/bin/python2
New python executable in ./bin/python2
Also creating executable in ./bin/python
Installing setuptools, pip...done.
bin/pip install -Ur requirements.txt
Obtaining file:///shared/contrib/pytest-django-master (from -r requirements.txt (line 1))
Running setup.py (path:/shared/contrib/pytest-django-master/setup.py) egg_info for package from file:///shared/contrib/pytest-django-master
Downloading/unpacking django (from -r requirements.txt (line 2))
Downloading Django-1.8.2-py2.py3-none-any.whl (6.2MB): 6.2MB downloaded
Downloading/unpacking django-configurations (from -r requirements.txt (line 3))
Downloading django_configurations-0.8-py2.py3-none-any.whl
Downloading/unpacking pytest-xdist (from -r requirements.txt (line 4))
Downloading pytest-xdist-1.12.tar.gz
Running setup.py (path:/shared/contrib/pytest-django-master/build/pytest-xdist/setup.py) egg_info for package pytest-xdist
no previously-included directories found matching '.hg'
Downloading/unpacking tox (from -r requirements.txt (line 5))
Downloading tox-2.0.2-py2.py3-none-any.whl
Downloading/unpacking wheel (from -r requirements.txt (line 6))
Downloading wheel-0.24.0-py2.py3-none-any.whl (63kB): 63kB downloaded
Downloading/unpacking twine (from -r requirements.txt (line 7))
Downloading twine-1.5.0-py2.py3-none-any.whl
Downloading/unpacking south (from -r requirements.txt (line 8))
Downloading South-1.0.2.tar.gz (96kB): 96kB downloaded
Running setup.py (path:/shared/contrib/pytest-django-master/build/south/setup.py) egg_info for package south
Downloading/unpacking pytest>=2.5 (from pytest-django==2.8.0->-r requirements.txt (line 1))
Downloading pytest-2.7.1-py2.py3-none-any.whl (126kB): 126kB downloaded
Downloading/unpacking execnet>=1.1 (from pytest-xdist->-r requirements.txt (line 4))
Downloading execnet-1.3.0-py2.py3-none-any.whl
Downloading/unpacking py>=1.4.22 (from pytest-xdist->-r requirements.txt (line 4))
Downloading py-1.4.28-py2.py3-none-any.whl (81kB): 81kB downloaded
Downloading/unpacking pluggy>=0.3.0,<0.4.0 (from tox->-r requirements.txt (line 5))
Downloading pluggy-0.3.0-py2.py3-none-any.whl
Downloading/unpacking virtualenv>=1.11.2 (from tox->-r requirements.txt (line 5))
Downloading virtualenv-13.0.3-py2.py3-none-any.whl (1.7MB): 1.7MB downloaded
Downloading/unpacking requests (from twine->-r requirements.txt (line 7))
Downloading requests-2.7.0-py2.py3-none-any.whl (470kB): 470kB downloaded
Downloading/unpacking pkginfo (from twine->-r requirements.txt (line 7))
Downloading pkginfo-1.2.1.tar.gz
Running setup.py (path:/shared/contrib/pytest-django-master/build/pkginfo/setup.py) egg_info for package pkginfo
Installing collected packages: django, django-configurations, pytest-xdist, tox, wheel, twine, south, pytest, pytest-django, execnet, py, pluggy, virtualenv, requests, pkginfo
Running setup.py install for pytest-xdist
no previously-included directories found matching '.hg'
Running setup.py install for south
Running setup.py develop for pytest-django
Creating /shared/contrib/pytest-django-master/lib/python2.7/site-packages/pytest-django.egg-link (link to .)
Adding pytest-django 2.8.0 to easy-install.pth file
Installed /shared/contrib/pytest-django-master
Running setup.py install for pkginfo
Installing pkginfo script to /shared/contrib/pytest-django-master/bin
Successfully installed django django-configurations pytest-xdist tox wheel twine south pytest pytest-django execnet py pluggy virtualenv requests pkginfo
Cleaning up...
touch bin/py.test
bin/pip install -e .
Obtaining file:///shared/contrib/pytest-django-master
Running setup.py (path:/shared/contrib/pytest-django-master/setup.py) egg_info for package from file:///shared/contrib/pytest-django-master
Requirement already satisfied (use --upgrade to upgrade): pytest>=2.5 in ./lib/python2.7/site-packages (from pytest-django==2.8.0)
Installing collected packages: pytest-django
Running setup.py develop for pytest-django
Creating /shared/contrib/pytest-django-master/lib/python2.7/site-packages/pytest-django.egg-link (link to .)
pytest-django 2.8.0 is already the active version in easy-install.pth
Installed /shared/contrib/pytest-django-master
Successfully installed pytest-django
Cleaning up...
bin/py.test
====================================================== test session starts ======================================================
platform linux2 -- Python 2.7.8 -- py-1.4.28 -- pytest-2.7.1
rootdir: /shared/contrib/pytest-django-master, inifile: setup.cfg
plugins: django, xdist
collected 114 items / 1 skipped
tests/test_database.py ........................
tests/test_db_name.py ...
tests/test_db_setup.py .F....sssssss..
tests/test_django_settings_module.py ............
tests/test_doctest.txt .
tests/test_environment.py ...........
tests/test_fixtures.py ....................s...
tests/test_manage_py_scan.py ...
tests/test_unittest.py ............
tests/test_urls.py ..
tests/test_without_django_loaded.py .......
=========================================================== FAILURES ============================================================
_________________________________________________________ test_db_reuse _________________________________________________________
django_testdir = <TmpTestdir local('/tmp/pytest-0/testdir/test_db_reuse0')>
def test_db_reuse(django_testdir):
"""
Test the re-use db functionality. This test requires a PostgreSQL server
to be available and the environment variables PG_HOST, PG_DB, PG_USER to
be defined.
"""
skip_if_sqlite_in_memory()
django_testdir.create_test_module('''
import pytest
from .app.models import Item
@pytest.mark.django_db
def test_db_can_be_accessed():
assert Item.objects.count() == 0
''')
# Use --create-db on the first run to make sure we are not just re-using a
# database from another test run
drop_database()
assert not db_exists()
# Do not pass in --create-db to make sure it is created when it
# does not exist
result_first = django_testdir.runpytest('-v', '--reuse-db')
assert result_first.ret == 0
result_first.stdout.fnmatch_lines([
"*test_db_can_be_accessed PASSED*",
])
> assert not mark_exists()
/shared/contrib/pytest-django-master/tests/test_db_setup.py:65:
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
/shared/contrib/pytest-django-master/pytest_django_test/db_helpers.py:174: in mark_exists
r = run_cmd('sqlite3', TEST_DB_NAME, 'SELECT 1 FROM mark_table')
/shared/contrib/pytest-django-master/pytest_django_test/db_helpers.py:35: in run_cmd
r = subprocess.Popen(args, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
/usr/lib/python2.7/subprocess.py:710: in __init__
errread, errwrite)
_ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
self = <subprocess.Popen object at 0x7fe41b705f90>
args = ['sqlite3', 'test_DBNAME_pytest_django_db_db_test', 'SELECT 1 FROM mark_table'], executable = 'sqlite3', preexec_fn = None
close_fds = False, cwd = None, env = None, universal_newlines = False, startupinfo = None, creationflags = 0, shell = False
to_close = set([9, 12]), p2cread = None, p2cwrite = None, c2pread = 9, c2pwrite = 11, errread = 12, errwrite = 13
def _execute_child(self, args, executable, preexec_fn, close_fds,
cwd, env, universal_newlines,
startupinfo, creationflags, shell, to_close,
p2cread, p2cwrite,
c2pread, c2pwrite,
errread, errwrite):
"""Execute program (POSIX version)"""
if isinstance(args, types.StringTypes):
args = [args]
else:
args = list(args)
if shell:
args = ["/bin/sh", "-c"] + args
if executable:
args[0] = executable
if executable is None:
executable = args[0]
def _close_in_parent(fd):
os.close(fd)
to_close.remove(fd)
# For transferring possible exec failure from child to parent
# The first char specifies the exception type: 0 means
# OSError, 1 means some other error.
errpipe_read, errpipe_write = self.pipe_cloexec()
try:
try:
gc_was_enabled = gc.isenabled()
# Disable gc to avoid bug where gc -> file_dealloc ->
# write to stderr -> hang. http://bugs.python.org/issue1336
gc.disable()
try:
self.pid = os.fork()
except:
if gc_was_enabled:
gc.enable()
raise
self._child_created = True
if self.pid == 0:
# Child
try:
# Close parent's pipe ends
if p2cwrite is not None:
os.close(p2cwrite)
if c2pread is not None:
os.close(c2pread)
if errread is not None:
os.close(errread)
os.close(errpipe_read)
# When duping fds, if there arises a situation
# where one of the fds is either 0, 1 or 2, it
# is possible that it is overwritten (#12607).
if c2pwrite == 0:
c2pwrite = os.dup(c2pwrite)
if errwrite == 0 or errwrite == 1:
errwrite = os.dup(errwrite)
# Dup fds for child
def _dup2(a, b):
# dup2() removes the CLOEXEC flag but
# we must do it ourselves if dup2()
# would be a no-op (issue #10806).
if a == b:
self._set_cloexec_flag(a, False)
elif a is not None:
os.dup2(a, b)
_dup2(p2cread, 0)
_dup2(c2pwrite, 1)
_dup2(errwrite, 2)
# Close pipe fds. Make sure we don't close the
# same fd more than once, or standard fds.
closed = { None }
for fd in [p2cread, c2pwrite, errwrite]:
if fd not in closed and fd > 2:
os.close(fd)
closed.add(fd)
if cwd is not None:
os.chdir(cwd)
if preexec_fn:
preexec_fn()
# Close all other fds, if asked for - after
# preexec_fn(), which may open FDs.
if close_fds:
self._close_fds(but=errpipe_write)
if env is None:
os.execvp(executable, args)
else:
os.execvpe(executable, args, env)
except:
exc_type, exc_value, tb = sys.exc_info()
# Save the traceback and attach it to the exception object
exc_lines = traceback.format_exception(exc_type,
exc_value,
tb)
exc_value.child_traceback = ''.join(exc_lines)
os.write(errpipe_write, pickle.dumps(exc_value))
# This exitcode won't be reported to applications, so it
# really doesn't matter what we return.
os._exit(255)
# Parent
if gc_was_enabled:
gc.enable()
finally:
# be sure the FD is closed no matter what
os.close(errpipe_write)
# Wait for exec to fail or succeed; possibly raising exception
# Exception limited to 1M
data = _eintr_retry_call(os.read, errpipe_read, 1048576)
finally:
if p2cread is not None and p2cwrite is not None:
_close_in_parent(p2cread)
if c2pwrite is not None and c2pread is not None:
_close_in_parent(c2pwrite)
if errwrite is not None and errread is not None:
_close_in_parent(errwrite)
# be sure the FD is closed no matter what
os.close(errpipe_read)
if data != "":
try:
_eintr_retry_call(os.waitpid, self.pid, 0)
except OSError as e:
if e.errno != errno.ECHILD:
raise
child_exception = pickle.loads(data)
> raise child_exception
E OSError: [Errno 2] No such file or directory
/usr/lib/python2.7/subprocess.py:1335: OSError
----------------------------------------------------- Captured stdout call ------------------------------------------------------
running ['/shared/contrib/pytest-django-master/bin/python2', '/shared/contrib/pytest-django-master/local/lib/python2.7/site-packages/pytest.py', '--basetemp=/tmp/pytest-0/testdir/test_db_reuse0/runpytest-0', '-v', '--reuse-db'] curdir= /tmp/pytest-0/testdir/test_db_reuse0
============================= test session starts ==============================
platform linux2 -- Python 2.7.8 -- py-1.4.28 -- pytest-2.7.1 -- /shared/contrib/pytest-django-master/bin/python2
rootdir: /tmp/pytest-0/testdir/test_db_reuse0, inifile:
plugins: xdist, django
collecting ... collected 1 items
tpkg/test_the_test.py::test_db_can_be_accessed PASSED
=========================== 1 passed in 0.11 seconds ===========================
----------------------------------------------------- Captured stderr call ------------------------------------------------------
nomatch: '*test_db_can_be_accessed PASSED*'
and: u'============================= test session starts =============================='
and: u'platform linux2 -- Python 2.7.8 -- py-1.4.28 -- pytest-2.7.1 -- /shared/contrib/pytest-django-master/bin/python2'
and: u'rootdir: /tmp/pytest-0/testdir/test_db_reuse0, inifile: '
and: u'plugins: xdist, django'
and: u'collecting ... collected 1 items'
and: u''
fnmatch: '*test_db_can_be_accessed PASSED*'
with: u'tpkg/test_the_test.py::test_db_can_be_accessed PASSED'
==================================================== short test summary info ====================================================
FAIL tests/test_db_setup.py::test_db_reuse
SKIP [1] /shared/contrib/pytest-django-master/local/lib/python2.7/site-packages/_pytest/skipping.py:140: Django > 1.3 required
SKIP [7] /shared/contrib/pytest-django-master/local/lib/python2.7/site-packages/_pytest/skipping.py:140: South does not support Django 1.7+
SKIP [1] /shared/contrib/pytest-django-master/tests/test_django_configurations.py:14: This version of django-configurations is incompatible with Django: https://github.com/jezdez/django-configurations/issues/65
======================================= 1 failed, 105 passed, 9 skipped in 24.96 seconds ========================================
Makefile:8: recipe for target 'test' failed
make: *** [test] Error 1