[BUG] filterwarnings does not filter out DeprecationWarnings from 3rd party libraries

[ghost]

I've encountered some strange behaviour when trying to suppress warnings in 3rd party libraries. It might be related to this numpy issue.

Here's a minimal example:

pytest.ini:

[pytest]
filterwarnings =
    ignore::DeprecationWarning

And a simple test test_foo.py:

import mlflow     # This seems to "break" the filterwarnings
import nbconvert  # An example lib that raises two DeprecationWarnings

def test_bar():
    assert True

When I run pytest, I get an unexpected warning:

============ test session starts ==================
platform linux -- Python 3.7.3, pytest-5.3.0, py-1.8.0, pluggy-0.12.0
rootdir: /hpcdata/a_username/foo, inifile: pytest.ini
plugins: arraydiff-0.3, doctestplus-0.3.0, openfiles-0.4.0, remotedata-0.3.1, pyfakefs-3.6.1
collected 1 item                                                                                                                                                                                         

test_foo.py .                                                                                                                                                                                      [100%]

============== warnings summary ==================
/....lib/python3.7/site-packages/nbformat/notebooknode.py:4
  /.../lib/python3.7/site-packages/nbformat/notebooknode.py:4: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
    from collections import Mapping

/.../lib/python3.7/site-packages/jinja2/utils.py:485
  /.../lib/python3.7/site-packages/jinja2/utils.py:485: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
    from collections import MutableMapping

-- Docs: https://docs.pytest.org/en/latest/warnings.html
============ 1 passed, 2 warnings in 2.64s ============

Output of pip list:

Package                            Version   
---------------------------------- ----------
alabaster                          0.7.12    
alembic                            1.1.0     
anaconda-client                    1.7.2     
anaconda-project                   0.8.3     
asn1crypto                         0.24.0    
astroid                            2.2.5     
astropy                            3.2.1     
atomicwrites                       1.3.0     
attrs                              19.1.0    
Babel                              2.7.0     
backcall                           0.1.0     
backports.os                       0.1.1     
backports.shutil-get-terminal-size 1.0.0     
bcrypt                             3.1.7     
beautifulsoup4                     4.8.0     
bitarray                           1.0.1     
bkcharts                           0.2       
bleach                             3.1.0     
bokeh                              1.3.4     
boto                               2.49.0    
Bottleneck                         1.2.1     
certifi                            2019.11.28
cffi                               1.12.3    
chardet                            3.0.4     
Click                              7.0       
cloudpickle                        1.2.1     
clyent                             1.2.2     
colorama                           0.4.1     
configparser                       3.8.1     
contextlib2                        0.5.5     
cryptography                       2.7       
cycler                             0.10.0    
Cython                             0.29.13   
cytoolz                            0.10.0    
dask                               2.2.0     
dask-jobqueue                      0.6.3     
databricks-cli                     0.8.7     
decorator                          4.4.0     
defusedxml                         0.5.0     
distributed                        2.2.0     
docker                             4.0.2     
docrep                             0.2.7     
docutils                           0.15.2    
entrypoints                        0.3       
et-xmlfile                         1.0.1     
fastcache                          1.1.0     
filelock                           3.0.10    
Flask                              1.1.1     
fsspec                             0.4.1     
gevent                             1.4.0     
gitdb2                             2.0.5     
GitPython                          3.0.2     
glob2                              0.7       
gmpy2                              2.1.0b1   
gorilla                            0.3.0     
graphviz                           0.12      
greenlet                           0.4.15    
gunicorn                           19.9.0    
h5py                               2.9.0     
heapdict                           1.0.0     
holoviews                          1.12.4    
html5lib                           1.0.1     
idna                               2.8       
imageio                            2.5.0     
imagesize                          1.1.0     
importlib-metadata                 0.18      
ipydatawidgets                     4.0.1     
ipykernel                          5.1.2     
ipympl                             0.3.3     
ipyscales                          0.3.0     
ipython                            7.7.0     
ipython-genutils                   0.2.0     
ipyvolume                          0.5.2     
ipywebrtc                          0.5.0     
ipywidgets                         7.5.1     
isort                              4.3.21    
itsdangerous                       1.1.0     
jdcal                              1.4.1     
jedi                               0.14.1    
jeepney                            0.4       
Jinja2                             2.10.1    
joblib                             0.13.2    
json5                              0.8.5     
jsonschema                         3.0.2     
jupyter-client                     5.3.1     
jupyter-console                    6.0.0     
jupyter-core                       4.4.0     
jupyterlab                         1.0.4     
jupyterlab-server                  1.0.0     
keyring                            19.0.2    
kiwisolver                         1.1.0     
lasio                              0.24.1    
lazy-object-proxy                  1.4.1     
libarchive-c                       2.8       
lief                               0.9.0     
llvmlite                           0.29.0    
locket                             0.2.0     
lxml                               4.4.0     
Mako                               1.1.0     
Markdown                           3.1.1     
MarkupSafe                         1.1.1     
matplotlib                         3.1.1     
mccabe                             0.6.1     
mistune                            0.8.4     
mkl-fft                            1.0.14    
mkl-random                         1.0.4     
mkl-service                        2.1.0     
mlflow                             1.2.0     
mock                               3.0.5     
more-itertools                     7.2.0     
mpmath                             1.1.0     
msgpack                            0.6.1     
multipledispatch                   0.6.0     
mysql-connector-python             8.0.18    
nbconvert                          5.6.0     
nbformat                           4.4.0     
networkx                           2.3       
nltk                               3.4.4     
nose                               1.3.7     
notebook                           6.0.0     
numba                              0.45.1    
numexpr                            2.6.9     
numpy                              1.16.4    
numpydoc                           0.9.1     
olefile                            0.46      
openpyxl                           2.6.2     
packaging                          19.0      
pandas                             0.25.0    
pandocfilters                      1.4.2     
panel                              0.6.2     
param                              1.9.1     
paramiko                           2.6.0     
parso                              0.5.1     
partd                              1.0.0     
path.py                            12.0.1    
pathlib2                           2.3.4     
patsy                              0.5.1     
pep8                               1.7.1     
pexpect                            4.7.0     
pickleshare                        0.7.5     
Pillow                             6.1.0     
pip                                19.2.1    
pkginfo                            1.5.0.1   
plotly                             4.1.1     
pluggy                             0.12.0    
ply                                3.11      
prometheus-client                  0.7.1     
prompt-toolkit                     2.0.9     
protobuf                           3.11.1    
psutil                             5.6.3     
PTable                             0.9.2     
ptyprocess                         0.6.0     
py                                 1.8.0     
pycodestyle                        2.5.0     
pycosat                            0.6.3     
pycparser                          2.19      
pycrypto                           2.6.1     
pyct                               0.4.6     
pycurl                             7.43.0.2  
pyfakefs                           3.6.1     
pyflakes                           2.1.1     
Pygments                           2.4.2     
pylint                             2.3.1     
PyNaCl                             1.3.0     
pyodbc                             4.0.27    
pyOpenSSL                          19.0.0    
pyparsing                          2.4.2     
pyproj                             2.2.1     
pyrsistent                         0.15.4    
PySocks                            1.7.0     
pytest                             5.3.0     
pytest-arraydiff                   0.3       
pytest-astropy                     0.5.0     
pytest-doctestplus                 0.3.0     
pytest-openfiles                   0.4.0     
pytest-remotedata                  0.3.1     
python-dateutil                    2.8.0     
python-editor                      1.0.4     
python-pptx                        0.6.18    
pythreejs                          2.1.1     
pytz                               2019.2    
pyviz-comms                        0.7.2     
PyWavelets                         1.0.3     
PyYAML                             5.1.2     
pyzmq                              18.0.2    
qgrid                              1.1.1     
QtAwesome                          0.6.0     
qtconsole                          4.5.2     
QtPy                               1.9.0     
querystring-parser                 1.2.4     
requests                           2.22.0    
retrying                           1.3.3     
rope                               0.14.0    
ruamel-yaml                        0.15.71   
scikit-image                       0.15.0    
scikit-learn                       0.21.3    
scipy                              1.3.0     
seaborn                            0.9.0     
SecretStorage                      3.1.1     
Send2Trash                         1.5.0     
setuptools                         41.0.1    
simplegeneric                      0.8.1     
simplejson                         3.16.0    
singledispatch                     3.4.0.3   
six                                1.12.0    
smmap2                             2.0.5     
snowballstemmer                    1.9.0     
sortedcollections                  1.1.2     
sortedcontainers                   2.1.0     
soupsieve                          1.9.2     
Sphinx                             2.1.2     
sphinxcontrib-applehelp            1.0.1     
sphinxcontrib-devhelp              1.0.1     
sphinxcontrib-htmlhelp             1.0.2     
sphinxcontrib-jsmath               1.0.1     
sphinxcontrib-qthelp               1.0.2     
sphinxcontrib-serializinghtml      1.1.1     
sphinxcontrib-websupport           1.1.2     
spyder                             3.3.6     
spyder-kernels                     0.5.1     
SQLAlchemy                         1.3.6     
sqlparse                           0.3.0     
sshtunnel                          0.1.5     
statsmodels                        0.10.1    
sympy                              1.4       
tables                             3.5.2     
tabulate                           0.8.3     
tblib                              1.4.0     
terminado                          0.8.2     
testpath                           0.4.2     
toolz                              0.10.0    
tornado                            6.0.3     
tqdm                               4.33.0    
traitlets                          4.3.2     
traittypes                         0.2.1     
unicodecsv                         0.14.1    
urllib3                            1.25.3    
wcwidth                            0.1.7     
webencodings                       0.5.1     
websocket-client                   0.56.0    
Werkzeug                           0.15.5    
wheel                              0.33.4    
widgetsnbextension                 3.5.1     
wrapt                              1.11.2    
wurlitzer                          1.0.3     
xlrd                               1.2.0     
XlsxWriter                         1.1.8     
xlwt                               1.3.0     
zict                               1.0.0     
zipp                               0.5.2     
zmapio                             0.3       

Thanks in advance for any help & advice you can give!

[asottile]

mlflow does a naughty thing and resets the global warning filter as a module import side-effect

there isn't really much pytest can do here to prevent/fix that -- I suggest creating an issue on mlflow to stop doing that

[ghost]

Thanks - I'll do that.

[okuuva]

I think this is still an issue, or construct does the same naughty thing in a different way since I wasn't able to find any hits with $ rg warnings in their repo.

Minimal example:

pytest.ini:

[pytest]
filterwarning =
    ignore::DeprecationWarning

Simple test test_bar.py:

import construct

def test_foo():
    assert True

Pytest shows an unexpected warning:

============================= test session starts ==============================
platform darwin -- Python 3.7.6, pytest-5.3.5, py-1.8.1, pluggy-0.13.1
rootdir: /Users/oula/gits/pytest-deprecation-warning-filter-no-work, inifile: pytest.ini
plugins: celery-4.4.0, cov-2.8.1
collected 1 item

test_bar.py .                                                            [100%]

=============================== warnings summary ===============================
/.../venv/lib/python3.7/site-packages/construct/core.py:3
  /.../venv/lib/python3.7/site-packages/construct/core.py:3: DeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
    import struct, io, binascii, itertools, collections, pickle, sys, os, tempfile, hashlib, importlib, imp

-- Docs: https://docs.pytest.org/en/latest/warnings.html
========================= 1 passed, 1 warning in 0.02s =========================

Upstream got rid of imp about a month ago and there hasn't been a release after that.

Pip list:

Package            Version
------------------ -------
attrs              19.3.0 
construct          2.10.56
importlib-metadata 1.5.0  
more-itertools     8.2.0  
packaging          20.1   
pip                20.0.2 
pluggy             0.13.1 
py                 1.8.1  
pyparsing          2.4.6  
pytest             5.3.5  
setuptools         41.2.0 
six                1.14.0 
wcwidth            0.1.8  
zipp               3.0.0  

[asottile]

the setting is filterwarnings ;)

[okuuva]

I could've sworn this worked yesterday. And it did before I c&p'd my pytest.ini from one project to another and managed to drop the s from the setting in the process. Sorry for bothering you guys with my idiocy 😅