Skip to content

DOC: Note on PyTables index issue and additional Contributing refinements #10069

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
May 7, 2015
Merged

DOC: Note on PyTables index issue and additional Contributing refinements #10069

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
30 changes: 22 additions & 8 deletions doc/source/contributing.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -112,8 +112,10 @@ want to clone your fork to your machine: ::
This creates the directory `pandas-yourname` and connects your repository to
the upstream (main project) *pandas* repository.

You will also need to hook up Travis-CI to your GitHub repository so the suite
is automatically run when a Pull Request is submitted. Instructions are `here
The testing suite will run automatically on Travis-CI once your Pull Request is
submitted. However, if you wish to run the test suite on a branch prior to
submitting the Pull Request, then Travis-CI needs to be hooked up to your
GitHub repository. Instructions are for doing so are `here
<http://about.travis-ci.org/docs/user/getting-started/>`_.

Creating a Branch
Expand All @@ -134,6 +136,17 @@ changes in this branch specific to one bug or feature so it is clear
what the branch brings to *pandas*. You can have many shiny-new-features
and switch in between them using the git checkout command.

To update this branch, you need to retrieve the changes from the master branch::

git fetch upstream
git rebase upstream/master

This will replay your commits on top of the lastest pandas git master. If this
leads to merge conflicts, you must resolve these before submitting your Pull
Request. If you have uncommitted changes, you will need to `stash` them prior
to updating. This will effectively store your changes and they can be reapplied
after updating.

.. _contributing.dev_env:

Creating a Development Environment
Expand Down Expand Up @@ -338,7 +351,7 @@ dependencies.
Building the documentation
~~~~~~~~~~~~~~~~~~~~~~~~~~

So how do you build the docs? Navigate to your local the folder
So how do you build the docs? Navigate to your local the folder
``pandas/doc/`` directory in the console and run::

python make.py html
Expand All @@ -358,8 +371,9 @@ If you want to do a full clean build, do::

Starting with 0.13.1 you can tell ``make.py`` to compile only a single section
of the docs, greatly reducing the turn-around time for checking your changes.
You will be prompted to delete `.rst` files that aren't required, since the
last committed version can always be restored from git.
You will be prompted to delete `.rst` files that aren't required. This is okay
since the prior version can be checked out from git, but make sure to
not commit the file deletions.

::

Expand Down Expand Up @@ -417,7 +431,7 @@ deprecation warnings where needed.
Test-driven Development/Writing Code
------------------------------------

*Pandas* is serious about `Test-driven Development (TDD)
*Pandas* is serious about testing and strongly encourages individuals to embrace `Test-driven Development (TDD)
<http://en.wikipedia.org/wiki/Test-driven_development>`_.
This development process "relies on the repetition of a very short development cycle:
first the developer writes an (initially failing) automated test case that defines a desired
Expand Down Expand Up @@ -550,8 +564,8 @@ Doing 'git status' again should give something like ::
# modified: /relative/path/to/file-you-added.py
#

Finally, commit your changes to your local repository with an explanatory message. An informal
commit message format is in effect for the project. Please try to adhere to it. Here are
Finally, commit your changes to your local repository with an explanatory message. *Pandas*
uses a convention for commit message prefixes and layout. Here are
some common prefixes along with general guidelines for when to use them:

* ENH: Enhancement, new functionality
Expand Down
2 changes: 1 addition & 1 deletion doc/source/index.rst.template
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -115,6 +115,7 @@ See the package overview for more detail about what's in the library.
{%if not single -%}
whatsnew
install
contributing
faq
overview
10min
Expand Down Expand Up @@ -149,7 +150,6 @@ See the package overview for more detail about what's in the library.
api
{% endif -%}
{%if not single -%}
contributing
internals
release
{% endif -%}
2 changes: 1 addition & 1 deletion doc/source/install.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -243,7 +243,7 @@ Optional Dependencies
* `Cython <http://www.cython.org>`__: Only necessary to build development
version. Version 0.19.1 or higher.
* `SciPy <http://www.scipy.org>`__: miscellaneous statistical functions
* `PyTables <http://www.pytables.org>`__: necessary for HDF5-based storage. Version 3.0.0 or higher required.
* `PyTables <http://www.pytables.org>`__: necessary for HDF5-based storage. Version 3.0.0 or higher required, Version 3.2.0 or higher highly recommended.
* `SQLAlchemy <http://www.sqlalchemy.org>`__: for SQL database support. Version 0.8.1 or higher recommended.
* `matplotlib <http://matplotlib.sourceforge.net/>`__: for plotting
* `statsmodels <http://statsmodels.sourceforge.net/>`__
Expand Down
4 changes: 4 additions & 0 deletions doc/source/io.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2364,6 +2364,10 @@ for some advanced strategies

As of version 0.15.0, pandas requires ``PyTables`` >= 3.0.0. Stores written with prior versions of pandas / ``PyTables`` >= 2.3 are fully compatible (this was the previous minimum ``PyTables`` required version).

.. warning::

There is a ``PyTables`` indexing bug which may appear when querying stores using an index. If you see a subset of results being returned, upgrade to ``PyTables`` >= 3.2. Stores created previously will need to be rewritten using the updated version.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is only true for when using compression, IIRC.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the "stores need to be rewritten" part? I rewrote the test table uncompressed in the old version and querying in the new version does not return all rows. I think it needs to be done regardless.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, ok. I just wanted to make it as specific as possible. This sounds like its ALWAYS a bug. IIRC I think you actually have to specify start or stop when you query. (e.g. your query needs to be something like: s.select('df',where='.....',start=10), right? (iow it has to be a chunked query)

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It doesn't need to be chunked. My sample was just s.select('df', where=Term(...)). I tried to make it non-specific by "may appear."

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

hmm, ok, that's fine then.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Since this is such a nuanced bug, I think it would be worth making 3.2 a requirement once that is officially out.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

sure, let's make an issue for 0.17.0 for that, maybe now just update install.rst to say highly recommened to use 3.2

.. ipython:: python
:suppress:
:okexcept:
Expand Down
2 changes: 1 addition & 1 deletion doc/source/whatsnew/v0.16.1.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -297,7 +297,7 @@ Bug Fixes




- Bug in PyTables queries that did not return proper results using the index (:issue:`8265`, :issue:`9676`)



Expand Down