Writing a setup.py is Hard

[dstufft]

Today most users tend to just cargo cult a setup.py around, never understanding most of the directives that actually go into one.

Solutions:

• Document Creating a New Package From Scratch
• "Package Init", Create a New Package Template?

Some of this already is starting to exist with the packaging guide's page on creating a package (See here), but that is incomplete.

[aaugustin]

Unless the distutils docs drastically improved since I last read them, the missing piece is a focused and correct distutils (or maybe setuptools?) tutorial.

How do I ship properly a dead-simple, Python-only, no-dependencies piece of code? This should fit in one or two screenfuls of docs, and is distinct from the reference docs.

[qwcode]

btw, see https://bitbucket.org/pypa/python-packaging-user-guide/issue/23/stop-recreating-the-pip-setuptools-docs-in

I want to basically dump the "Packaging Guides" attempt at yet another setuptools tutorial.
setuptools needs to own that. It already has one. Let's improve that.

As for distutils, any of it's docs need to be boldly red flagged with "Don't Start Here". Distutils docs need to exist primarily for the api at this point.
https://bitbucket.org/pypa/python-packaging-user-guide/issue/29/modernize-distutils-docs

[ncoghlan]

"Don't start here" is a large part of the PEP 453 plan for the core docs: http://www.python.org/dev/peps/pep-0453/#documentation

Essentially, the distutils docs become a "for packaging tool developers only" reference, while the real cross-version packaging guide becomes independent of CPython.

We may also want to point people in the direction of Audrey Roy's cookiecutter project - it's a nice way to jumpstart a lot of boilerplate packaging and collaboration information.

[ncoghlan]

CPython issue for the PEP 453 "Installing Python Modules" docs update: http://bugs.python.org/issue19407

[ncoghlan]

The distributing Python modules docs aren't in scope for PEP 453, but with the precedent set for the installing side, it's easy to make the case for delegating the build side to external version independent docs as well.

[dstufft]

I've gotten more response about how writing a setup.py is hard with some specific criticism, which included the cargo cult nature of setup.py

[dstufft]

Someone else called out how brittle things are, even a well written setup.py can easily break, sometimes silently, if you rearrange things at all.

[aaugustin]

It's still unclear to me how we went from "never use setuptools, it's too weird, it monkey-patches things to spread itself as soon as it's installed on your system, etc." to "use setuptools". Technically, that must have happened around the setuptools + distribute merge, but isn't setuptools still essentially the same as when its use was discouraged?

From the confused-packager-department... I'm still carefully using distutils.core and not setuptools to avoid inflicting it to users of my packages...

[qwcode]

went from "never use setuptools, it's too weird"

there was propaganda about using distribute over setuptools, because distribute was more open and supported py3, but both still had the monkey patching and were/are "weird.". Just using distutils is a non-starter for many projects, because it doesn't offer "install_requires".

[qwcode]

The big shift for people I think is going from "just wait, the distutils2 revolution is coming in the stdlib", to "distutils2 is not coming, and there is no plan atm to release a replacement into the stdlib, it's ok/safe to use setuptools for packaging/building right now; we're working on it"

[aaugustin]

It's the first time I hear about "distutils2".

Even though I play the devil's advocate in this thread, I'm not totally clueless about packaging.

[qwcode]

It's the first time I hear about "distutils2".

the distutils2 effort (or "packaging", the name it was going to be called in the stdlib) went on for years from roughly 2009 to 2012.

here's a link to a graphic about it in the "Hitchhiker's Guide to Packaging" (which the new "Python Packaging User guide" replaces.) http://guide.python-distribute.org/introduction.html#current-state-of-packaging

[ncoghlan]

The main change was that we (wearing my CPython core developer hat) figured
out most of the really problematic defaults were in easy_install rather
than being inherent to using setuptools on the build side. So
setuptools+pip is substantially better behaved than easy_install (the
latter is hard to change due to backwards compatibility concerns).

setuptools still has a lot of rough edges, but we need it (or an
equivalent) as a low barrier to entry way to produce usable metadata for
pure distutils projects.

[hickford]

You're right, writing a setup.py is intimiding for new users.

Nodejs has a wonderful wizard npm init https://www.npmjs.org/doc/cli/npm-init.html

This will ask you a bunch of questions, and then write a package.json for you.

If you already have a package.json file, it'll read that first, and default to the options in there. It is strictly additive, so it does not delete options from your package.json without a really good reason to do so.

Also Npm's package.json is much simpler to work with than setup.py because it's a lifeless json file rather than a Turing-complete Python script.

The Npm documentation is fantastic too:

• https://www.npmjs.org/doc/misc/npm-developers.html
• https://docs.npmjs.com/files/package.json

[ionelmc]

There's https://pypi.python.org/pypi/d2to1 that would make this declarative (similar to package.json). Why hasn't it gained much traction ?