Why is Big-endian powerpc no longer supported ?

[amckinstry]

Sorry about the "non-bug" nature of this issue, but I'm working on OpenMPI 3.0.0 for Debian, and see BE powerpc is no longer supported.
Given that other powerpc and other BE systems are supported, I'm curious about this, and wondering whats going on.

[gpaulsen]

Thanks for asking your question. I'm sorry that dropping support for Big-endian powerpc has affected you. The community makes every effort to support as wide of a selection of platforms as possible, with our limited volunteered resources.

During the development of Open MPI v3.0.0, during our Open MPI supported platforms discussion, we discussed who might be willing to work on and support Open MPI on Big-Endian Power PC platforms. IBM committed to support the ppc64le (Little Endian) platform on Linux, however all IBM roadmaps only have Little Endian products. Due to perceived lack of interest, IBM chose not to support Big Endian Platforms.

Another aspect to the removal of support is due to our new improved continuous integration testing automation. Whenever anyone creates a pull request to either master branch or a release branch, the automation automatically begins a build and test of that pull request on all supported platforms (existing across various different member organizations). This automation has drastically improved quality to the point where we now define our support statement for Open MPI such that we only support platforms that we have added to this infrastructure and test regularly.

The Open MPI community is interested in supporting as many platforms as practical, and would be interested in working with anyone who is interested in supporting a platform, provided we can obtain a testing platform, and a contact person or organization.

If you are still interested, please reply, and we can continue discussing.

[amckinstry]

Thanks for this reply.

My concern is that, from a Debian Linux perspective, we support and test multiple architectures that are not part of the CI infrastructure upstream (see https://buildd.debian.org/status/package.php?p=openmpi); we then run our own tests on these.

PPC on Big-endian systems is arguably one of the better supported of the lesser archs. It seems strange that we will support alpha, hppa, sparc64, etc (without the latest fabrics, of course) but not powerpc.

This testing on other archs has two main benefits beyond supporting the tiny number of users doing HPC on minority archs. It shakes out the bugs latent in the codebase, and then in doing so, it makes future ports to new archs (arm64, etc) possible as developers know the codebase actually supports the standards rather than having many assumptions ("All the worlds a Vax", as they used to say).

Can we downgrade the failure to build on BE PPC from an hard error to a warning - "WARNING: this architecture is not tested and officially supported" ?

[gpaulsen]

I think the work required is a bit more involved than just changing the error to a warning. I thought I remembered some serious bug reports against PowerPC Big Endian that would need to be resolved.

[glaubitz]

IBM committed to support the ppc64le (Little Endian) platform on Linux, however all IBM roadmaps only have Little Endian products. Due to perceived lack of interest, IBM chose not to support Big Endian Platforms.

Of course, it was IBM. They also forcefully removed PowerPC BE support in Golang for no reason (I still have a working fork which I regularly rebase against master) and also almost objected my work of zfs on linux on PowerPC BE.

I think the work required is a bit more involved than just changing the error to a warning. I thought I remembered some serious bug reports against PowerPC Big Endian that would need to be resolved.

We in Debian are happy to support big-endian PowerPC targets and if there are bugs to be fixed, there are many people who are happy to help. It would be great if this could be turned into a warning again and if there are issues, we will be seeing them on our infrastructure and provide patches to fix them.

Thanks!

[opoplawski]

Fedora still builds for big-endian powerpc. I'm not sure what will be more painful - dealing with a buggy open-mpi on BE powerpc or putting in all of the ExcludeArch ppc64 statements in all of the dependent libraries. FWIW - BE powerpc64 is one of the three architectures that Fedora's CI system runs builds for, see https://apps.fedoraproject.org/koschei/package/openmpi. You'll also see there that there are sporadic test failures for ppc64.

[opoplawski]

Hmm, really surprised to see this change introduced in 2.1.2 - ostensibly a bug-fix release.

[gpaulsen]

The Open MPI community has a policy to discontinue support in OpenMPI v3.0.0 for any platform that does not have regular continuous integration testing for new pull requests, and also a clearly identified maintainer who is willing to step up and help debug and fix problems found on that platform. Unfortunately ppc64 Big Endian was one of the architectures that no one in the community stepped up to this level of support.

The Open MPI Community welcomes additional continuous integration platforms, help maintaining those platforms, submitting CI test results to the community jenkins server, and help in debugging issues found on those platforms during continuous integration. If you're interested, please let us know.

[rhc54]

I believe the question wasn't about v3.0.0, but rather why it disappeared in v2.1.2 - which was supposed to be a bug-fix release and should have fallen under the compatibility promise.

[glaubitz]

The Open MPI community has a policy to discontinue support in OpenMPI v3.0.0 for any platform that does not have regular continuous integration testing for new pull requests, and also a clearly identified maintainer who is willing to step up and help debug and fix problems found on that platform.

What's wrong with keeping the code if it doesn't hurt the other architectures?

I don't understand a policy like this. There is code that people are using, so why not just keep it?

I would understand this argument if the code would actually hurt other targets. But as long as that doesn't happen, why not just leave it in and accept drive-by patches to fix issues.

These super-strict policies just result in lots of frustration in downstream projects.

Debian and other downstreams are willing to help and provide patches, but please understand that OpenMPI isn't the only upstream project on this planet and if every other project posed such high standards on their downstream projects, we could throw away all ports except x86_64 and arm64.

We are helping whereever we can. Contributing patches to almost any important project in the community. Heck, I have even become upstream committer in OpenJDK and Firefox because I am so busy contributing such patches. But we (I) can't be upstream maintainer and committer in every project on the planet.

Really, please don't do that. It's incredibly frustrating.

[amckinstry]

@gpaulsen , for reference you can see the build systems we use here:
https://buildd.debian.org/status/package.php?p=openmpi&suite=experimental
This are the build logs for Debian for various archs; change "experimental" to "sid" to see the 2.1* track. Typically we triage bugs and report non-Debian specific ones (we maintain a handful of Debian-specific patches, eg. for multiarch support) upstream here.

In building packages, we also run any test suites contained within these. Periodically, but in particular as part of transitions (major library changes) we rebuild everything and test everything - eg. for openmpi2 -> openmpi3 testing I've rebuilt all MPI packages in Debian against openmpi3 - minor bugs ensued - eg. python-escript presumed Openmpi version numbers are always dotted integers, which 3.0.1rc1 broke - minor patch sent to them for this, etc.

So we are typically doing a lot of testing that is not seen upstream (here) unless bugs are found and the change is in openmpi not the client package (or below, on a lib used by openmpi).

It would be useful if there was a reference here to the changeset that dropped BE support, and the bugs that this closed.