Meta-ticket: improve conda setup

[tobiasdiez]

• Conda: Add primecountpy #33330: primecountpy is not installed in the conda env because it is not listed in environment.yml
• conda-forge-standard: Runtime error in sage.libs.giac #33056: giac installed via conda is not usable and leads to many doctest failures
• Fix for rename of conda e-antic package to libeantic #33358: fix for downgrades in src/environment-optional.yml
• Fixes for the conda-for-Sage-developers installation method, add GH Actions workflow #30845: Fix the conda-for-Sage-developers installation method, add GH Actions workflow
• Updated https://wiki.sagemath.org/Conda to match Fixes for the conda-for-Sage-developers installation method, add GH Actions workflow #30845
  • use configure --prefix=$CONDA_PREFIX
  • use configure --with-system-...=force
  • use conda for all python dependencies instead of relying on pip -r requirements.txt
• add more details on conda-based installations of Sage #33426 Migrate documentation from wiki https://wiki.sagemath.org/Conda to developer and installation docs
• Update build/pkgs/matplotlib/install-requires.txt and distros/conda.txt #33642 Update build/pkgs/matplotlib/install-requires.txt
• More updates for installation and developer guides #33088 More updates for installation and developer guides
• Add pip packages to conda environments, add missing conda.txt files #33613 Pip package git_trac_command, add pip packages to conda environments
• On WSL, gc and thus ecl are not found, as discussed at #30845 comment:152
• Make sure conda installs only python packages that are satisfying the version constraints specified in install_requires. Notably, ptyprocess, iniconfig, sphinx, pytest, rpy2 are currently reinstalled by pip. Also add --no-dependencies to pip install in the conda ci workflow. This is Conda config does not respect constraints in install_requires #35074.

Fix remaining tests that fail:

• sage distutils classes

File "src/sage/interfaces/sage0.py", line 208, in sage.interfaces.sage0.Sage.cputime
Failed example:
    sage0.cputime()     # random output
Exception raised:
    Traceback (most recent call last):
      File "/home/runner/work/sage/sage/src/sage/doctest/forker.py", line 695, in _run
        self.compile_and_execute(example, compiler, test.globs)
      File "/home/runner/work/sage/sage/src/sage/doctest/forker.py", line 1093, in compile_and_execute
        exec(compiled, globs)
      File "<doctest sage.interfaces.sage0.Sage.cputime[0]>", line 1, in <module>
        sage0.cputime()     # random output
      File "/home/runner/work/sage/sage/src/sage/interfaces/sage0.py", line 219, in cputime
        return float(s)
    ValueError: could not convert string to float: 'iniconda/envs/sage-build/lib/python3.8/site-packages/setuptools/_distutils/version.py:351: DeprecationWarning: distutils Version classes are deprecated. Use packaging.version instead.\n  other = LooseVersion(other)\n1.60718'

• conda: 3d graphics do not show #34298: threejs path

 File "src/sage/repl/ipython_kernel/install.py", line 127, in sage.repl.ipython_kernel.install.SageKernelSpec.use_local_threejs
Failed example:
    os.path.isdir(threejs)
Expected:
    True
Got:
    False

CC: @isuruf @dimpase @saraedum @mkoeppe @slel

Component: build

Issue created by migration from https://trac.sagemath.org/ticket/33331

[tobiasdiez]

Description changed:

--- 
+++ 
@@ -3,6 +3,7 @@
 - Documentation doesn't mention the use of `--prefix=$CONDA_PREFIX` (without this things like `ecl-config` are not found)
 - The correct version of rpy2, scipy, sphinx among other packages is not installed in the conda env, so add version constraints in the corresponding `conda.txt` files
 - #33330: primecountpy is not installed in the conda env because its not listed in `environment.yml`
+- #33056: giac installed via conda is not usable and leads to many doctest failures
 - Installation with `src/environment-optional.yml` uses rather old packages such as lcalc 1 and pari 2.11.4. This leads to issues such as the one below. Or that the package is then rejected by `configure` as being too old. This is most likely an issue with some of the optional packages that get installed and that still relies on an old version of some conda package, which then leads to various downgrades. See discussion at #30845.
 
 ```

[tobiasdiez]

Description changed:

--- 
+++ 
@@ -1,6 +1,6 @@
 - Update documentation to use conda to manage python dependencies instead of relying on `pip -r requirements.txt`
 - Migrate documentation from wiki https://wiki.sagemath.org/Conda to developer docs
-- Documentation doesn't mention the use of `--prefix=$CONDA_PREFIX` (without this things like `ecl-config` are not found)
+- On WSL, gc and thus ecl are not found, see discussion starting with [#30845 comment:152](https://github.com/sagemath/sage/issues/30845#comment:152)
 - The correct version of rpy2, scipy, sphinx among other packages is not installed in the conda env, so add version constraints in the corresponding `conda.txt` files
 - #33330: primecountpy is not installed in the conda env because its not listed in `environment.yml`
 - #33056: giac installed via conda is not usable and leads to many doctest failures

[isuruf]

Description changed:

--- 
+++ 
@@ -4,15 +4,6 @@
 - The correct version of rpy2, scipy, sphinx among other packages is not installed in the conda env, so add version constraints in the corresponding `conda.txt` files
 - #33330: primecountpy is not installed in the conda env because its not listed in `environment.yml`
 - #33056: giac installed via conda is not usable and leads to many doctest failures
-- Installation with `src/environment-optional.yml` uses rather old packages such as lcalc 1 and pari 2.11.4. This leads to issues such as the one below. Or that the package is then rejected by `configure` as being too old. This is most likely an issue with some of the optional packages that get installed and that still relies on an old version of some conda package, which then leads to various downgrades. See discussion at #30845.
+- #33358: fix for downgrades in src/environment-optional.yml
+}}}
 
-```
-building 'sage.libs.lcalc.lcalc_Lfunction' extension
-    cc1plus: warning: command line option '-Wstrict-prototypes' is valid for C/ObjC but not for C++
-    In file included from sage/libs/lcalc/lcalc_Lfunction.cpp:740:
-    sage/libs/lcalc/lcalc_sage.h:1:10: fatal error: lcalc/L.h: No such file or directory
-        1 | #include "lcalc/L.h"
-          |          ^~~~~~~~~~~
-    compilation terminated.
-```
-

[isuruf]

comment:4

The correct version of rpy2, scipy, sphinx among other packages is not installed in the conda env, so add version constraints in the corresponding conda.txt files

@tobiasdiez, is this fixed by #33358 too?

[tobiasdiez]

comment:5

Replying to @isuruf:

The correct version of rpy2, scipy, sphinx among other packages is not installed in the conda env, so add version constraints in the corresponding conda.txt files

@tobiasdiez, is this fixed by #33358 too?

Nope, still present. See e.g. output of https://pipelines.actions.githubusercontent.com/wUvxDab7ZCURYAl3HeSwsAoJVAdvXg6u4iTbsP5NChUqrJ4TKs/_apis/pipelines/1/runs/9243/signedlogcontent/3?urlExpires=2022-02-17T12%3A02%3A43.1351322Z&urlSigningMethod=HMACV1&urlSignature=ildfTJhaQ2KGVywf69MDFQSVOl7GTONDSdIOurIphKI%3D

Conda is installing rpy2 v3.4.5, while sagelib install requires specify <3.4,>=3.3. Thus pip will download an older version. I think it should be enough to specify the same upper constraint (for the few python packages that have one) in the distros/conda.txt.

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,5 +1,5 @@
 - Update documentation to use conda to manage python dependencies instead of relying on `pip -r requirements.txt`
-- Migrate documentation from wiki https://wiki.sagemath.org/Conda to developer docs
+- #33426 Migrate documentation from wiki https://wiki.sagemath.org/Conda to developer docs
 - On WSL, gc and thus ecl are not found, see discussion starting with [#30845 comment:152](https://github.com/sagemath/sage/issues/30845#comment:152)
 - The correct version of rpy2, scipy, sphinx among other packages is not installed in the conda env, so add version constraints in the corresponding `conda.txt` files
 - #33330: primecountpy is not installed in the conda env because its not listed in `environment.yml`

[dimpase]

Description changed:

--- 
+++ 
@@ -1,5 +1,5 @@
 - Update documentation to use conda to manage python dependencies instead of relying on `pip -r requirements.txt`
-- #33426 Migrate documentation from wiki https://wiki.sagemath.org/Conda to developer docs
+- #33426 Migrate documentation from wiki https://wiki.sagemath.org/Conda to developer and installation docs
 - On WSL, gc and thus ecl are not found, see discussion starting with [#30845 comment:152](https://github.com/sagemath/sage/issues/30845#comment:152)
 - The correct version of rpy2, scipy, sphinx among other packages is not installed in the conda env, so add version constraints in the corresponding `conda.txt` files
 - #33330: primecountpy is not installed in the conda env because its not listed in `environment.yml`

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,4 +1,6 @@
 - Update documentation to use conda to manage python dependencies instead of relying on `pip -r requirements.txt`
+- Documentation doesn't mention the use of `--prefix=$CONDA_PREFIX` (without this things like `ecl-config` are not found)
+
 - #33426 Migrate documentation from wiki https://wiki.sagemath.org/Conda to developer and installation docs
 - On WSL, gc and thus ecl are not found, see discussion starting with [#30845 comment:152](https://github.com/sagemath/sage/issues/30845#comment:152)
 - The correct version of rpy2, scipy, sphinx among other packages is not installed in the conda env, so add version constraints in the corresponding `conda.txt` files

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,3 +1,4 @@
+- #30845 Fixes for the conda-for-Sage-developers installation method, add GH Actions workflow
 - Update documentation to use conda to manage python dependencies instead of relying on `pip -r requirements.txt`
 - Documentation doesn't mention the use of `--prefix=$CONDA_PREFIX` (without this things like `ecl-config` are not found)
 

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,3 +1,6 @@
+- #33330: primecountpy is not installed in the conda env because its not listed in `environment.yml`
+- #33056: giac installed via conda is not usable and leads to many doctest failures
+- #33358: fix for downgrades in src/environment-optional.yml
 - #30845 Fixes for the conda-for-Sage-developers installation method, add GH Actions workflow
 - Update documentation to use conda to manage python dependencies instead of relying on `pip -r requirements.txt`
 - Documentation doesn't mention the use of `--prefix=$CONDA_PREFIX` (without this things like `ecl-config` are not found)
@@ -5,8 +8,5 @@
 - #33426 Migrate documentation from wiki https://wiki.sagemath.org/Conda to developer and installation docs
 - On WSL, gc and thus ecl are not found, see discussion starting with [#30845 comment:152](https://github.com/sagemath/sage/issues/30845#comment:152)
 - The correct version of rpy2, scipy, sphinx among other packages is not installed in the conda env, so add version constraints in the corresponding `conda.txt` files
-- #33330: primecountpy is not installed in the conda env because its not listed in `environment.yml`
-- #33056: giac installed via conda is not usable and leads to many doctest failures
-- #33358: fix for downgrades in src/environment-optional.yml
-}}}
 
+

[mkoeppe]

Description changed:

--- 
+++ 
@@ -1,10 +1,11 @@
 - #33330: primecountpy is not installed in the conda env because its not listed in `environment.yml`
 - #33056: giac installed via conda is not usable and leads to many doctest failures
 - #33358: fix for downgrades in src/environment-optional.yml
-- #30845 Fixes for the conda-for-Sage-developers installation method, add GH Actions workflow
-- Update documentation to use conda to manage python dependencies instead of relying on `pip -r requirements.txt`
-- Documentation doesn't mention the use of `--prefix=$CONDA_PREFIX` (without this things like `ecl-config` are not found)
-
+- #30845: Fixes for the conda-for-Sage-developers installation method, add GH Actions workflow
+- Updated https://wiki.sagemath.org/Conda to match #30845
+  - use `configure --prefix=$CONDA_PREFIX`
+  - use `configure --with-system-...=force`
+  - use conda for all python dependencies instead of relying on `pip -r requirements.txt`
 - #33426 Migrate documentation from wiki https://wiki.sagemath.org/Conda to developer and installation docs
 - On WSL, gc and thus ecl are not found, see discussion starting with [#30845 comment:152](https://github.com/sagemath/sage/issues/30845#comment:152)
 - The correct version of rpy2, scipy, sphinx among other packages is not installed in the conda env, so add version constraints in the corresponding `conda.txt` files