#288 Expose c++ integrator to python and remove scipy dependence #290

rmjarvis · 2012-09-28T19:51:11Z

This turned out to be a lot easier than I thought, so this is a pretty short pull request.

The upshot is that the following now works:

>>> import galsim
>>> def func(x): return x**2
... 
>>> galsim.int1d(func,0,1)
0.33333333333333337
>>> galsim.int1d(func,0,2)
2.666666666666667
>>> galsim.int1d(func,-1,1)
0.66666666666666674

I also updated the Da in Cosmology to use this rather than the scipy.quad function, which removes our only scipy dependence.

(#288)

reikonakajima · 2012-09-29T07:48:39Z

It ran successfully on my Mac OS X 10.7.4 and Linux, but it's failing the nosetests in 10.5.8:

scons: done reading SConscript files.
scons: Building targets ...
run_tests(["tests/tests.log"], ["bin/test_main"])

Starting python tests...
........................................................................................Nosetests terminated by signal  10

Starting cpp tests...
Running 8 test cases...

*** No errors detected
test_main finished successfully.

scons: done building targets.

rmjarvis · 2012-09-29T15:21:41Z

Reiko, can you check if the above python session for integrating x^2 works for you?

Also, in the tests directory, you can try python test_lensing.py directly, which might output a better error message than just "terminated by signal 10". If not, you can try using gdb:

gdb python
run test_lensing.py
bt

which might tell us something useful.

(#288)

rmandelb · 2012-09-30T01:56:53Z

Very nice! This compiles and all tests pass on my Mac and the Linux cluster at CMU.

pmelchior · 2012-09-30T05:21:29Z

Thanks, Mike. This makes the NFW halo class much nicer to use (no scipy).

Leave only one thing to desire: pre-lensing coordinates instead of post-lensing coordinates. I know how this can be done, but this will have wait a little until I can pick it up.

reikonakajima · 2012-09-30T06:40:22Z

Hi Mike:

So, the int1d crashes:

reiko@edna~/2code/GalSim% python
Python 2.7.3 (default, Sep 11 2012, 10:36:56) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import galsim
>>> def func(x): return x**2
... 
>>> galsim.int1d(func,0,1)
Bus error

so I've tried gdb. I got a bunch of Could not find object file warnings when I loaded gdb python. Not sure if it's very informative:

reiko@edna~/2code/GalSim/tests% gdb python
GNU gdb 6.3.50-20050815 (Apple version gdb-966) (Tue Mar 10 02:43:13 UTC 2009)
Copyright 2004 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-apple-darwin"...Reading symbols for shared libraries .
warning: Could not find object file "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_python27/python27/work/Python-2.7.3/libpython2.7.a(getbuildinfo.o)" - no debug information available for "./Modules/getbuildinfo.c".

[lots of similar warnings]

warning: Could not find object file "/opt/local/var/macports/build/_opt_local_var_macports_sources_rsync.macports.org_release_ports_lang_python27/python27/work/Python-2.7.3/libpython2.7.a(xxsubtype.o)" - no debug information available for "./Modules/xxsubtype.c".

... done

(gdb) run test_lensing.py
Starting program: /opt/local/bin/python test_lensing.py
Reading symbols for shared libraries +++. done

Program received signal SIGTRAP, Trace/breakpoint trap.
0x8fe01010 in __dyld__dyld_start ()
(gdb) bt
#0  0x8fe01010 in __dyld__dyld_start ()
Cannot access memory at address 0x4
(gdb)

rmjarvis · 2012-09-30T13:34:30Z

Reiko, the fact that it crashes at dyld_dyld_start sounds like maybe it's trying to use an old installation. So maybe scons got confused and didn't relink everything correctly. Could you try recompiling completely from scratch:

scons -c
scons -c install
scons
scons install
scons tests

reikonakajima · 2012-09-30T15:06:06Z

Mike:
No, it didn't work. I still get the same error.

…d a bunch of print statements to help diagnose Reiko's bus error. (#288)

rmjarvis · 2012-09-30T17:15:00Z

Hmm. I'm not sure then...

Try the version I just pushed. I made a python layer version of int1d (now called galsim.integ.int1d rather than just galsim.int1d) that provides an interface to the C++ layer version. I don't think that will solve your problem, but it lets me put print statements just before the call to the c++ layer version. I also added a bunch of print statement in the c++ layer. So hopefully we can find the exact line where the bus error is happening and maybe have a better guess as to what might be causing it.

rmjarvis · 2012-09-30T17:16:35Z

Oh, and to avoid nosetests swallowing the python output, better to check this in your own python session:

import galsim
func = lambda x : x**2
galsim.integ.int1d(func,0,1)

should do the trick.

reikonakajima · 2012-10-01T06:58:28Z

Hi Mike. Looks good---it seems to pass all tests (but it has a spurious output):

Starting python tests...
........................................................................................Start PyInt1d
func = _func = ..................................................
----------------------------------------------------------------------
Ran 138 tests in 59.124s

OK
Nosetests finished successfully.

Starting cpp tests...
Running 8 test cases...

*** No errors detected
test_main finished successfully.

scons: done building targets.
reiko@edna~/2code/GalSim% python
Python 2.7.3 (default, Sep 11 2012, 10:36:56) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import galsim
>>> func = lambda x : x**2
>>> galsim.integ.int1d(func,0,1)
Start python int1d
min =  0.0
max =  1.0
rel_err =  1e-06
abs_err =  1e-12
Start PyInt1d
func = _func = 0.33333333333333337
>>>

Thanks!

(#288)

rmjarvis · 2012-10-01T07:24:26Z

Great. I wasn't actually expecting that to fix the problem. Merely to provide some information to help diagnose the problem. I wonder what aspect of what I changed was the cure... :)

Anyway, I removed the print statements, so it's probably worth making sure it still works now. If so, I'd say we're good to go.

reikonakajima · 2012-10-01T08:57:26Z

I don't know why "python layer version of int1d" works either. Something that may be relevant, though, is that my Python2.7 was compiled with gcc 4.0.1, while I compile GalSim with gcc4.7.1.

I've rerun all tests, and all tests pass.

rmjarvis · 2012-10-01T12:10:02Z

That explains why your print statements didn't actually go through (python and c++ linking different clibs often cause I/O clashes). But I don't know why it would have caused a bus error for the int1d stuff. Anyway, glad it works now.

rmjarvis · 2012-10-01T17:01:34Z

Any more comments about this one? Barney and Rachel (when you get back online), you might want to look over the diff again, in particular the new file integ.py which has a python doc string now. See if that looks ok.

Also, when working on Reiko's problem, I ended up switching the python function from galsim.int1d to galsim.integ.int1d, which matches the c++ namespace, so I think that's probably preferred. But let me know if you want to go back to the shorter version.

barnabytprowe · 2012-10-01T18:38:34Z

Hi Mike... I'll take a good look at this later tonight and make comments (if any seem necessary). Shouldn't take too long.

barnabytprowe · 2012-10-01T22:58:12Z

galsim/integ.py

I pushed a fix for what I'm guessing is a typo in the specification of default rel_err=1.e-12 for abs_err... Please check!

Yes. Thank you.

barnabytprowe · 2012-10-01T23:43:14Z

It all looks very good to me... The only additional thing, which I think could be considered optional but would certainly not hurt, would be some simple unit tests for the integrator in a standalone fashion. A couple of functions would do (maybe one simple, one a little more oscillatory and one with at least one infinite limit to test that our defaults are good). If you're busy I volunteer, too...

(I also owe you some changes to INSTALL.md by the way describing the new fink pathway... It's on the list for tomorrow!)

rmjarvis · 2012-10-02T00:32:27Z

That sounds worthwhile. I never did get around to writing unit tests for it. So if you're volunteering... :) A starting point could be the IntExample.cpp file in the examples directory for a C++ unit test. It includes an infinite integral. It wouldn't be too hard to convert these to python. There isn't anything there that's oscillatory yet.

barnabytprowe · 2012-10-02T13:03:16Z

OK, cool, I'll push some unit tests this afternoon!

…e limits (#288)

…ite and infinite limits, another quite a bit harder (more oscillatory) (#288)

(#288)

barnabytprowe · 2012-10-02T18:16:09Z

Hi Mike, I've added a few short tests... A nice simple function and a rather harder one with many (ever more rapid as x increases) zero crossings, using three sets of finite and three sets of infinite limits. It's only four tests, so if you would like to see more let me know and I can do a few, but I think the behaviour is already relatively impressive.

I also made an edit to the docstring of integ.py - it struck me that the user ought to know what values above which numbers are interpreted as being infinite (this is in Int.h as MOCK_INF2).

(#288)

reikonakajima · 2012-10-03T15:43:53Z

Hi Barney:

test1 passes.
test2 passes.
test3 fails:

Starting python tests...
.............................................................................................Python(7002) malloc: *** error for object 0x4aca4ac: Non-aligned pointer being freed
*** set a breakpoint in malloc_error_break to debug
Nosetests terminated by signal  6

Starting cpp tests...
Running 8 test cases...

*** No errors detected
test_main finished successfully.

barnabytprowe · 2012-10-03T16:06:15Z

Hmmm, I just realised that test 2 was stupid as it didn't actually run any code that would need the runtime error. I'll be back with a better suggestion in a sec...

rmjarvis · 2012-10-03T16:17:10Z

It's ok Barney. I think the problem stems from g++ 4.7's ostringstream being incompatible with earlier versions (e.g. the one that python was compiled with). I was able to reproduce the problem by using g++ 4.7 for GalSim with my system python (which uses gcc 4.2). It may not be ostringstream specifically, but when I change how the IntFailure exception is created by removing the ostringstream stuff, then the exception passes through the python-c++ boundary correctly. I'll push the fix in a few minutes.

(#288)

rmjarvis · 2012-10-03T16:19:29Z

OK, Reiko, give it a try now.

barnabytprowe · 2012-10-03T16:20:18Z

OK...

Go back to test 1, and try the following in the python interpreter:

>>> import galsim

>>> def func(x): return 1./x**2

>>> result = galsim.integ.int1d(func, 0, 2.e10)

You should see something like...

terminate called throwing an exception
Abort trap: 6

...and an ugly crash.

Now try the same thing in test 2 (test 3 should give the same results). In test 2's commit, I get (using 4.7.2 gcc-built galsim and gcc 4.0.1 EDP):

browe$ python
Enthought Python Distribution -- www.enthought.com
Version: 7.1-2 (64-bit)

Python 2.7.2 |EPD 7.1-2 (64-bit)| (default, Jul 27 2011, 14:50:45) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "packages", "demo" or "enthought" for more information.
>>> import galsim
>>> def func(x): return 1./x**2
... 
>>> result = galsim.integ.int1d(func, 0, 2.e10)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "galsim/integ.py", line 32, in int1d
    success, result = _galsim.PyInt1d(func,min,max,rel_err,abs_err)
  File "<stdin>", line 1, in func
ZeroDivisionError: float division by zero
>>>

... Oh, I see Mike has just posted telling me not to worry! Well, maybe this will turn out useful...

barnabytprowe · 2012-10-03T16:21:42Z

Sorry for the confusion Reiko, try scons tests on the newest version of the branch here (as per Mike's request just above) and ignore me for the time being!

reikonakajima · 2012-10-03T16:38:16Z

Hi Mike & Barney,
I've run it, and it still fails, but fails differently:

Starting python tests...
.............................................................................................terminate called after throwing an instance of 'galsim::integ::IntFailure'
  what():  Bad integrand behavior found in the integration interval in intGKP
Nosetests terminated by signal  6

Starting cpp tests...
terminate called after throwing an instance of 'galsim::integ::IntFailure'
  what():  Bad integrand behavior found in the integration interval in intGKP
Running 12 test cases...
terminate called recursively
terminate called recursively
terminate called recursively
terminate called recursively
[ad infinitum]

reikonakajima · 2012-10-03T16:48:08Z

Running your tests, Barney, I get:
(1) test1

reiko@edna~/2code/GalSim% python
Python 2.7.3 (default, Sep 11 2012, 10:36:56) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import galsim
>>> def func(x): return 1./x**2
... 
>>> result = galsim.integ.int1d(func, 0, 2.e10)
terminate called after throwing an instance of 'boost::python::error_already_set'
Abort

(2) test2, same as above

reiko@edna~/2code/GalSim% python
Python 2.7.3 (default, Sep 11 2012, 10:36:56) 
[GCC 4.0.1 (Apple Inc. build 5493)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import galsim
>>> def func(x): return 1./x**2
... 
>>> result = galsim.integ.int1d(func, 0, 2.e10)
terminate called after throwing an instance of 'boost::python::error_already_set'
Abort

Could it be that I need to worry about the boost.python gcc compilation version too?
I have boost @1.50.0_0+no_single+no_static+python27.

rmjarvis · 2012-10-03T17:02:27Z

Hmm.. That means that neither the try/catch block in pysrc/Integ.cpp, nor the boost.testing try/catch block are able to catch the exception.

I found some articles about exceptions not being caught across shared libraries (thrown in one, caught in another), so I'll try to see what kind of fix is recommended to get that to work.

rmjarvis · 2012-10-03T17:23:00Z

Reiko, can you please post the results of these two commands:

otool -L lib/libgalsim.dylib 
otool -L galsim/_galsim.so

reikonakajima · 2012-10-03T17:58:31Z

Sorry, the last post (deleted) was from my other laptop. The correct outputs are:

reiko@edna~/2code/GalSim% otool -L lib/libgalsim.dylib
lib/libgalsim.dylib:
    /Users/reiko/2code/GalSim/lib/libgalsim.0.dylib (compatibility version 0.1.0, current version 0.1.0)
    /opt/local/lib/libfftw3.3.dylib (compatibility version 7.0.0, current version 7.2.0)
    /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib (compatibility version 1.0.0, current version 218.0.0)
    /opt/local/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.17.0)
    /usr/local/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
    /opt/local/lib/gcc47/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.7)
reiko@edna~/2code/GalSim% otool -L galsim/_galsim.so
galsim/_galsim.so:
    /opt/local/lib/libboost_python-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
    /opt/local/Library/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.0)
    /opt/local/lib/libfftw3.3.dylib (compatibility version 7.0.0, current version 7.2.0)
    /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib (compatibility version 1.0.0, current version 218.0.0)
    /Users/reiko/2code/GalSim/lib/libgalsim.0.dylib (compatibility version 0.1.0, current version 0.1.0)
    /opt/local/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.17.0)
    /usr/local/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
    /opt/local/lib/gcc47/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.7)

rmjarvis · 2012-10-03T18:25:52Z

Thanks. I think the problem is that you have two versions of libgcc_s in there. The gcc47 one is probably the one we want. Not sure why the other one is getting linked as well.

Also, it's a bit strange that tmv isn't listed there. Can you post what command scons uses to link the library lib/libgalsim.0.dylib. (It should start with g++ -o lib/libgalsim.0.dylib ...)

reikonakajima · 2012-10-03T18:37:01Z

Is this it?

g++ -o lib/libgalsim.0.dylib -Wl,-no_compact_unwind -dynamiclib -dynamic -Wl,
-install_name,/Users/reiko/2code/GalSim/lib/libgalsim.0.dylib -Wl,-compatibility_version,0.1 -Wl,
-current_version,0.1 src/.obj/BinomFact.os src/.obj/FFT.os src/.obj/Image.os src/.obj/Interpolant.os
src/.obj/Laguerre.os src/.obj/OneDimensionalDeviate.os src/.obj/PhotonArray.os src/.obj/SBProfile.os 
src/.obj/SBBox.os src/.obj/SBGaussian.os src/.obj/SBExponential.os src/.obj/SBSersic.os src/.obj/SBMoffat.os 
src/.obj/SBAiry.os src/.obj/SBLaguerre.os src/.obj/SBAdd.os src/.obj/SBConvolve.os src/.obj/SBTransform.os 
src/.obj/SBInterpolatedImage.os src/.obj/SBDeconvolve.os src/.obj/SBKolmogorov.os src/.obj/CppShear.os 
src/.obj/Table.os src/.obj/RealSpaceConvolve.os src/.obj/Random.os src/hsm/.obj/PSFCorr.os -Llib 
-L/usr/local/lib -L/opt/local/lib -ltmv_symband -lpthread -lfftw3 -ltmv -lblas

rmjarvis · 2012-10-03T18:46:24Z

Which thing required /usr/local/lib? Is that where TMV is installed probably?

Also, /usr/local/lib is an odd place to have a gcc_s library. Do you have a version of gcc installed there?

Anyway, my best guess as to what is going on is that having -L/usr/local/lib specified means that the linker includes that libgcc_s library, even though g++ later also adds its own libgcc_s library, and the two of them are conflicting in how they handle exceptions.

I think the best solution is to figure out what installed a libgcc_s library in /usr/local/lib and uninstall it, since it's (probably) not the system gcc and it's not the MacPort gcc. So I don't think it's anything you are currently using.

If that's not possible, then you can try installing TMV in a different location so TMV_DIR won't be /usr/local/ and it won't need to include -L/usr/local/lib in the link line.

reikonakajima · 2012-10-03T19:04:31Z

Which thing required /usr/local/lib? Is that where TMV is installed probably?

Yes.

Also, /usr/local/lib is an odd place to have a gcc_s library. Do you have a version of gcc installed there?

I agree, that's strange. I don't think it should be there (port installs in /opt/local/lib, and the default is in /usr/lib).

Anyway, my best guess as to what is going on is that having -L/usr/local/lib specified means that the linker includes that libgcc_s library, even though g++ later also adds its own libgcc_s library, and the two of them are conflicting in how they handle exceptions.
I think the best solution is to figure out what installed a libgcc_s library in /usr/local/lib and uninstall it, since it's (probably) not the system gcc and it's not the MacPort gcc. So I don't think it's anything you are currently using.

I hid the /usr/local/lib version of all gcc libs, but now it's doing this (it found the other version):

reiko@edna~/2code/GalSim% otool -L lib/libgalsim.dylib
lib/libgalsim.dylib:
    /Users/reiko/2code/GalSim/lib/libgalsim.0.dylib (compatibility version 0.1.0, current version 0.1.0)
    /opt/local/lib/libfftw3.3.dylib (compatibility version 7.0.0, current version 7.2.0)
    /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib (compatibility version 1.0.0, current version 218.0.0)
    /opt/local/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.17.0)
    /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
    /opt/local/lib/gcc47/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.7)

and the errors I get from running the tests are the same:

Starting python tests...
.............................................................................................terminate called after throwing an instance of 'galsim::integ::IntFailure'
  what():  Bad integrand behavior found in the integration interval in intGKP
Nosetests terminated by signal  6

Starting cpp tests...
terminate called after throwing an instance of 'galsim::integ::IntFailure'
  what():  Bad integrand behavior found in the integration interval in intGKP
Running 12 test cases...
terminate called recursively
terminate called recursively
terminate called recursively
[ad infinitum]

rmjarvis · 2012-10-03T19:22:09Z

Here's another idea. Not sure if it will work:

scons EXTRA_FLAGS='-Wl,-reexport_library,/opt/local/lib/gcc47/libgcc_s.1.dylib'

rmjarvis · 2012-10-03T19:23:36Z

You can also try installing GalSim with the system g++:

scons CXX='/usr/bin/g++'

(And don't use the above EXTRA_FLAGS in this case.)

reikonakajima · 2012-10-03T19:39:41Z

The first one with the EXTRA_FLAGS didn't work:

reiko@edna~/2code/GalSim% otool -L galsim/_galsim.so                                                  galsim/_galsim.so:
    /opt/local/lib/gcc47/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
    /opt/local/lib/libboost_python-mt.dylib (compatibility version 0.0.0, current version 0.0.0)
    /opt/local/Library/Frameworks/Python.framework/Versions/2.7/Python (compatibility version 2.7.0, current version 2.7.0)
    /opt/local/lib/libfftw3.3.dylib (compatibility version 7.0.0, current version 7.2.0)
    /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib (compatibility version 1.0.0, current version 218.0.0)
    /Users/reiko/2code/GalSim/lib/libgalsim.0.dylib (compatibility version 0.1.0, current version 0.1.0)
    /opt/local/lib/libstdc++.6.dylib (compatibility version 7.0.0, current version 7.17.0)
    /usr/lib/libgcc_s.1.dylib (compatibility version 1.0.0, current version 1.0.0)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 111.1.7)

The second suggestion should work, but it will probably clash with FFTW and/or TMV, so it'll take some time to re-install these and re-run. I'll get back to you when I'm done.

reikonakajima · 2012-10-03T20:26:55Z

Hi Mike---short story: compiling GalSim with gcc 4.0.1 works. All my other libraries are compiled with this (Python, Boost, FFTW), except possibly TMV. I'm happy to stay in this configuration.

Some side notes, which I know you're aware at least of the second point:

(1) To make compilation work with gcc v4.0.1, I had to remove EXTRA_FLAGS='-Wl,-no_compact_unwind' (apparently it's only relevant for gcc v4.7.1?), which was tripping up linking (FFTW tests) in scons.

(2) I thought I was having scons problem with BOOST and FFTW. It turns out that I just needed to run

/bin/rm -r .sconf_temp

to reset the scons tests.

(3) I had a funny problem with scons -c when my config file changed to a new CXX:

reiko@edna~/2code/GalSim% scons -c 
scons: Reading SConscript files ...
SCons is version 2.2.0 using python version 2.7.3
Python is from /opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7
Using the following (non-default) scons options:
   CXX = /usr/bin/g++
   TMV_DIR = /usr/local
   FFTW_DIR = /local
   NOSETESTS = /opt/local/bin/nosetests-2.7
These can be edited directly in the file gs_scons.conf.
Type scons -h for a full list of available options.
OSError: [Errno 13] Permission denied: '.sconf_temp/conftest_0':
  File "/Users/reiko/2code/GalSim/SConstruct", line 1300:
    ClearCache()
  File "/Users/reiko/2code/GalSim/SConstruct", line 125:
    shutil.rmtree(".sconf_temp")
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 250:
    onerror(os.remove, fullname, sys.exc_info())
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 248:
    os.remove(fullname)

which seemed to cause the problem above. This problem seems to originate from (somehow) scons seeing my other gcc, which is /opt/local/bin/gcc (which gets linked to the preferred gcc version)--if I reset the /opt/local/bin/gcc link (which is done via a sudo port select --set gcc gcc40 command), then the scons -c command works without a hitch.

reikonakajima · 2012-10-03T20:39:35Z

I'd like to mention the problem I had with EXTRA_FLAGS='-Wl,-no_compact_unwind' in the Installation FAQ, but I'm not sure how the wording should go, since this problem seems specific to my system, and possibly not others. I've edited the FAQ, but could @rmjarvis please check the wording? (I've edited it directly on the webpage---perhaps I should have done it in this branch. Sorry)

rmjarvis · 2012-10-03T20:48:00Z

Actually, I think your last item about being unable to delete .sconf_temp/conftest_0 is because you did /bin/rm -r sconf_temp so the file wasn't there already when it was trying to remove it. I usually recommend /bin/rm -r .scon* to get rid of both that and the file .sconsign.dblite, so SCons doesn't think the conftest files exist.

reikonakajima · 2012-10-03T21:25:32Z

Hi Mike: Actually, the ordering of command execution was such that I tried scons -c first, didn't work (gave error #3), and resorted to /bin/rm -r .sconf_temp. I scrolled back to see what I've actually done. There were no rm .sconf_temp, but I found that I got a similar error even when I ran scons:

reiko@edna~/2code/GalSim% scons
scons: Reading SConscript files ...
SCons is version 2.2.0 using python version 2.7.3
Python is from /opt/local/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7
Using the following (non-default) scons options:
   CXX = /usr/bin/g++
   TMV_DIR = /usr/local
   FFTW_DIR = /opt/local
   NOSETESTS = /opt/local/bin/nosetests-2.7
These can be edited directly in the file gs_scons.conf.
Type scons -h for a full list of available options.
OSError: [Errno 13] Permission denied: '.sconf_temp/conftest_0':
  File "/Users/reiko/2code/GalSim/SConstruct", line 1300:
    ClearCache()
  File "/Users/reiko/2code/GalSim/SConstruct", line 125:
    shutil.rmtree(".sconf_temp")
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 250:
    onerror(os.remove, fullname, sys.exc_info())
  File "/opt/local/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/shutil.py", line 248:
    os.remove(fullname)

This "OSError" message cleared with a following scons -c, but had linking issues, presumably because the files in .sconf_temp was not the correct version.

It seems that, although scons -c should take care of updating files in .sconf_temp/ as well as the file .sconsign.dblite, sometimes it fails to do so. So if I'm having problem with scons, it's generally a good idea to say /bin/rm -r .scon*. Is that right? (Thanks!)

rmjarvis · 2012-10-03T21:40:14Z

So if I'm having problem with scons, it's generally a good idea to say /bin/rm -r .scon*. Is that right? (Thanks!)

Yes. This should probably be in the FAQ. It's especially relevant when you change significant things about your system outside of the GalSim directory -- install new libraries, remove libraries, change symlinks, etc.

rmjarvis · 2012-10-03T22:18:34Z

Reiko, is everything working for you now here? I added some text to the Install FAQ about this if you want to check it out and make sure I covered everything.

Then if all's well, I think we can go ahead and merge this.

reikonakajima · 2012-10-04T06:22:32Z

Yes Mike, it's all working. I see the Installation FAQ entry---thanks! Please go ahead and merge.

barnabytprowe · 2012-10-04T09:09:28Z

I second the merge (and thanks for the FAQ updates Reiko!)

On Thu, Oct 4, 2012 at 7:22 AM, Reiko Nakajima [email protected]:

Yes Mike, it's all working. I see the Installation FAQ entry---thanks!
Please go ahead and merge.

—
Reply to this email directly or view it on GitHubhttps://github.com//pull/290#issuecomment-9131784.

Barnaby Rowe
Postdoctoral Research Associate

Department of Physics & Astronomy
University College London
Gower Street
London WC1E 6BT
United Kingdom

#288 Expose c++ integrator to python and remove scipy dependence

rmjarvis added 2 commits September 28, 2012 15:24

Export int1d into python

33aa141

(#288)

Export int1d into python take 2

23fa3a6

(#288)

fix typo

ec58e89

(#288)

Make a python layer version of int1d to call the C++ version. Also ad…

8fb8fdb

…d a bunch of print statements to help diagnose Reiko's bus error. (#288)

Remove diagnostic print statements.

ff57043

(#288)

please check: fixed rel_err/abs_err typo in int1d docstring

cb5db03

barnabytprowe reviewed Oct 1, 2012
View reviewed changes

barnabytprowe added 3 commits October 2, 2012 18:00

added info to the int1d docstrings to describe how to specify infinit…

832ee1e

…e limits (#288)

added test_integ.py... currently contains four tests, one easy on fin…

cdaa738

…ite and infinite limits, another quite a bit harder (more oscillatory) (#288)

added test_integ to run_all_tests properly

1b1e2e9

(#288)

fixed error in err_msg for sin(x^2)*exp(-|x|) finite interval tests

ae664bd

(#288)

Remove ostringstream from the code that builds the IntFailure messages

001be44

(#288)

Merge branch 'master' into #288

7d62f38

rmjarvis added a commit that referenced this pull request Oct 4, 2012

Merge pull request #290 from GalSim-developers/#288

a58571a

#288 Expose c++ integrator to python and remove scipy dependence

rmjarvis merged commit a58571a into master Oct 4, 2012

#288 Expose c++ integrator to python and remove scipy dependence #290

#288 Expose c++ integrator to python and remove scipy dependence #290

Uh oh!

Conversation

rmjarvis commented Sep 28, 2012

Uh oh!

reikonakajima commented Sep 29, 2012

Uh oh!

rmjarvis commented Sep 29, 2012

Uh oh!

rmandelb commented Sep 30, 2012

Uh oh!

pmelchior commented Sep 30, 2012

Uh oh!

reikonakajima commented Sep 30, 2012

Uh oh!

rmjarvis commented Sep 30, 2012

Uh oh!

reikonakajima commented Sep 30, 2012

Uh oh!

rmjarvis commented Sep 30, 2012

Uh oh!

rmjarvis commented Sep 30, 2012

Uh oh!

reikonakajima commented Oct 1, 2012

Uh oh!

rmjarvis commented Oct 1, 2012

Uh oh!

reikonakajima commented Oct 1, 2012

Uh oh!

rmjarvis commented Oct 1, 2012

Uh oh!

rmjarvis commented Oct 1, 2012

Uh oh!

barnabytprowe commented Oct 1, 2012

Uh oh!

barnabytprowe Oct 1, 2012

Choose a reason for hiding this comment

Uh oh!

rmjarvis Oct 1, 2012

Choose a reason for hiding this comment

Uh oh!

barnabytprowe commented Oct 1, 2012

Uh oh!

rmjarvis commented Oct 2, 2012

Uh oh!

barnabytprowe commented Oct 2, 2012

Uh oh!

barnabytprowe commented Oct 2, 2012

Uh oh!

reikonakajima commented Oct 3, 2012

Uh oh!

barnabytprowe commented Oct 3, 2012

Uh oh!

rmjarvis commented Oct 3, 2012

Uh oh!

rmjarvis commented Oct 3, 2012

Uh oh!

barnabytprowe commented Oct 3, 2012

Uh oh!

barnabytprowe commented Oct 3, 2012

Uh oh!

reikonakajima commented Oct 3, 2012

Uh oh!

reikonakajima commented Oct 3, 2012

Uh oh!

rmjarvis commented Oct 3, 2012

Uh oh!

rmjarvis commented Oct 3, 2012

Uh oh!

reikonakajima commented Oct 3, 2012

Uh oh!

rmjarvis commented Oct 3, 2012

Uh oh!

reikonakajima commented Oct 3, 2012

Uh oh!

rmjarvis commented Oct 3, 2012

Uh oh!

reikonakajima commented Oct 3, 2012

Uh oh!