- cctbxbb - phenix-online.org

problem when using Python to convert from numpy array to flex.int array
by Jun Aishima 03 May '16

03 May '16

I have used the bootstrap method to build cctbx with my own Python (in a virtual environment). I setup my virtual environment, then source $CCTBX/build/setpaths.sh (bash). I am currently trying to turn a detector image (uint16) read in with fabio, which results in a numpy ndarray, and then turn that into a scitbx flex array. Since I've had problems doing this in my system, I've written a little script that reproduces the problem: ********** import fabio file=fabio.open('testcrystal_1_001.img') # adsc detector image file.data[0] from scitbx.array_family import flex flex.int(file.data[3071]) # this is a 3072x3072 image. For some reason, row 0 causes no problems. ********** At the end of this email is the stack trace I get. Has anybody had similar issues? I first saw this when I was trying to convert the entire image into a flex array, but now I've seen it with only one row [3071], while still working for [0:1500], so it doesn't seem like a simple memory issue. Any help would be appreciated. Jun Aishima ********** show_stack(1): <stdin>(1) <module> libc backtrace (23 frames, most recent call last): /home/aishimaj/spotter/bin/python() [0x414199] /lib64/libc.so.6(__libc_start_main+0xfd) [0x366d21ed5d] /home/aishimaj/spotter/bin/python(Py_Main+0xb22) [0x4150b2] /home/aishimaj/spotter/bin/python(PyRun_AnyFileExFlags+0x6b) [0x4ce82b] /home/aishimaj/spotter/bin/python(PyRun_InteractiveLoopFlags+0x4e) [0x4ce01e] /home/aishimaj/spotter/bin/python(PyRun_InteractiveOneFlags+0x1ff) [0x4cdd9f] /home/aishimaj/spotter/bin/python(PyEval_EvalCode+0x32) [0x4ad362] /home/aishimaj/spotter/bin/python(PyEval_EvalCodeEx+0x877) [0x4ad257] /home/aishimaj/spotter/bin/python(PyEval_EvalFrameEx+0x6a0b) [0x4aa8bb] /home/aishimaj/spotter/bin/python(PyObject_Call+0x47) [0x419ba7] /home/aishimaj/spotter/bin/python() [0x46df9f] /home/aishimaj/spotter/bin/python() [0x47623f] /home/aishimaj/spotter/bin/python(PyObject_Call+0x47) [0x419ba7] /home/aishimaj/spotter/bin/python() [0x42296f] /home/aishimaj/spotter/bin/python(PyObject_Call+0x47) [0x419ba7] /beamline/phenix/bootstrap/build/lib/libboost_python.so(+0x2d088) [0x7f5561101088] /beamline/phenix/bootstrap/build/lib/libboost_python.so(boost::python::handle_exception_impl(boost::function0<void>)+0x43) [0x7f5561112013] /beamline/phenix/bootstrap/build/lib/libboost_python.so(+0x2fe08) [0x7f5561103e08] /beamline/phenix/bootstrap/build/lib/libboost_python.so(boost::python::objects::function::call(_object*, _object*) const+0xef) [0x7f55611031ff] /beamline/phenix/bootstrap/build/lib/scitbx_array_family_flex_ext.so(boost::python::objects::signature_py_function_impl<boost::python::detail::caller<scitbx::af::versa<int, scitbx::af::flex_grid<scitbx::af::small<long, 10ul> > >* (*)(boost::python::numeric::array const&), boost::python::detail::constructor_policy<boost::python::default_call_policies>, boost::mpl::vector2<scitbx::af::versa<int, scitbx::af::flex_grid<scitbx::af::small<long, 10ul> > >*, boost::python::numeric::array const&> >, boost::mpl::v_item<void, boost::mpl::v_item<boost::python::api::object, boost::mpl::v_mask<boost::mpl::vector2<scitbx::af::versa<int, scitbx::af::flex_grid<scitbx::af::small<long, 10ul> > >*, boost::python::numeric::array const&>, 1>, 1>, 1> >::operator()(_object*, _object*)+0x52) [0x7f555ed68452] /beamline/phenix/bootstrap/build/lib/scitbx_array_family_flex_ext.so(scitbx::af::boost_python::flex_int_from_numpy_array(boost::python::numeric::array const&)+0x29) [0x7f555f039f79] /beamline/phenix/bootstrap/build/lib/scitbx_array_family_flex_ext.so(scitbx::af::versa<int, scitbx::af::flex_grid<scitbx::af::small<long, 10ul> > > scitbx::af::boost_python::versa_flex_from_numpy_array<int>(boost::python::numeric::array const&)+0x1c87) [0x7f555f0415c7] /lib64/libc.so.6() [0x366d2326a0] Segmentation fault (Python and libc call stacks above) This crash may be due to a problem in any imported Python module, including modules which are not part of the cctbx project. To disable the traps leading to this message, define these environment variables (e.g. assign the value 1): BOOST_ADAPTBX_FPE_DEFAULT BOOST_ADAPTBX_SIGNALS_DEFAULT This will NOT solve the problem, just mask it, but may allow you to proceed in case it is not critical. Jun Aishima | Post Doc - MX | Australian Synchrotron p: (03) 8540 4114 | f: (03) 8540 4200 jun.aishima(a)synchrotron.org.au<mailto:[email protected]> | www.synchrotron.org.au 800 Blackburn Road, Clayton, Victoria 3168

3 6

Algorithm developer postdoc opening at Berkeley National Lab
by Nicholas Sauter 28 Apr '16

28 Apr '16

Berkeley Lab's Molecular Biophysics and Integrated Bioimaging Division <http://biosciences.lbl.gov/divisions/mbib/> is looking for a Computational Biologist Postdoctoral Fellow to work in the Sauter group. We are seeking an algorithm developer for the increasingly complex analysis of large diffraction datasets in structural biology. Current projects utilize XFEL crystallography and spectroscopy to investigate the photosynthetic mechanism of water splitting and to probe other metalloenzyme reactions. We also wish to test whether diffuse scattering can reveal correlated atomic motions in crystals. Many problems remain to be solved, including the details of how to optimally merge datasets from thousands of crystals. Our software development projects (including packages such as DIALS <http://dials.lbl.gov> and cctbx.xfel <http://cci.lbl.gov/xfel>) have been highlighted in several high-impact publications listed HERE <http://biosciences.lbl.gov/profiles/nicholas-sauter>. Candidates should have expertise in one or more computational techniques including, but not limited to, ray tracing to test underlying physical models of the diffraction, Bayesian approaches for refining model parameters, macromolecular modeling and refinement, neural networks for interpreting image features, as well as signal processing and denoising methods. Extensive data analysis experience in crystallography or from more general bioimaging backgrounds are welcome. We particularly encourage strong mathematical intuition and a track record of bringing new ideas and tools to fruition, as evidenced through written publication and clear presentation. Candidates should send an expression of interest, CV, and list of three references to *Nick Sauter, nksauter(a)lbl.gov <nksauter(a)lbl.gov>.* Further details are posted at https://lbl.taleo.net/careersection/2/jobdetail.ftl?lang=en&job=82354. Nicholas K. Sauter, Ph. D. Computer Staff Scientist, Molecular Biophysics and Integrated Bioimaging Division Lawrence Berkeley National Laboratory 1 Cyclotron Rd., Bldg. 33R0345 Berkeley, CA 94720

1 0

Re: [cctbxbb] new fail on windows
by R. D. Oeffner 25 Apr '16

25 Apr '16

Markus, I'd say that this is a wrong take on the current state of affairs regarding supporting Windows. I believe line 22 in easy_mp.py you refer to is supposed to be one of the last warnings one would get if one were to use some of the functions there with Windows in an unsupported manner, say if objects can't be pickled. I don't think it means this file was written with non-windows support intentionally. I appreciate people code on different platforms but the way of working shouldn't be aloof denial of the existence of Windows. In the short term until you get a Windows PC perhaps Billy Poon might grant you access to one of the virtual Windows systems he has set up for testing. Otherwise I'm happy to run tests for you every now and then on my PC. Regards, Rob -----Original Message----- From: cctbxbb-request(a)phenix-online.org Sent: Saturday, April 23, 2016 20:00 To: cctbxbb(a)phenix-online.org Subject: cctbxbb Digest, Vol 85, Issue 23 Send cctbxbb mailing list submissions to cctbxbb(a)phenix-online.org To subscribe or unsubscribe via the World Wide Web, visit http://phenix-online.org/mailman/listinfo/cctbxbb or, via email, send a message with subject or body 'help' to cctbxbb-request(a)phenix-online.org You can reach the person managing the list at cctbxbb-owner(a)phenix-online.org When replying, please edit your Subject line so it is more specific than "Re: Contents of cctbxbb digest..." Today's Topics: 1. Re: new fail on windows (markus.gerstel(a)diamond.ac.uk) ---------------------------------------------------------------------- Message: 1 Date: Fri, 22 Apr 2016 21:26:38 +0000 From: <markus.gerstel(a)diamond.ac.uk> To: <cctbxbb(a)phenix-online.org> Subject: Re: [cctbxbb] new fail on windows Message-ID: <415F7F573406BB498256C68158D456746125CE10(a)EXCHMBX01.fed.cclrc.ac.uk> Content-Type: text/plain; charset=iso-8859-1 Hi Nigel, Should be resolved now. It is true that we do not currently test on windows. In this particular case it turns out neither does anyone else, cf. https://sourceforge.net/p/cctbx/code/HEAD/tree/trunk/libtbx/easy_mp.py line 22 which causes the other easy_mp tests to be skipped on Windows. This will make it kind of difficult to support windows software building on top of it, so at some point this needs to be addressed properly. Best wishes & good weekend -Markus ________________________________ From: cctbxbb-bounces(a)phenix-online.org [cctbxbb-bounces(a)phenix-online.org] on behalf of Nigel Moriarty [nwmoriarty(a)lbl.gov] Sent: Friday, April 22, 2016 17:39 To: cctbx mailing list Subject: Re: [cctbxbb] new fail on windows On Fri, Apr 22, 2016 at 1:35 AM, <markus.gerstel(a)diamond.ac.uk<mailto:[email protected]>> wrote: Hopefully this is fixed with r24353. Afraid not. Is it true that you are not testing this windows feature on a windows machine? Regarding junit, I'll talk with Aaron when he returns. Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov<http://CCI.LBL.gov> -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom ------------------------------ _______________________________________________ cctbxbb mailing list cctbxbb(a)phenix-online.org http://phenix-online.org/mailman/listinfo/cctbxbb End of cctbxbb Digest, Vol 85, Issue 23 ***************************************

2 1

new fail on windows
by Nigel Moriarty 23 Apr '16

23 Apr '16

******************************************************************************** ERROR: 1 TEST FAILURES. PLEASE FIX BEFORE COMMITTING CODE. ******************************************************************************** Standard error: Traceback (most recent call last): File "c:\users\builder\slave\cctbx-nightly-intel-windows\modules\cctbx_project\libtbx\tst_easy_mp.py", line 210, in <module> run(args=sys.argv[1:]) File "c:\users\builder\slave\cctbx-nightly-intel-windows\modules\cctbx_project\libtbx\tst_easy_mp.py", line 205, in run check_if_stacktrace_is_propagated_properly(method='multiprocessing') File "c:\users\builder\slave\cctbx-nightly-intel-windows\modules\cctbx_project\libtbx\tst_easy_mp.py", line 187, in check_if_stacktrace_is_propagated_properly assert isinstance(e, exceptions.ZeroDivisionError) or "ZeroDivisionError" in str(e), "Exception type mismatch: %s" % str(e) AssertionError: Exception type mismatch: Can't pickle <function may_divide_by_zero at 0x01C06470>: it's not found as __main__.may_divide_by_zero Traceback (most recent call last): File "<string>", line 1, in <module> File "C:\Users\builder\slave\cctbx-nightly-intel-windows\base\bin\python\lib\multiprocessing\forking.py", line 381, in main self = load(from_parent) File "C:\Users\builder\slave\cctbx-nightly-intel-windows\base\bin\python\lib\pickle.py", line 1378, in load return Unpickler(file).load() File "C:\Users\builder\slave\cctbx-nightly-intel-windows\base\bin\python\lib\pickle.py", line 858, in load dispatch[key](self) File "C:\Users\builder\slave\cctbx-nightly-intel-windows\base\bin\python\lib\pickle.py", line 880, in load_eof raise EOFError EOFError Cheers Nigel --- Nigel W. Moriarty Building 33R0349, Molecular Biophysics and Integrated Bioimaging Lawrence Berkeley National Laboratory Berkeley, CA 94720-8235 Phone : 510-486-5709 Email : NWMoriarty(a)LBL.gov Fax : 510-486-5909 Web : CCI.LBL.gov

3 4

Re: [cctbxbb] cctbxbb Digest, Vol 85, Issue 21
by R.D. Oeffner 22 Apr '16

22 Apr '16

Hi Markus, I was gonna ask why don't you guys set up a windows machine and test for the cctbx crash on that? The VC++9.0 compiler is free and if you use the python that comes in a phenix for windows installation the necessary modules are all present. Rob On 22/04/2016 09:35, cctbxbb-request(a)phenix-online.org wrote: > Send cctbxbb mailing list submissions to > cctbxbb(a)phenix-online.org > > To subscribe or unsubscribe via the World Wide Web, visit > http://phenix-online.org/mailman/listinfo/cctbxbb > or, via email, send a message with subject or body 'help' to > cctbxbb-request(a)phenix-online.org > > You can reach the person managing the list at > cctbxbb-owner(a)phenix-online.org > > When replying, please edit your Subject line so it is more specific > than "Re: Contents of cctbxbb digest..." > > > Today's Topics: > > 1. Re: new fail on windows (markus.gerstel(a)diamond.ac.uk) > > > ---------------------------------------------------------------------- > > Message: 1 > Date: Fri, 22 Apr 2016 08:35:42 +0000 > From: <markus.gerstel(a)diamond.ac.uk> > To: <cctbxbb(a)phenix-online.org> > Subject: Re: [cctbxbb] new fail on windows > Message-ID: > <415F7F573406BB498256C68158D456746125C23B(a)EXCHMBX01.fed.cclrc.ac.uk> > Content-Type: text/plain; charset="utf-8" > > Hi Nigel, > > Hopefully this is fixed with r24353. > > One question: Do you think it would be possible to install > https://pypi.python.org/pypi/junit-xml on your build machines, run > libtbx.run_tests_parallel with the output_junit_xml=True option, and > collect the output.xml files somewhere where we can download them? > > At the moment we do monitor the buildbot status - but only in a very > coarse way. We only see test failures on a high level, see attached > screenshot. As you can see there are a number of things broken, some > of them for quite some time, so because on that level there is always > something guaranteed to fail the overall status we see is always set > to ?warning?. I don?t think any of us bothers anymore looking into the > details, a number of errors are about things we can?t do anything > about anyway. > > However, if you could give us a collection of output.xml?s we could, > instead of taking the overall build status, use the test results and > set up a DIALS-East internal mail on the buildbot test status. Then we > would pick up these things earlier. > > Best wishes, > -Markus > > From: cctbxbb-bounces(a)phenix-online.org > [mailto:[email protected]] On Behalf Of Nigel Moriarty > Sent: 21 April 2016 17:41 > To: cctbx mailing list > Subject: [cctbxbb] new fail on windows > > > ******************************************************************************** > > ERROR: 1 TEST FAILURES. PLEASE FIX BEFORE COMMITTING CODE. > > ******************************************************************************** > > > > Standard error: > > Traceback (most recent call last): > > File > "c:\users\builder\slave\cctbx-nightly-intel-windows\modules\cctbx_project\libtbx\tst_easy_mp.py", > line 210, in <module> > > run(args=sys.argv[1:]) > > File > "c:\users\builder\slave\cctbx-nightly-intel-windows\modules\cctbx_project\libtbx\tst_easy_mp.py", > line 205, in run > > > check_if_stacktrace_is_propagated_properly(method='multiprocessing') > > File > "c:\users\builder\slave\cctbx-nightly-intel-windows\modules\cctbx_project\libtbx\tst_easy_mp.py", > line 187, in check_if_stacktrace_is_propagated_properly > > assert isinstance(e, exceptions.ZeroDivisionError) or > "ZeroDivisionError" in str(e), "Exception type mismatch: %s" % str(e) > > AssertionError: Exception type mismatch: Can't pickle <function > may_divide_by_zero at 0x01C06470>: it's not found as > __main__.may_divide_by_zero > > Traceback (most recent call last): > > File "<string>", line 1, in <module> > > File > "C:\Users\builder\slave\cctbx-nightly-intel-windows\base\bin\python\lib\multiprocessing\forking.py", > line 381, in main > > self = load(from_parent) > > File > "C:\Users\builder\slave\cctbx-nightly-intel-windows\base\bin\python\lib\pickle.py", > line 1378, in load > > return Unpickler(file).load() > > File > "C:\Users\builder\slave\cctbx-nightly-intel-windows\base\bin\python\lib\pickle.py", > line 858, in load > > dispatch[key](self) > > File > "C:\Users\builder\slave\cctbx-nightly-intel-windows\base\bin\python\lib\pickle.py", > line 880, in load_eof > > raise EOFError > > EOFError > Cheers > > Nigel > > --- > Nigel W. Moriarty > Building 33R0349, Molecular Biophysics and Integrated Bioimaging > Lawrence Berkeley National Laboratory > Berkeley, CA 94720-8235 > Phone : 510-486-5709 Email : > NWMoriarty(a)LBL.gov<mailto:[email protected]> > Fax : 510-486-5909 Web : CCI.LBL.gov<http://CCI.LBL.gov> > > -- > This e-mail and any attachments may contain confidential, copyright > and or privileged material, and are for the use of the intended > addressee only. If you are not the intended addressee or an authorised > recipient of the addressee please notify us of receipt by returning > the e-mail and do not use, copy, retain, distribute or disclose the > information in or attached to the e-mail. > Any opinions expressed within this e-mail are those of the individual > and not necessarily of Diamond Light Source Ltd. > Diamond Light Source Ltd. cannot guarantee that this e-mail or any > attachments are free from viruses and we cannot accept liability for > any damage which you may sustain as a result of software viruses which > may be transmitted in or with the message. > Diamond Light Source Limited (company no. 4375679). Registered in > England and Wales with its registered office at Diamond House, Harwell > Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United > Kingdom >

2 1

Trapping NaN or not trapping NaN
by Luc Bourhis 13 Apr '16

13 Apr '16

Richard and I have been hammering at our build system and code to try to solve several crashes on OS X with Xcode 7.3. Several of them came from the fact that we trap NaN whereas Xcode 7.3 aggressive optimisations produce code that may create a NaN but still deliver the correct final result. A trivial example came from cctbx/xray/conversions.h. We have if (f_sq > 0) f = std::sqrt(f_sq); else f = 0; With the version of clang shipping with Xcode 7.3, both branches are evaluated in parallel before selecting the result based on the test f_sq > 0. This is done so as not to stall the pipeline and this can result in much faster code indeed. But that means that every time f_sq < 0, a NaN will be produced, and in the context of our tests, this means we crash. Richard and I avoided the elephant in the room, stop to trap NaN, and as a result came with uglier and uglier fixes. Moreover, we are now at the mercy of the compiler applying such an optimisation to another file at the next update of Xcode, which would then force us to fix the compilation of said file in the relevant SConscript. It’s a loosing game. Thus I reckon that at least on OS X, we should stop trapping NaN. That will leave us with the true clang regression for which I will try to submit reduced cases to the clang bugzilla. Best wishes, Luc

5 7

Boost broken on mac?
by markus.gerstel＠diamond.ac.uk 12 Apr '16

12 Apr '16

Dials west is currently observing test failures on Mac related to Boost. To exclude any errors on our side I have run things twice clean from bootstrap: One of a number of affected tests: libtbx.python cctbx/xray/boost_python/tst_xray.py show_stack(1): /Users/dlshudson/jenkins_slave/workspace/dials_package_macosx/build_dials/modules/cctbx_project/cctbx/xray/boost_python/tst_xray.py(381) exercise_conversions show_stack(2): /Users/dlshudson/jenkins_slave/workspace/dials_package_macosx/build_dials/modules/cctbx_project/cctbx/xray/boost_python/tst_xray.py(1993) run show_stack(3): /Users/dlshudson/jenkins_slave/workspace/dials_package_macosx/build_dials/modules/cctbx_project/cctbx/xray/boost_python/tst_xray.py(2009) <module> libc backtrace (29 frames, most recent call last): 30 Python 0x0000000100000d71 Python + 3441 29 Python 0x0000000100000e58 Python + 3672 28 Python 0x00000001000ef06f Py_Main + 3135 27 Python 0x00000001000d78be PyRun_SimpleFileExFlags + 702 26 Python 0x00000001000d7d84 PyRun_FileExFlags + 164 25 Python 0x00000001000aea06 PyEval_EvalCode + 54 24 Python 0x00000001000af244 PyEval_EvalCodeEx + 2100 23 Python 0x00000001000b25c8 PyEval_EvalFrameEx + 12280 22 Python 0x00000001000b6ab6 fast_function + 118 21 Python 0x00000001000af244 PyEval_EvalCodeEx + 2100 20 Python 0x00000001000b25c8 PyEval_EvalFrameEx + 12280 19 Python 0x00000001000b6ab6 fast_function + 118 18 Python 0x00000001000af244 PyEval_EvalCodeEx + 2100 17 Python 0x00000001000b4198 PyEval_EvalFrameEx + 19400 16 Python 0x0000000100011023 PyObject_Call + 99 15 Python 0x000000010006ad8b type_call + 347 14 Python 0x00000001000708af slot_tp_init + 175 13 Python 0x0000000100011023 PyObject_Call + 99 12 Python 0x000000010001f0d8 instancemethod_call + 232 11 Python 0x0000000100011023 PyObject_Call + 99 10 libboost_python.dylib 0x0000000101888663 boost::python::objects::function_call(_object*, _object*, _object*) + 83 9 libboost_python.dylib 0x000000010188ce71 boost::python::handle_exception_impl(boost::function0<void>) + 81 8 libboost_python.dylib 0x00000001018889ca boost::detail::function::void_function_ref_invoker0<boost::python::objects::(anonymous namespace)::bind_return, void>::invoke(boost::detail::function::function_buffer&) + 26 7 libboost_python.dylib 0x000000010188693a boost::python::objects::function::call(_object*, _object*) const + 874 6 cctbx_xray_ext.so 0x0000000107c8e1c5 boost::python::detail::caller_arity<3u>::impl<void (*)(_object*, scitbx::af::const_ref<double, scitbx::af::trivial_accessor> const&, scitbx::af::const_ref<double, scitbx::af::trivial_accessor> const&), boost::python::default_call_policies, boost::mpl::vector4<void, _object*, scitbx::af::const_ref<double, scitbx::af::trivial_accessor> const&, scitbx::af::const_ref<double, scitbx::af::trivial_accessor> const&> >::operator()(_object*, _object*) + 229 5 cctbx_xray_ext.so 0x0000000107c91f44 boost::python::objects::make_holder<2>::apply<boost::python::objects::value_holder<cctbx::xray::array_f_sq_as_f<cctbx::xray::f_sq_as_f_xtal_3_7, double> >, boost::mpl::joint_view<boost::python::detail::drop1<boost::python::detail::type_list<scitbx::af::const_ref<double, scitbx::af::trivial_accessor> const&, scitbx::af::const_ref<double, scitbx::af::trivial_accessor> const&, boost::python::optional<double const&, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_>, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_> >, boost::python::optional<double const&, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_, mpl_::void_> > >::execute(_object*, scitbx::af::const_ref<double, scitbx::af::trivial_accessor> const&, scitbx::af::const_ref<double, scitbx::af::trivial_accessor> const&) + 100 4 ??? 0x0000000000000002 0x0 + 2 3 libsystem_platform.dylib 0x00007fff8caabeaa _sigtramp + 26 2 boost_python_meta_ext.so 0x0000000101841370 initboost_python_meta_ext + 0 Floating-point error (Python and libc call stacks above) This crash may be due to a problem in any imported Python module, including modules which are not part of the cctbx project. To disable the traps leading to this message, define these environment variables (e.g. assign the value 1): BOOST_ADAPTBX_FPE_DEFAULT BOOST_ADAPTBX_SIGNALS_DEFAULT This will NOT solve the problem, just mask it, but may allow you to proceed in case it is not critical. Any ideas? Best wishes, -Markus -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

6 35

cctbx can't build without iotbx
by Luc Bourhis 06 Apr '16

06 Apr '16

Hi all, am I getting mad or % libtbx.configure —only cctbx % libtbx.scons will result in a linking error because of the line marked with <<<<< in cctbx/maptbx/SConscript ? if (not env_etc.no_boost_python): Import("env_cctbx_boost_python_ext") env_bpl = env_cctbx_boost_python_ext.Clone() env_etc.include_registry.append( env=env_bpl, paths=env_etc.cctbx_common_includes) env_bpl.Prepend(LIBS=["asymmetric_map", "iotbx_xplor", "cctbx_sgtbx_asu”, <<<<<<<<<<<<<<<<<<<<<<<<<<<<< "cctbx", "omptbx"]) env_bpl.SharedLibrary( target="#lib/cctbx_asymmetric_map_ext", source=["asymmetric_map_ext.cpp"]) This code was introduced in Dec 2013!!! The only workaround % libtbx.configure cctbx iotbx So much for the modularisation with the libtbx_config’s… Best wishes, Luc

2 1

Re: [cctbxbb] sequential revision number from git repo
by markus.gerstel＠diamond.ac.uk 06 Apr '16

06 Apr '16

Hi Johan, No, not really. .gitversion is not in the repository and only kept (and updated) locally. Basically the logic is: Can I determine the current version number (ie do I have git installed and the repository present) - if so, write .gitversion. Otherwise, attempt to read .gitversion and use the information therein (this covers the route via the installer). Hence, if you download the zip file frm github you will not have a .gitversion file. But really in this case we can assume that the user knows what they're doing. -Markus On 5 Apr 2016 16:34, "Hattne, Johan" <hattnej(a)janelia.hhmi.org> wrote: Would this not mean that .gitversion is updated on every commit, at least if as in our case, it not only records the release version but also the current commit? @Rob: Apologies for my liberal use of “hook”. What our build system does is that it generates what essentially amounts to Markus’s .gitversion whenever we run “make dist” (using “git describe” or some such, it need not involve the network). There is little need for this file when compiling from a sources under git control, because it can be done on the fly. // Cheers; Johan > On Apr 5, 2016, at 09:47, markus.gerstel(a)diamond.ac.uk wrote: > > Hi Johan, > > We generate a file named '.gitversion' that lives in the main directory. That way the version number survives being packed up in an installer. > If you download the code via the "Download ZIP" button on Github there is no way (as far as I know) to recover any version information, so we fall back to a default of "DIALS 1.dev" ("DIALS 1.1" for the release branch) > > -Markus > > -----Original Message----- > From: cctbxbb-bounces(a)phenix-online.org [mailto:[email protected]] On Behalf Of Hattne, Johan > Sent: 05 April 2016 14:21 > To: cctbx mailing list > Subject: Re: [cctbxbb] sequential revision number from git repo > > Hi Markus; > > I didn’t actually read the code, but it looks like we’re doing something similar over here, and I find the the tricky bit is how to deal with any kind of releases that don’t have all the little git bits. We sort that out by having hooks in the “make dist” equivalent, which basically generate a version file that is included in the release tarball. > > However, it was recently pointed out to us that that all falls apart if someone uses the "Download ZIP” button from the code tab, because then the hooks are not run. I haven’t even attempted to RTFM around this, and if you or anybody else here has a solution I won’t have to. > > Do you by any chance? > > // Cheers; Johan > > >> On Apr 5, 2016, at 08:55, markus.gerstel(a)diamond.ac.uk wrote: >> >> Hi Rob, >> >> This is close to what we do with DIALS. >> If you run dials.version you may get e.g. >> DIALS 1.dev.132-g7042805 >> So this is the 132nd commit after the commit we tagged "1.dev", which >> happened when we branched for the 1.1 stable release so we could keep >> the version numbers sufficiently different >> >> If you run DIALS 1.1, for example the version from CCP4, you might get >> DIALS 1.1.3-gb5528a5-release The 3rd commit on the 1.1 release branch. >> >> Same with xia2: >> XIA2 0.4.0.301-g7f16ab6 >> >> In xia2 we also add the branch name if it is not 'master', so the version distributed with DIALS 1.1 is: >> XIA2 0.4.0.289-g2a1e557-dials-1.1 >> >> We kept the commit ID in - and I would recommend to keep it with cctbx, too. >> The reason for this is: >> - The number on its own does not uniquely identify a commit, as you >> might be on a different branch (although we do put in the branch >> names, too) >> - Because then I can go to >> https://github.com/dials/dials/commit/b5528a5 and see everything about >> that commit >> - Because then I can do git checkout b55528a5 and instantly have that particular code version on my machine to fiddle with things. >> >> To see the magic happen, check >> https://github.com/dials/dials/blob/master/util/version.py or >> https://github.com/xia2/xia2/blob/master/XIA2Version.py >> >> -Markus >> >> >> -----Original Message----- >> From: cctbxbb-bounces(a)phenix-online.org >> [mailto:[email protected]] On Behalf Of R.D. Oeffner >> Sent: 05 April 2016 13:38 >> To: cctbxbb(a)phenix-online.org >> Subject: [cctbxbb] sequential revision number from git repo >> >> Hi, >> >> We are slowly moving phaser svn to git. One issue that we would like to address is if there is a practical way of conveying a linear commit history from the git repo. >> >> One approach is to just count the number of revisions that was made to the master branch in the git repo with "git rev-list --count HEAD" >> and then pretend that number is a revision number. This enables us to tell users "This bug was fixed after revision 6456 of phaser" rather than "This bug was fixed after commit 29b0a1c... of phaser". Is there any pitfalls with this approach? >> >> What workarounds do other people using git come up with? >> >> >> >> Rob >> >> >> -- >> Robert Oeffner, Ph.D. >> Research Associate, The Read Group >> Department of Haematology, >> Cambridge Institute for Medical Research University of Cambridge >> Cambridge Biomedical Campus Wellcome Trust/MRC Building Hills Road >> Cambridge CB2 0XY >> >> www.cimr.cam.ac.uk/investigators/read/index.html<http://www.cimr.cam.ac.uk/investigators/read/index.html> >> tel: +44(0)1223 763234 >> mobile: +44(0)7712 887162 >> _______________________________________________ >> cctbxbb mailing list >> cctbxbb(a)phenix-online.org >> http://phenix-online.org/mailman/listinfo/cctbxbb >> >> -- >> This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. >> Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. >> Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. >> Diamond Light Source Limited (company no. 4375679). Registered in >> England and Wales with its registered office at Diamond House, Harwell >> Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United >> Kingdom >> >> >> _______________________________________________ >> cctbxbb mailing list >> cctbxbb(a)phenix-online.org >> http://phenix-online.org/mailman/listinfo/cctbxbb > > Research Specialist @ Gonen Lab ____________________________________________________ > Janelia Research Campus * 19700 Helix Drive Ashburn, VA 20147 * +1 (571) 209-4000 extension 3376 > > > _______________________________________________ > cctbxbb mailing list > cctbxbb(a)phenix-online.org > http://phenix-online.org/mailman/listinfo/cctbxbb > > -- > This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. > Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. > Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. > Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom > > _______________________________________________ > cctbxbb mailing list > cctbxbb(a)phenix-online.org > http://phenix-online.org/mailman/listinfo/cctbxbb Research Specialist @ Gonen Lab ____________________________________________________ Janelia Research Campus * 19700 Helix Drive Ashburn, VA 20147 * +1 (571) 209-4000 extension 3376 _______________________________________________ cctbxbb mailing list cctbxbb(a)phenix-online.org http://phenix-online.org/mailman/listinfo/cctbxbb

3 3

CCTBX.XFEL problems with synchrotron stills
by Schulz, Eike-Christian 06 Apr '16

06 Apr '16

Dear all, I am new to serial crystallography and trying to process still images from a synchrotron source (Pilatus 6M detector) using CCTBX. Obviously the process is not glitch-free and I was hoping to get some help here. The indexing routine starts, indexes the images correctly and and produces a log file but no .pickle files are created in the results/000/ directory as described in the wiki entry. I assume that there is a software module that is not properly loaded during the process since I get following error message during indexing: ---------BEGIN Integrate one frame 75 warsaw0048_00075.pickle EFFECTIVE TILING 0 0 2282 2282 Indexing error: No module named cxi_user Is there a way to find out if all dependencies are matched ? I suppose CCTBX.XFEL depends on some (specific) python libraries … ? Or am I completely wrong and the error is something different ? With best regards Eike

3 4