- cctbxbb - phenix-online.org

proposal to remove iotbx.shelx.from_ins
by Mirko Scholz 12 Jun '12

12 Jun '12

Hello, since I am new here it seems an odd proposal, but could you not remove iotbx.shelx.from_ins.from_ins since its functionality is covered much better by xray.structure.from_shelx (at least it seems to me). The following is roughly identical with the first statement being inaccurate (anisotropic vs isotropic displacements parameters) and wrong (structures containing disorder) xs0 = iotbx.shelx.from_ins.from_ins(file_name='foo.ins') fc0 = xs0.structure_factors(d_min=0.7, algorithm='direct').f_calc() xs1 = cctbx.xray.structure.from_shelx(file=open('foo.ins')) fc1 = xs1.structure_factors(d_min=0.7, algorithm='direct').f_calc() As someone who (seldom) uses cctbx I stumbled first over iotbx.shelx.from_ins and only noticed after a while that it does not handle disorder properly (problem is in ins_record.if_10_added). from_ins is used only here ./iotbx/command_line/emma.py ./cctbx/adp_restraints/tst_ext.py Where it should be possible to substitute it though I have not tested it. Cordially Mirko Scholz

4 4

controlling types over the C++/Python interface
by David Waterman 16 May '12

16 May '12

Hi, I'm starting to explore the use of boost.python in cctbx. I've written a simple C++ class and exposed its accessor methods as attributes using the .add_property syntax in the wrapper. In the C++ class my member data and accessor return types are, for example, scitbx::vec3<double>. When I access the attribute in Python with the usual dot operator I get a tuple. What I'd really like though is an object of type <class 'scitbx.matrix.col'>, which is what I passed to the constructor of the object in Python. So the conversion from Python to C++ does what I want, but not the other way round. How do I take control of the type of the attribute in Python, without having to explicitly construct a scitbx.matrix.col from the tuple? It's the same case if I use .def and expose the accessor as a method instead of an attribute. I admit I don't fully understand the details of conversions in either direction. I found I had to add from cctbx.array_family import flex to the package __init__.py where my C++ extension is imported, so that I could pass scitbx.matrix.col objects (or scitbx.matrix.rec, or tuples) to the constructor in Python and not get a Boost.Python.ArgumentError. I've not yet figured out what this import is doing though. Apologies for the remedial question. These are still early days for me. Thanks -- David

5 12

Niggli-reduced cell C++ implementation
by Martin Uhrin 24 Apr '12

24 Apr '12

Dear Cctbx community, Firstly I'd like to say thank you to Rolf, Nicholas and Paul for their expertly thought through implementation of the reduced cell algorithm. I've found it to be extremely useful for my work. My code is all in C++ and I'd like to be able to use the Krivy-Gruber algorithm. My understanding is that only the reduced (Buerger) unit cell algorithm is implemented in C++ [1] which guarantees shortest lengths but not unique angles. From my understanding the Krivy-Gruber would also guarantee me uniqueness of unit cell angles, however this is only implemented in Python [2]. Sorry to be so verbose, I just wanted to check that I was on the right page. Would it be possible for me to implement the Krivy-Gruber in C++ by adding in the epsilon_relative to the parameter and following the procedure found in the python version? Many thanks, -Martin [1] http://cctbx.sourceforge.net/current/c_plus_plus/classcctbx_1_1uctbx_1_1fas… [2] http://cctbx.sourceforge.net/current/python/cctbx.uctbx.krivy_gruber_1976.h… -- Martin Uhrin Tel: +44 207 679 3466 Department of Physics & Astronomy Fax:+44 207 679 0595 University College London martin.uhrin.10(a)ucl.ac.uk Gower St, London, WC1E 6BT, U.K. http://www.cmmp.ucl.ac.uk

3 8

Handling unmerged reflections from MTZ
by Graeme.Winter＠Diamond.ac.uk 24 Apr '12

24 Apr '12

Hi Folks, I have been writing very bad Python code to handle unmerged MTZ files using CCTBX - in essence copying all of the measurements to lists for each Miller index. Inefficient. Has anyone looked a neater way of handling these files perhaps with doing the "hard work" from C++? Reading the file is very easy, but managing the data is harder. Essentially I would like to be able to read in a scaled-summed-but-unmerged MTZ file (equivalent to unmerged scale pack no-merge-original-index) and get back all observations of a given HKL, which I can then subsequently analyse… I could crack on and write something but it strikes me as a good idea to ask first. Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 2

Branches
by Graeme.Winter＠Diamond.ac.uk 22 Apr '12

22 Apr '12

Hi Folks, Quick question - does anyone ever use branches with the CCTBX repository? Just at the moment I'm up to a few things which I would like to be able to commit, but I don't want to break the existing code... however as a project it does not seem CCTBX uses branches much... Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 9

Type checking in Phil...
by Graeme.Winter＠Diamond.ac.uk 11 Apr '12

11 Apr '12

… is there any? This does not give an error: from libtbx.phil import parse working_phil = parse(""" parameter { toy = 10 .type = int } """) if __name__ == '__main__': working_phil.show() # this should crash right? working_phil = working_phil.fetch(parse('parameter.toy=foo')) working_phil.show() Output: cctbx.python toy_phil.py parameter { toy = 10 } parameter { toy = foo } Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 3

Build on OS X Lion
by Graeme.Winter＠Diamond.ac.uk 23 Feb '12

23 Feb '12

Hi Folks, Just ran the cctbx_svn_getting_started script to do a build on a new lion system and I got this error - had anyone else got this working correctly on such a system? Template definition not found is a little beyond my usual C++ debugging skills. Many thanks, Graeme c++ -o scitbx/math/boost_python/tensor_rank_2.o -c -DBOOST_ALL_NO_LIB -DBOOST_DISABLE_THREADS -DBOOST_PYTHON_MAX_BASES=2 -I/Users/graeme/svn/cctbx/sources/boost -fPIC -fno-strict-aliasing -w -no-cpp-precomp -DNDEBUG -O3 -ffast-math -Iinclude -I/Users/graeme/svn/cctbx/sources/include -I/Users/graeme/svn/cctbx/sources/cctbx_project/include -I/Users/graeme/svn/cctbx/sources/cctbx_project -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/tensor_rank_2.cpp clang: warning: argument unused during compilation: '-no-cpp-precomp' In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9: In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:339:16: error: call to function 'bessel_j_small_z_series' that is neither visible in the template definition nor found by argument-dependent lookup Jv = bessel_j_small_z_series(v, x, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:119:4: note: in instantiation of function template specialization 'boost::math::detail::bessel_jy<long double, boost::math::policies::policy<boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> >' requested here bessel_jy(v, x, &j, &y, need_j, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:165:9: note: in instantiation of function template specialization 'boost::math::detail::cyl_bessel_j_imp<long double, boost::math::policies::policy<boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> >' requested here * cyl_bessel_j_imp(T(T(n)+T(0.5f)), x, bessel_no_int_tag(), pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:449:65: note: in instantiation of function template specialization 'boost::math::detail::sph_bessel_j_imp<long double, boost::math::policies::policy<boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> >' requested here return policies::checked_narrowing_cast<result_type, Policy>(detail:... ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:455:11: note: in instantiation of function template specialization 'boost::math::sph_bessel<double, boost::math::policies::policy<boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy, boost::math::policies::default_policy> >' requested here return sph_bessel(v, x, policies::policy<>()); ^ /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:291:13: note: in instantiation of function template specialization 'boost::math::sph_bessel<double>' requested here return( sph_bessel(l,x) ); ^ /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:363:9: note: in instantiation of function template specialization 'scitbx::math::bessel::spherical_bessel<double>' requested here bessel::spherical_bessel); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:47:10: note: 'bessel_j_small_z_series' should be declared prior to the call site or in namespace 'boost::math::policies' inline T bessel_j_small_z_series(T v, T x, const Policy& pol) ^ In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9: In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:346:16: error: call to function 'bessel_y_small_z_series' that is neither visible in the template definition nor found by argument-dependent lookup Yv = bessel_y_small_z_series(v, x, &Yv_scale, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:139:10: note: 'bessel_y_small_z_series' should be declared prior to the call site or in namespace 'boost::math::policies' inline T bessel_y_small_z_series(T v, T x, T* pscale, const Policy& pol) ^ In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9: In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:356:16: error: call to function 'bessel_j_small_z_series' that is neither visible in the template definition nor found by argument-dependent lookup Jv = bessel_j_small_z_series(v, x, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:47:10: note: 'bessel_j_small_z_series' should be declared prior to the call site or in namespace 'boost::math::policies' inline T bessel_j_small_z_series(T v, T x, const Policy& pol) ^ In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/math_ext.cpp:3: In file included from /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/bessel.h:9: In file included from /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/bessel.hpp:17: /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy.hpp:363:16: error: call to function 'bessel_yn_small_z' that is neither visible in the template definition nor found by argument-dependent lookup Yv = bessel_yn_small_z(n, x, &Yv_scale, pol); ^ /Users/graeme/svn/cctbx/sources/boost/boost/math/special_functions/detail/bessel_jy_series.hpp:213:3: note: 'bessel_yn_small_z' should be declared prior to the call site or in namespace 'boost::math::policies' T bessel_yn_small_z(int n, T z, T* scale, const Policy& pol) ^ 4 errors generated. c++ -o scitbx/math/boost_python/chebyshev.o -c -DBOOST_ALL_NO_LIB -DBOOST_DISABLE_THREADS -DBOOST_PYTHON_MAX_BASES=2 -I/Users/graeme/svn/cctbx/sources/boost -fPIC -fno-strict-aliasing -w -no-cpp-precomp -DNDEBUG -O3 -ffast-math -Iinclude -I/Users/graeme/svn/cctbx/sources/include -I/Users/graeme/svn/cctbx/sources/cctbx_project/include -I/Users/graeme/svn/cctbx/sources/cctbx_project -I/System/Library/Frameworks/Python.framework/Versions/2.7/include/python2.7 /Users/graeme/svn/cctbx/sources/cctbx_project/scitbx/math/boost_python/chebyshev.cpp clang: warning: argument unused during compilation: '-no-cpp-precomp' scons: *** [scitbx/math/boost_python/math_ext.o] Error 1 scons: building terminated because of errors. usr+sys time: 23.10 seconds wall clock time: 3 minutes 39.02 seconds (219.02 seconds total) -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

3 6

Comparison of Crystal Structures
by David Lonie 13 Jan '12

13 Jan '12

Hi list, I have written an open-source (BSD-licensed) algorithm to compare two crystal structure descriptions and identify whether they describe the same underlying structure. The algorithm is capable of detecting degenerate descriptions (meaning unit cell vectors + atomic positions and types) even in the presence of the "real-world" issues confronting computational crystallographers (choice of unit cell representation and offset, rotations, reflections, and numerical noise). The algorithm returns a boolean match using customizable tolerances. I just wanted to inform the mailing list of this, in case such functionality is needed by your user community. The code, XtalComp, is written in C++ and is easy to interface with, having no external dependencies. The liberal license would allow it to be included with the cctbx sources and interfaced with cctbx code if desired. The code uses the CMake build system, but the simple nature of the sources would allow trivial porting to SCons. There is a web interface to the code here: http://xtalopt.openmolecules.net/xtalcomp/xtalcomp.html Also on that page are links to the journal article describing the algorithm, as well as a free download of the preprint version of the article. The code is available on Github (http://www.github.com/dlonie/XtalComp). Thanks for your time, and I hope that someone out there will find this code useful :-) Keep up the good work! Dave

2 1

Attitude of rotation, Python to C++
by Graeme.Winter＠Diamond.ac.uk 13 Jan '12

13 Jan '12

Hi Folks, Just started moving code from Python to C++ - and seeing a 40x speedup which is nice - however I noticed something odd. scitbx.matrix.col.rotate(axis, angle) has the reversed attitude to scitbx::vec3<double>.rotate(scitbx::vec3<double>, double angle) viz: this python code s = (self._ub * hkl).rotate(self._axis, angle) q = (s + self._s0).normalize() # check if diffracted ray parallel to detector face q_dot_n = q.dot(detector_normal) if q_dot_n == 0: continue r = (q * distance / q_dot_n) - self._detector_origin x = r.dot(self._detector_fast) y = r.dot(self._detector_slow) if x < self._limits[0] or y < self._limits[2]: continue if x > self._limits[1] or y > self._limits[3]: continue observed_reflection_positions.append((hkl, x, y, angle)) gives the same result as scitbx::vec3<double> s, q, r; double x, y, q_dot_n; /* this is rather weird - the attitude of rotation is back to front in the c++ code */ s = (ub * hkl).rotate(axis, - angle); q = (s + s0).normalize(); q_dot_n = q * normal; if (q_dot_n == 0) return false; r = (q * distance / q_dot_n) - origin; x = r * fast; y = r * slow; if (x < limits[0]) return false; if (y < limits[2]) return false; if (x > limits[1]) return false; if (y > limits[3]) return false; prediction[0] = x; prediction[1] = y; where I had to use -angle. Code is linked into same routines - used identically. With -angle I get numerically identical results (which is nice) Anyone got any clues on this one? Thanks, Graeme -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom

2 2

How to use the binner in cctbx.miller?
by Jan Marten Simons 03 Jan '12

03 Jan '12

Hi, as documentation is lacking on this: How can I use the binner from cctbx.miller to bin reflections into different groups according to their d-value and would it be possible afterwards to get integral intesities of each binned group easily? Thanks, Jan

3 3