Hello All, I spent some time to build cctbx on an OS X 10.7 box for a non-system interpreter. I ran into some kinks along the way, so I thought I'd share my notes in case anyone wants to do the same. These notes are in reStructuredText, so it was no extra work to turn them into the attached HTML file, which might make reading a little easier. James ==================================== Build Notes for cctbx on OS X 10.7.5 ==================================== These notes describe a successful build of cctbx against a specific python interpreter, namely `Enthought python`_ 7.3-2 for x86_64. Although working, versions of cctbx are available for both the system python or with a dedicated python, these canned solutions are not optimal. First, it's not advisable to modify system utilities. Although cctbx seems well-behaved in this regard, other packages may not be so well behaved. Second, the canned approach requires building many of the numerous packages that come with bundeld python distributions like the Enthought distribution (some such packages exceed even cctbx in their difficulty to build). Third, if every package required its own python, every script would need to fire-up dozens of separate interpreters just to do its job, resulting in significant computational overhead with consummately slower execution times. .. _`Enthought python`: http://www.enthought.com/products/epd.php 1. The default c++ compiler for OS X 10.7 is actually clang (``/usr/bin/c++``). Using clang exposes invalid template code. The first of many errors is:: In file included from /opt/cctbx/cctbx_sources/mmtbx/bulk_solvent/bulk_solvent_ext.cpp:7: /opt/cctbx/cctbx_sources/mmtbx/bulk_solvent/bulk_solvent.h:496:21: error: call to function 'r_factor' that is neither visible in the template definition nor found by argument-dependent lookup r.push_back(r_factor(f_obs, f_model.const_ref())); These types of errors are documented at the page http://clang.llvm.org/compatibility.html#dep_lookup .. note:: The installation directory is ``/opt/cctbx``. 2. I tried to use the HPC compilers at http://hpc.sourceforge.net/, but they do not recognize the ``-no-cpp-precomp`` flag. The web page https://svn.boost.org/trac/boost/ticket/6697 suggests the problem is with boost. However, according to the comments therein, this should have been fixed in boost 1.50. I am building against boost 1.52. Perplexed about the source of this compiler flag, I tracked it down to the file ``cctbx_sources/libtbx/SConscript``, after spending much of a day learning how scons goes about its business. There are two possible fixes: - Comment the following line in ``cctbx_sources/libtbx/SConscript``:: # env_etc.ccflags_base.extend(["-no-cpp-precomp"]) - Link ``/usr/bin/g++`` to ``c++`` in a way that it will be found first in the path. For example:: % ln -s /usr/bin/g++ ${HOME}/bin/c++ 3. I first fixed the ``-no-cpp-precomp`` error by the former, commenting the offending line in ``cctbx_sources/libtbx/SConscript`` but this failed with the error:: /opt/cctbx/cctbx_sources/scitbx/array_family/memory.h:35:47: error: operator '&&' has no right operand #elif defined(_XOPEN_SOURCE) && _XOPEN_SOURCE >= 600 && defined(_POSIX_ADVISORY_INFO) ^ The page http://forums.fedoraforum.org/archive/index.php/t-52929.html suggests that the naked ``_XOPEN_SOURCE`` wrecks the conditional when ``_XOPEN_SOURCE`` is not defined. One possible way to overcome this problem is to change the offending line into a nested if/then, hoping that the comparison never gets evaluated under circumstances when ``_XOPEN_SOURCE`` is not defined:: #elif defined(_XOPEN_SOURCE) #if _XOPEN_SOURCE >= 600 && defined(_POSIX_ADVISORY_INFO) #define SCITBX_AF_HAS_ALIGNED_MALLOC 1 #define SCITBX_AF_USE_POSIX_FOR_ALIGNED_MALLOC 1 #endif #endif However, this was unnecessary because the second fix to the ``-no-cpp-precomp`` error (linking ``/usr/bin/g++`` to ``c++``) worked and avoided changing the preprocessor source. 4. Left to its own devices, scons could not find my ``fftw3.h``, which I have installed (via fink) to ``/sw/include/fftw3.h``. To fix this problem, I inserted the second of the following two lines into the file ``cctbx_sources/libtbx/SConscript``:: env_etc.ccflags_base.extend(["-no-cpp-precomp"]) env_etc.ccflags_base.extend(["-I/sw/include"]) Although this allowed scons to find the header, it could not find the actual library, requiring the insertion of the second of the following lines into the file ``cctbx_sources/libtbx/SConscript``:: env_etc.shlinkflags_bpl = [ "-L/sw/lib", "-w", # suppress "source/lib does not exist" warning "-bundle", "-undefined", "dynamic_lookup"]
Hi James,
1. The default c++ compiler for OS X 10.7 is actually clang (``/usr/bin/c++``). Using clang exposes invalid template code. The first of many errors is::
In file included from /opt/cctbx/cctbx_sources/mmtbx/bulk_solvent/bulk_solvent_ext.cpp:7: /opt/cctbx/cctbx_sources/mmtbx/bulk_solvent/bulk_solvent.h:496:21: error: call to function 'r_factor' that is neither visible in the template definition nor found by argument-dependent lookup r.push_back(r_factor(f_obs, f_model.const_ref()));
we definitively need to fix that then. I'll give it a go over the weekend. Best wishes, Luc
Hi James,
Can I ask what version of the cctbx are you attempting to build? For some
time (at least several months) it has been possible to build the cctbx (and
the whole of Phenix) using the default clang on the latest Mac OS X. Most
of the Phenix developers here in Berkeley are now using clang on the latest
OS X, so someone would almost certainly notice immediately if something
wasn't building correctly. The latest "released" build of the cctbx (dating
from May 2012!) does not build with clang as I recall. Unfortunately we do
not have an official release mechanism for the cctbx, beyond Nat deciding
when to mark one of the nightly builds as a release. Clearly the current
"released" version is rather old and we should replace it with a more
recent version. For now I would recommend trying with one of the more
recent nightly builds (e.g.
http://cci.lbl.gov/cctbx_build/show_results.cgi?build_tag=2013_03_22_0005),
or checking the code out directly from svn.
Cheers,
Richard
On 22 March 2013 03:36, Luc Bourhis
Hi James,
1. The default c++ compiler for OS X 10.7 is actually clang (``/usr/bin/c++``). Using clang exposes invalid template code. The first of many errors is::
In file included from /opt/cctbx/cctbx_sources/mmtbx/bulk_solvent/bulk_solvent_ext.cpp:7: /opt/cctbx/cctbx_sources/mmtbx/bulk_solvent/bulk_solvent.h:496:21: error: call to function 'r_factor' that is neither visible in the template definition nor found by argument-dependent lookup r.push_back(r_factor(f_obs, f_model.const_ref()));
we definitively need to fix that then. I'll give it a go over the weekend.
Best wishes,
Luc
_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb
On Fri, Mar 22, 2013 at 11:48 AM, Richard Gildea
Can I ask what version of the cctbx are you attempting to build? For some time (at least several months) it has been possible to build the cctbx (and the whole of Phenix) using the default clang on the latest Mac OS X. Most of the Phenix developers here in Berkeley are now using clang on the latest OS X, so someone would almost certainly notice immediately if something wasn't building correctly. The latest "released" build of the cctbx (dating from May 2012!) does not build with clang as I recall. Unfortunately we do not have an official release mechanism for the cctbx, beyond Nat deciding when to mark one of the nightly builds as a release. Clearly the current "released" version is rather old and we should replace it with a more recent version.
Done - it's the same build that the 1.8.2 release is derived from. -Nat
Hi Richard, I'm positive I did the most naïve thing (i.e. what a typical user would do) and 1. googled "cctbx" 2. clicked on the top hit (http://cctbx.sourceforge.net/) 3. clicked on downloads (http://cci.lbl.gov/cctbx_build/) 4. scrolled down to sources and clicked the first link (http://cci.lbl.gov/cctbx_build/results/2012_05_08_2305/cctbx_python_273_bund...) Note that the build tag (2012_05_08_2305) of the sources match the precompiled binaries. This tag also matches the most recent of the "Local build times" panel. I didn't notice the "Show unreleased builds" link, but I'm not sure it would have made much difference because it doesn't explain what those builds are. A helpful addition to the download page might be conspicuous instructions for an svn/git/hg/etc checkout of the "latest-greatest" development tree, if the sources are preferable to the released builds. Many projects support that mechanism of distribution. For some projects, like git itself (http://git-scm.com/downloads), this is the recommended way of acquiring the sources. If the package needs to be bundled into an archive, maybe there should be a conspicuous "nightly build" link. James On Mar 22, 2013, at 12:48 PM, Richard Gildea wrote:
Hi James,
Can I ask what version of the cctbx are you attempting to build? For some time (at least several months) it has been possible to build the cctbx (and the whole of Phenix) using the default clang on the latest Mac OS X. Most of the Phenix developers here in Berkeley are now using clang on the latest OS X, so someone would almost certainly notice immediately if something wasn't building correctly. The latest "released" build of the cctbx (dating from May 2012!) does not build with clang as I recall. Unfortunately we do not have an official release mechanism for the cctbx, beyond Nat deciding when to mark one of the nightly builds as a release. Clearly the current "released" version is rather old and we should replace it with a more recent version. For now I would recommend trying with one of the more recent nightly builds (e.g. http://cci.lbl.gov/cctbx_build/show_results.cgi?build_tag=2013_03_22_0005), or checking the code out directly from svn.
Cheers,
Richard
On 22 March 2013 03:36, Luc Bourhis
wrote: Hi James, 1. The default c++ compiler for OS X 10.7 is actually clang (``/usr/bin/c++``). Using clang exposes invalid template code. The first of many errors is::
In file included from /opt/cctbx/cctbx_sources/mmtbx/bulk_solvent/bulk_solvent_ext.cpp:7: /opt/cctbx/cctbx_sources/mmtbx/bulk_solvent/bulk_solvent.h:496:21: error: call to function 'r_factor' that is neither visible in the template definition nor found by argument-dependent lookup r.push_back(r_factor(f_obs, f_model.const_ref()));
we definitively need to fix that then. I'll give it a go over the weekend.
Best wishes,
Luc
_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb
_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb
On Fri, Mar 22, 2013 at 12:39 PM, James Stroud
Hi Richard,
I'm positive I did the most naïve thing (i.e. what a typical user would do) and
1. googled "cctbx" 2. clicked on the top hit (http://cctbx.sourceforge.net/) 3. clicked on downloads (http://cci.lbl.gov/cctbx_build/) 4. scrolled down to sources and clicked the first link (http://cci.lbl.gov/cctbx_build/results/2012_05_08_2305/cctbx_python_273_bund...)
This should now be changed to the build from late February (2013).
A helpful addition to the download page might be conspicuous instructions for an svn/git/hg/etc checkout of the "latest-greatest" development tree, if the sources are preferable to the released builds.
Unfortunately at present the builds include a lot of stuff that isn't in cctbx proper, including some SVN trees which aren't publicly available. I'd like to change this, but we need to discuss it internally.
Many projects support that mechanism of distribution. For some projects, like git itself (http://git-scm.com/downloads), this is the recommended way of acquiring the sources. If the package needs to be bundled into an archive, maybe there should be a conspicuous "nightly build" link.
Agreed. -Nat
participants (4)
-
James Stroud
-
Luc Bourhis
-
Nathaniel Echols
-
Richard Gildea