Could you re-do the patch to reduce repetitive code?

E.g.

-if (env_etc.static_libraries): builder = envlm.StaticLibrary -else: builder = envlm.SharedLibrary +if (env_etc.static_libraries): + builder = envlm.StaticLibrary +else: + if hasattr(envlm, 'VersionedSharedLibrary'): + builder = envlm.VersionedSharedLibrary + else: + builder = envlm.SharedLibrary +

builder = env_etc.get_library_builder(env=envlm)

with the currently repeated code centralized in libtbx/SConscript under

def get_library_builder(env):

If you send me your sourceforge user name I'll enable svn write permission for you, so that you can check in the patch directly. I can try, but I don't have much experience with Scons so far, therefore I'd prefer not to write directly to svn, so I don't break anything :D But if someone could review my patch before, I can do it. My account is 'raybuntu'

We don't depend on PyCifRW anymore. Instead we are now using iotbx.cif; http://cctbx.sf.net/iotbx_cif/

Ok, I just found that in your cctbx_bundle.tar.gz and thought it's still active.

When you copy .py files into the /usr directories, could you automatically insert from __future__ import division as the first line of all .py files?

In case of a Debian package I'd have to write a patch, since Debian doesn't allow direct manipulation of upstream tarball's. But I could do it very easy and fast with my zsh and sed: ''' cd cctbx_sources sed -i '1i from __future__ import division' **/*.py ''' Probably a setup.py could correct that too.

This question has been asked several times before. It would indeed be good to have optional support for something that works like "python setup.py install". I'm not sure what's the best way to implement this. Maybe start with the scripts that you have already? Would it make sense to check them in under an new directory? (Note that we already have rpmbuild at the top level; does this help in any way?)

My hacks to get things to the places they belong are some basic shell tricks that are very bad and I'm not satisfied with them. Today I started making a setup.py.

The libtbx/configure framework is designed to avoid copies of Python sources. If you run "python setup.py install" copies are made. This is extremely confusing in development. I.e. we really need to keep the libtbx/configure framework, but it would be good to also have scripts that copy into the /usr directories for needs like yours.

OK, I have an opinion how this could be done (just an idea): The libtbx/configure is not bad. Maybe it could produce a 'setup.py'. The SConscripts should offer an optional install target (scons supports that). This install target copies the shared libraries to $prefix/lib, extensions to $prefix/lib/python.X.Y/pymodules, runs a 'setup.py' which installs all modules to $prefix/share/pyshared. So nothing would really change the way it is the only thing would be an optional: "scons install PREFIX=/usr" to copy the files. OK the paths it installs are very Debian specific here but I think we can find a solution here, mostly because Debian has a helper script that can correct paths very good. I also think we need to distinguish cctbx developer and end user. The end user is not really interested in cctbx_sources. Either she/he is a C ++ or a Python developer. So when she/he is a Python developer he just needs the modules, extensions and shared libraries. The *.h, *.hpp, *.c, *.cpp files doesn't interested her/him. On the other hand the C++ Dev. only needs the shared libraries + some header files. Radi

I've added you to the list of users with svn write permissions. If your proposed change works on your platform, please check it in. In this case it will probably work everywhere. It is easy to temporarily back out the changes again if necessary. (Please see libtbx/development/dev_guidelines.txt if you get a chance.) OK, I will change my patch and upload it then.

Great. If you like check it in under libtbx. That would enable others to help out; I don't have much spare time but I'll definitely look and maybe there are a few quick things I can help with.

I need to test that first and then I can share it.

Do we need the dependency analysis for this? I'd expect that the install operation finishes in a few seconds. It is probably easier to write a plain Python script not involving SCons. I guess it's probably easier to make an install target in scons. Because scons always knows what was built.

That's true, but it will make things more complicated. The C++ sources can be very useful as a reference even for pure Python users. For the first pass I suggest copying the complete source tree. Getting all include files will need extra attention. Some are generated when you run libtbx/configure.py, under <build>/include. That reminds me of something. There were some other problems too. The SConscripts are trying to build libantlr3, libcbf and libclipper and then link against them. But this libraries already exist in most distributions. I solved it commenting (patch) out the builder for Debian, but this is no solution for upstream. I don't know how this could be solved. Maybe the "get_library_builder(env)" could check for libraries first and only build if they're missing? Also you are building libann that is manipulated with the self_include part. This conflicts with the libann a distribution is offering. I solved it for my Debian package by building and linking it statically to the extensions. Would it be possible to rename the lib to i.e. libann-adaptbx? Also I had to rename libclipper to libclipper-core, libantlr3 to libantlr3c.

I realize this is a lot of changes I'm requesting and they are only suggestions, I don't wanna push anything. It's not a big deal for me to patch this only for my package all the time, so I wouldn't mind if you don't like my ideas.

Hi Ralf,

Renaming all the libraries you list is fine.

My main worry is API compatibility with all those libraries. These libraries are tiny in comparison to everything else, therefore we just compile them from scratch. Managing compatibility with system libraries could introduce a lot of complications eating away your time. Is the minute saving in disk space and compilation time worth the trouble?

That's why the soname and soversion would be important here. Probably you are right with the time and disk space. I'll just compile all python extensions statically. So maybe that patch I submitted is not needed anymore. Radi