bootstrap external dependencies
Good morning, last night we ran into a problem with some of the remaining external dependencies in the bootstrap script base install step. The sphinx package will currently download 11 files during its installation (list below). As with all other downloads I would like to be able to download these files beforehand and then pass them to our build systems directly, so that the subsequent builds are no longer dependant on network connectivity or pypi server reliability. I would like to propose a python-extra package for the bootstrap script, that checks the existence of a base_tmp/python_extra directory, and, if found, would install all python packages in that folder. This would be an easy facility to automatically build extra python packages (at DLS we already add a few extra packages, such as junit-xml, so this would actually simplify our build process), and would not affect any other bootstrap users. Alternatively, since the dependencies are fixed as they come from the Sphinx archive on the cci servers, we could package them up as well and put static build instructions into build_sphinx() in install_base_packages.py. Thoughts? -Markus The packages are: https://pypi.python.org/packages/source/J/Jinja2/Jinja2-2.7.3.tar.gz#md5=b9d... https://pypi.python.org/packages/source/d/docutils/docutils-0.12.tar.gz#md5=... https://pypi.python.org/packages/source/P/Pygments/Pygments-2.0.2.tar.gz#md5... https://pypi.python.org/packages/source/M/MarkupSafe/MarkupSafe-0.23.tar.gz#... https://pypi.python.org/packages/source/n/nose/nose-1.3.7.tar.gz#md5=4d3ad0f... https://pypi.python.org/packages/source/p/pyparsing/pyparsing-2.0.3.tar.gz#m... https://pypi.python.org/packages/source/t/tornado/tornado-4.2.tar.gz#md5=c54... https://pypi.python.org/packages/source/p/python-dateutil/python-dateutil-2.... https://pypi.python.org/packages/source/c/certifi/certifi-2015.04.28.tar.gz#... https://pypi.python.org/packages/source/b/backports.ssl_match_hostname/backp... https://pypi.python.org/packages/source/s/six/six-1.9.0.tar.gz#md5=476881ef4... -- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
Markus
This seems fine to me. As you stated, as long as it doesn't affect other
bootstrap users or the builds, it should be fine.
Cheers
Nigel
---
Nigel W. Moriarty
Building 64R0246B, Physical Biosciences Division
Lawrence Berkeley National Laboratory
Berkeley, CA 94720-8235
Phone : 510-486-5709 Email : [email protected]
Fax : 510-486-5909 Web : CCI.LBL.gov
On Tue, Jun 16, 2015 at 1:22 AM,
Good morning,
last night we ran into a problem with some of the remaining external dependencies in the bootstrap script base install step. The sphinx package will currently download 11 files during its installation (list below). As with all other downloads I would like to be able to download these files beforehand and then pass them to our build systems directly, so that the subsequent builds are no longer dependant on network connectivity or pypi server reliability.
I would like to propose a python-extra package for the bootstrap script, that checks the existence of a base_tmp/python_extra directory, and, if found, would install all python packages in that folder. This would be an easy facility to automatically build extra python packages (at DLS we already add a few extra packages, such as junit-xml, so this would actually simplify our build process), and would not affect any other bootstrap users.
Alternatively, since the dependencies are fixed as they come from the Sphinx archive on the cci servers, we could package them up as well and put static build instructions into build_sphinx() in install_base_packages.py.
Thoughts?
-Markus
The packages are:
https://pypi.python.org/packages/source/J/Jinja2/Jinja2-2.7.3.tar.gz#md5=b9d...
https://pypi.python.org/packages/source/d/docutils/docutils-0.12.tar.gz#md5=...
https://pypi.python.org/packages/source/P/Pygments/Pygments-2.0.2.tar.gz#md5...
https://pypi.python.org/packages/source/M/MarkupSafe/MarkupSafe-0.23.tar.gz#...
https://pypi.python.org/packages/source/n/nose/nose-1.3.7.tar.gz#md5=4d3ad0f...
https://pypi.python.org/packages/source/p/pyparsing/pyparsing-2.0.3.tar.gz#m...
https://pypi.python.org/packages/source/t/tornado/tornado-4.2.tar.gz#md5=c54...
https://pypi.python.org/packages/source/p/python-dateutil/python-dateutil-2....
https://pypi.python.org/packages/source/c/certifi/certifi-2015.04.28.tar.gz#...
https://pypi.python.org/packages/source/b/backports.ssl_match_hostname/backp...
https://pypi.python.org/packages/source/s/six/six-1.9.0.tar.gz#md5=476881ef4...
-- This e-mail and any attachments may contain confidential, copyright and or privileged material, and are for the use of the intended addressee only. If you are not the intended addressee or an authorised recipient of the addressee please notify us of receipt by returning the e-mail and do not use, copy, retain, distribute or disclose the information in or attached to the e-mail. Any opinions expressed within this e-mail are those of the individual and not necessarily of Diamond Light Source Ltd. Diamond Light Source Ltd. cannot guarantee that this e-mail or any attachments are free from viruses and we cannot accept liability for any damage which you may sustain as a result of software viruses which may be transmitted in or with the message. Diamond Light Source Limited (company no. 4375679). Registered in England and Wales with its registered office at Diamond House, Harwell Science and Innovation Campus, Didcot, Oxfordshire, OX11 0DE, United Kingdom
_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb
participants (2)
-
markus.gerstel@diamond.ac.uk
-
Nigel Moriarty