Hi,
Whilst looking into performance of various DIALS routines recently, I noticed that the flex.int.as_double() conversion takes roughly 3x as long as the equivalent functionality in numpy:
(the choice of a 2527x2463 array is simply because this is the size of the raw data array for a Pilatus 6M detector)
import time
from dials.array_family import flex
n = 100
a = flex.int(flex.grid(2527, 2463))
t0 = time.time()
for i in range(n):
b = a.as_double()
t1 = time.time()
t = t1-t0
print "flex.int.as_double(): %.2fs (%.3fs/call)" %(t, t/n)
import numpy as np
a = np.random.randint(100, size=a.all())
t0 = time.time()
for i in range(n):
b = a.astype(np.float64)
t1 = time.time()
t = t1-t0
print "numpy.astype(np.float64): %.2fs (%.3fs/call)" %(t, t/n)
$ libtbx.python time_as_double.py
flex.int.as_double(): 5.76s (0.058s/call)
numpy.astype(np.float64): 1.86s (0.019s/call)
This is the current version of the as_double function:
static flex_double
as_double(f_t const& a)
{
shared_plain<double> result(a.size(), init_functor_null<double>());
for(std::size_t i=0;i
Hi Richard,
On 11 Oct 2016, at 15:11, [email protected] wrote:
shared_plain<double> result(a.begin(), a.end());
eventually, that results in calling std::uninitialized_copy. This has a significant overhead compared to std::copy which could be used here for simple types such as integers and doubles, probably resulting in another big boost. This could be done with hairy template specialisations I guess but not sure it’s worth the sweat!!
Richard,
No comment except "Great work" figuring this out!
Nick
Nicholas K. Sauter, Ph. D.
Senior Staff Scientist, Molecular Biophysics & Integrated Bioimaging
Division
Lawrence Berkeley National Laboratory
1 Cyclotron Rd., Bldg. 33R0345
Berkeley, CA 94720
(510) 486-5713
On Tue, Oct 11, 2016 at 9:44 AM, Luc Bourhis
Hi Richard,
On 11 Oct 2016, at 15:11, [email protected] wrote:
shared_plain<double> result(a.begin(), a.end());
eventually, that results in calling std::uninitialized_copy. This has a significant overhead compared to std::copy which could be used here for simple types such as integers and doubles, probably resulting in another big boost. This could be done with hairy template specialisations I guess but not sure it’s worth the sweat!!
_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb
Further timing comparisons on linux (RHEL6):
Before:
$ libtbx.python time_as_double.py
flex.int.as_double(): 8.99s (0.090s/call)
numpy.astype(np.float64): 2.34s (0.023s/call)
After:
$ libtbx.python time_as_double.py
flex.int.as_double(): 1.79s (0.018s/call)
numpy.astype(np.float64): 2.33s (0.023s/call)
If no one objects I will commit this change tomorrow. All the tests (scitbx, cctbx, dxtbx, dials, xia2, etc appear to pass with the modified code).
Cheers,
Richard
Dr Richard Gildea
Data Analysis Scientist
Tel: +441235 77 8078
Diamond Light Source Ltd.
Diamond House
Harwell Science & Innovation Campus
Didcot
Oxfordshire
OX11 0DE
________________________________
From: [email protected] [[email protected]] on behalf of Nicholas Sauter [[email protected]]
Sent: 11 October 2016 19:55
To: cctbx mailing list
Subject: Re: [cctbxbb] flex.int.as_double() performance
Richard,
No comment except "Great work" figuring this out!
Nick
Nicholas K. Sauter, Ph. D.
Senior Staff Scientist, Molecular Biophysics & Integrated Bioimaging Division
Lawrence Berkeley National Laboratory
1 Cyclotron Rd., Bldg. 33R0345
Berkeley, CA 94720
(510) 486-5713
On Tue, Oct 11, 2016 at 9:44 AM, Luc Bourhis
participants (3)
-
Luc Bourhis -
Nicholas Sauter -
richard.gildea＠diamond.ac.uk