Hi all, It seems that a flex.double array containing "nan" values doesn't survive pickling as in: oeffner@grove:~$ cctbx.python Python 2.7.15 (default, Sep 11 2018, 23:48:45) [GCC 4.4.7 20120313 (Red Hat 4.4.7-18)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from cctbx.array_family import flex import pickle flex_nans = flex.double(3, float('nan')) list(flex_nans)
[nan, nan, nan]
list(pickle.loads( pickle.dumps(flex_nans))) [0.0, 0.0, 0.0]
If the flex double is cast into a python list then it does survive pickling as in:
list(pickle.loads( pickle.dumps(list( flex_nans ))))
[nan, nan, nan]
I would like to use nan values to indicate missing or invalid data values. Would anyone have a suggestion how to go about this? Many thanks, Rob -- Robert Oeffner, Ph.D. Research Associate, The Read Group Department of Haematology, Cambridge Institute for Medical Research University of Cambridge Cambridge Biomedical Campus The Keith Peters Building Hills Road Cambridge CB2 0XY www.cimr.cam.ac.uk/investigators/read/index.html tel: +44(0)1223 763234
Hi Rob,
We looked a little into this and it seems like the byte string interface
will preserve NAN. We're going to dig deeper into the serialization code
for a fix.
Anyway, here is a code snippet for using the byte string interface. The
byte string can be pickled, so a workaround for flex would be to add an
extra conversion step to byte string before pickling.
from __future__ import print_function
import pickle
from scitbx.array_family import flex
# NAN in flex.double
a = flex.random_double(5)
n = float('nan')
a[3] = n
# pickle test
p = pickle.dumps(a)
ap = pickle.loads(p)
# byte string test
b = a.copy_to_byte_str()
ab = flex.double_from_byte_str(b)
# pickled byte string test
pb = pickle.dumps(b)
apb = flex.double_from_byte_str(pickle.loads(pb))
# check arrays
print('Original:', list(a))
print('Pickle: ', list(ap), 'Oh no!')
print('Bytes: ', list(ab))
print('Bytes2: ', list(apb))
--
Billy K. Poon
Research Scientist, Molecular Biophysics and Integrated Bioimaging
Lawrence Berkeley National Laboratory
1 Cyclotron Road, M/S 33R0345
Berkeley, CA 94720
Tel: (510) 486-5709
Fax: (510) 486-5909
Web: https://phenix-online.org
On Mon, Jun 10, 2019 at 7:10 AM Robert Oeffner
Hi all,
It seems that a flex.double array containing "nan" values doesn't survive pickling as in:
oeffner@grove:~$ cctbx.python Python 2.7.15 (default, Sep 11 2018, 23:48:45) [GCC 4.4.7 20120313 (Red Hat 4.4.7-18)] on linux2 Type "help", "copyright", "credits" or "license" for more information.
from cctbx.array_family import flex import pickle flex_nans = flex.double(3, float('nan')) list(flex_nans)
[nan, nan, nan]
list(pickle.loads( pickle.dumps(flex_nans))) [0.0, 0.0, 0.0]
If the flex double is cast into a python list then it does survive pickling as in:
list(pickle.loads( pickle.dumps(list( flex_nans ))))
[nan, nan, nan]
I would like to use nan values to indicate missing or invalid data values. Would anyone have a suggestion how to go about this?
Many thanks,
Rob
-- Robert Oeffner, Ph.D. Research Associate, The Read Group Department of Haematology, Cambridge Institute for Medical Research University of Cambridge Cambridge Biomedical Campus The Keith Peters Building Hills Road Cambridge CB2 0XY
www.cimr.cam.ac.uk/investigators/read/index.html tel: +44(0)1223 763234
_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb
participants (2)
-
Billy Poon -
Robert Oeffner