Right, typically if I don't know the cell, I'll index a bunch of images with no target, then use cluster.unit_cell on the results to get a consensus cell, then re-index with that cell.

For dials.stills_process, the default of fft1d works pretty well generally, but to get the best results, indexing.method_list=fft1d,fft3d might be better.  There, if the first method fails the second is tried.  real_space_grid_search can also be used.

Getting a specific image in a composite file (such as hdf5) isn't a use case I've hit often, so I don't have an easy solution there.  It's possible an image range parameter to dials.import would be useful?

-Aaron

On Sat, Feb 3, 2018 at 1:02 AM, Graeme Winter <graeme.winter@gmail.com> wrote:
Hi Aaron

(and DIALS)

For stills / single images - should dials.index switch by magic to default fft1d? 

Of is that the responsibility of e.g. xia2 / cctbx.xfel to worry about?

It has to be said, more often than not, particularly for screening, we do not know cell / symmetry a priori (at diamond anyways) 

A related question

If I have a data block / strong pickle from 1000 stills, how can I index e.g. image 672? Do I have to split this into 1000 separate things?

@JH - thanks for this - I think some rigorous testing will be useful here. 

Cheers Graeme

On 3 Feb 2018, at 08:37, Aaron Brewster <asbrewster@lbl.gov> wrote:

Hi James, thanks.  I looked at random image 15 and after trying a bunch of stuff it indexes fine if I use indexing.method=fft3d.  The default for stills processing is fft1d, so I find this super interesting.

For what it's worth, I pretty much never process stills without a target cell.  I generated ~30 random images using your script and indexed them, then I got the unit cells with "dials.show *refined*.json | grep cell".  They were all about the same.  So I re-indexed random image 15 using fft1d but I also specified known_symmetry.cell=50,60,70,90,90,90 and it indexed fine.

So, either indexing.method=fft3d or known_symmetry.cell=50,60,70,90,90,90 is sufficient to index random image 15.  Certainly for experimental data I would recommend using known_symmetry as the results are always better.  However, I'd like to look deeper at fft1d vs fft3d at some point.

Cc'ing dials-support.

Thanks!
-Aaron



On Fri, Feb 2, 2018 at 1:35 PM, James Holton <jmholton@lbl.gov> wrote:
While testing the new simTBX diffraction image simulator Aaron and I found that some rather beautiful-looking images still won't index.  Subsequently, I followed up with mosflm, which does succeed in these cases, even without prior knowledge of the cell or space group.

I have now run this 1000 times, varying only the crystal orientation, and found 171 cases of dials.stills_process failing to index the simulated image whereas mosflm had no trouble, and 5 cases where mosflm failed and dials.stills_process succeeded.  There were no overlapping cases where both programs failed.

I have tarballed up the relevant files here:
http://bl831.als.lbl.gov/~jamesh/bugreports/dials_index_020218.tgz

The tarball contains a "runme.com" shell script for reproducing these results, and also the tst_nanoBragg_forindex.py jiffy for making a test image given a provided random-number seed.  For example:

libtbx.python ./tst_nanoBragg_forindex.py random 15

will create a file called noiseimage_001.cbf that dials.stills_process cannot index (at least in my hands).  For those of you who don't have mosflm installed, the tarball incluses a copy, and the "autoindex.com" script should run on any linux system.

Any ideas as to why there is this discrepancy?  Feels like an opportunity for a non-trivial improvement.

Something to ponder over the weekend, I suppose.  Note that I recently checked in a few bug fixes to simTBX, but this analysis is unaffected by them, so you shouldn't have to update your build to reproduce this.

Cheers,

-James Holton
MAD Scientist



_______________________________________________
cctbxbb mailing list
cctbxbb@phenix-online.org
http://phenix-online.org/mailman/listinfo/cctbxbb


------------------------------------------------------------------------------
Check out the vibrant tech community on one of the world's most
engaging tech sites, Slashdot.org! http://sdm.link/slashdot_______________________________________________
DIALS-support mailing list
DIALS-support@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/dials-support