Re: [cctbxbb] [Dials-support] indexing success rates

Right, typically if I don't know the cell, I'll index a bunch of images with no target, then use cluster.unit_cell on the results to get a consensus cell, then re-index with that cell. For dials.stills_process, the default of fft1d works pretty well generally, but to get the best results, indexing.method_list=fft1d,fft3d might be better. There, if the first method fails the second is tried. real_space_grid_search can also be used. Getting a specific image in a composite file (such as hdf5) isn't a use case I've hit often, so I don't have an easy solution there. It's possible an image range parameter to dials.import would be useful? -Aaron On Sat, Feb 3, 2018 at 1:02 AM, Graeme Winter wrote:

Hi Aaron

(and DIALS)

For stills / single images - should dials.index switch by magic to default fft1d?

Of is that the responsibility of e.g. xia2 / cctbx.xfel to worry about?

It has to be said, more often than not, particularly for screening, we do not know cell / symmetry a priori (at diamond anyways)

A related question

If I have a data block / strong pickle from 1000 stills, how can I index e.g. image 672? Do I have to split this into 1000 separate things?

@JH - thanks for this - I think some rigorous testing will be useful here.

Cheers Graeme

On 3 Feb 2018, at 08:37, Aaron Brewster wrote:

Hi James, thanks. I looked at random image 15 and after trying a bunch of stuff it indexes fine if I use indexing.method=fft3d. The default for stills processing is fft1d, so I find this super interesting.

For what it's worth, I pretty much never process stills without a target cell. I generated ~30 random images using your script and indexed them, then I got the unit cells with "dials.show *refined*.json | grep cell". They were all about the same. So I re-indexed random image 15 using fft1d but I also specified known_symmetry.cell=50,60,70,90,90,90 and it indexed fine.

So, either indexing.method=fft3d or known_symmetry.cell=50,60,70,90,90,90 is sufficient to index random image 15. Certainly for experimental data I would recommend using known_symmetry as the results are always better. However, I'd like to look deeper at fft1d vs fft3d at some point.

Cc'ing dials-support.

Thanks! -Aaron

On Fri, Feb 2, 2018 at 1:35 PM, James Holton wrote:

...

While testing the new simTBX diffraction image simulator Aaron and I found that some rather beautiful-looking images still won't index. Subsequently, I followed up with mosflm, which does succeed in these cases, even without prior knowledge of the cell or space group.

I have now run this 1000 times, varying only the crystal orientation, and found 171 cases of dials.stills_process failing to index the simulated image whereas mosflm had no trouble, and 5 cases where mosflm failed and dials.stills_process succeeded. There were no overlapping cases where both programs failed.

I have tarballed up the relevant files here: http://bl831.als.lbl.gov/~jamesh/bugreports/dials_index_020218.tgz

The tarball contains a "runme.com" shell script for reproducing these results, and also the tst_nanoBragg_forindex.py jiffy for making a test image given a provided random-number seed. For example:

libtbx.python ./tst_nanoBragg_forindex.py random 15

will create a file called noiseimage_001.cbf that dials.stills_process cannot index (at least in my hands). For those of you who don't have mosflm installed, the tarball incluses a copy, and the "autoindex.com" script should run on any linux system.

Any ideas as to why there is this discrepancy? Feels like an opportunity for a non-trivial improvement.

Something to ponder over the weekend, I suppose. Note that I recently checked in a few bug fixes to simTBX, but this analysis is unaffected by them, so you shouldn't have to update your build to reproduce this.

Cheers,

-James Holton MAD Scientist

_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb

------------------------------------------------------------ ------------------ Check out the vibrant tech community on one of the world's most engaging tech sites, Slashdot.org! http://sdm.link/slashdot______ _________________________________________ DIALS-support mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/dials-support