Getting r1_factor of miller arrays with different count of miller indices
Hi, I'm facing a new challenge while working with cctbx: Imagine a set of (possibly incomplete) measured intesities, or integrated intensities from xrd powder patterns (I_obs) and a fitting crystal symmetry (xtal_symm). Now if I want to check if a given structure would generate the same intensities. (--> low R1) the following code exibits the problem: # -*- coding: utf-8 -*- from __future__ import division from cctbx import xray from cctbx import crystal from cctbx.array_family import flex from libtbx import Auto obs_file = "Test.hkl" # in shelx hklf4 format xtal_symm = crystal.symmetry( unit_cell=(4.000, 5.4321, 7.531, 90.0, 90.0, 90.0), space_group_symbol="P mm2") trial_structure = xray.structure( special_position_settings=crystal.special_position_settings( crystal_symmetry=xtal_symm), scatterers=flex.xray_scatterer([ xray.scatterer( label="Si", site=(0.0,0.0,0.0), u=0.2)])) # load (integral) intensities from diffraction data from iotbx import reflection_file_reader rfl = reflection_file_reader.any_reflection_file("amplitudes="+obs_file, ensure_read_access=True) I_obs = rfl.as_miller_arrays(crystal_symmetry=xtal_symm, force_symmetry=False, merge_equivalents=True, base_array_info=None)[0] I_obs = I_obs.discard_sigmas() f_obs = I_obs.as_amplitude_array() f_calc = trial_structure.structure_factors(d_min=1.0).f_calc() f_calc.merge_equivalents() R1 = f_calc.r1_factor(f_obs, scale_factor=Auto, assume_index_matching=False) gives: " assert other.indices().size() == self.indices().size() AssertionError " Is there some way to get the R1_factor for this kind of scenario? Cheers and thanks in advance, Jan
On Fri, Jul 6, 2012 at 11:59 AM, Jan Marten Simons
f_calc = trial_structure.structure_factors(d_min=1.0).f_calc() f_calc.merge_equivalents()
R1 = f_calc.r1_factor(f_obs, scale_factor=Auto, assume_index_matching=False)
gives: " assert other.indices().size() == self.indices().size() AssertionError "
Is there some way to get the R1_factor for this kind of scenario?
Yes, you need to filter out the missing reflections: f_calc.common_set(other=f_obs).r1_factor(f_obs, scale_factor=Auto, assume_index_matching=False) If that doesn't work, add this line before it: f_obs = f_obs.common_set(other=f_calc) -Nat
Hi Jan,
You will need to get a matching set of indices for both arrays before
calculating the R1 factor. The simplest way to do this would be by calling
the miller.array.common_sets() function:
e.g. where f_obs and f_calc are two arrays that are not necessarily
matching sets of indices:
f_obs, f_calc = f_obs.common_sets(f_calc)
Hope that helps,
Richard
On 6 July 2012 11:59, Jan Marten Simons
Hi,
I'm facing a new challenge while working with cctbx:
Imagine a set of (possibly incomplete) measured intesities, or integrated intensities from xrd powder patterns (I_obs) and a fitting crystal symmetry (xtal_symm). Now if I want to check if a given structure would generate the same intensities. (--> low R1) the following code exibits the problem:
# -*- coding: utf-8 -*- from __future__ import division from cctbx import xray from cctbx import crystal from cctbx.array_family import flex from libtbx import Auto
obs_file = "Test.hkl" # in shelx hklf4 format xtal_symm = crystal.symmetry( unit_cell=(4.000, 5.4321, 7.531, 90.0, 90.0, 90.0), space_group_symbol="P mm2")
trial_structure = xray.structure( special_position_settings=crystal.special_position_settings( crystal_symmetry=xtal_symm), scatterers=flex.xray_scatterer([ xray.scatterer( label="Si", site=(0.0,0.0,0.0), u=0.2)]))
# load (integral) intensities from diffraction data from iotbx import reflection_file_reader rfl = reflection_file_reader.any_reflection_file("amplitudes="+obs_file, ensure_read_access=True) I_obs = rfl.as_miller_arrays(crystal_symmetry=xtal_symm, force_symmetry=False, merge_equivalents=True, base_array_info=None)[0] I_obs = I_obs.discard_sigmas() f_obs = I_obs.as_amplitude_array()
f_calc = trial_structure.structure_factors(d_min=1.0).f_calc() f_calc.merge_equivalents()
R1 = f_calc.r1_factor(f_obs, scale_factor=Auto, assume_index_matching=False)
gives: " assert other.indices().size() == self.indices().size() AssertionError "
Is there some way to get the R1_factor for this kind of scenario?
Cheers and thanks in advance, Jan _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb
On Fri, Jul 6, 2012 at 12:04 PM, Richard Gildea
You will need to get a matching set of indices for both arrays before calculating the R1 factor. The simplest way to do this would be by calling the miller.array.common_sets() function:
e.g. where f_obs and f_calc are two arrays that are not necessarily matching sets of indices:
f_obs, f_calc = f_obs.common_sets(f_calc)
Wow, I didn't even notice this... Richard is correct, ignore my answer. -Nat
Just to avoid any confusion between mine and Nat's responses, there are two
similarly named functions in miller.array, common_set and common_sets.
Thus the following two lines (note singular common_set):
f_obs = f_obs.common_set(f_calc)
f_calc = f_calc.common_set(f_obs)
is equivalent to the one-liner (note plural common_sets):
f_obs, f_calc = f_obs.common_sets(f_calc)
Cheers,
Richard
On 6 July 2012 12:04, Richard Gildea
Hi Jan,
You will need to get a matching set of indices for both arrays before calculating the R1 factor. The simplest way to do this would be by calling the miller.array.common_sets() function:
e.g. where f_obs and f_calc are two arrays that are not necessarily matching sets of indices:
f_obs, f_calc = f_obs.common_sets(f_calc)
Hope that helps,
Richard
On 6 July 2012 11:59, Jan Marten Simons
wrote: Hi,
I'm facing a new challenge while working with cctbx:
Imagine a set of (possibly incomplete) measured intesities, or integrated intensities from xrd powder patterns (I_obs) and a fitting crystal symmetry (xtal_symm). Now if I want to check if a given structure would generate the same intensities. (--> low R1) the following code exibits the problem:
# -*- coding: utf-8 -*- from __future__ import division from cctbx import xray from cctbx import crystal from cctbx.array_family import flex from libtbx import Auto
obs_file = "Test.hkl" # in shelx hklf4 format xtal_symm = crystal.symmetry( unit_cell=(4.000, 5.4321, 7.531, 90.0, 90.0, 90.0), space_group_symbol="P mm2")
trial_structure = xray.structure( special_position_settings=crystal.special_position_settings( crystal_symmetry=xtal_symm), scatterers=flex.xray_scatterer([ xray.scatterer( label="Si", site=(0.0,0.0,0.0), u=0.2)]))
# load (integral) intensities from diffraction data from iotbx import reflection_file_reader rfl = reflection_file_reader.any_reflection_file("amplitudes="+obs_file, ensure_read_access=True) I_obs = rfl.as_miller_arrays(crystal_symmetry=xtal_symm, force_symmetry=False, merge_equivalents=True, base_array_info=None)[0] I_obs = I_obs.discard_sigmas() f_obs = I_obs.as_amplitude_array()
f_calc = trial_structure.structure_factors(d_min=1.0).f_calc() f_calc.merge_equivalents()
R1 = f_calc.r1_factor(f_obs, scale_factor=Auto, assume_index_matching=False)
gives: " assert other.indices().size() == self.indices().size() AssertionError "
Is there some way to get the R1_factor for this kind of scenario?
Cheers and thanks in advance, Jan _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb
Hi, can anyone tell me what's r1_factor and how it is different from classical text-book definition of R-factor, for example: http://en.wikipedia.org/wiki/R-factor_(crystallography) (or for those who doesn't trust wiki: page 147 in Blundell&Johnson (1976)) ? Are there r2_factor, r3_factor, ..., etc ? Thanks, Pavel On 7/6/12 12:09 PM, Richard Gildea wrote:
Just to avoid any confusion between mine and Nat's responses, there are two similarly named functions in miller.array, common_set and common_sets.
Thus the following two lines (note singular common_set):
f_obs = f_obs.common_set(f_calc) f_calc = f_calc.common_set(f_obs)
is equivalent to the one-liner (note plural common_sets):
f_obs, f_calc = f_obs.common_sets(f_calc)
Cheers,
Richard
On 6 July 2012 12:04, Richard Gildea
mailto:[email protected]> wrote: Hi Jan,
You will need to get a matching set of indices for both arrays before calculating the R1 factor. The simplest way to do this would be by calling the miller.array.common_sets() function:
e.g. where f_obs and f_calc are two arrays that are not necessarily matching sets of indices:
f_obs, f_calc = f_obs.common_sets(f_calc)
Hope that helps,
Richard
On 6 July 2012 11:59, Jan Marten Simons
mailto:[email protected]> wrote: Hi,
I'm facing a new challenge while working with cctbx:
Imagine a set of (possibly incomplete) measured intesities, or integrated intensities from xrd powder patterns (I_obs) and a fitting crystal symmetry (xtal_symm). Now if I want to check if a given structure would generate the same intensities. (--> low R1) the following code exibits the problem:
# -*- coding: utf-8 -*- from __future__ import division from cctbx import xray from cctbx import crystal from cctbx.array_family import flex from libtbx import Auto
obs_file = "Test.hkl" # in shelx hklf4 format xtal_symm = crystal.symmetry( unit_cell=(4.000, 5.4321, 7.531, 90.0, 90.0, 90.0), space_group_symbol="P mm2")
trial_structure = xray.structure( special_position_settings=crystal.special_position_settings( crystal_symmetry=xtal_symm), scatterers=flex.xray_scatterer([ xray.scatterer( label="Si", site=(0.0,0.0,0.0), u=0.2)]))
# load (integral) intensities from diffraction data from iotbx import reflection_file_reader rfl = reflection_file_reader.any_reflection_file("amplitudes="+obs_file, ensure_read_access=True) I_obs = rfl.as_miller_arrays(crystal_symmetry=xtal_symm, force_symmetry=False, merge_equivalents=True, base_array_info=None)[0] I_obs = I_obs.discard_sigmas() f_obs = I_obs.as_amplitude_array()
f_calc = trial_structure.structure_factors(d_min=1.0).f_calc() f_calc.merge_equivalents()
R1 = f_calc.r1_factor(f_obs, scale_factor=Auto, assume_index_matching=False)
gives: " assert other.indices().size() == self.indices().size() AssertionError "
Is there some way to get the R1_factor for this kind of scenario?
Cheers and thanks in advance, Jan _______________________________________________ cctbxbb mailing list [email protected] mailto:[email protected] http://phenix-online.org/mailman/listinfo/cctbxbb
_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb
Hi Pavel,
Small molecule crystallographers often refer to the R-factor as the
R1-factor. The '1' just means that is calculated using the amplitudes
without squaring. An R2-factor would be obtained by replacing the
amplitudes by the squares of the amplitudes and taking the square root. See
for example equations 7 and 8 in this paper by David Watkin (
http://dx.doi.org/10.1107/S0021889808007279). The terminology R1 factor is
also used in the SHELX manual.
Cheers,
Richard
On 6 July 2012 12:23, Pavel Afonine
Hi,
can anyone tell me what's r1_factor and how it is different from classical text-book definition of R-factor, for example:
http://en.wikipedia.org/wiki/R-factor_(crystallography)
(or for those who doesn't trust wiki: page 147 in Blundell&Johnson (1976))
?
Are there r2_factor, r3_factor, ..., etc ?
Thanks, Pavel
On 7/6/12 12:09 PM, Richard Gildea wrote:
Just to avoid any confusion between mine and Nat's responses, there are two similarly named functions in miller.array, common_set and common_sets.
Thus the following two lines (note singular common_set):
f_obs = f_obs.common_set(f_calc) f_calc = f_calc.common_set(f_obs)
is equivalent to the one-liner (note plural common_sets):
f_obs, f_calc = f_obs.common_sets(f_calc)
Cheers,
Richard
On 6 July 2012 12:04, Richard Gildea
wrote: Hi Jan,
You will need to get a matching set of indices for both arrays before calculating the R1 factor. The simplest way to do this would be by calling the miller.array.common_sets() function:
e.g. where f_obs and f_calc are two arrays that are not necessarily matching sets of indices:
f_obs, f_calc = f_obs.common_sets(f_calc)
Hope that helps,
Richard
On 6 July 2012 11:59, Jan Marten Simons
wrote: Hi,
I'm facing a new challenge while working with cctbx:
Imagine a set of (possibly incomplete) measured intesities, or integrated intensities from xrd powder patterns (I_obs) and a fitting crystal symmetry (xtal_symm). Now if I want to check if a given structure would generate the same intensities. (--> low R1) the following code exibits the problem:
# -*- coding: utf-8 -*- from __future__ import division from cctbx import xray from cctbx import crystal from cctbx.array_family import flex from libtbx import Auto
obs_file = "Test.hkl" # in shelx hklf4 format xtal_symm = crystal.symmetry( unit_cell=(4.000, 5.4321, 7.531, 90.0, 90.0, 90.0), space_group_symbol="P mm2")
trial_structure = xray.structure( special_position_settings=crystal.special_position_settings( crystal_symmetry=xtal_symm), scatterers=flex.xray_scatterer([ xray.scatterer( label="Si", site=(0.0,0.0,0.0), u=0.2)]))
# load (integral) intensities from diffraction data from iotbx import reflection_file_reader rfl = reflection_file_reader.any_reflection_file("amplitudes="+obs_file,
ensure_read_access=True) I_obs = rfl.as_miller_arrays(crystal_symmetry=xtal_symm, force_symmetry=False, merge_equivalents=True, base_array_info=None)[0] I_obs = I_obs.discard_sigmas() f_obs = I_obs.as_amplitude_array()
f_calc = trial_structure.structure_factors(d_min=1.0).f_calc() f_calc.merge_equivalents()
R1 = f_calc.r1_factor(f_obs, scale_factor=Auto, assume_index_matching=False)
gives: " assert other.indices().size() == self.indices().size() AssertionError "
Is there some way to get the R1_factor for this kind of scenario?
Cheers and thanks in advance, Jan _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb
_______________________________________________ cctbxbb mailing [email protected]http://phenix-online.org/mailman/listinfo/cctbxbb
_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb
Actually, the R2-factor (or more commonly the wR2-factor) is calculated
using the squared differences in the intensities rather than the squared
differences in the amplitudes as I stated in my previous email. See also
section 2.7 in the SHELX manual.
Cheers,
Richard
On 6 July 2012 12:34, Richard Gildea
Hi Pavel,
Small molecule crystallographers often refer to the R-factor as the R1-factor. The '1' just means that is calculated using the amplitudes without squaring. An R2-factor would be obtained by replacing the amplitudes by the squares of the amplitudes and taking the square root. See for example equations 7 and 8 in this paper by David Watkin ( http://dx.doi.org/10.1107/S0021889808007279). The terminology R1 factor is also used in the SHELX manual.
Cheers,
Richard
On 6 July 2012 12:23, Pavel Afonine
wrote: Hi,
can anyone tell me what's r1_factor and how it is different from classical text-book definition of R-factor, for example:
http://en.wikipedia.org/wiki/R-factor_(crystallography)
(or for those who doesn't trust wiki: page 147 in Blundell&Johnson (1976))
?
Are there r2_factor, r3_factor, ..., etc ?
Thanks, Pavel
On 7/6/12 12:09 PM, Richard Gildea wrote:
Just to avoid any confusion between mine and Nat's responses, there are two similarly named functions in miller.array, common_set and common_sets.
Thus the following two lines (note singular common_set):
f_obs = f_obs.common_set(f_calc) f_calc = f_calc.common_set(f_obs)
is equivalent to the one-liner (note plural common_sets):
f_obs, f_calc = f_obs.common_sets(f_calc)
Cheers,
Richard
On 6 July 2012 12:04, Richard Gildea
wrote: Hi Jan,
You will need to get a matching set of indices for both arrays before calculating the R1 factor. The simplest way to do this would be by calling the miller.array.common_sets() function:
e.g. where f_obs and f_calc are two arrays that are not necessarily matching sets of indices:
f_obs, f_calc = f_obs.common_sets(f_calc)
Hope that helps,
Richard
On 6 July 2012 11:59, Jan Marten Simons
wrote: Hi,
I'm facing a new challenge while working with cctbx:
Imagine a set of (possibly incomplete) measured intesities, or integrated intensities from xrd powder patterns (I_obs) and a fitting crystal symmetry (xtal_symm). Now if I want to check if a given structure would generate the same intensities. (--> low R1) the following code exibits the problem:
# -*- coding: utf-8 -*- from __future__ import division from cctbx import xray from cctbx import crystal from cctbx.array_family import flex from libtbx import Auto
obs_file = "Test.hkl" # in shelx hklf4 format xtal_symm = crystal.symmetry( unit_cell=(4.000, 5.4321, 7.531, 90.0, 90.0, 90.0), space_group_symbol="P mm2")
trial_structure = xray.structure( special_position_settings=crystal.special_position_settings( crystal_symmetry=xtal_symm), scatterers=flex.xray_scatterer([ xray.scatterer( label="Si", site=(0.0,0.0,0.0), u=0.2)]))
# load (integral) intensities from diffraction data from iotbx import reflection_file_reader rfl = reflection_file_reader.any_reflection_file("amplitudes="+obs_file,
ensure_read_access=True) I_obs = rfl.as_miller_arrays(crystal_symmetry=xtal_symm, force_symmetry=False, merge_equivalents=True, base_array_info=None)[0] I_obs = I_obs.discard_sigmas() f_obs = I_obs.as_amplitude_array()
f_calc = trial_structure.structure_factors(d_min=1.0).f_calc() f_calc.merge_equivalents()
R1 = f_calc.r1_factor(f_obs, scale_factor=Auto, assume_index_matching=False)
gives: " assert other.indices().size() == self.indices().size() AssertionError "
Is there some way to get the R1_factor for this kind of scenario?
Cheers and thanks in advance, Jan _______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb
_______________________________________________ cctbxbb mailing [email protected]http://phenix-online.org/mailman/listinfo/cctbxbb
_______________________________________________ cctbxbb mailing list [email protected] http://phenix-online.org/mailman/listinfo/cctbxbb
Am Freitag 06 Juli 2012 21:04:23 schrieb Richard Gildea:
Hi Jan,
You will need to get a matching set of indices for both arrays before calculating the R1 factor. The simplest way to do this would be by calling the miller.array.common_sets() function:
e.g. where f_obs and f_calc are two arrays that are not necessarily matching sets of indices:
f_obs, f_calc = f_obs.common_sets(f_calc)
Hope that helps,
It helps in case of a complete set of observations. and if you don't mind I'll add an option for this to the r1_factor function. But in case of a reflection which would be present in the model structure at a d value within d_min, but which was not observed in the experiment due to structure factor cancellation or as its intensity/snr/I_over_sigma was to low this kind of treatment would give a lower r1 factor than a correct treatment of this situation. Do you have any idea on this? Cheers and thanks in advance, Jan
On 6 July 2012 11:59, Jan Marten Simons
wrote: Hi,
I'm facing a new challenge while working with cctbx:
Imagine a set of (possibly incomplete) measured intesities, or integrated intensities from xrd powder patterns (I_obs) and a fitting crystal symmetry (xtal_symm). Now if I want to check if a given structure would generate the same intensities. (--> low R1) the following code exibits the problem:
participants (4)
-
Jan Marten Simons
-
Nathaniel Echols
-
Pavel Afonine
-
Richard Gildea