Re: [phenixbb] Fwd: [ccp4bb] Phenix; converting a map to structure factors
Hi Mark, Tom copied me your email.
As I'm sure you are aware there is tremendous interest in building into and refining against low resolution (by X-ray standards) EM maps. Could you comment to the phenix BB about whether its appropriate to convert maps to reciprocal space and use ML refinement?
While it is a possible way of doing this (and this is how it's been done in the past) I don't think it is a good route to follow. In case of cryo-EM your data is map, so why not just refine against the map? Why take a detour through reciprocal-space by converting map into structure factors (=manipulating your original data) and doing reciprocal space using refinement programs that were designed for crystallography? With this in mind we have developed a new refinement tool in Phenix (it is still in active development!) called phenix.real_space_refine that is designed to refine atomic models directly into maps (any map: X-ray, neutron, cryo-EM, low or high resolution), for quick summary see http://phenix-online.org/presentations/real_space_refine.pdf as well as documentation page (at the bottom): http://phenix-online.org/documentation/ and newsletter article (page 43): http://phenix-online.org/newsletter/CCN_2013_07.pdf Let me know if you have any questions or comments! Pavel
Hi Pavel, Would reciprocal space refinement improve maps - or would all the improvement come from model bias? i.e. would maps look nicer, but with no new information content? I guess the difference from X-ray is that while each HKL has contributions from all atoms in the ASU, for an EM map converted to structure factors this is not true. Also, EM maps have substantial local resolution differences, and I don't understand how this would impact SF calculation. I know that some in the EM community, who are not used to hands on (i.e. coot/O etc) real space refinement, are disappointed by the small radius of convergence for automatic real space refinement that is not guided by a human being sitting at a graphics terminal. I doubt that reciprocal space refinement would change that, but would Fo-Fc maps make it easier to choose between good and bad fits to EM maps? I'm hoping that people with more experience will chime in; I believe this is an active issue for CCP4 (refmac) users as well. Mark ________________________________________ From: Pavel Afonine [[email protected]] Sent: Friday, July 10, 2015 9:56 PM To: Terwilliger, Thomas Charles; Mayer, Mark (NIH/NICHD) [E]; PHENIX user mailing list Subject: Re: Fwd: [phenixbb] [ccp4bb] Phenix; converting a map to structure factors Hi Mark, Tom copied me your email. As I'm sure you are aware there is tremendous interest in building into and refining against low resolution (by X-ray standards) EM maps. Could you comment to the phenix BB about whether its appropriate to convert maps to reciprocal space and use ML refinement? While it is a possible way of doing this (and this is how it's been done in the past) I don't think it is a good route to follow. In case of cryo-EM your data is map, so why not just refine against the map? Why take a detour through reciprocal-space by converting map into structure factors (=manipulating your original data) and doing reciprocal space using refinement programs that were designed for crystallography? With this in mind we have developed a new refinement tool in Phenix (it is still in active development!) called phenix.real_space_refine that is designed to refine atomic models directly into maps (any map: X-ray, neutron, cryo-EM, low or high resolution), for quick summary see http://phenix-online.org/presentations/real_space_refine.pdf as well as documentation page (at the bottom): http://phenix-online.org/documentation/ and newsletter article (page 43): http://phenix-online.org/newsletter/CCN_2013_07.pdf Let me know if you have any questions or comments! Pavel
Hi Mark,
Would reciprocal space refinement improve maps - or would all the improvement come from model bias? i.e. would maps look nicer, but with no new information content?
cryo-EM map is your data, similarly to Fobs in case of X-ray. We never change Fobs during structure solution and refinement, likewise cryo-EM map should not be changed. What happens when you use reciprocal space refinement programs to refine against "Fobs and phases" generated from cryo-EM map is that the program will calculate 2mFo-DFc and mFo-DFc maps using model phases combined with phases from the map. Obviously such calculated maps will be model biased. I would say this is not desirable. (Oh and don't forget - you need to use electron form-factors if you refine against the cryo-EM map in reciprocal space!).
I guess the difference from X-ray is that while each HKL has contributions from all atoms in the ASU, for an EM map converted to structure factors this is not true. Also, EM maps have substantial local resolution differences, and I don't understand how this would impact SF calculation.
This brings another point.. If you calculate a box of reflections from the map, then conversion Map <> Structure Factors is lossless and biunique. The problem is that when we convert map to structure factors of certain resolution we essentially carve a sphere with reflections out of the box; this is because reciprocal-space refinement programs are designed to work with sphere of reflections not the box. This manipulation result in loss of information: Map1 -> sphere of reflections -> Map2 with Map1 being not the same as Map2.
I know that some in the EM community, who are not used to hands on (i.e. coot/O etc) real space refinement, are disappointed by the small radius of convergence for automatic real space refinement that is not guided by a human being sitting at a graphics terminal.
One of the great things about real-space refinement is that it can be done locally. In particular this means optimization methods with larger convergence radius can be used: for example, one can systematically sample chi angles of a residue side chain to find the best fitting rotamer. Another example: data / restraints target optimal weights can be obtain in a matter of seconds, while obtaining optimal weight in reciprocal space refinement may be a very time consuming task. I can bring many more examples like these. In real-space refinement one can use refinement target functions that are much faster to compute (such as sum of map values at atom centers) compared to reciprocal space LS or ML functions. This allows programs to try more things during refinement and eventually arrive at a better result. We use all this in phenix.real_space_refine. Of course a human being moving atoms by hand at a graphics is the optimization tool with perhaps largest convergence radius. I would say in terms of convergence radius real-space refinement sits in between reciprocal-space refinement and human being. Pavel
Hi Pavel and Sacha, This is great - thanks for explaining why its probably not a good idea to do reciprocal space refinement using EM maps as a starting point for structure factor calculation. My last gasp:what about case where you have a not perfect model built into EM map, real space refined locally, and then would like to do a global geometry optimization? Mark ________________________________________ From: Pavel Afonine [[email protected]] Sent: Saturday, July 11, 2015 2:51 PM To: Mayer, Mark (NIH/NICHD) [E]; Terwilliger, Thomas Charles; PHENIX user mailing list Subject: Re: Fwd: [phenixbb] [ccp4bb] Phenix; converting a map to structure factors Hi Mark,
Would reciprocal space refinement improve maps - or would all the improvement come from model bias? i.e. would maps look nicer, but with no new information content?
cryo-EM map is your data, similarly to Fobs in case of X-ray. We never change Fobs during structure solution and refinement, likewise cryo-EM map should not be changed. What happens when you use reciprocal space refinement programs to refine against "Fobs and phases" generated from cryo-EM map is that the program will calculate 2mFo-DFc and mFo-DFc maps using model phases combined with phases from the map. Obviously such calculated maps will be model biased. I would say this is not desirable. (Oh and don't forget - you need to use electron form-factors if you refine against the cryo-EM map in reciprocal space!).
I guess the difference from X-ray is that while each HKL has contributions from all atoms in the ASU, for an EM map converted to structure factors this is not true. Also, EM maps have substantial local resolution differences, and I don't understand how this would impact SF calculation.
This brings another point.. If you calculate a box of reflections from the map, then conversion Map <> Structure Factors is lossless and biunique. The problem is that when we convert map to structure factors of certain resolution we essentially carve a sphere with reflections out of the box; this is because reciprocal-space refinement programs are designed to work with sphere of reflections not the box. This manipulation result in loss of information: Map1 -> sphere of reflections -> Map2 with Map1 being not the same as Map2.
I know that some in the EM community, who are not used to hands on (i.e. coot/O etc) real space refinement, are disappointed by the small radius of convergence for automatic real space refinement that is not guided by a human being sitting at a graphics terminal.
One of the great things about real-space refinement is that it can be done locally. In particular this means optimization methods with larger convergence radius can be used: for example, one can systematically sample chi angles of a residue side chain to find the best fitting rotamer. Another example: data / restraints target optimal weights can be obtain in a matter of seconds, while obtaining optimal weight in reciprocal space refinement may be a very time consuming task. I can bring many more examples like these. In real-space refinement one can use refinement target functions that are much faster to compute (such as sum of map values at atom centers) compared to reciprocal space LS or ML functions. This allows programs to try more things during refinement and eventually arrive at a better result. We use all this in phenix.real_space_refine. Of course a human being moving atoms by hand at a graphics is the optimization tool with perhaps largest convergence radius. I would say in terms of convergence radius real-space refinement sits in between reciprocal-space refinement and human being. Pavel
Hi Mark,
My last gasp:what about case where you have a not perfect model built into EM map, real space refined locally, and then would like to do a global geometry optimization?
phenix.real_space_refine uses a combination of local and global refinement methods so I would say just iterate refinement and manual rebuilding until you are done. Pavel
participants (2)
-
Mayer, Mark (NIH/NICHD) [E]
-
Pavel Afonine