The charge-flipping method of Oszlányi & Süto  (2004) has been implemented using the keywords shown in Table 11‑2.  Also included is the use of the tangent formula (Hauptman & Karle, 1956) within the iterative charge-flipping process. Equations appearing in charge-flipping keywords can be functions of the items shown in Table 11‑1. At the end of a charge flipping process a file with the same name as that given by cf_hkl_file is created but with a *.FC extension. Almost all of the charge-flipping keywords can be equations allowing for great flexibility in regards to changing resolution etc… on the fly. Table 11‑3 lists charge-flipping examples found in the CF directory.

 

 

|
Table 11‑2. Keywords that can be used in charge-flipping. ||

 

CF works particularlly well on data at good resolution (<1Å resolution). For data at poor resolution or for difficult structures then inclusion of the tangent formula can facilitate solution and sharpen electron densities, see example CF-1A7Y.INP. Powder diffraction data usually fall under the poor resolution/data quality category and as such additional symmetry restraints using symmetry_obey_0_to_1 can sharpen electron densities. Example CF-ALVO4.INP demonstrates the use of the tangent formula on powder data.

The choice and amount of perturbation necessary for finding a solution are important considerations. Not enough perturbation leads to the system being trapped within a local parameter space; too much perturbation may lead to a solution not being found and in addition contrast in R-factors prior to and at convergence are diminished leading to difficult to identify solutions. Many of the examples in the CF directory uses the Ramp macro to gradually vary control parameters, here are some examples:

fraction_density_to_flip = Ramp(0.85, 0.8, 100);

fraction_reflections_weak = Ramp(0.5, 0, 100);

flip_regime_2 = Ramp(1, 0, 200);

flip_regime_3 = Ramp(0.25, 0.5, 200);  

symmetry_obey_0_to_1 = Ramp(0.5, 1, 100);

tangent_scale_difference_by = Ramp(0, 1, 100);

Choosing control parameters in this manner gradually decreases perturbation allowing for solutions to be found and identified. This is similar to a simulated annealing process where temperatures start at high values and then progressively lowered.

Perturbations can be categorized as being of either phase, structure factor intensity or electron density perturbations as shown in Table 11‑2. There are two built in flipping regimes, flip_regime_2 and flip_regime_3, and one user defined regime flip_equation. Only one can be used and they all modify the electron density. In the absence of a flipping regime the following is used:

where d corresponds to the electron density threshold.

Using the tangent formula on either difficult structures or on data at poor resolution often leads to uranium atom solutions. Uranium atom solutions can be avoided by modifying the electron density using a flipping regime that dampens high electron densities or by using pick_atoms.

Using a large number of triplets per E_h value (a value for tangent_max_triplets_per_h greater than 100) reduces perturbation, increases occurrences of uranium atom solutions and increases the chances of finding a solution after an initial phase randomization. A large number of triplets would typically be used for poor resolution data; correspondingly a flipping regime that avoids uranium atom solutions should be chosen.

Perturbations mostly increase randomness in the system with the exceptions of the tangent formula, scale_density_below_threshold and histogram_match_scale_fwhm.

By default CF uses the minimum observed d spacing to define the Ewald sphere; alternatively min_d can be used. The Ewald sphere can be increased using extend_calculated_sphere_to; this inserts missing reflections and gives them the status of “weak” reflections. Weak reflections are also inserted for missing reflections within the Ewald sphere. Weak reflection phases and structure factors can be modified using scale_weak_reflections and add_to_phases_of_weak_reflections.

Reflections that have zero intensities according to the space group are not included in CF; correspondingly the number of observed reflections removed are reported. Structure factor intensities  within a family of reflections are determined by averaging the observed structure factors intensities. This averaging is also performed on calculated intensities each CF iteration for weak reflections.

Changing the space group is possible; changing the space group to a higher symmetry from that as implied in the input hkl file often makes sense. Changing the space group to a lower symmetry implies less symmetry and is useful for checking whether a significantly better R-factor is realized.

Typically a fraction of observed reflections are given the status of “weak” using fraction_reflections_weak. When a solution is found and CF terminates a *.FC file is saved; this file comprises structures factors that produced the best R-factor. A new CF process can be initiated with phase information saved in the *.FC file using the Restart_CF macro. To further complete the structure the new CF process may for example reduce perturbations in order to sharpen the electron density.

For powder data it is usually best to maximize the number of constraints due to poor data quality; it is also best to use *.A files as generated by a Pawley refinement and to then use cf_in_A_matrix. No weak observed reflections within the observed Ewald sphere should be assigned by setting fraction_reflections_weak to zero. Instead weak reflections can be included by extending the Ewald sphere with something like:

extend_calculated_sphere_to 1

add_to_phases_of_weak_reflections = 90 Ramp(1, 0, 100);

If the Ewald sphere is extended such that the weak reflections are many then some of these weak reflections could well be of high intensity. Subsequently offsetting high intensity weak reflections by a constant could lead to too much perturbation and thus the following may be preferential:

extend_calculated_sphere_to 1

add_to_phases_of_weak_reflections = Rand(-180,180) Ramp(1, 0, 100);

In a Pawley refinement the calculated intensities at the low d-spcaing edge of are often in error to a large extent; it is therefore best to remove these reflections using delete_observed_reflections, for example example:

delete_observed_reflections = D_spacing < 1.134;

A typical first try INP file template for powders is as follows:

macro Nr { 100 }

charge_flipping

cf_in_A_matrix PAWLEY_FILE.A

space_group $

a # b # c al # be # ga #

delete_observed_reflections = D_spacing < #;

extend_calculated_sphere_to #

add_to_phases_of_weak_reflections = 90 Ramp(1, 0, Nr);

flip_regime_2 = Ramp(1, 0, Nr);

symmetry_obey_0_to_1 = Ramp(0.5, 1, Nr);

Tangent(.3, 30)

min_grid_spacing .3

load f_atom_type f_atom_quantity { … }

Normalized structure factors enhances the chances of finding a solution (Oszlányi & Süto, 2006) and are realized by inclusion of f_atom_type’s and when correct_for_temperature_effects is non-zero. Example CF-1A7Y-GABOR.INP implements the algorithm of Oszlányi & Süto (2005) with normalized structure factors. In the original algorithm the amount of charge flipped is a function of the maximum electron density; this can be realized using:

user_threshold = 0.2 Get(max_density_at_cycle_iter_0);

Get(max_density_at_cycle_iter_0) is a different value at the start of each CF process as phases are chosen randomly. An alternative means of defining the threshold is:

fraction_density_to_flip 0.83

The CF process is sensitive to the threshold value. A value of 0.83 for fraction_density_to_flip is optimum for 1a7y and produces a solution in ~1000 iterations.  A solution is not found however at 0.75 or 0.85. To overcome this sensitivity the fraction_density_to_flip parameter could be ramped as a function of iteration from a high value to a low value, or,

fraction_density_to_flip = Ramp(0.85, 0.8, 100);

Implementation of such a ramp solves 1a7y in ~2000 iterations.

F000 is allowed to float when scale_F000 is set to 1. In the Oszlányi & Süto (2005) algorithm a floating F000 produces the best results for some structures but not for others (see section 11.2.3).

When the electron density is perturbed then a floating F000 often produces unfavourable oscillations in R-factors. In general the electron density is best left unperturbed when scale_F000 is non-zero.

Example CF-1A7Y-GABOR.INP does not seem to solve at a lower resolution, try for example:

delete_observed_reflections = D_spacing < 1.1;

On the other hand when scale_F000 is zero then electron density perturbations are possible; CF_1A7Y.INP solves 1a7y at 1.1 Angstrom (ie. include “delete_observed_reflections = D_spacing < 1.1”);  CF_1A7Y.INP uses flip_regime_2 and the tangent formula.

The effects of CF keywords can be investigated by inclusion/exclusion of keywords or by changing equations. This section lists some investigative examples and highlights the use of keywords necessary to solve particular examples found in the CF directory.

Example CF-1A7Y-OMIT.INP uses pick_atoms to modify the peaks of the highest intensity atoms as follows:

 

pick_atoms *

choose_to 5

omit = Rand(1, 1.1);

This example additionally uses the tangent formula and 1a7y solves in ~100 iterations and with a large contrast in R-factors before and at converegnce. Another means to modify the peaks are:

pick_atoms *

choose_to 5

insert = Rand(-.1, 1);

The insert case is slightly slower than the omit case as the 5 atoms are first omitted before insertion. Each case however solves 1a7y in a similar number of iterations.

Example CF-1A7Y-NO-TANGENT.INP is similar but without the tangent formula, 1a7y in this case solves in ~1000 iterations.

In CF-ALVO4.INP comment out the Tangent line as follows:

‘ Tangent(.5, 50)

Run CF-ALVO4.INP and turn on Octahedra viewing in the OpenGL window. Visual inspection of picked atoms should show electron densities that are not recognizable as correct solutions.

Include the Tangent line and rerun; after a minute or two and at the bottom of the Ramps visual inspection of picked atoms should show a well defined solution.

Thus use of the tangent formula assists in solving CF-ALVO4.INP.

CF works particularly well on pseudo symmetric structures (Oszlányi et el., 2006). Example CF-PN-02.INP is an oxide structure that contains 441 atom in the asymmetric unit (Lister et el., 2004); run CF to convergence. Pick atoms and turn on Octahedra viewing; all polyhedra should be well formed. Thus CF works extremely fast and trivializes the solving of such structures. The contents of the INP file is as follows:

charge_flipping

cf_hkl_file 020pn.hkl

space_group Pn

a 24.1332

b 19.5793

c 25.1091

be 99.962

fraction_reflections_weak 0.4

symmetry_obey_0_to_1 .3

Tangent(.25, 30)

load f_atom_type f_atom_quantity

{

MO = 42 2;

P  = (126 - 42) 2;

O  = (441 - 126) 2;

}

The tangent formula is used to assist symmetry_obey_0_to_1 and to assist in finding the solution faster; it is not necessary for this example however. The Oszlányi & Süto (2005) algorithm can be used by replaceing symmetry_obey_0_to_1 and the Tangent line with the following:

scale_F000 1

fraction_reflections_weak .4

add_to_phases_of_weak_reflections 90

user_threshold = 0.15 Get(max_density_at_cycle_iter_0);

Slow convergence is then observed and the reason is the use of F000. This is opposite to the case of 1a7y in CF-1A7Y-GABOR.INP where F000 is necessary. Setting scale_F000 to zero greatly increases the rate of convergence.

When symmetry_obey_0_to_1 is defined origin finding is performed each iteration of charge flipping. Symmetry elements of the space group are used in finding an origin. On finding an origin the electron density is shifted to a position that best matches the symmetry of the space group. Additionally a restraint is placed on the electron density pixels forcing symmetry to be obeyed.

Run CF-AE14.INP to convergence; notice the P-1 symmetry. Remove symmetry_obey_0_to_1 and run to convergence; the origin should now be arbitrary.

Run CF-AE5.INP until a solution is found; terminate CF, this saves the phase information to the file AE5.FC.

Copy AE5.FC to AE5-SAVE.FC.

Place the following lines into the file CF-AE5-POOR.INP

set_initial_phases_to ae5-save.fc

randomize_initial_phases_by 0

This simply starts CF with optimum phase values. Also include the following line:

symmetry_obey_0_to_1 .75

Run CF-AE5-POOR.INP; the atom positions after picking should visually produce the correct result. Comment out symmetry_obey_0_to_1 and rerun CF-AE5-POOR.INP. R-factors should diverge and picked atoms should show a non-solution. Thus symmetry_obey_0_to_1 assists in solving CF-AE5-POOR.INP.

Include symmetry_obey_0_to_1 and remove set_initial_phases_to and randomize_initial_phases_by and then rerun CF-AE5-POOR.INP. A solution should be obtained in a few minutes. Note that in this example the default flipping regime leads to regular occurrences of uranium atom solutions; this can be trivially ascertained by viewing the electron density. To reduce the occurrences of uranium atom solutions the following flipping regime is used:

flip_regime_3 0.5

Example CF-AE9-POOR.INP demonstrates the limit to which the present CF implementation can operate. Single crystal data is purposely chosen to isolate resolution effects and not intensity errors. The tangent formula is critical where without it the CF process is extremely perturbed and unstable. “flip_regime_3 .5” is used due to occurrences of uranium atom solutions.

There are no ramps, instead the CF process is restarted when the R-factor fails to decrease for 100 consecutive iterations, or,

break_cycle_if_true = Get(iters_since_last_best) > 100;

randomize_phases_on_new_cycle_by = Rand(-180, 180);

Half of the observed reflections are considered weak and additionally missing reflections up to 1 Angstrom are included and considered weak using:

fraction_reflections_weak 0.5

extend_calculated_sphere_to 1

The intensities of weak reflections are left untouched and instead a Pi/2 phase shift is randomly applied to ~30% of weak reflections as follows:

add_to_phases_of_weak_reflections = If(Rand(0, 1) < .3, 90, 0);

A symmetry_obey_0_to_1 of 0.7 is used not merely to find an origin but rather to prevent the electron density from straying.

Run CF-AE9-POOR.INP and a solution should be clearly recognizable after a few minutes. Change/remove keywords and rerun to view effects. Examples CF-CIME-POOR.INP and CF-AE5-POOR.INP are similar.

CF-SUCROSE.INP without scale_density_below_threshold=0 exhibits large oscillations in R-factors resulting in difficult to identify solutions; this can be prevented by increasing the amount of charge flipped and including scale_density_below_threshold=0,  for example

fraction_density_to_flip 0.83

scale_density_below_threshold 0

When scale_density_below_threshold=0 is used the percentage of charge that is less than the threshold before the application of scale_density_below_threshold is reported; the difference between this reported value and (1-fraction_density_to_flip) gives the amount of flipped pixels that survived scale_density_below_threshold. At fraction_density_to_flip of 0.83 approximately 23% of pixels survives scale_density_below_threshold=0 which in effect means that only 23% of pixels are actually flipped out of the original 83%.

picked_atoms is used as a perturbation where 30% of atoms are omitted using:

pick_atoms *

activate = Cycle_Iter == 0;

insert = If(Rand(0, 1) > 0.3, 10, 0);

Note that atoms are inserted at an intensity that is 10 times the average intensity. This increases the weight of inserted atoms relative to electron density noise. It also initially gives more weight to weak reflections.

Use of scale_density_below_threshold often results in CF requiring more interations to solution; a solution however is preferable to no solution.

The act of flipping introduces an appreciable amount of unwanted high frequencies in the structure factors. This effect can be reduced by dampening high frequencies using apply_exp_scale which is ON by default. apply_exp_scale changes R-factors and not phases, directions taken by CF are unchanged.

Run CF-1A7Y.INP until convergence. The difference in R-factors before and at convergence should be ~0.39 (ie. 0.81 and 0.42). Turn OFF apply_exp_scale by including the following line:

apply_exp_scale 0

Rerun CF-1A7Y.INP until convergence. The difference in R-factors before and at convergence should now be ~0.29 (ie. 0.81 and 0.52). Thus apply_exp_scale increases contrast in R-factors. Note that most of the increase seems to be realized from d-spacings less than 1 Angstrom.

 

[add_to_cloud_N !E]

[add_to_cloud_when !E]

The current cloud is added to the GUI cloud creating a running average cloud for display purposes. add_to_cloud_N corresponds to the number of most recent clouds to include in the running average. add_to_cloud_when determines when the current cloud is to be included in the running average; here’s an example:

add_to_cloud_N 10

add_to_cloud_when = Mod(Cycle_Iter, 2);

Averaged clouds eliminate noise and is effective if the cloud remains stationery which is generally the case. Note that add_to_phases_of_weak_reflections can produce translations of the cloud and should not be included when averaging clouds.

[add_to_phases_of_weak_reflections !E]

Allows for modification to phases of weak reflections. For example, to add p/2 to the phases of weak reflections then the following could be used:

add_to_phases_of_weak_reflections 90

When add_to_phases_of_weak_reflections is defined then the intensities of weak reflections are not set to zero; instead they are left untouched meaning that their intensities are set to the values as determined by the inverse Fourier transform. See also scale_weak_reflections.

[apply_exp_scale !E]

Determines a and b each CF iteration such that the following is a minimum:

R-factor = ∑| a Exp(b / D_spacing^2) Fc – Fo |

where Fc and Fo are the calculated and observed moduli respectively. Use of apply_exp_scale corrects R-factors in case of an incorrect temperature factor correction as applied when normalizing structure factors. Use of apply_exp_scale typically increases the difference between R-factors prior to and at convergence. apply_exp_scale is used by default, setting it to zero prevents its use.

 [cf_hkl_file $file]

Defines the input hkl file.

[cf_in_A_matrix $file [scale_Aij !E] ]

Data input is from a file created using out_A_matrix from a previous Pawley refinement. The correlations in $file are used to partition intensities during each iteration of charge-flipping. This partitioning is applied to structure factors as used by CF and as used by the tangent formula.

scale_Aij can be used to modify the A matrix off-diagonal coefficients, here are some examples:

scale_Aij = Get(Aij);

scale_Aij = Get(Aij)^2; ‘ the default

scale_Aij = 0; ‘ Equivalent to using a Pawley generated hkl file

CF on powder data can also be initiated using standard hkl files.

[break_cycle_if_true !E]

Interrupts charge flipping to execute randomize_phases_on_new_cycle_by. Cycle_Iter is set to zero and Cycle is incremented.

[correct_for_atomic_scattering_factors !E]

Structure factors are normalized when non-zero and when f_atom_type’s are defined. By default structure factors are normalized.

[correct_for_temperature_effects !E]

Attempts to remove isotropic temperature effects from the structure factors. correct_for_temperature_effects is ON by default, setting it to zero will prevent the correction. Normalized structure factors are realized when correct_for_temperature_effects is ON and the unit cell contents is defined using f_atom_type and f_atom_quantity.

[delete_observed_reflections !E]

Reflections are deleted before entering CF according to delete_observed_reflections; it can be a function of D_spacing, for example:

delete_observed_reflections = D_spacing < 1.1;

Once deleted, observed reflections cannot be reinstated by changing min_d.

[extend_calculated_sphere_to !E]

Used to sharpen electron density clouds by filling in missing reflections; added reflections are given the status of “weak”. extend_calculated_sphere_to can be used in conjunction with scale_weak_reflections and add_to_phases_of_weak_reflections to modify “weak” reflection magnitudes and phases respectively (see section 11.2.6); here’s an example:

extend_calculated_sphere_to 1

add_to_phases_of_weak_reflections = If(Rand(0, 1) < .3, 90, 0);

[f_atom_type $type f_atom_quantity !E]…

Defines atom types and number of atoms within the unit cell; used by the tangent formula in determining E_h values and by the OpenGL display for picking atoms. For the tangent formula then realtive quantities are important.

[find_origin !E]

If defined and non-zero then origin finding is turned ON. symmetry_obey_0_to_1 defines find_origin by default. symmetry_obey_0_to_1 can be used without find_origin by defining and setting find_origin to zero.

[flip_equation !E]

Allwows for a user defined flip; here’s an example:

flip_equation =

If(Get(density) < Get(threshold), -Get(density), Get(density));

[flip_regime_2 !E]

The elctron density is modified according to Eq. (11‑1) and then further modified using:

flip_regime_2 is typically ramped from 1 to 0.

[flip_regime_3 !E]

The elctron density is modified according to Eq. (11‑1) and then further modified using:

A value of 0.5 for flip_regime_3 introduces little perturbation whilst reducing the occurance of uranium atom solutions. It is recommended that flip_regime_3 be used in cases where flip_regime_2 produces uranium atom solutions. An additional perturbation, such as “add_to_phases_of_weak_reflections=90;” may be necessary.

[fraction_density_to_flip !E]

The amount of charge flipped is fractionally based. A value of 0.6, for example, sets the threshold d such that the sign of the lowest 60% of charge is changed. Get(threshold) can be used to retrieve d.

[fraction_reflections_weak !E]

Defines the fraction of observed reflections to flag as “weak”. When scale_weak_reflections, add_to_phases_of_weak_reflections and extend_calculated_sphere_to are all not defined then intensities of weak reflections are set to zero effectively removing them from the charge flipping process. Otherwise intensities of weak reflections are not set to zero; instead they are left untouched prior to scale_weak_reflections and add_to_phases_of_weak_reflections and space group family averaging.

[histogram_match_scale_fwhm !E]

[hm_size_limit_in_fwhm !E]

[hm_covalent_fwhm !E]

An implementation of Histogram Matching (Baerlocher et al., 2007) where the distribution of pixels within the unit cell is restrained to one that mathes Gaussian atoms with intensities corresponding to the atoms defined by f_atom_type‘s. The Histogram matching operation is performed when  histogram_match_scale_fwhm evaluates to a non-zero value. Subsequently the full width at half maximum (FWHM) of the Gaussians (obtained from the file ATOM_RADIUS.DEF) is scaled by histogram_match_scale_fwhm. hm_size_limit_in_fwhm corresponds to the extent to which the Gaussians are calculated in units of FWHM. Covalent radii is used if hm_covalent_fwhm evaluates to a non-zero value otherwise ionic radii is used. An example use is as follows:

histogram_match_scale_fwhm = If(Mod(Cycle_Iter, 3), 0, 1);

hm_size_limit_in_fwhm 1

hm_covalent_fwhm 1

Reported on is the fraction of pixels modified; values of 1 for both histogram_match_scale_fwhm and hm_size_limit_in_fwhm seem optimal where typically ~15 to 20% of pixels are modified. Use of histogram matching should produce R-factors at convergence that are equal to or than less R-factors produced when not using histogram matching. Histogram matching sharpens the electron density cloud for data  at poor resolution (see examples CF-CIME-HISTO.INP and CF-CIME-POOR-HISTO.INP).

 [min_d !E]

Determines in Angstroms the resolution of observed reflections to work with; only observed reflections with a d-spacing above min_d are considered. min_d is evaluated each CF iteration. Get(num_observed_reflections_above_d_min) is updated when a change in min_d is detected. See also extend_calculated_sphere_to and delete_observed_reflections.

[min_grid_spacing !E]

If defined then the grid spacing used is set to the smaller of min_d/2 and min_grid_spacing; useful for obtaining many grid points for graphical purposes.

[neutron_data]

Signals that the input data is of neutron type. Used in the picking of atoms and additionally E_h values are not corrected from any defined f_atom_type and f_atom_quantity keywords.

[pick_atoms $atoms]…

[activate !E]

[choose_from !E]

[choose_to !E]

[choose_randomly !E]

[omit !E]

[displace !E]

[insert !E]

pick_atoms modifies the electron density based on picked atoms. $atom corresponds to the atom types to be operated on; it can contain the wild card character ‘*’ and the negation character ‘!’, see section 7.6 for details. The operations of pick_atoms are invoked when activate evaluates to a non-zero value; for example,

pick_atoms “O C”

activate = Mod(Cycle_Iter, 20) == 0;

The picking routine will attempts to locate the atom types found in $atom based on the intensities of picked atoms and the scattering power of the atoms defined in f_atom_type. For example,

load f_atom_type f_atom_quantity { Ca 2 O 10 C 12 }

pick_atoms “O C”

Here 2 Ca atoms are first picked and then 10 O atoms and then 12 C atoms. The picked atoms operated on will be the O and C atoms with the Ca atoms ignored.

choose_from and choose_to can be used to limit the number of atoms operated on. Note, that picked atoms within a particular pick_atoms are sorted in decreasing intensity order. For example, to not operate on the first thee O atoms and the last 2 C atoms then the following could be used:

choose_from 4

choose_to 20

choose_randomly further reduces the atoms operated on and is exected after choose_from and choose_to.

omit removes operated on atoms from the electron density. Atoms can be partially removed by setting omit to values less than 1. Values greater than 1 can also be used, the effect is to change the sign of the electron density. omit operating on a few of the highest intensity atoms is an extremely effective means of preventing the occurance of uranium atom solutions, see CF-1A7Y-OMIT.INP; for example:

pick_atoms *

choose_to 5

omit = Rand(1, 1.1);

Omitting atoms randomly is a technique referred to as “random omit maps“ in ShelXD, (Schneider and Sheldrick, 2002).

insert inserts operated on atoms; a value of 1 inserts the atoms with an intensity that is equal to the average of the picked atoms. Values of less than 1 descreases the intensity of the inserted atoms. When insert is defined then omit is internally defined if it does not already exist. Thus, atoms are removed before insertion by default.

displace (in Angstroms) displaces atom positions from their picked positions; it is evaluated before insert. For example, to randomly displace atoms by 0.3 Angstroms then the following could be used:

displace = Rand(0.4, 0.6);

insert 1

There can be more than one occurance of pick_atoms, for example to limit uranium atom solutions then the follow can be used:

pick_atoms *

choose_to 5

insert = Rand(-.1, 1);

To further randomly remove ~33% of atoms then the following could additionally be used:

break_cycle_if_true = Get(iters_since_last_best) > 10;

pick_atoms *

activate = Cycle_Iter == 0;

insert = If(Rand(0, 1) > 0.33, 10, 0);

Note that in this example atoms are inserted at ten times the average picked intensity; this simply gives more weight to picked atoms relative to electron density noise. Additionaly weak reflections are also given more weighting.

[pick_atoms_when !E]

Atoms are picked in the OpenGL display when pick_atoms_when evaluates to a non-zero value; here’s an example:

pick_atoms_when = Mod(Cycle_Iter + 1, 10) == 0;

Note that picking can be manually initiated from the Cloud dialog of the OpenGL display. A text description of picked atoms can be obtained by opening the “Temporary output” text window of the OpenGL window.

[randomize_initial_phases_by !E]

Initializes  phases. To start a process with already saved phase information then the following could be used:

set_initial_phases_to aleady_saved.fc

randomize_initial_phases_by 0 ' this has a default of Rand(-180,180)

[randomize_phases_on_new_cycle_by !E]

randomize_phases_on_new_cycle_by = Rand(-180, 180); ‘ an example

[scale_density_below_threshold !E]

Electron density pixels that are less than the threshold value are scaled by scale_density_below_threshold. Values for scale_density_below_threshold thata re less than 1 tends to sharpen the electron density and to reduce large oscillations in R-factors; the latter occurs for bad data, see example CF-SUCROSE.INP. A value of zero for scale_density_below_threshold results in “low density elimination“ simlar to that of Shiono & Woolfson (1992).

[scale_E !E]

Normalized structure factors (E_h values) are a function of correct_for_temperature_effects and unit cell contents. scale_E allows for an additional scaling of E_h values.

[scale_F !E]

CF works with normalized structure factors by default. scale_F is an additional scaling of structure factors. The defualt scale_F broadens electron density peaks to avoid pixilation effects and is given by:

scale_F = Exp(-0.2 Get(d_squared_inverse));

[scale_F000 !E]

Scale should be set to 1 for compliance with the algorithm of Oszlányi & Süto (2004). When scale_F000 is non_zero then modifcations to the electron density produces unfavourable effects.

[scale_weak_reflections !E]

By default weak reflection structure factors are set to zero; however when either scale_weak_reflections or add_to_phases_of_weak_reflections is defined then weak reflections structure factors are instead modified accordingly, for example:

scale_weak_reflections = Rand(-0.2, 0.4);

scale_weak_reflections or add_to_phases_of_weak_reflections can be a function of D_spacing.

[set_initial_phases_to $file]

[modify_initial_phases !E]

Sets initial phases to those appearing in $file. Typically $file corresponds to a *.FC file saved in a previous charge-flipping process. modify_initial_phases is executed each iteration of CF; it can be used to restrain the phases of $file. For example,

modify_initial_phases =

Get(initial_phase) + Min(Abs(Get(phase_difference)), 45);

where phase_difference corresponds to the difference between the current phase and the initial phase; it has a value between ±90º. modify_initial_phases can be used to constrain phases to those as determined by HRTEM (Baerlocher et al., 2007).

[space_group $]

If defined then the cf_hkl_file is assumed to comprise merged hkls corresponding to the defined space group; otherwise the cf_hkl_file is assumed to be of space group type P1.

[symmetry_obey_0_to_1 !E]

If a space group is defined then symmetry is adhered to according to symmetry_obey_0_to_1.  symmetry_obey_0_to_1 can be through of as a real space electron density restraint; its value should range between 0 and 1. If 1 then symmetry is obeyed 100%. symmetry_obey_0_to_1 is implemented follows:

For a particular set of equivalent grid points as determined by the equivalent positions of the space group an average density r_avg is obtained. The electron densities on the grid points are then adjusted as follows:

r_new = r (1 - symmetry_obey_0_to_1) + symmetry_obey_0_to_1 r_avg

The text output 'symmetry error' as displayed when symmetry_obey_0_to_1 is used is defined as follows:

'symmetry error' =

where the summation is over all of the electron density grid points.

symmetry_obey_0_to_1 defines find_origin by default. find_origin is applied before symmetry_obey_0_to_1. find_origin shifts the electron density such that an approximate error in 'symmetry error' is minimized; thus find_origin assists in the symmetry_obey_0_to_1 restraint.

[tangent_num_h_read !E]

[tangent_num_k_read !E]

[tangent_num_h_keep !E]

[tangent_max_triplets_per_h !E]

[tangent_min_triplets_per_h !E]

[tangent_scale_difference_by !E]

tangent_num_h_read and tangent_num_k_read defines the number of highest h and highest k reflections to read in determining triplets.

tangent_num_h_keep defines the number of highest h reflections to include for tangent formula updating.

tangent_max_triplets_per_h and tangent_min_triplets_per_h defines the maximum and minium number of triplets per reflection h. Reflections with less that tangent_min_triplets_per_h are not included for tangent formula updating.

tangent_scale_difference_by corresponds to S in the following:

[user_threshold !E]

By default Get(threshold) is determined using fraction_density_to_flip. When defined then user_threshold overrides fraction_density_to_flip. Electron density pixels are normalized to have a maximum value of 1, thus typical values for user_threshold range between 0 and 0.1.

[use_Fc]

Sets initial phases to those saved in a previous *.FC file. The FC file used corresponds to the same name as the data file, defined using cf_hkl_file or cf_in_A_matrix, but with a FC extension. use_Fc is similar to set_initial_phases_to except that the file used is implied.

[verbose #]

A value of 1 outputs text in a verbose manner. A value of 0 outputs text only when a R-factor less that a previous value is encountered within a particular Cyle.

[view_cloud !E]

Informs a detected GUI to display the electron density. Here are some examples:

view_cloud 1 ' Update cloud every charge-flipping iteration

view_cloud = Mod(Cycle_Iter, 10) == 0;

 

The following algorithm is based on the iterative method of Coelho (2003). Unlike lp_serach it requires the extraction of d-spacings. The INDEXING directory contains example INP files, for example:

index_zero_error

try_space_groups “2 75”

load index_d {

   8.912

   7.126

   4.296

   …

}

Individual space groups can be tried or for simplicity all of the Bravais lattices can be tried by placing them in the INP file using the standard macros as follows:

Bravais_Cubic_sgs

Bravais_Trigonal_Hexagonal_sgs

Bravais_Tetragonal_sgs

Bravais_Orthorhombic_sgs

Bravais_Monoclinic_sgs

Bravais_Triclinic_sgs

On termination of Indexing a *.NDX file is created with a name corresponding to the name of the INP file and placed in the same directory as the INP file. The *.NDX file contains solutions found as well as a detailed summary of the best 20 solutions. Here’s an example of an NDX file:

' Indexing method - Alan Coelho (2003), J. Appl. Cryst. 36, 86-95

' Time: 2.015 seconds

 

     'Sg     Status UNI      Vol       Gof     Zero      Lps…

 

Indexing_Solutions_With_Zero_Error_2 {

 

   0) P42/nmc    3   0    1187.321    38.82   0.0000    11.1924  …

   1) P42/nmc    3   0    1187.057    38.64   0.0000    11.1896  …  

   2) P42/nmc    3   0    1187.458    38.61   0.0000    11.1914  …  

…

}

/*

======================================================================

   0) P-1         0     985.652    30.80   0.0111     7.0877  …  

 

   h   k   l       dc       do    do-dc     2Thc     2Tho   2Tho-2Thc

   0   0   1   15.857   15.830   -0.027    5.569    5.578    0.009

   0   1   0    8.765    8.750   -0.015   10.084   10.101    0.017

   0   0   2    7.928    7.910   -0.018   11.151   11.177    0.026

   0   1   1    7.788    7.780   -0.008   11.352   11.364    0.012

   0  -1   1    7.559    7.560    0.001   11.698   11.696   -0.002

…

*/

Details of solutions can be obtained at a later stage by including solution lines found in the NDX file into the INP file. For example, supposing details of solutions 50 and 51 were sought then the following (see example INDEXING\EX10.INP) could be used:

index_lam 1.540596

index_zero_error     

try_space_groups 2

 

Indexing_Solutions_With_Zero_Error_2 {

  50) P-1        1   0    2064.788     9.74   0.0000   …

  51) P-1        3   0    3128.349     9.61   0.0115   …

}

load index_d {

   15.83 good

    8.75

    7.91

    …

}

After running this INP file a *.DET file is created containing details of the supplied solutions.

The data structures for indexing are as follows:

Tindexing

[index_lam  !E1.540596]

[index_min_lp !E2]  [index_max_lp !E]

[index_max_Nc_on_No !E5]

[index_max_number_of_solutions #3000]

[index_max_th2_error !E0.05]

[index_max_zero_error #0.2]

[index_th2  !E | index_d !E]…

[index_I  E1 [good]]

[index_x0 !E]

[index_zero_error]

[seed]

[try_space_groups $]...

[x_angle_scaler #0.1]

[x_scaler #]

Values for most keywords are automatically determined or have default values (appearing as numbers above) adequate for difficult indexing problems. In the following example from UPPW (service provided by Armel Le Bail to the SDPD mailing list at http://sdpd.univ-lemans.fr/uppw/) only a few keywords are necessary. Also note the use of the dummy keyword; this allows for the exclusion of 2q and I values without having to edit the columns of data.

 

seed

index_lam  0.79776

index_zero_error   

index_max_Nc_on_No 6

try_space_groups 3

load index_th2 dummy dummy index_I dummy  {

    ' d (A)  2Theta     Height      Area     FWHM

     1.724  26.50645    2758.3   23303.7    0.0450

     2.646  17.27733  150393.8  747063.6    0.0250

     3.235  14.13204   98668.8  493153.7    0.0250

     3.417  13.37776   11102.6   53185.0    0.0250

     5.190   8.80955     782.7    3910.9    0.0250

     …

}

[index_lam  !E1.540596]

Defines the wavelength in Å.

[index_min_lp !E2.5] [index_max_lp !E]

Defines the minimum and maximum allowed lattice parameters. Typically the maximum is determined automatically.

[index_max_Nc_on_No !E5]

Determines the maximum ratio of the number of calculated to observed lines. The value of 6 allows for up to 83% of missing lines.

[index_max_number_of_solutions #1000]

The number of best solutions to keep.

[index_max_th2_error !E0.05]

Used for determining impurity lines (un-indexed lines UNI in *.NDX). Large values, 1 for example, forces the consideration of more observed input lines. For example if it is know that there are none or maybe just one impurity line then a large value for index_max_th2_error will speed up the indexing procedure.

[index_max_zero_error !E0.2]

Excludes solutions with zero errors greater than index_max_zero_error.

[index_th2  !E | index_d !E]…

[index_I  E1 [good]]

index_th2 or index_d defines a reflection entry in 2q degrees or d-spacing in Å.

index_I is typically set to the area under the peak; it is used to weight the reflection.

good signals that the corresponding d-spacing is not an impurity line. A single use of good on a large d-spacing decreases the number of possible solutions and hence speeds up the indexing process (see examples INDEXING\EX10.INP).

[index_x0 !E]

Defines X_hh in the reciprocal lattice equation:

In a triclinic lattice the highest d-spacing can probably be indexed as 100 or 200 etc. Thus

index_x0 = 1/(d_max)^2;

speeds up the indexing process (if, in this case, the first line can be indexed as 100) and additionally the chances of finding the correct solution is enhanced. Example EX13.INP demonstrates this. Note that if the data is in 2Th degrees then the following can be used:

index_x0 = (2 Sin(2Th_min Pi/360) / wavelength))^2;

The two macros Index_x0_from_d and Index_x0_from_th2 simplify the use of index_x0.

[index_zero_error]

Includes a zero error.

[seed]

Seeds the random number generator.

[try_space_groups $]…

[x_angle_scaler #0.1]

[x_scaler #]

Defines the space groups to be searched. The macros Bravais_Cubic_sgs etc… (see TOPAS.INC) defines lowest symmetry Bravais space groups. It is almost always sufficient to use only these. Higher symmetry space groups for the Bravais lattices corresponding to the 10 best solutions is subsequently searched. Here are some examples of using try_space_groups.

Below is a list showing which space groups have identical hkls in regards to powder data.

x_scaler is a scaling factor used for determining the number of steps to search in parameter space. x_scaler needs to be less than 1. Increasing x_scaler searches parameter space in finer detail. Default values are as follows:

Cubic                            0.99

Hexagonal/Trigonal         0.95

Tetragonal                     0.95

Orthorhombic                 0.89

Monoclinic                     0.85

Triclinic                         0.72

x_angle_scaler is a scaling factor for determining the number of angular steps for monoclinic and triclinic space groups. Small values, 0.05 for example, increases the number of angular steps. The dult value of 0.1 is usually sufficient.

Here are two example output lines from an NDX file.

0) P42/nmc  3   0    1187.124    38.82   0.0000    11.1904    11.1904     9.4799     90.000     90.000     90.000 ' ===  24  19

6) P-421c   3   0    1187.124    35.67   0.0000    11.1904    11.1904     9.4799     90.000     90.000     90.000 ' ===  24  19

Ø       The 1^st column corresponds to the rank of the solution.

Ø       The 2^nd corresponds to the space group.

Ø       The 3^rd corresponds to the Status of the solution with meaning of the number as follows:

Ø       The 4^th  column corresponds to the number of un-indexed lines.

Ø       The 5^th column corresponds to the volume of the lattice.

Ø       The 6^th corresponds to the goodness of fit value.

Ø       The 7^th corresponds to the zero error if index_zero_error is included.

Ø       Columns 8 to 13 contains the lattice parameters.

The last 2 columns contain the number of non-zero h² + k² + h k and l² values used in the indexed lines. These represent the hkl coefficient for X0 and X1 respectively for Trigonal/Hexagonal systems. When one of these numbers are zero then the corresponding lattice parameters is not represented and the number is therefore displayed as the negative number of –999. This facility is particularly useful for identifying dominant zones. For example, if the smallest lattice parameter is 3Å and the smallest d-spacings is 4Å then it is impossible to determine the small lattice parameter. In these cases values of –999 will be obtained.

The following table gives the hkl coefficients corresponding to the X_nn reciprocal lattice parameters for the 7 crystal systems.

The most probable cause of failure is the inclusion of too many d-spacings. If it is assumed that the smallest lattice parameter is greater than 3Å then it is problematic to include d-spacings with values less than about 2.5Å when there are already 30 to 40 reflections with d values greater than 2.5Å. Some of the problems caused by very low d-spacings are:

Ø       The number of calculated lines increases dramatically and thus index_max_Nc_on_No will need to be increased.

Ø       The low d-spacings are probably inaccurate due to peak overlap at the high angles they are observed at.

A situation where it is necessary to include low d-spacings is when there are only a few d-spacings available as in higher symmetry lattices.

a, b and c lattice vectors can be converted to Cartesian coordinates with a collinear with the Cartesian x axis and b coplanar with the Cartesian x-y plane as follows:

where

a_x = a

b_x = b cos(g),   b_y = b sin(g)

c_x = c cos(b),   c_y =  c (cos(a) – cos(b) cos(g)) / sin(g),   c_z² = c² - (c_x)²– (c_y)²

a, b, c are the lattice parameters and a, b, g the lattice angles. The reciprocal lattice vectors A, B, and C calculated from the lattice vectors of Eq. (12‑1) become:

A= A_x i + A_y j  + A_zk

B = B_y j + B_z k

C = C_z

The equation relating a particular d-spacing d_hkl to a particular hkl in terms of the reciprocal lattice parameters is:

 

where

The command line program tc.exe provides for batch mode operation. Running tc.exe without arguments displays help information. Running an INP file is as follows:

tc pbso4

Macros can be passed to the command line. One use for this is to pass a file name to an INP file as follows:

1)   Create a TEMPLATE.INP file with the required refinement details, this should look something like the following:

xdd FILE

etc…

2)   TEMPLATE.INP is fed to tc.exe by command line and the word FILE (within TEMPLATE.INP) is expanded to whatever the macro on the command line is. For example,

tc …\file_directory\TEMPLATE.INP “macro FILE { file.xy }”

The macro called FILE is described on the command line within quotation marks. On running tc.exe the word 'FILE' occurring in TEMPATE.INP is expanded to 'file.xy'. Note that more than one macro can be described on the command line.

To process a whole directory of data files, say *.XY file for example, then:

1)       From the file directory execute the DOS command:

dir *.xy > …\main_ta_directory\XY.BAT

      The XY.BAT file will then reside in the main TA directory.

2)   Edit …\main_ta_directory\XY.BAT to look like the following:

tc …\file_directory\template “macro FILE { file1.xy }”

copy …\file_directory\template.out …\file_directory\file1.out

tc …\file_directory\template.inp “macro FILE { file2.xy }”

copy …\file_directory\template.out …\file_directory\file2.out

etc….

After each run of tc.exe a TEMPLATE.OUT file is created containing refined results. This file is copied to another file “file1.out”, “file2.out” etc… in order to save it from being overwritten.

After running XY.BAT a number of *.OUT files is created one for each *.XY file.

In summary TC.EXE receives TEMPLATE.INP to process. Words occurring in TEMPLATE.INP are expanded depending on the macros described on the command line.

 

Baerlocher, C; Gramm, F.; Massüger, L; McCusker, L; He, Z; Hovmöller, S & Zou, X. (2007). SCIENCE VOL 315 23 FEBRUARY 2007

Baerlocher, C.; McCusker, L.B.; & Palatinus, L. (2007).  Z.  Kristallogr. 222, 47-53

Balzar, D. (1999). Microstructure Analysis from Diffraction, edited by R. L. Snyder, H. J. Bunge, and J. Fiala, International Union of Crystallography, 1999. “Voigt-function model in diffraction line-broadening analysis”

Bergmann, J., Kleeberg, R., Haase, A. & Breidenstein, B. (2000). Mat. Sci. Forum, 347-349, 303-308. “Advanced Fundamental Parameters Model for Improved Profile Analysis”.

Brindley, G.W. (1945). Phil. Mag., 36, 347-369. “The effect of grain or particle size on X-ray reflections from mixed powders and alloys, considered in relation to the quantitative determination of crystalline substances by X-ray methods”

Broyden, C.G., J. Inst. Maths. Applics., Vol. 6, pp 76-90, 1970. “The Convergence of a Class of Double-rank Minimization Algorithms,”

Cagliotti, G., Paoletti, A. & Ricci, F.P (1958). Nucl. Inst., 3, 223-228. “Choice of collimators for a crystal spectrometer for neutron diffraction”

Coelho, A. A. (2005). J. Appl. Cryst. 38, 455-461. “A bound constrained conjugate gradient solution method as applied to crystallographic refinement problems”

Coelho, A. A,. (2003). J. Appl. Cryst., 36, 86–95. “Indexing of powder diffraction patterns by iterative use of singular value decomposition”.

Coelho, A. A. & Cheary, R. W. (1997). Computer Physics Communications, 104, 15-22. “A fast and simple method for calculating electrostatic potentials”

Cheary, R.W. & Coelho, A.A. (1998a). J. Appl. Cryst., 31, 851-861. “Axial divergence in a conventional X‑ray powder diffractometer I. Theoretical foundations”

Cheary, R.W. & Coelho, A.A. (1998b). J. Appl. Cryst., 31, 862-868. “Axial divergence in a conventional X‑ray powder diffractometer II. Implementation and comparison with experiment”

Baerlocher, C; Gramm, F.; Massüger, L; McCusker, L; He, Z; Hovmöller, S & Zou, X. (2007). SCIENCE VOL 315 23 FEBRUARY 2007.

Chernick, M.R. (1999). Bootstrap Methods, A Practitioner’s Guide, Wiley, New York.

DiCiccio, T.J. & Efron, B. (1996). Bootstrap confidence intervals (with discussion), Statistical Science 11, 189–228.

Durbin, J. & Watson, G.S. (1971). Biometrika, 58, 1-19. “Testing for Serial Correlation in Least Square Regression, III”

Efron, B. & Tibshirani, R. (1986). Bootstrap methods for standard errors, confidence intervals and other measures of statistical accuracy, Statistical Science 1, 54–77.

Fletcher, R. (1970). Computer Journal, Vol. 13, pp 317-322. “A New Approach to Variable Metric Algorithms,”

Finger, L.W., Cox, D.E & Jephcoat, A.P. (1994). J. Appl. Cryst., 27, 892-900. “A correction for powder diffraction peak asymmetry due to axial divergence”

Flack, H. D. (1983). Acta Cryst. A39, 876-881

Goldfarb, D. (1970). Mathematics of Computing, Vol. 24, pp 23-26. “A Family of Variable Metric Updates Derived by Variational Means”

Hauptman, H. & Karle, J. (1956). Acta Cryst.  9, 635

Hill, R.J. & Flack, H.D. (1987). J. Appl. Cryst., 20, 356-361. “The Use of the Durbin-Watson d Statistic in Rietveld Analysis”

Hölzer, G., Fritsch, M., Deutsch, M., Härtwig, J. & Förster, E. (1997).Physical Review A, 56, 4554-4568. “Ka_1,2and Kb_1,2X-ray emission lines of the 3d transition metals”

Järvinen, M. (1993). J. Appl. Cryst., 26, 525-531. “Application of symmetrized harmonics expansion to correction of the preferred orientation effect”

Johnson, C.K. & Levy, H.A. (1974).International Tables for X-ray Crystallography, IV, 311 - 336. “Thermal-motion analysis using Bragg diffraction data”

Le Bail, A. & Jouanneaux, A. (1997). J. Appl. Cryst., 30, 265-271. “A Qualitative Account for Anisotropic Broadening in Whole-Powder-Diffraction-Pattern Fitting by Second-Rank Tensors”

Lister, S. E.; Radosavljevic Evans, I.;  Howard, J. A. K.; Coelho A. and Evans, J. S. O. (2004). Chemical Communications, Issue 22.

March, A. (1932). Z. Krist., 81, 285-297. “Mathematische Theorie der Regelung nach der Korngestalt bei affiner Deformation”

Marquardt, D. W. (1963). J. Soc. Ind. Appl. Math., 11(2), 431-331. “An algorythm for least-squares estimation of nonlinear parameters”

Oszlányi, G. & Süto A. (2004).  Acta Cryst. A60, 134-141

Oszlányi, G. & Süto A. (2005).  Acta Cryst. A61, 147-152

Oszlányi, G. & Süto A. (2006).  Acta Cryst. A63, 156–163

Oszlányi, G.; Süto A.; Czugler, M. & Parkanyi, L. (2006). J. AM. CHEM. SOC. 9 VOL. 128, NO. 26, 8393. “Charge Flipping at Work: A Case of Pseudosymmetry”.

Shanno, D.F. (1970). Mathematics of Computing, Vol. 24, pp 647-656. “Conditioning of Quasi-Newton Methods for Function Minimization”

Favre-Nicolin, V.  and Cerny, R. (2002) EPDIC 8 proceedings. “Fox: Modular Approach to Crystal Structure Determination from Powder Diffraction”

Sabine, T. M., Hunter, B. A., Sabine, W. R., Ball, C. J. (1998):  J. Appl. Cryst. 31, 47-51

Schneider, T. R. & Sheldrick, G. M. (2002). Acta Cryst. D58, 1772-1779. “Substructure solution with SHELXD“

Shiono, M. & Woolfson, M. M. (1992). Acta Cryst. A48, 451-456

Young, R.A. (1993). The Rietveld Method, edited by R.A. Young, IUCr Book Series, Oxford University Press 1993, 1-39. “Introduction to the Rietveld method”