What is precision?

In our blog article today, we would like to take a short look on the analytical method validation parameter “precision” and explain the different terms repeatability, intermediate precision and reproducibility.

Precision is a measure of closeness among the data obtained through a series of experiments under similar conditions. For analytical methods, it is the closeness among the results obtained under the specified conditions of the method. Precision aims to display the random error that could occur in a method. It targets the question of how close the results are to each other.

Precision, under any circumstances should not be mistaken with trueness. A method can be precise but not true ("accurate") or vice versa or both. Check the example below for a better understanding:

Figure 1: Explains the difference between Precision and Trueness with the example of a footballer practicing 9 spot kicks. 1. Ball was hit in the goal but in different areas; 2. Ball was hit towards the left side of the goal; 3. Ball was hit outside the goal but on the left side; 4. Ball was hit in all directions outside the target goal area.

 

In Figure 1, we see at 1 that the results of our method result in the correct value on average (since all are inside the goal = true), but the individual values show a high scatter (= imprecise). With 3 we have a very precise method, because all results are very close to each other, but unfortunately it measures wrong (because all are outside the goal). 2 represents the ideal case and 4 requires optimization.

Precision of analytical methods is defined in terms of repeatability, intermediate precision and reproducibility.

	Day 1	Day 2	Analyst 1	Analyst 2	Instru-ment 1	Instru-ment 2	Lab 1	Lab 2	Scatter of the results
Repeatability	x		x		x		x
Intermediate precision	x	x	x	x	x	x	x
Reproducibility							x	x

Table 1: Reflects a general scheme of precision experiments.

 

Repeatability

Repeatability tells us about the closeness of the analytical results after a series of repeated experiments in a short time interval. Repeatability is also referred as intra-assay precision. In this, the method is performed multiple times at the same day by the same operator (= repeated measurements of a sample directly one after the other), i.e. the measurements are subject to very similar conditions. As per the method validation guideline ICH Q2(R1), repeatability can only be determined with a minimum of 9 determinations (3 concentration x 3 replicates) or 6 determinations at 100% of the test concentration.

Figure 2: Depicts the idea of repeatability. Same footballer takes the spot kicks at different times on the same day.

 

Intermediate precision

Intermediate precision reflects the variations within the laboratory by analyzing the closeness of analytical results that are obtained when a method is applied on different days, by different operators using different instruments (but usually identical in construction, i.e. same model purchased from the same manufacturer). Hence, the intermediate precision is compared day to day, analyst to analyst, instrument to instrument. Alternatively, intermediate precision may also be evaluated in a so-called “matrix approach” consisting of 6 experiments covering all aspects together. The scatter of the intermediate precision results is generally a bit larger than that of the repeatability precision, since, e.g., in addition to slight differences in the way different analysts work, slight differences in the room temperature could also occur, if, e.g., on the second day the sun really burns through the window while the first day was cloudy.

Figure 3: Illustrates the idea of Day to Day intermediate precision. Same footballer takes the spot kicks at two different days.

Figure 4: Shows the idea of Analyst to Analyst intermediate precision. Two different footballers take the spot kicks keeping all other factors constant.

Figure 5: Details the idea of Instrument to Instrument intermediate precision. Same footballer takes the spot kicks using two different balls.

 

Reproducibility

Reproducibility is a kind of precision that displays differences in a method by comparing analytical results obtained by a method performed in different laboratories, thus different places. Here, not only different analysts, but also environmental conditions and, for example, devices from different manufacturers play a role. It should not be mixed up with method transfer, which is an important (mandatory) parameter for drugs and their active pharmaceutical ingredients that are being launched in different countries or continents or are being manufactured in a new place than the one reported before. To make it complicated: The previous statement is not quite correct, because a co-validation as part of a method transfer aims exactly at the reproducibility between the participating laboratories. The reproducibility essentially shows that the performance of the analytical method is independent to the location of the laboratory and to different laboratories, respectively. Therefore, during a method validation within one lab, reproducibility does not play a role.

Figure 6: Details the idea of reproducibility (inter-laboratory) precision. Same footballer takes the spot kicks in two different locations, although it is usually not the same footballer.

 

Reproducibility is not used only to measure the performance of a method. Many laboratories use this parameter to assess their performance of the quality test. A proficiency test like a universal Ring test could be performed for such purpose. A Ring test is an inter laboratory test that allows to estimate the performance of a method (and of the testing laboratory, respectively) based on trials done with similar homogenous samples. The goal of such test is to improve the quality and efficiency of the laboratory performance (e.g. in case of being an accredited testing laboratory). Ring tests are essential to prove competence to accreditation authorities, customers, and regulatory bodies. Often Ring tests are performed for methods which should become compendial methods, thus being listed in a pharmacopoeia like e.g. in the Ph.Eur..

 

Results from precision tests are analyzed statistically. The ICH Q2(R1) suggests the result should be presented in terms of standard deviation, relative standard deviation (coefficient of variance), and confidence interval for each type of precision. The confidence interval is a useful statistical tool to assess precision. In other words, it allows us to know the range of the method within which it can produce precise and reliable data. Regarding precision, it may not be useful to apply it to repeatability in case only 6 determinations at 100% test concentrations may have been done, but it can be worth to calculate an “overall” CI, considering both the values obtained during repeatability as well as the ones of intermediate precision. Further information about confidence intervals in the context of method validation and precision can be found here.

If chromatographic separation methods are used to determine the content of drug substances listed in the Ph. Eur., the Ph. Eur. also specifies maximum values for the relative standard deviation (RSD) depending on the number of injections (section 2.2.46). This aspect is taken up by the ZLG Aide-mémoire 07123101. Furthermore, the ZLG Aide-mémoire 07123101 makes the clear statement that an "exclusive determination of the "device precision" (repeated measurement of the processed sample or a standard) is not sufficient to determine the precision".