Table 1 Validation criteria for analytical performance metrics

Analytical sensitivity

Likelihood that the assay will detect a target (for example, organism variant, sequence region, functional element, and so on) in a sample (that is, target), if present; can include target attribution when defined as strain- or isolate-level detection. Also known as the true positive rate. Calculated by dividing number of true positives by the sum of true positive and false negatives (TP/(TP + FN)).

Analytical specificity

Likelihood that the assay will not detect a target, if not in the sample; can include false target attribution. Also known as the true negative rate. Calculated by dividing true negatives by the sum of true negatives plus false positives (TN/(TN + FP)). May be impractical to calculate for methods designed to detect the known universe of organisms.

Precision

The degree that individual measurements of the same sample are similar with regard to the presence and absence of target. Determined by the distribution of random errors and not the true or underlying value.

Accuracy

Degree that the material measured is similar to its true value. Calculated by (TP + TN)/(TP + FP + FN + TN).

Reproducibility

The degree to which the same result(s) is obtained for a sample when the assay is repeated between/among different operators and/or detection instruments.

Repeatability

The degree to which the same result(s) is obtained for a sample when the assay is repeated by the same operator and/or detection instrument.

Limit of detection

Minimum level of input material for a target as a proportion of the total at which all replicates are consistently positive for that target.

Reportable range

The region(s) of genome(s) that are sequenced and from which information is drawn for comparison or attribution.

False positive rate

The rate at which a target is incorrectly called as present. Also known as Type I error. Calculated as 1 – specificity

False negative rate

The rate at which a target organism is incorrectly called as absent. Also known as Type II error. Calculated as 1 – sensitivity.

Assay robustness

Stability of analytical performance under variable conditions, that is, likelihood of assay success.

Reference materials^a

Materials/samples used to test the performance of the assay (for example, reference panels of the target and mock or non-probative materials) relevant to the intended application of the assay.

Databases^a

Collection of data and reference genomes, genes and genomic elements to be used for interpretation of results.

Interpretation criteria for results^a

Analysis (quantitative or qualitative) used and confidence level of a result (match, association, most recent common ancestor, and so on).