Validation

Introduction

The Validation tab gives you a complete overview of PeptideShaker's matches validation results. Note that you can edit the settings used when creating the projects in the advanced validation settings.

In this tab, for each group you can:

More about Validation


Group Selection

The standard groups inspected by PeptideShaker are PSMs, peptides and proteins. However, using the default settings, if statistical significance is ensured PSMs will be separated according to their charge. Similarly, peptides can be separated based on their modification status.

This grouping strategy allows you to increase the sensitivity of the processing without compromising robustness. Note however that changes at the PSM level can affect the Peptide and Protein levels. Similarly, changes at the Peptide level can affect the Protein level.

It is thus important to apply changes at the lower levels first!

For more information about peptide grouping see Vaudel et al: Peptide identification quality control, Proteomics 2011;11(10):2105-14.



More about Validation


Identification Summary

The identification summary provides essential metrics for the selected group:



More about Validation


Validation Parameters

The score threshold used, illustrated by a red vertical line in the confidence plot, can be changed to meet three types of requirements:


By default the threshold is set to 1% FDR for all levels. You can change this setting when creating the project in the advanced identification parameters.



More about Validation


Score vs. Confidence

This plot displays the confidence plotted against the score of the selected group's identifications.

The red vertical line indicates the chosen threshold. The red area on the right of the threshold illustrates the share of retained true positives. The green area on the left of the threshold illustrates the share of potential true positives not validated, i.e., the false negatives.

Tip: It is important to verify that the confidence reaches 0, otherwise the total number of true positives will be under-estimated.

No red line is displayed? You should use a less restrictive threshold.



More about Validation


Target and Decoy

This plot displays the distribution of target and decoy hits, in green and red, respectively, plotted against the identification score of the selected group.



More about Validation


FDR vs. Coverage

This plot displays the coverage which can be expected, the proportion of retained true positives (1-FNR), plotted against the FDR necessary to obtain such a coverage. In other words it is a ROC curve for the selected group, or a cost benefit plot.

A point indicates the performance at the selected threshold. Note that the point can diverge from the curve for small datasets.



More about Validation