[Info box: Careful: in study design, retrospective does not necessarily mean that leftover samples are used. Retrospective studies can be conducted with fresh samples as well, if the clinical condition of the donor was known.]

In a prospective study, the main difference is that we do not know the clinical condition of the donor before they are included in the study. Donors would be selected according to other inclusion criteria, e.g., certain symptoms. If we were to include 300 donors with flu-like symptoms, we would not be able to predict how many of them would be positive or negative. The outcome might only be 44 positives and 256 negatives because a lot of donors were infected with other respiratory pathogens. In such a case, the collection must be continued until the desired number of Flu A positive samples is acquired. Therefore, the total necessary sample number for prospective studies is usually higher than for retrospective studies.
So why bother with prospective study designs? From a purely academical point of view, prospective studies generate much more valid data. The separation of positives and negatives in a retrospective study design is prone to selection bias due to the inclusion of clearly positive or negative samples. Consequently, we are making it easy for our index test (= the test that is to be validated). In a prospective study design, we include samples from donors who may be just barely positive or who are positive for similar biomarkers that could lead to false-positive results. Here, we are properly challenging our index test.
Be that as it may, the effort, time, and budget that is necessary to have a prospective clinical performance study conducted, is substantial. Therefore, many manufacturers accept the downsides of a retrospective study design to the benefit of quicker and cheaper results.
Careful, though: It is expected that in the near future this ‘easy way’ will not be viable anymore.
According to IVDR article 57.2, ‘where appropriate, performance studies shall be performed in circumstances similar to the normal conditions of use of the device’. For most IVDs, the normal condition of use would be in consequence of a certain medical indication. This implies a study design in which the true condition of the donors is unknown during testing with the index test, hence a prospective design. One of the main goals of the IVDR is that manufacturers pay heed to this logic.
While the expression ‘where appropriate’ seems to give us some leeway, it is becoming apparent that the legislators’ preference for prospective studies will be more and more enforced by secondary legal acts.
A first example is EU common list of COVID-19 antigen tests, published by the European Commission Directorate-General for Health and Food Safety. This document contained requirements for SARS-CoV‑2 assay validations which were de facto binding for the industry. Notably, it also differentiated between two categories of assays: Category A listed only products that had been validated through prospective clinical field studies, while Category B contained those with retrospective in vitro studies. EU Member States were ‘strongly encouraged’ by the authors to use only Category A devices for COVID test certificates which excluded Category B devices from a major reimbursement opportunity.
We expect that other future publications will go in the same direction. While it may not become illegal to conduct retrospective performance studies, the preference for prospective studies by key opinion leaders, expert panels or governmental authorities will oftentimes make this more complex and expensive study design the only option for manufacturers.
Long story short: It is tempting to facilitate clinical validation through a retrospective study design. However, the risks that are associated with this decision should not be neglected. The time will come when retrospective clinical data is simply not enough anymore. A prospective study may cost more and take longer but you gain first-rate data and are safe from any future enforcement actions.

Today, MDCG-guideline 2021–21, around which all our validation efforts revolved for over two years, is part of EU Regulation 2022/1107 (‘Common Specifications for certain class D in vitro diagnostic medical devices’). This document comprises twelve annexes containing tables with official (and quite challenging) sample size requirements for the state-of-the-art validation of devices intended for detection of blood group antigens, HIV, hepatitis, and other infectious diseases. These Common Specifications can greatly facilitate study design, as the number and specifications of samples to be procured are clear from the start. However, they place high demands on manufacturers and apply to only a dozen medical indications.

This means that for the vast majority of IVD devices, clinical performance study design still relies on statistical sample size calculation. We don’t want to delve into the depths of this mathematical field here (which fills countless books and publications), but rather give a couple of hints to make life easier of a non-statistician.
First of all, there is not one correct way to conduct a sample size calculation. There are multiple statistical tests that apply to different study designs. For IVD devices, however, it usually (but not always) comes down to two types of statistical tests that aim at achieving the following study goals:

1. Approach: I want to estimate the performance of my product with a reasonable margin of uncertainty, OR
2. Approach: I want to show that the performance of my product does not fall below a certain value.

These goals sound similar, but the formulas they use and the claims that they provide evidence for are different. If the intended purpose of your product stipulates being non-inferior to a state-of-the-art device, an estimation of the performance with a two-sided margin of uncertainty (the first approach) may not be the best way to calculate sample size for its clinical study. What you want to know after all is, how many donors you need to recruit to show that your device is no less sensitive than a competitor’s device. This circumstance is much better accounted for by the second approach.

Knowing which statistical approach for sample size calculation is appropriate for my study is not much use to me if I don’t know the formulae or how to apply them (which is true for most of us). Fortunately, there are several statistical publications that provide tables with pre-calculated sample sizes for different expected sensitivity/specificity-values, margins of uncertainty (= width of the confidence interval; applies to the first approach), levels of disease prevalence, or minimal accepted sensitivity/specificity-values (applies to the second approach).

[Infobox: Yes, this means that the outcome of the study (i.e., the performance of your product) must already be known during sample size planning. Remember that the goal of sample size planning is finding a number of samples that is large enough to sufficiently support your performance claim, but not so large that the study becomes economically and ethically unreasonable. The expected outcome can be deduced from pre-studies or findings from your product development phase, but also from competitors’ performance studies. For instance, if the state-of-the-art in medicine requires that your product has a certain minimum sensitivity, you know that this sensitivity should be the expected outcome of your study, otherwise your product would not be usable (AND you know that you should choose the second approach to sample size calculation).]

Some exemplary publications are:

• K. Hajian-Tilaki / Journal of Biomedical Informatics 48 (2014) 193–204
• A. Flahault et al. / Journal of Clinical Epidemiology 58 (2005) 859–862
• F. Krummenauer, H‑U. Kauczor / Fortschr Röntgenstr 174 (2002) 1438–1444 (German)

Using sample sizes from these or similar publications will not compromise your conformity assessment as long as you provide justification for the sample size chosen. The current state-of-the-art of your product type or findings from pre-studies can be used as justification.
Of course, multiple factors not mentioned here have an influence on sample size estimation (e.g., power and p‑value). Also, this article cannot apply to quantitative or semi-quantitative assays. For simple qualitative IVD assays, however, the statistical complexity is limited, which allows the generalisations made here in the first place. For anything more complex, consulting a statistician is always the only safe way.

The intended purpose of the vast majority of IVD devices, however, requires a more complex sample procurement strategy. For instance, a clinical performance study for a pregnancy test would require urine samples from women at specific time points before amenorrhea. This background data is usually not available for archived urine samples in biobanks. For a D‑dimer rapid test, one would need samples from patients with suspected deep venous thrombosis collected within a narrow time frame. A point-of-care white blood cell counting device must use non-frozen freshly drawn blood for analysis. For other products, it may be vital to know whether the donor was smoker or non-smoker, what their ethnicity was, or whether they were taking certain food supplements. They might need a specific novel sample type, certain pre-analytical conditions or simply a limited number of freeze-thaw cycles. The list goes on and on.

To put it in a nutshell: The sample requirements of most IVD devices are simply too specific to be coincidentally found in a biobank’s freezer. Anything that deviates from the current state-of-the-art medical routine is usually not collected in hospitals or elsewhere and therefore do not end up at a biobank.
Therefore, what manufacturers need, are service providers that are capable of establishing de novo procurement ways according to the intended purpose of the individual product that is to be validated.
At in.vent Diagnostica, we call this Targeted Sample Procurement.
Obviously, such services are more costly and time consuming than picking samples from a freezer. The alternative, however, might be that the notified body rejects your validation data due to unrealistic testing circumstances. A lot of IVD manufacturers will have to change their habits to avoid this dramatic outcome.
Do you need samples that correspond specifically to your product’s intended purpose? Contact us via email or give us a call, we would love to help you out!