Biosecurity Commons Support Portal

Overview of Proof of Freedom

"Proof of Freedom" is a term used to describe the evidence that a specific pest or disease is absent from a particular geographical region. This concept, however, is complex and nuanced. It involves balancing the amount of effort put into surveillance against the potential cost of missing a pest. The key to demonstrating proof a freedom requires a combination of good surveillance strategies, statistical analysis, and an understanding of the potential risks and costs involved. Proof of freedom is crucial in ensuring biosecurity, facilitating international trade, and protecting native flora and fauna.

It is almost impossible to definitively prove that a pest is entirely absent from a vast region, such as Australia. That level of proof would require flawless surveillance methods and a comprehensive, simultaneous census of all susceptible hosts or locations across the nation. Such perfect surveillance is often unachievable. Instead, a pest's absence is inferred based on a certain level of confidence, which is calculated on the assumption that we would have detected the pest if it were present. 

Surveillance serves a critical role in asserting proof of freedom and area freedom. For example, the Department uses information on pest freedom in negotiating market access that supports Australian exporting industries. Surveillance data is pivotal in providing solid proof that aids these negotiations.

Claims of area freedom from invasive species must be backed by compelling evidence to satisfy trade market guidelines and avoid costs associated with unmanaged pest incursions. This evidence usually includes analyzing surveillance data using statistical and computational techniques, which provide satisfactory confidence that the pest is not present.

Figure 1. An example of generalised workflow for proof of freedom.

The confidence in evidence that an area is free of a specific pest is determined by three factors. First, the estimated prevalence of the pest which surveillance efforts are targeted must be selected. For example, regulators might decide that the costs or eradicating a pest would still be low if 1% of the locations or hosts have pests. At 1% prevalence managers may know that it would still be easy to eradicate the pest if detected, so the prevalence rate is the maximum tolerable presence of a pest before it becomes a significant or costly threat. Second, an understanding is required of the effectiveness of our detection efforts or the surveillance sensitivity. In other words, surveillance sensitivity is the likelihood of detecting the pest if present. Surveillance sensitivity can be challenging to quantify especially as it may vary spatially, but that understanding is critical to high confidence in reported proof of freedom. Third, the extent and variation in survey effort is needed to get accurate estimates of proof of freedom confidence. As more work is put into finding the pest, the more likeli it is to find the pest. Finally, results are sensitive to all three of these factors, and over time we will look to help users develop these critical inputs on the platform.

We propose that when the future risk of damages is high, we should strive to be more confident about the pest's absence. In contrast, when the risk is lower, our level of confidence need not be as high. This principle is derived from a cost-benefit analysis where we balance the costs of increased surveillance with the potential damages from a pest incursion. Surveillance efforts will need to be greater when the risk of impacts from a pest or disease are high. Getting quantitatively strong evidence of proof of freedom can be achieved fairly easily on the platform, but the robustness of that result will require a robust understanding of pest prevalence, surveillance sensitivity and surveillance effort.

In conclusion, Proof of Freedom, or Area Freedom, in biosecurity is not about absolute certainty of a pest's absence, but rather about managing and reducing risks based on confidence levels derived from surveillance data. It requires a strategic approach that balances the sensitivity of detection, the maximum tolerable prevalence of the pest, and the surveillance effort, all while considering the potential economic impacts and the varying levels of future risks. It is a dynamic and ongoing process that calls for continuous surveillance and improvement in detection methods.

Table 1. Literature on approaches to Proof of Freedom.

Step 1 – Review available evidence, knowledge, data & surveillance outputs

Keep in mind it is easy to get a result by using the dashboard tools available at Biosecurity Commons, but a robust Proof of Freedom requires the best available data on surveillance, surveillance effort and a reasonable estimate of the pest prevalence you are trying to detect.  If those are used to generate a surveillance design where risks and costs are well considered, the resulting confidence in proof of freedom will also be robust. Considerations of risks of a pest's establishment and costs of not identifying a pest that is actually present are critical to achieve results that meet the required thresholds of evidence.

Step 2 – Provide the context of your Proof of Freedom workflow

Taking time to fill out the context allows users to capture the kind of proof of freedom workflow that was run as well as the original units used within the experiment’s data.  This can be helpful to record if a user is returning to a saved project or for users sharing a project. These setting do not alter the results. 

Surveillance type: “survey” Other types of surveillance that could be used include traps, samples, reports, mixed and other. 

Surveillance quantity unit: “hours” Other types of units which could be used to describe the quantity of surveillance to be allocated to each site (or grid cell) include units, traps, detectors, or samples. 

Cost unit: “$”  The descriptive unit to describe costs or benefit savings can be described as either hours or dollars “$”. 

Distance/area unit: “meters” The unit to describe spatial distance can be set to either meters or kilometers. 

Time unit: “years” The units used to describe time could also include: months, weeks, days or hours. 

Step 3 – Select the method of Proof of Freedom

Currently, two types of methods are available, “Hypothesis testing freedom design” or “Bayesian freedom design”.

Hypothesis Testing Freedom Design "Represents area freedom design functionality utilizing hypothesis testing approaches to assess the likelihood of an invasive species being present when it has not been detected for a sequence of time intervals or applications of a surveillance system." Expect a result where one column has the iteration number and the adjacent column shows the reducing p-value going from 1.0 to 0.00001 with zero representing perfect evidence that a pest would not be missed if it was present given that level of effort.

Bayesian Freedom Design "Represents area freedom design functionality utilizing Bayesian approaches to assess the likelihood of freedom, or an invasive species being absent when it has not been detected for a sequence of time intervals or applications of a surveillance system." Expect a resulting table where one column has the iteration number, and the adjacent column where each row shows the increasing confidence in detecting a pest with each iteration. Numbers will be growing from zero to 0.99 with 1.0 or 100% confidence.

Step 4 – Select the input parameters for the proof of freedom workflow

    Bayesian freedom design

Detection input

    Detection record: This can include a table of previous survey efforts where the first column is the survey number, and the second column 'detected' indicates a 0 when the pest was not detected, and a 1 when the pest was present. 

                Probability freedom: The prior probability of invasive species freedom or absence used in the first iteration of the Bayesian process. Values are typically estimated via expert elicitation. Default is 0.5 for an uninformed prior.

                Stopping condition: This dropdown can set the stopping condition as 'Number of iterations' or 'Target confidence'.   

Detection Probability

                Probability detect: The probability of detecting the invasive species given its presence. Also known as system sensitivity or detection confidence for a surveillance system. Default is NULL implying only detection records are available. Temporally changing values may be provided by a numeric vector, the length of which should be sufficient for the expected number of 'iterations', given the specified stopping criteria, else the last value of the vector is repeated.

                Probability persist: The probability that the invasive species persists at each time interval (specified by the 'time_unit' parameter in the 'context'). Default is 1 implies that the invasive species will persist across time intervals if present, representing the worst case scenario when persistence probability is unknown. Only utilized when 'pr_detect' is given. Temporally changing values may be provided by a numeric vector, the length of which should be sufficient for the expected number of 'iterations', given the specified stopping criteria, else the last value of the vector is repeated.

                Probability freedom: The prior probability of invasive species freedom or absence used in the first iteration of the Bayesian process. Values are typically estimated via expert elicitation. Default is 0.5 for an uninformed prior.

                Stopping condition: This dropdown can set the stopping condition as 'Number of iterations' or 'Target confidence'.   

    Hypothesis Testing Freedom Design

Detection input

    Detection record: This can include a table of previous survey efforts where the first column is the survey number, and the second column 'detected' indicates a 0 when the pest was not detected, and a 1 when the pest was present. 

                Stopping condition: This dropdown can set the stopping condition as 'Number of iterations' or 'Target confidence'.   

Detection Probability

                Probability detect: The probability of detecting the invasive species given its presence. Also known as system sensitivity or detection confidence for a surveillance system. Default is NULL implying only detection records are available. Temporally changing values may be provided by a numeric vector, the length of which should be sufficient for the expected number of 'iterations', given the specified stopping criteria, else the last value of the vector is repeated.

                Probability persist: The probability that the invasive species persists at each time interval (specified by the 'time_unit' parameter in the 'context'). Default is 1 implies that the invasive species will persist across time intervals if present, representing the worst case scenario when persistence probability is unknown. Only utilized when 'pr_detect' is given. Temporally changing values may be provided by a numeric vector, the length of which should be sufficient for the expected number of 'iterations', given the specified stopping criteria, else the last value of the vector is repeated.

                Stopping condition: This dropdown can set the stopping condition as 'Number of iterations' or 'Target confidence'.   

Step 5 – Review results & share or save project with new name

Results can be viewed at the top of the workflow tree, and if you want to share the project or save the project under a new name click on the 'Manage' drop down button, and either select 'Save Project As New' or 'Share Project'.  See here for more information on how to save or share data results and projects.

Literature Cited

Anderson, D. P., Ramsey, D. S. L., Nugent, G., Bosson, M., Livingstone, P., Martin, P. A. J., Sergeant, E., Gormley, A. M., & Warburton, B. (2013). A novel approach to assess the probability of disease eradication from a wild-animal reservoir host. Epidemiology and Infection, 141(7), 1509–1521. https://doi.org/10.1017/S095026881200310X 

Barclay, H. J., & Hargrove, J. W. (2005). Probability models to facilitate a declaration of pest-free status, with special reference to tsetse (Diptera: Glossinidae). BULLETIN OF ENTOMOLOGICAL RESEARCH -LONDON-, 95(1), 1–12. https://doi.org/10.1079/BER2004331 

Barclay, H. J., & Humble, L. (2009). Probability models to facilitate a declaration that an exotic insect species has not yet invaded an area. Biological Invasions, 11(6), 1267–1280. https://doi.org/10.1007/s10530-008-9331-1 

Barnes, B., Parsa, M., Giannini, F., & Ramsey, D. (2021). Analytical Bayesian models to quantify pest eradication success or species absence using zero-sighting records. Theoretical Population Biology, 144, 70–80. https://doi.org/10.1016/j.tpb.2021.10.001 

Martin, P. A. J., Cameron, A. R., & Greiner, M. (2007). Demonstrating freedom from disease using multiple complex data sources: 1: A new methodology based on scenario trees. Preventive Veterinary Medicine, 79(2), 71–97. https://doi.org/10.1016/j.prevetmed.2006.09.008 

Martin, T. (2017). Surveillance for Detection of Pests and Diseases: How Sure Can We Be of Their Absence? In A. Robinson, T. Walshe, M. Burgman, & M. Nunn (Eds.), Invasive Species: Risk Assessment and Management (pp. 348-384). Cambridge: Cambridge University Press. https://doi.org/10.1017/9781139019606.018 

Rout, T. M. (2009). Declaring Eradication of Invasive Species: A Review of Methods for Transparent Decision-Making. Plant Protection Quarterly, 24(3), 92–94. https://search.informit.org/doi/10.3316/INFORMIT.735300979129582 

Rout, T. M., Salomon, Y., & McCarthy, M. A. (2009a). Using Sighting Records to Declare Eradication of an Invasive Species. Journal of Applied Ecology, 46(1), 110–117. https://doi.org/10.1111/j.1365-2664.2008.01586.x 

Rout, T. M., Thompson, C. J., & McCarthy, M. A. (2009b). Robust Decisions for Declaring Eradication of Invasive Species. Journal of Applied Ecology, 46(4), 782–786. https://doi.org/10.1111/j.1365-2664.2009.01678.x 

Rout, T. M., Kirkwood, R., Sutherland, D. R., Murphy, S., & McCarthy, M. A. (2014). When to declare successful eradication of an invasive predator? Animal Conservation, 17(2), 132–125. https://doi.org/10.1111/acv.12065 

Solow, A. R. (1993). Inferring Extinction from Sighting Data. Ecology, 74(3), 962–964. https://doi.org/10.2307/1940821