Package List and Search

Background

This web site arose after a review of Australia’s stock assessment practices. From this study, many Australian assessments were defined as bespoke. Although there was often good reason for this, several assessments could have been undertaken using peer-reviewed, well tested and documented, freely available packages, with a large support base of users and developers. When assessment scientists were asked why bespoke packages were used extensively in Australia, a key issue was finding packages that would suit their use and whether training was available. This web site, we hope, will be a tool for all assessment scientists wanting to find a package that is appropriate for their needs. However, we do not intend for this web site to lead you to a single “best” assessment. This might be the role of other tools. We provide our recommendations regarding packages, but this remains our own opinions.

The project conducted an in-depth search of freely available packages. Initially, about 135 packages were found. Of these, about 63 were removed from the list. The main reasons for the removals were that the packages were either:

1. No longer supported or freely available,
2. Had since been replaced by a more modern approach by the developer (or others),
3. Not ultimately an assessment (as defined on the Purpose of this site page) or
4. Not a package – rather bespoke code and as such not supported by the developer for general use.

Most of the removed package fell into the fourth group. It is unfortunate that so many very good and generalizable methods were not developed further into packages for the stock assessment community.

Another observation during this project is that more data-rich packages are being (or have been) modified to be able to undertake data-limited and -moderate assessments. This is a very exciting development.

This web site is regularly reviewed and updated, including  the list of packages, their specifications and web links. New packages are added and some are deleted if inactive for a long period or no longer supported by the author(s). Presently there are about 77 assessment packages that appear in the table below.

We would like to acknowledge the initial inputs from CAPAM, especially drs Mark Maunder and Simon Hoyle who created the original list and information. Of course, the detailed information is usually provided by the project team with help from the package developer, whose inputs have been invaluable.

The full methods are described in Dichmont, et al. 2021. Collating stock assessment packages to improve stock assessments. Fisheries Research. Volume 236, April 2021, 105844

Package types

A first step was to classify the packages into different types. There are:

• Catch curve: catch curve analysis is a method for estimating the total mortality of a stock (Z) from the slope of the relative numbers present in each age or length class.
• Catch only: there is a large group of packages that use mainly or only catch data to assess the status of a stock. It should be noted that catch only methods include expansive assumptions such as an open access and unmanaged fishery. In most cases in the world, the assumptions behind catch only methods do not apply and can results in imprecise and biased estimates of stock status. Several papers have been written on these methods, including Free et al (2019 – https://doi.org/10.1016/j.fishres.2019.105452), Pons et al (2020 – https://doi.org/10.1139/cjfas-2019-0276) and Dai et al (2023 – https://doi.org/10.1016/j.fishres.2022.106520). These methods should therefore be used with caution.
• Delay difference: these assessment approaches are mid-way in complexity between surplus production models and full age-structured assessments. Delay difference models represent the population using two stages – recruits and spawners.
• Depletion model: few packages fell in this category. This assessment methods model in-season catch dynamics and are different enough to stand on their own.
• Integrated assessment: these are assessments that are based on age- or length- structured (or both) population dynamics models and integrates data from various disparate sources into a single framework for parameter estimation.
• Length-based: these are data-limited or -moderate approaches that use length data to estimate mortality and length-based reference points.
• Mean length: these data-limited assessments use a time series of mean lengths to estimate mortality and length-based reference points.
• Size-structured: are data-rich integrated assessments that are similar to age-based integrated assessments, but are aimed at hard to age species such as crustaceans and abalone that use at their basis size-frequency data. Only one package falls into this category despite these assessments being very common.
• Surplus production: surplus production models are one of the simplest approaches that nevertheless represent stock dynamics. Here the population is placed in a dynamic pool. It does not keep track of age or length structure. A new category of surplus production models has been created – age-structured production models – to allow for age and index data models.
• VPA: Virtual Population Analyses are a class of age-structured models that are based on the assumption that the catch-at-age data are known with negligible error. Many assessments previously conducted using VPA are now being conducted using integrated methods.

The following flow diagram illustrates the ten classes, with a few package examples:

Package specifications

In order to facilitate the use of this web site, we thought it would be useful to have reasonably detailed specifications for each package. These can be found by clicking the Details button for a specific package in the table below. We thank the developers for their input (either as part of a CAPAM census to which we added further information) or by the project team. This dataset has been checked by several developers and other stock assessment experts, but may still need input from anyone who notices discrepancies. The Contact us page is available for this purpose. Another option is to use this Survey where you can add the requested details in a more structured manner (our preferred way).

Package status

The final step was to Comment on the package and provide a Status remark. The categories are:

• Supported: A package that is maintained and is based on a statistically and mathematically appropriate analysis method (about 20 packages).
• Supported and Recommended: Packages classified within each class of assessment methods as ‘Supported and Recommended’ using the the qualitative criteria described in Table 3 of the above-mentioned paper (noting that not all criteria are applicable to all types of assessments) (~31 packages):
  • their support of multiple functional forms for biological processes (e.g., types of selectivity patterns for methods based on age- and size-structured models;
  • whether allowance can be made for process and observation error for surplus production models), and the breadth of the types of data that can be used for parameter estimation;
  • whether the packages / methods have been subjected to testing using simulation by analysts other than the package developers and are being applied in more than a single case;
  • whether documentation is available in the form of peer-reviewed publications;
  • whether a technical user manual and documentation are available;
  • the size of the current user base that identified potential bugs and provides suggestions for updates, and the responsiveness of the development team to these comments; and
  • whether the method applies state-of-the-art statistical estimation methods and can adequately quantify the uncertainty of model outputs. When multiple implementations of the same basic approach are available, we have recommended the version we have found easiest to use. Note that the methods are recommended within assessment types and are all hence not equally preferred. For example, we would also advocate the use of assessment methods that utilize all of the available data over methods that rely only on catch data.
• Inactive: The package has not been modified for several years (~8 packages).
• Not evaluated: Packages that are new and are not yet actively used or peer-reviewed (~3 packages). This status may change over time.
• Still under development: A new package that will replace an existing package, but the process is not yet completed (presently no packages).
• Superseded: The method is no longer being updated by the developer, but is still being used in some cases (~14 packages).

The below table is a summary of a much larger table containing extensive specification for each package we have included. They range from data-limited to data-rich. If you see any missing packages or any of the data is incorrect please Contact Us.

The complete list with specifications can be downloaded here: Packages spreadsheet

Table filters

Some basic instructions about the table:

1. You can sort any of the column headings.
2. You can filter various aspects of the table. The filter options allow you to click more than one option.
3. You can search for words in the table. This search is quite broad.
4. Package status may be useful to narrow the list of packages down, noting that the category “Supported and Recommended” are packages we recommend. This filter can be combined with the filer “Type” which will help to further reduce the final list of packages.
5. The full table of package information is downloadable at the bottom of the page.

There are 2 columns with live links:

1. The last column (Details) links to a detailed description and allied specifications of the chosen package.
2. The second column (Web link) links to the package.

December 2023