Abstract

SISSI (Shortcut In Sample Size Identification) is an operational tool designed to estimate the optimal sample size in experimental data collection. It implements a jackknife visual evaluation system (Confalonieri et al., 2004, 2006), based on intensive use of the data sample by systematically taking sub-samples (virtual samples) of the original data and calculating the mean for each of sub-samples. This approach does not get across the typical limitations of conventional methods, because for its application to be reliable the data are not required to meet any statistical assumption. Visual, easy-to-interpret provisions are supplied to display the variation of means and standard deviations as size of virtual samples increases. An automatic option for identification of optimal sample size is given, targeting at the size for which it becomes negligible the rate of change of means. A manual option can be applied alternatively. A conventional approach to sample size identification is also given, based on Student-t distribution. The software is developed in Visual Basic and runs under Windows operating systems.


Sample statistics

Mean, standard deviation and number of data of the input file are computed.


Jackknife statistics

Jackknife-based estimates of standard deviation and standard error are computed as follows:

1. An input data set of N observed elements is used.

2. Leaving out each of the observations in turn, for each reduced body of data (sub-sample) the statistics of interest (mean and standard deviations) are computed. These statistics are called q-j, j=1. These statistics are the result of leaving out the jth subset.

3. Pseudo-values qj* are defined by the formula:

4. Mean and standard error of the N pseudo-values are computed using the usual formulas. The mean provides the jackknife estimate of the statistic qj*; the standard error is an estimate of the population standard error of the statistic. Jackknife-estimated mean is equal to the sample mean (see Sample statistics).


Resampling

The original sample (data in input) of N elements is divided into groups of k elements. virtual samples (combinations without repetitions) of N-k elements are generated by eliminating [image] times k different values from the original sample. The generation of virtual samples is repeated N-1 times with k assuming values from 1 to N-2, for a total of [image] virtual samples. Mean and standard deviation are computed for all the generated samples and displayed on graph.

When a large data set is used in input, long computation time may be an important drawback. To prevent this, the sample sizes evaluated are progressively reduced. Each of possible sample sizes from 2 to N-1 are evaluated for input samples up to 20 units. A step of 2 is chosen for computation when input file includes 21-40 data; a step of 3 is chosen for computation when input file includes 41-60 data, and so on. Missing values are covered by a linear connection between two consecutive sample sizes.


Data trimming

Data trimming is preliminary to both automatic and manual sample size determination. This is meant to discard extreme means (the highest and the lowest ones) from further computations, after the generation of virtual samples has been performed. Exclusion of 0-10% of extreme mean values is allowed.


Automatic sample size determination

Analyzing the trends of the means obtained from resampling-based generation for increasing values of N-k, the optimum sample size is considered the N-k value for which the variability between the means do not really decrease anymore with increasing sample size. The automatic procedure consists of selecting (N-k)’ out of those N-k higher than 2 and lower than N-2. Four weighted linear regressions are performed over the means as follows: (i) the first runs over the highest values of the N-k≤(N-k)’; (ii) likewise the first, the second runs over the lowest values; (iii) the third runs the highest values of the N-k>(N-k)’; (iv) likewise the third, the fourth runs the lowest values. A global parameter (SR2) is computed by summing the coefficient of determination (R2) of the four regressions. The reiteration of the steps from i to iv for all the possible (N-k)’ allows the identification of the optimal sample size, that is (N-k)’ where SR2 is the highest. A trimming process allows to leaving extreme samples (e.g. external to 95% confidence interval) out of computation. The process is reiterated until the next sample size does not produce SR2 larger than 5% than the previous.


Manual sample size determination

The user is allowed to adjust manually the resampling-estimated sample size via automatic procedure. This is meant to further reduce sample size if the variability achieved (expressed as % coefficient of variation) is expected to be low enough to fall within what is considered by the researcher to be acceptable. A trimming process can be preliminarily applied to the generated means.


Student-t sample size determination

Determination of sample size (n) from a conventional approach based on two-tail, one-sample Student-t distribution is given according to the following:

where tn-1, a is the Student-t value for n-1 degrees of freedom and a significance level, D is the maximum acceptable error (difference between sample mean and population mean), s is an estimate of population standard deviation (standard deviation from the input sampling units is given by default). An iteration process is performed starting with degrees of freedom equal to n=N-1 (N being the size of the evaluation sample) and terminating until n does not change anymore.

The user is requested to input the system with the maximum acceptable error and an estimate of the variance that is known to be associated to the process under study. Student-t determination is provided as a comparison term for the automatic estimate, but its validity is subject to the occurrence of normal data distribution.


Coefficients of variation

Once the optimum sample size is identified via automatic or manual procedure, then minimum and maximum coefficients of variation (CV, % standard deviations over the mean) are computed. They are computed from the lowest and the highest standard deviations correspondingly to the sample size determined. Both lowest and highest standard deviations are divided by the grand mean (mean of the means of virtual samples at the sample size determined).


Inputs

Input files are Excel spreadsheets (.xls). The format required is a single column (column A) of numbers with no header. Numbers represent sampling units to be used for sample size determination, also identified as pre-sampling units. Input data are displayed
on a grid in the work area. The grid can be manipulated by be entering new data or changing existing data, and saved into an .xls file.


Outputs

The outputs are displayed on the screen, in the relevant boxes of the work area (frame "Results"). An output file of the procedure's settings (including the input data) can be created by the Exporting options. A text editor can be used to open the generated file. Configuration and results are listed as in the exemplary file below:

************************************************************************************************************************************************************

Date: 5/26/2006

Time: 12:20:19 PM

Name of the opened file: E:\Documenti\Privata\Statistics\Confalonieri\SISSI\SISSI\Example 1.xls

************************************************************************************************************************************************************

SAMPLING UNITS:

9.70800

11.85200

12.80200

12.28600

9.33300

11.47500

12.25100

10.48000

9.31500

12.54100

12.87300

11.19200

SAMPLE STATISTICS

mean: 11.342

standard deviation: 1.331

number: 12

JACKKNIFE STATISTICS

standard deviation: 1.345

standard error: .384

CONFIGURATION

Virtual samples number: 250

Trimming: 5 %

************************************************************************************************************************************************************

RESULTS:

************************************************************************************************************************************************************

Automatic sample size: 8

Means chart

Up-left line: y = 12.358 -.095 * x

Down-left line: y = 10.198 +.112 * x

Up-right line: y = 11.691

Down-left line: y = 10.983

Average mean: 11.3252

Maximum mean: 11.6801

Minimum mean: 10.9534

Average standard deviation: 1.3276

Maximum standard deviation: 1.5017

Minimum standard deviation: 1.1202

************************************************************************************************************************************************************

Manual sample size: 4

Average mean: 11.3425

Maximum mean: 12.0154

Minimum mean: 10.6386

Average standard deviation: 1.3251

Maximum standard deviation: 1.6818

Minimum standard deviation: .6917

************************************************************************************************************************************************************

************************************************************************************************************************************************************

Student-t sample size determination

Alpha: 0.05

Maximum accepted error: 2

Student-t sample size: 5

Average mean: 11.3183

Maximum mean: 11.8879

Minimum mean: 10.7737

Average standard deviation: 1.3204

Maximum standard deviation: 1.5699

Minimum standard deviation: .913

************************************************************************************************************************************************************


Saving

The input sampling unit can be modified after input data are loaded onto the grid, and saved as Excel file by File/Save/Pre-sampling results, or clicking on the relative icon (see Toolbar). Once the grid is modified the file must be saved to enable all computations required.


Exporting

A text file (.dat) reporting the procedure's settings and results can be created by Export/Configuration and Results, or clicking on the relative icon (see Toolbar). See Outputs for an example of exported file.


Work area

The work area is made up of a single window, including separate frames, a menu bar, and a toolbar.

To use the menu bar: choose the appropriate menu and select an option from the drop-down list.
To use the toolbar: click on the button on the toolbar that corresponds with the option that you want to use.
To run the functions click on the relevant buttons or move the cursor on the bar of manual sample size determination.


Frames

The work area is structured into 11 frames:

  • Sampling units

  • Sample statistics

  • Jackknife estimates

  • Virtual samples

  • Visual jackknife

  • Trimming

  • Automatic sample size determination

  • Manual sample size determination

  • Coefficients of variation

  • Student-t sample size determination

  • Results

Sampling units
This frame consists of a grid including the data from the input file. Once an input file is open the data are displayed on a grid, named "Sampling units". It contains the sample of input data, which will be resampled according to jackknife procedure. Data in the grid can be changed by the user. Changed data must be saved (Saving) if computations are required on them. The grid is cleared clicking on the icon "New" (see Toolbar).

Sample statistics
Mean, standard deviation and number of sample data are computed (see Sample statistics) and reported in the relevant boxes.

Jackknife estimates
Standard deviation, standard error associated to the mean, and standard error associated to the standard deviation are computed (see Jackknife statistics) and reported in the relevant boxes.

Virtual samples
After loading the grid, the maximum number of virtual samples possibly generated from the input data is displayed. This is the maximum number of combinations generated from N sampling units divided into groups of k elements; for instance, the maximum number is 924 with N=12, and it corresponds to the number of possible combinations with k=6. The maximum number may be considerably high. So, a reduced number is recommended to be inputted to prevent excessive computational requirements without reducing the reliability of results. A number in the range 200-1000 may be suitable, as deduced from literature (Efron and Tibshirani, 1993; Manly, 1997): 250 is given by default. Clicking on the button "Generate the virtual samples" the generation process will start off.

If a number of virtual examples exceeding the maximum number is inputted, a warning message displays:

Redundant samples do not contribute to computations, unless a small influence on the coefficients of variation.

A similar display appears any time the number of virtual examples exceeds 250, warning the user about an increasingly time-consuming process:

Visual jackknife
This frame displays the results of jackknife-based resampling in the form of two graphs. The top graph reports the variability of standard deviations against sample sizes. The bottom graph reports the variability of means against sample sizes. Vertical bars will be displayed on the graphs to indicate the optimum sample size estimated via alternative approaches (automatic, manual, conventional). This frame also includes the frame for manual sample size determination.

Settings for the charts can be customized double-clicking on them and using the options made available by a chart designer.

Trimming
The user may want to discard a certain percentage (in the range 0-10%) of the lowest and the highest values of the generated means, prior to computing the optimum sample size via either automatic or manual procedure: 5% is given by default. It means 2.5% of the highest and 2.5% of the lowest means are discarded. The user may want not to discard any data by setting to 0 the trimming option. Clicking on the button "Trim the generated samples" the discarding process will be carried out.

Automatic sample size determination
Optimum sample size via automatic procedure is determined once the trimming option has been run, clicking on the button "Automatic".

The manual option is always suggested and the following message appears:

If the input file is too small, the automatic option easily fails in identifying the optimum sample size (error message below), and the manual option becomes necessary.

Manual sample size determination
A slider is given to set manually sample sizes. This is especially useful in order to adjust sample size after its determination via automatic option. The slider can be moved either backward ('liberal' solution: lessening of automatically determined sample size) or forward ('conservative' solution: augmentation of automatically determined sample size).

Coefficients of variation
Once the optimum sample size is identified, then minimum and maximum coefficients of variation (CV, % standard deviations over the mean) are displayed. Lowest and highest coefficients of variation are given in the azure boxes. The same values in the pink boxes refer to the manual sample size determination. As the user moves backward or forward the bar for manual sample size determination, CV values in the pink boxes change accordingly.

Student-t sample size determination
This option relies upon the normal distribution of data. The user may want to compare the result from this conventional approach to the one from the automatic or the manual option. Three inputs are required: standard deviation of data, type-I error probability level (Alpha) and maximum accepted error (maximum accepted difference between sample and population means). Sample standard deviation is given by default. The user may want to change it if he/she has some knowledge about it. The maximum accepted error is user-defined. Probability level is selected out of three options (0.10, 0.05, 0.01) in a drop-down list (0.05 is the default probability level). Estimation of sample size is performed by clicking on the button "Student-t".

Small values of the maximum accepted error may produce an estimate of sample size exceeding the input sample size. An error message is displayed in such cases:

Results
Results are the estimated sample sizes. Three outcomes may be given and reported in the relevant boxes, i.e. sample sizes from automatic option, manual option, or Student-t based option.


Menu bar

The menu bar allows the user to select menu items to accomplish a variety of tasks.

File From the File menu, the user can open input files (Open), create new input files (New), save input files (Save/Pre-sampling results), and exit the program (Exit). Keyboard buttons can be alternatively used for quick activation of some of the menu items: Ctrl+O (Open), Ctrl+N (New).

Export From the Export menu (Configuration and Results) the user can generate a text file (.dat) reporting configuration settings and a summary of results.

? From the ? menu, the user can open the help file (Contents) and the about form of SISSI (About SISSI). The F1 button can be alternatively used for quick opening of the help file.


Toolbar

The toolbar contains a set of icons, which the user can click on to perform a range of tasks. Short tips texts appear when the cursor is left a few seconds on each icon.

open an input file

save an input file

create an input file (it clears the open grid)

export a configuration & results file

open the help file


Diagram

A synthetic diagram of the software SISSI is given by the figure below.


Getting started

An exemplary working session is described in the following paragraphs:

  • Open SISSI

  • Open input file

  • Generate virtual samples

  • Determine sample size

  • Export configuration and results


Open SISSI

  • Double click on the file SISSI 1.00.exe in the installation directory (..\program files\SISSI\), or on the icon of the software at your desktop and in the start menu. The splash screen appears.

  • Click on the splash screen to access the work area.


Open input file

After Open SISSI:

  • Click Open by File in the menu bar, or click on the relevant icon in the toolbar. A file requestor appears.

  • Select the sample file Example 1.xls and click on OK. The grid is loaded of a sample of 12 data, and both sample statistics and jackknife statistics are computed and reported in the relevant boxes. Sample statistics are: mean equal to 11.342, standard deviation equal to 1.331, and number of data equal to 12. Jackknife statistics: are standard deviation 1.345, standard error associated to the mean equal to 0.384, standard error associated to the standard deviation equal to 0.19.


Generate virtual samples

After Open input file:

  • A number of 9.24E+02 virtual samples displays in the frame Virtual samples, but 250 is the recommended number.

    Click on the button "Generate the virtual samples" (frame Virtual samples) to generate 250 virtual samples. The process may required a few seconds. A screen like the following appears while processing:

    The user may want to stop the process and return to the work area clicking on the button Abort.

  • Start time and end times of generation are given at the bottom of the work area. Two graphs are displayed in frame Visual jackknife as the process terminates. The top graph reports the distribution of standard deviations for each sample size. The bottom graph reports the distribution of means for each sample size.


Determine sample size

After Generate virtual samples:

  • Click on button "Trim the generated samples" (frame Trimming) to discard 5% of the generated means (see Data trimming).

  • Click on the button "Automatic" (frame Automatic sample size determination) and then OK to estimate the optimum sample size from visual jackknife procedure (see Automatic sample size determination). Look at the vertical bar on the two graphs of frame Visual jackknife. Sample size equal to 8 is displayed on the graphs. Average mean (11.3252) and standard deviation (1.3276) for the estimated sample size are also displayed. You may obtain a different sample size because 250 virtual samples have been randomly selected out of 9.24E+02 possible virtual samples (see Generate virtual samples).

    The estimated sample size and given in the box Visual jackknife (auto) of frame Results.

    Look at CV values (frame Coefficients of variation). They range between 9.89 and 13.26. You may obtain different values as a consequence of the reduced number of virtual samples generated.

  • Move the bar placed between the two graphs (frame Manual sample size determination). A new vertical bar is displayed on the two graphs, giving a new sample size. If, for instance, the bar is placed at sample size equal to 4, the new average values of mean (11.3425) and standard deviation (1.3251) for the estimated sample size are displayed.

    The sample size determined via manual option is displayed in the box Visual jackknife (manual) of frame Results.

    Coefficients of variation     (CV) also change as the new sample size is given. In this case, CV range becomes wider: from 6.12 to 14.89. You may obtain different values as a consequence of the reduced number of virtual samples generated.

  • Enter 2 in the box "Max. accepted error" of frame Student-t sample size determination, leave standard deviation equal to 1.33 and Alpha=0.05, and click on the button "Student-t". A third vertical bar is displayed on the two graphs, giving an estimate of sample size equal to 5. Screenshot below reports sample sizes from the three option: automatic (blu bar), manual (red bar), Student-t based (green bar).

    This estimated sample size value is also reported in the box Student-t based of frame Results.

  • Enter 0.5 in the box "Max. accepted error" and repeat the above procedure. A message error is displayed because estimated sample size is larger than the size of input sample:


Export configuration and results

After Determine sample size:

  • Click on Export/Configuration and Results by the menu bar, or click on the relative icon in the toolbar. A file saver displays. Write down a filename and click on Save to save the text file in the directory chosen (see Saving).

  • Open the file saved by a text editor. For the most part, the file reports the same inputs and outputs as displayed in the work area. Other reports are regression parameters estimated for identification of optimum sample size via automatic determination. For instance, Up-left line: y = 12.358 -.095 * x refers to regression line interpolating the highest means for sample sizes less than the optimum. Equation of Down-left line is the equivalent for the lowest means. The other two lines (Up-right line, Down-left line) are horizontal interpolations, starting at the optimum sample size. Maximum and minimum values of mean and standard deviation for each sample size estimated are also given.


About

The current version of SISSI was developed by:

European Commission Directorate General Joint Research Centre
Institute for the Protection and Security of the Citizen - Agriculture and Fisheries Unit
Institute for Health and Consumer Protection - Biotechnology and GMOs Unit
21020 Ispra (VA), Italy

University of Milan
Department of Crop Science
20133 Milan, Italy

Development Roberto Confalonieri, Marco Acutis, Gianni Bellocchi, Giampiero Genovese

Programming Roberto Confalonieri

This software is made available free of charge to non-profit users. Comments about SISSI can be sent to roberto.confalonieri@jrc.it.

DISCLAIMER Joint Research Centre and University of Milan cannot be held responsible for any inconvenient resulting from the installation or the use of this program.


References

Confalonieri, R. 2004. A jackknife-derived visual approach for sample size determination. Riv. Ital. Agrometeorol. 1:9-13.

Confalonieri, R., D. Stroppiana, M. Boschetti, D. Gusberti, S. Bocchi and M. Acutis. 2006. Analysis of rice sample size variability due to development stage, nitrogen fertilization, sowing technique and variety using the visual jackknife. Field Crops Res. 97:135-141.

Efron, B., and R.J. Tibshirani. 1993. An introduction to bootstrap. Chapman and Hall/CRC, New York, USA.

Manly, B.F.J. 1997. Randomization and Monte Carlo methods in biology. Chapman and Hall/CRC, New York, USA.


Acknowledgement

A long list of people concerned in field and laboratory researches has been involved with testing the software under a variety of alternative conditions.


Installation

A setup file (SISSI setup.exe) allows installing SISSI. The icon of the software will be shown on the desktop:

All the files will be saved in the default directory ..\program files\SISSI\. A sub-directory ..\sample files\ is also installed, containing three exemplary *.xls input files (Example 1.xls, Example 2.xls, Example 3.xls). Example 1.xls contains 12 sampling units, Example 2.xls contains 100 sampling units, Example 3.xls contains 7 sampling units. The first and the second file are exemplary of a small and a big sample, respectively. The third file is exemplary of a very small sample, where the automatic option easily fails in identifying the optimum sample size.