26.1 Descriptive Statistics

One principal goal of descriptive statistics is to represent the essence of a large data set concisely. Octave provides the mean, median, and mode functions which all summarize a data set with just a single number corresponding to the central tendency of the data.

 
: m = mean (x)
: m = mean (x, dim)
: m = mean (x, vecdim)
: m = mean (x, "all")
: m = mean (…, nanflag)
: m = mean (…, outtype)

Compute the mean of the elements of x.

If x is a vector, then mean (x) returns the mean of the elements in x defined as

mean (x) = SUM_i x(i) / N

where N is the number of elements in x.

If x is an array, then mean(x) computes the mean along the first non-singleton dimension of x.

The optional variable dim forces mean to operate over the specified dimension, which must be a positive integer-valued number. Specifying any singleton dimension in x, including any dimension exceeding ndims (x), will result in a mean equal to x.

Specifying the dimensions as vecdim, a vector of non-repeating dimensions, will return the mean over the array slice defined by vecdim. If vecdim indexes all dimensions of x, then it is equivalent to the option "all". Any dimension in vecdim greater than ndims (x) is ignored.

Specifying the dimension as "all" will force mean to operate on all elements of x, and is equivalent to mean (x(:)).

The optional input outtype specifies the data type that is returned. outtype can take the following values:

'default' : Output is of type double, unless the input is

single in which case the output is of type single.

'double' : Output is of type double.
'native' : Output is of the same type as the input as reported

by (class (x)), unless the input is logical in which case the output is of type double.

The optional variable nanflag specifies whether to include or exclude NaN values from the calculation using any of the previously specified input argument combinations. The default value for nanflag is "includenan" which keeps NaN values in the calculation. To exclude NaN values set the value of nanflag to "omitnan". The output will still contain NaN values if x consists of all NaN values in the operating dimension.

See also: median, mode, movmean.

 
: m = median (x)
: m = median (x, dim)
: m = median (x, vecdim)
: m = median (x, "all")
: m = median (…, nanflag)
: m = median (…, outtype)

Compute the median value of the elements of x.

When the elements of x are sorted, say s = sort (x), the median is defined as

             |  s(ceil (N/2))          N odd
median (x) = |
             | (s(N/2) + s(N/2+1))/2   N even

If x is an array, then median (x) operates along the first non-singleton dimension of x.

The optional variable dim forces median to operate over the specified dimension, which must be a positive integer-valued number. Specifying any singleton dimension in x, including any dimension exceeding ndims (x), will result in a median equal to x.

Specifying the dimensions as vecdim, a vector of non-repeating dimensions, will return the median over the array slice defined by vecdim. If vecdim indexes all dimensions of x, then it is equivalent to the option "all". Any dimension in vecdim greater than ndims (x) is ignored.

Specifying the dimension as "all" will force median to operate on all elements of x, and is equivalent to median (x(:)).

median (…, outtype) returns the median with a specified data type, using any of the input arguments in the previous syntaxes. outtype can take the following values:

"default"

Output is of type double, unless the input is single in which case the output is of type single.

"double"

Output is of type double.

"native".

Output is of the same type as the input (class (x)), unless the input is logical in which case the output is of type double.

The optional variable nanflag specifies whether to include or exclude NaN values from the calculation using any of the previously specified input argument combinations. The default value for nanflag is "includenan" which keeps NaN values in the calculation. To exclude NaN values set the value of nanflag to "omitnan". The output will still contain NaN values if x consists of all NaN values in the operating dimension.

See also: mean, mode, movmedian.

 
: m = mode (x)
: m = mode (x, dim)
: [m, f, c] = mode (…)

Compute the most frequently occurring value in a dataset (mode).

mode determines the frequency of values along the first non-singleton dimension and returns the value with the highest frequency. If two, or more, values have the same frequency mode returns the smallest.

If the optional argument dim is given, operate along this dimension.

The return variable f is the number of occurrences of the mode in the dataset.

The cell array c contains all of the elements with the maximum frequency.

See also: mean, median.

Using just one number, such as the mean, to represent an entire data set may not give an accurate picture of the data. One way to characterize the fit is to measure the dispersion of the data. Octave provides several functions for measuring dispersion.

 
: [s, l] = bounds (x)
: [s, l] = bounds (x, dim)
: [s, l] = bounds (…, "nanflag")

Return the smallest and largest values of the input data x.

If x is a vector, the bounds are calculated over the elements of x. If x is a matrix, the bounds are calculated for each column. For a multi-dimensional array, the bounds are calculated over the first non-singleton dimension.

If the optional argument dim is given, operate along this dimension.

The optional argument "nanflag" defaults to "omitnan" which does not include NaN values in the result. If the argument "includenan" is given, and there is a NaN present, then the result for both smallest (s) and largest (l) elements will be NaN.

The bounds are a quickly computed measure of the dispersion of a data set, but are less accurate than iqr if there are outlying data points.

See also: range, iqr, mad, std.

 
: y = range (x)
: y = range (x, dim)

Return the range, i.e., the difference between the maximum and the minimum of the input data.

If x is a vector, the range is calculated over the elements of x. If x is a matrix, the range is calculated over each column of x.

If the optional argument dim is given, operate along this dimension.

The range is a quickly computed measure of the dispersion of a data set, but is less accurate than iqr if there are outlying data points.

See also: bounds, iqr, mad, std.

 
: Z = iqr (x)
: Z = iqr (x, dim)
: Z = iqr (x, "ALL")

Return the interquartile range of x, defined as the distance between the 25th and 75th percentile values of x calculated using: quantile (x, [0.25 0.75])

If x is a vector, iqr (x) will operate on the data in x.

If x is a matrix, iqr (x) will operate independently on each column in x returning a row vector Z.

If x is a n-dimensional array, iqr (x) will operate independently on the first non-singleton dimension in x, returning an array Z the same shape as x with the non-singleton dimenion reduced to 1.

The optional variable dim can be used to force iqr to operate over the specified dimension. dim can either be a scalar dimension or a vector of non-repeating dimensions over which to operate. In either case dim must be positive integers. A vector dim concatenates all specified dimensions for independent operation by iqr.

Specifying dimension "ALL" will force iqr to operate on all elements of x, and is equivalent to iqr (x(:)). Similarly, specifying a vector dimension including all non-singleton dimensions of x is equivalent to iqr (x, "ALL").

If x is a scalar, or only singleton dimensions are specified for dim, the output will be zeros (size (x)).

As a measure of dispersion, the interquartile range is less affected by outliers than either range or std.

See also: bounds, mad, range, std, prctile, quantile.

 
: m = mad (x)
: m = mad (x, opt)
: m = mad (x, opt, dim)
: m = mad (x, opt, vecdim)
: m = mad (x, opt, "all")

Compute the mean or median absolute deviation (MAD) of the elements of x.

The mean absolute deviation is defined as

mad = mean (abs (x - mean (x)))

The median absolute deviation is defined as

mad = median (abs (x - median (x)))

If x is a vector, compute mad for each element in x. If x is an array the calculation is performed over the first non-singleton dimension.

mad excludes NaN values from calculation similar to using the omitnan option in var, mean, and median.

The optional argument opt determines whether mean or median absolute deviation is calculated. The default is 0 which corresponds to mean absolute deviation; a value of 1 corresponds to median absolute deviation. Passing an empty input [] defaults to mean absolute deviation (opt = 0).

The optional argument dim forces mad to operate along the specified dimension. Specifying any singleton dimension in x, including any dimension exceeding ndims (x), will result in an output of 0.

Specifying the dimension as vecdim, a vector of non-repeating dimensions, will return the mad over the array slice defined by vecdim. If vecdim indexes all dimensions of x, then it is equivalent to the option "all". Any dimension included in vecdim greater than ndims (x) is ignored.

Specifying the dimension as "all" will force mad to operate on all elements of x, and is equivalent to mad (x(:)).

As a measure of dispersion, mad is less affected by outliers than std.

See also: bounds, range, iqr, std, mean, median.

 
: y = meansq (x)
: y = meansq (x, dim)

Compute the mean square of the elements of the vector x.

The mean square is defined as

meansq (x) = 1/N SUM_i x(i)^2

where N is the length of the x vector.

If x is a matrix, return a row vector containing the mean square of each column.

If the optional argument dim is given, operate along this dimension.

See also: var, std, moment.

 
: s = std (x)
: s = std (x, w)
: s = std (x, w, dim)
: s = std (x, w, vecdim)
: s = std (x, w, "ALL")
: s = std (…, nanflag)
: [s, m] = std (…)

Compute the standard deviation of the elements of the vector x.

The standard deviation is defined as

std (x) = sqrt ((1 / (N-1)) * SUM_i ((x(i) - mean(x))^2))

where N is the number of elements of x.

If x is an array, compute the standard deviation along the first non-singleton dimensions of x.

The optional argument w determines the weighting scheme to use. Valid values are:

0 [default]:

Normalize with N-1 (population standard deviation). This provides the square root of the best unbiased estimator of the standard deviation.

1:

Normalize with N (sample standard deviation). This provides the square root of the second moment around the mean.

a vector:

Compute the weighted standard deviation with non-negative weights. The length of w must equal the size of x in the operating dimension. NaN values are permitted in w, will be multiplied with the associated values in x, and can be excluded by the nanflag option.

an array:

Similar to vector weights, but w must be the same size as x. If the operating dimension is supplied as vecdim or "all" and w is not a scalar, w must be an same-sized array.

Note: w must always be specified before specifying any of the following dimension options. To use the default value for w you may pass an empty input argument [].

The optional variable dim forces std to operate over the specified dimension, which must be a positive integer-valued number. Specifying any singleton dimension in x, including any dimension exceeding ndims (x), will result in a standard deviation of 0.

Specifying the dimensions as vecdim, a vector of non-repeating dimensions, will return the standard deviation calculated over the array slice defined by vecdim. If vecdim indexes all dimensions of x, then it is equivalent to the option "all". Any dimension in vecdim greater than ndims (x) is ignored.

Specifying the dimension as "all" will force std to operate on all elements of x, and is equivalent to std (x(:)).

The optional variable nanflag specifies whether to include or exclude NaN values from the calculation using any of the previously specified input argument combinations. The default value for nanflag is "includenan" which keeps NaN values in the calculation. To exclude NaN values set the value of nanflag to "omitnan". The output will still contain NaN values if x consists of all NaN values in the operating dimension.

The optional second output variable m contains the mean of the elements of x used to calculate the standard deviation. If v is the weighted standard deviation, then m is also the weighted mean.

See also: var, bounds, mad, range, iqr, mean, median.

In addition to knowing the size of a dispersion it is useful to know the shape of the data set. For example, are data points massed to the left or right of the mean? Octave provides several common measures to describe the shape of the data set. Octave can also calculate moments allowing arbitrary shape measures to be developed.

 
: v = var (x)
: v = var (x, w)
: v = var (x, w, dim)
: v = var (x, w, vecdim)
: v = var (x, w, "all")
: v = var (…, nanflag)
: [v, m] = var (…)

Compute the variance of the elements of the vector x.

The variance is defined as

var (x) = (1 / (N-1)) * SUM_i ((x(i) - mean(x))^2)

where N is the number of elements of x.

If x is an array, compute the variance along the first non-singleton dimensions of x.

The optional argument w determines the weighting scheme to use. Valid values are:

0 [default]:

Normalize with N-1 (population variance). This provides the square root of the best unbiased estimator of the variance.

1:

Normalize with N (sample variance). This provides the square root of the second moment around the mean.

a vector:

Compute the weighted variance with non-negative weights. The length of w must equal the size of x in the operating dimension. NaN values are permitted in w, will be multiplied with the associated values in x, and can be excluded by the nanflag option.

an array:

Similar to vector weights, but w must be the same size as x. If the operating dimension is supplied as vecdim or "all" and w is not a scalar, w must be an same-sized array.

Note: w must always be specified before specifying any of the following dimension options. To use the default value for w you may pass an empty input argument [].

The optional variable dim forces var to operate over the specified dimension, which must be a positive integer-valued number. Specifying any singleton dimension in x, including any dimension exceeding ndims (x), will result in a variance of 0.

Specifying the dimensions as vecdim, a vector of non-repeating dimensions, will return the variance calculated over the array slice defined by vecdim. If vecdim indexes all dimensions of x, then it is equivalent to the option "all". Any dimension in vecdim greater than ndims (x) is ignored.

Specifying the dimension as "all" will force var to operate on all elements of x, and is equivalent to var (x(:)).

The optional variable nanflag specifies whether to include or exclude NaN values from the calculation using any of the previously specified input argument combinations. The default value for nanflag is "includenan" which keeps NaN values in the calculation. To exclude NaN values set the value of nanflag to "omitnan". The output will still contain NaN values if x consists of all NaN values in the operating dimension.

The optional second output variable m contains the mean of the elements of x used to calculate the variance. If v is the weighted variance, then m is also the weighted mean.

See also: std, mean, cov, skewness, kurtosis, moment.

 
: y = skewness (x)
: y = skewness (x, flag)
: y = skewness (x, flag, dim)

Compute the sample skewness of the elements of x.

The sample skewness is defined as

               mean ((x - mean (x)).^3)
skewness (X) = ------------------------.
                      std (x).^3

The optional argument flag controls which normalization is used. If flag is equal to 1 (default value, used when flag is omitted or empty), return the sample skewness as defined above. If flag is equal to 0, return the adjusted skewness coefficient instead:

                  sqrt (N*(N-1))   mean ((x - mean (x)).^3)
skewness (X, 0) = -------------- * ------------------------.
                      (N - 2)             std (x).^3

where N is the length of the x vector.

The adjusted skewness coefficient is obtained by replacing the sample second and third central moments by their bias-corrected versions.

If x is a matrix, or more generally a multi-dimensional array, return the skewness along the first non-singleton dimension. If the optional dim argument is given, operate along this dimension.

See also: var, kurtosis, moment.

 
: y = kurtosis (x)
: y = kurtosis (x, flag)
: y = kurtosis (x, flag, dim)

Compute the sample kurtosis of the elements of x.

The sample kurtosis is defined as

     mean ((x - mean (x)).^4)
k1 = ------------------------
            std (x).^4

The optional argument flag controls which normalization is used. If flag is equal to 1 (default value, used when flag is omitted or empty), return the sample kurtosis as defined above. If flag is equal to 0, return the "bias-corrected" kurtosis coefficient instead:

              N - 1
k0 = 3 + -------------- * ((N + 1) * k1 - 3 * (N - 1))
         (N - 2)(N - 3)

where N is the length of the x vector.

The bias-corrected kurtosis coefficient is obtained by replacing the sample second and fourth central moments by their unbiased versions. It is an unbiased estimate of the population kurtosis for normal populations.

If x is a matrix, or more generally a multi-dimensional array, return the kurtosis along the first non-singleton dimension. If the optional dim argument is given, operate along this dimension.

See also: var, skewness, moment.

 
: m = moment (x, p)
: m = moment (x, p, type)
: m = moment (x, p, dim)
: m = moment (x, p, type, dim)
: m = moment (x, p, dim, type)

Compute the p-th central moment of the vector x.

The p-th central moment of x is defined as:

1/N SUM_i (x(i) - mean(x))^p

where N is the length of the x vector.

If x is a matrix, return the row vector containing the p-th central moment of each column.

If the optional argument dim is given, operate along this dimension.

The optional string type specifies the type of moment to be computed. Valid options are:

"c"

Central Moment (default).

"a"
"ac"

Absolute Central Moment. The moment about the mean ignoring sign defined as

1/N SUM_i (abs (x(i) - mean(x)))^p
"r"

Raw Moment. The moment about zero defined as

moment (x) = 1/N SUM_i x(i)^p
"ar"

Absolute Raw Moment. The moment about zero ignoring sign defined as

1/N SUM_i ( abs (x(i)) )^p

If both type and dim are given they may appear in any order.

See also: var, skewness, kurtosis.

 
: q = quantile (x)
: q = quantile (x, p)
: q = quantile (x, p, dim)
: q = quantile (x, p, dim, method)

For a sample, x, calculate the quantiles, q, corresponding to the cumulative probability values in p. All non-numeric values (NaNs) of x are ignored.

If x is a matrix, compute the quantiles for each column and return them in a matrix, such that the i-th row of q contains the p(i)th quantiles of each column of x.

If p is unspecified, return the quantiles for [0.00 0.25 0.50 0.75 1.00]. The optional argument dim determines the dimension along which the quantiles are calculated. If dim is omitted it defaults to the first non-singleton dimension.

The methods available to calculate sample quantiles are the nine methods used by R (https://www.r-project.org/). The default value is method = 5.

Discontinuous sample quantile methods 1, 2, and 3

  1. Method 1: Inverse of empirical distribution function.
  2. Method 2: Similar to method 1 but with averaging at discontinuities.
  3. Method 3: SAS definition: nearest even order statistic.

Continuous sample quantile methods 4 through 9, where p(k) is the linear interpolation function respecting each method’s representative cdf.

  1. Method 4: p(k) = k / N. That is, linear interpolation of the empirical cdf, where N is the length of P.
  2. Method 5: p(k) = (k - 0.5) / N. That is, a piecewise linear function where the knots are the values midway through the steps of the empirical cdf.
  3. Method 6: p(k) = k / (N + 1).
  4. Method 7: p(k) = (k - 1) / (N - 1).
  5. Method 8: p(k) = (k - 1/3) / (N + 1/3). The resulting quantile estimates are approximately median-unbiased regardless of the distribution of x.
  6. Method 9: p(k) = (k - 3/8) / (N + 1/4). The resulting quantile estimates are approximately unbiased for the expected order statistics if x is normally distributed.

Hyndman and Fan (1996) recommend method 8. Maxima, S, and R (versions prior to 2.0.0) use 7 as their default. Minitab and SPSS use method 6. MATLAB uses method 5.

References:

  • Becker, R. A., Chambers, J. M. and Wilks, A. R. (1988) The New S Language. Wadsworth & Brooks/Cole.
  • Hyndman, R. J. and Fan, Y. (1996) Sample quantiles in statistical packages, American Statistician, 50, 361–365.
  • R: A Language and Environment for Statistical Computing; https://cran.r-project.org/doc/manuals/fullrefman.pdf.

Examples:

x = randi (1000, [10, 1]);  # Create empirical data in range 1-1000
q = quantile (x, [0, 1]);   # Return minimum, maximum of distribution
q = quantile (x, [0.25 0.5 0.75]); # Return quartiles of distribution

See also: prctile.

 
: q = prctile (x)
: q = prctile (x, p)
: q = prctile (x, p, dim)

For a sample x, compute the quantiles, q, corresponding to the cumulative probability values, p, in percent.

If x is a matrix, compute the percentiles for each column and return them in a matrix, such that the i-th row of q contains the p(i)th percentiles of each column of x.

If p is unspecified, return the quantiles for [0 25 50 75 100].

The optional argument dim determines the dimension along which the percentiles are calculated. If dim is omitted it defaults to the first non-singleton dimension.

Programming Note: All non-numeric values (NaNs) of x are ignored.

See also: quantile.

A summary view of a data set can be generated quickly with the statistics function.

 
: stats = statistics (x)
: stats = statistics (x, dim)

Return a vector with the minimum, first quartile, median, third quartile, maximum, mean, standard deviation, skewness, and kurtosis of the elements of the vector x.

If x is a matrix, calculate statistics over the first non-singleton dimension.

If the optional argument dim is given, operate along this dimension.

See also: min, max, median, mean, std, skewness, kurtosis.