4.1 Methods for Computing p-values

The numerical computation of p-values is done by the Fitter class via the methods

     double calc_nested_lrt_pvalue(Model& fullmodel,
                                   Model& constrainedmodel)
     double calc_lrt_pvalue(Model& model1, Model& model2,
                            int argc=0, char** argv=0)

Both methods return the computed p-value. More information about the last p-value computation can be obtained with the methods

     double Fitter::pvalue()
     double Fitter::pvalue_error()
     bool Fitter::reached_precision_goal()
     double Fitter::failed_shot_ratio()

The pvalue method simply returns the result of the last p-value computation. Since the p-value is calculated by numerical Monte Carlo integration it has a statistical error, which is returned by the pvalue_error method. The method reached_precision_goal returns true if the desired precision goal (as set by dvegas_precision, see Options for p-value Computations) has been reached in the last p-value integration and false otherwise. Each evaluation of the integrand requires two minimisations, and a small fraction of these minimisations will usually fail. The fraction of “failed shots” in the last p-value computation is returned by the failed_shot_ratio method. Note that, if you use parallelised integration (see Parallel Computation) the counting of failed shots does not work and the number returned by failed_shot_ratio is usually too small.

The calc_nested_lrt_pvalue method is used for comparing nested models, and constrainedmodel must be a constrained version of fullmodel. If you defined copy constructors and virtual assignment operators for your model classes (see Writing your own Model Class) you will probably want to construct constrainedmodel like this:

     MyModel constrainedmodel = fullmodel;
     constrainedmodel.fix(MyModel::P_MYSECONDPAR);
     constrainedmodel.fix(MyModel::P_MYLASTPAR);
     ...

The calc_nested_lrt_pvalue method checks if the two models have the same number of parameters (as returned by Model::nparameters()) and if all the parameters fixed in fullmodel are also fixed in constrainedmodel, but otherwise it is your responsibility to ensure that the two models are indeed nested. The calc_nested_lrt_pvalue method the performs a likelihood ratio test, using the model constrainedmodel with its current parameters as the null hypothesis. The difference

     constrainedmodel.chisquare() - fullmodel.chisquare()

is taken as the \Delta\chi^2 value obtained from the measured data. So, usually, you will want to fit fullmodel and constrainedmodel to your data (see Fitting a Model) before passing them to calc_nested_lrt_pvalue.

The method

     double calc_lrt_pvalue(Model& model1, Model& model2,
                            int argc=0, char** argv=0)

is used for comparing unrelated models. Here, the only limitation is that both models must provide the same number of observables. It is your responsibility to ensure that the integer values used to identify the observables are the same in both models. The order of the first two arguments is not important in this case: the model with the bigger chi-square value (as returned by Model::chisquare()) and its current parameter values is used as null hypothesis. As before, the difference of the two chisquare() members is assumed to be the \Delta\chi^2 value obtained from the measured data. The last two arguments of calc_lrt_pvalue are for parallelising the p-value computation. They are discussed in Parallel Computation. If they are omitted, no parallelisation is used.