eval_scripts package

Submodules

eval_scripts.complexity module

eval_scripts.complexity.all_in_one_test(testprocess, testmodules)
eval_scripts.complexity.all_in_one_train(trainprocess, trainmodules)
eval_scripts.complexity.getallparams(li)

eval_scripts.performance module

eval_scripts.performance.AUPRC(pts)
eval_scripts.performance.accuracy(truth, pred)
eval_scripts.performance.compute_metrics(truth, pred, names)

Compute a dict of named classification metrics for label tensors.

Parameters:

  • truth – ground truth class labels, shape (N,)

  • pred – predicted class labels, shape (N,)

  • names – iterable of metric names (case-insensitive). Supported: “UA”/”UAR” (unweighted accuracy, macro-averaged recall), “WA”/”Accuracy” (weighted accuracy, i.e. standard accuracy), “F1” (macro-averaged F1 score).

Returns:

dict mapping each requested name to its computed value.

eval_scripts.performance.eval_affect(truths, results, exclude_zero=True)
eval_scripts.performance.f1_score(truth, pred, average)
eval_scripts.performance.ptsort(tu)
eval_scripts.performance.unweighted_accuracy(truth, pred)

Unweighted accuracy: macro-averaged per-class recall.

Also commonly reported as UAR (Unweighted Average Recall) in the speech/affect emotion recognition literature - both names refer to the same computation (sklearn’s balanced accuracy).

eval_scripts.robustness module

eval_scripts.robustness.effective_robustness(robustness_result, task)

Compute the effective robustenss metric given the performance of the method on the task.

eval_scripts.robustness.effective_robustness_helper(robustness_result, task)

Helper function that computes the effective robustness metric as the performance difference compared to late fusion method.

Parameters:

  • robustness_result – Performance of the method on datasets applied with different level of noises.

  • task – Name of the task on which the method is evaluated.

eval_scripts.robustness.get_robustness_metric(robustness_result, task, metric)

Compute robustness metric given specific method performance and the task.

Parameters:

  • robustness_result – Performance of the method on datasets applied with different level of noises.

  • task – Name of the task on which the method is evaluated.

  • metric – Type of robustness metric to be computed. ( “effective” / “relative” )

eval_scripts.robustness.maxmin_normalize(result, task)

Normalize the metric for robustness comparison across all methods.

Parameters:

  • result – Un-normalized robustness metrics of all methods on the given task.

  • task – Name of the task.

eval_scripts.robustness.relative_robustness(robustness_result, task)

Compute the relative robustenss metric given the performance of the method on the task.

eval_scripts.robustness.relative_robustness_helper(robustness_result, task)

Helper function that computes the relative robustness metric as the area under the performance curve.

Parameters:

robustness_result – Performance of the method on datasets applied with different level of noises.

eval_scripts.robustness.single_plot(robustness_result, task, xlabel, ylabel, fig_name, method)

Produce performance vs. robustness plot of a single method.

Parameters:

  • robustness_result – Performance of the method on dataset applied with different level of noises.

  • task – Name of the task on which the method is evaluated.

  • xlabel – Label of x-axis to be appeared in the plot.

  • ylabel – Label of y-axis to be appeared in the plot.

  • fig_name – Name of plot to be saved.

  • method – Name of the method.

Module contents