Previous topic

testcollection module

Next topic

sound module

This Page

evaluate module

class bregman.evaluate.Evaluator(adb, ground_truth, seq_length=None, query_duration=None, tempo=None, tempo_range=None)
A class for evaluation of an audioDB (adb) database instance
Evaluator initialization:
 required parameters:
    adb - the full path of an audioDB database instance
    ground_truth - a list of database indices for a class of query
 optional parameters:
    seq_length OR query_duration - length of queries w.r.t. adb time stamps
    tempo OR tempo_range - tempo/tempi over which to search
evaluate(seq_length=None, query_duration=None, tempo=1.0, gt_only=True)
Evaluate loop over ground truth: query_duration varies with respect to tempo:
  query_duration - fractional seconds (requires adb.delta_time)
   OR seq_length - integer length of query sequence
  gt_only = if True, return only ground-truth results otherwise return full database results
evaluate_tempo_range(seq_length=None, query_duration=None, tempo_range=[, 1.0], gt_only=True)
Loop over tempo range evaluating and integrating results:
 query_duration = fractional seconds (requires adb.delta_time)
  OR seq_length - integer length of query sequence
 tempo_range = list of tempo proportions relative to nominal 1.0
 gt_only = if True, return only ground-truth results otherwise return full database results        
find_gt_ranks(out_ranks, ground_truth_keys=None)
Return ranks matrix for ground-truth columns only
get_adb_lists()
return two lists of database keys, lengths
get_gt_lists()
return two lists of ground truth keys, lengths
Initializes the evaluation loop search parameters
 build sequence length lower-bound list of included GT items >= seq_length
 build sequence length upper-bound list of excluded database items < seq_length
 set adb.configQuery parameters based on seq_length, tempo, and ground_truth
returns gt_lower_bound_list, gt_orig_list
lower_bound_list_by_length(lst, length, tempo=1.0)
truncate (key, length) tuples by lower bound len
return two lists incl_keys, incl_lens
reduce_evaluation_to_gt(out_ranks, out_dists, seq_length=None, query_duration=None)
Pick out only ground truth from rank results:
 query_duration - fractional seconds (requires adb.delta_time)
  OR seq_length - integer length of query sequence
report()
Print the mean rank between query runs (at each tempo) and by pure distance
run(seq_length=None, query_duration=None, tempo=None, tempo_range=None)
Run the evaluation using optional parameters
  query_duration - length of query in seconds
  tempo - change tempo of query by this proportion: e.g. 1.2 is 20% faster
  tempo_range - a list of tempo change queries to sweep
set_adb(adb)
set the audioDB instance for this evaluator
adb - the full path of an audioDB database instance
set_ground_truth(ground_truth)
set the ground truth for this instance
ground_truth - a list of database indices for a class of query
set_optional_parameters(seq_length, query_duration, tempo, tempo_range)
set optional parameters for evaluation:
  seq_length  - length of query in feature frames 
  query_duration - length of query in seconds
  tempo - change tempo of query by this proportion: e.g. 1.2 is 20% faster
  tempo_range - a list of tempo change queries to sweep
set_seq_length(seq_length, query_duration)
check if we are using duration (in seconds) and set seq_length
according to adb.delta_time, otherwise just use seq_length
upper_bound_list_by_length(lst, length)
truncate (key, length) tuples by upper bound length
return excl_keys, excl_lens
class bregman.evaluate.TimbreChannelEvaluator(adb, ground_truth, timbre_channels, delta_time=None, seq_length=None, query_duration=None, tempo=None, tempo_range=None)
An evaluator class for timbre channels.

Inputs:

adb - an AudioDB database in timbre-channel layout:
   ttttt0c0: track#tc
       h0 p0
       h1 p0
       h2 p0
   ttttt0c1: track#tc
       h0 p0
       h1 p0
       h2 p0
   ...
ground_truth - list of ground_truth key indices in range 0 ... ntracks-1
timbre_channels - number of timbre channels in audioDB database
delta_time - per-frame delta time [0.1s]
seq_length - query sequence length (in frames)
query_duration - alternative query sequence length (in seconds)
tempo - relative tempo scale of search [2.0 = half speed, 1.0 = no scaling, 0.5 = double speed]
tempo_range - range of relative tempos to search
evaluate(seq_length=None, query_duration=None, tempo=1.0, gt_only=True, ground_truth=None, res_name=None)
Evaluate ranks and distances for each ground-truth pattern for retrieval of other GT patterns.
Overridden method from Evaluator for timbre channels
 Map distances and weights for each query in each channel
 Reduce using Bhattacharyya distance metric and base-line averaging metric.

 Sets self.ranks_list, self.dists_list
 Display ranked results as in print_results()

Returns a tuple (ranks,dists)
Each element of the tuple contains len(test_set_list) lists, 
 each list contains ground-truth ranks, and whole-dataset distances, of each test-set 
 in the test_set_list
static f_measure(ranks)
Return the standard F-measure for the given ranks
get_adb_lists(tc)
return two lists of database keys, lengths for timbre channel tc
get_gt_lists(tc)
return two lists of ground truth keys, lengths for timbre channel tc
Initialize search parameters: include list for t_chan, exclude list for !t_chan, Key format is nnnnnncc where nnnnnn is the track id, and cc is the tc id.
static prec_rec(ranks)
Return precision and recall arrays for ranks array data    
rank_by_distance_avg(qkeys, ikeys, rkeys, dists)
Reduce timbre-channel distances to ranks list by ground-truth key indices
Kullback distances
rank_by_distance_bhatt(qkeys, ikeys, rkeys, dists)
Reduce timbre-channel distances to ranks list by ground-truth key indices
Bhattacharyya distance on timbre-channel probabilities and Kullback distances