README.md

Cross-species F0 estimation, dataset and study of baseline algorithms

Use a crepe model pretrained on animal signals to analyse your own signals

TODO

Reproduce paper experiments

metadata.py

Stores a dictionary of datasets and characteristics (SR, NFFT, path to access soundfiles, and downsampling factor for ultra/infra-sonic signals) Convenient to iterate over the whole dataset

from metadata import species
for specie in species:
    wavpath, FS, nfft, downsample, step = species[specie].values()
    # iterate over files (one per vocalisation)
    for fn in tqdm(glob(wavpath), desc=specie):
        sig, fs = sf.read(fn) # read soundfile
        annot = pd.read_csv(f'{fn[:-4]}.csv') # read annotations (one column Time in seconds, one column Freq in Herz)
        preds = pd.read_csv(f'{fn[:-4]}_preds.csv') # read the file gathering per algorithm f0 predictions

print_annot.py

For each vocalisation, prints a spectrogram and overlaid annotations as .png file stored in the annot_pngs folder.

run_all.py

Runs all baseline algorithms over the dataset.

• praat (praat-parselmouth implem)
• pyin (librosa implem)
• crepe (torchcrepe implem)
• crepe finetuned (torchcrepe implem)
• crepe finetuned over all species except the target
• crepe finetuned only on the target species
• crepe (original tensorflow implem https://arxiv.org/abs/1802.06182)
• basic pitch (https://arxiv.org/abs/2203.09893)
• pesto (https://arxiv.org/abs/2309.02265)
• pesto finetuned over the target species

This scripts stores predictions along with resampled annotations in {basename}_preds.csv files

print_annot.py

For each vocalisation, prints a spectrogram and overlaid annotations as .png file stored in the annot_pngs folder. Similarly, print_preds.py prints spectrograms for a given species but also includes predictions from a chosen algorithm.

eval_all.py

Evaluates each algorithms over the dataset using {basename}_preds.csv files, with a threshold of 50 cents for accuracies. For each algorithms and species, this outputs ROC optimal thresholds, Recall, False alarm, Pitch accuracy, and Chroma accuracy. /!\ These metrics are mesured per vocalisation before being averaged. Scores are stored in scores/{specie}_scores.csv files

vocalisation_characterisation.py

Evaluates metrics for each annotated temporal bin:

• the presence of a sub-harmonic following this paper
• the salience of the annotation as the ratio of the energy of the f0 (one tone around the annotation) and its surrounding (one octave around the annotation)
• the harmonicity of the annotation as the ratio between the energy of all harmonics and that of all harmonics except the fundamental These values are stored in the SHR, harmonicity and salience columns of the {basename}_preds.csv files