IPI extractor core
ipi_extract.py is a graphic utility made for the manual extraction of sperm whale inter pulse interval (IPI).
Prerequisite
This program uses the following libraries (other versions might work):
- numpy 1.16
- matplotlib 3.1
- pandas 0.24
- pydub 0.23
- pysoundfile 0.9
- pytables 3.6
- python 3.6
- scipy 1.2
Note that in order to play the sound signal, pydub requires additional libraries
Options
There are two ways to set the options. The first way is by setting the option in the command line.
The list of available options can be obtained by typing
python ipi_extract.py -h
If no options are given pass via the command line, then the script will assume that the second way is used. In that case, the script will ask the user the value for each of the options.
e.g. What is the input file path?
Usage
The GUI can be split into three parts. At the top, a segment of 20ms of the sound signal is displayed. The middle part is made of three groups of plots. A group is a square of 2 by 2 plots, each displaying a different representation of the same click. The button part is made of additional interactive buttons. The code is written in a way that allows the implementation of other widgets in a script that would import this (see IPI_bombyx.py as an example)
The annotations are saved once the GUI is closed.
Sectioning clicks
The top part is composed of two arrows and the sound signal. Clicking one of the arrows will translate the signal by 15s in the corresponding direction. Clicking on the sound signal will select the click that will be analyzed. The click selected will be at the maximum near the location clicked (0.1 s). The click will then be displayed in one of the 3 groups of plots selected by the radio button below them. While hovering above the signal, the cursor will have the color of the selected group. Clicking on a new click will change the click to be analyzed. An already analyzed click can be clicked again. Unselected clicks will be black if any fields have been annotated, and purple otherwise.
Note that a click can be assigned to two groups of plots. This currently causes desyncs in the display of the annotation. In a future update, this might either be fixed, or the possibilities of selecting the same click for multiple groups will be removed
Assigning labels
Once assigned to a group, a click will be shown with 4 representations. A group of plots is composed of the following representations:
| Signal | Spectrogram |
|---|---|
| Autocorrelation | Cepstrum |
Clicking on any of the plots (meaning also in the non-current groups) will update the corresponding values.
Not all plots need to be labeled. Nonlabeled plots will have a nan value in the database.
A vertical bar will follow the cursor while hovering above one of the plots. On the other plots of the same groups, vertical bars are also displayed at the corresponding current hovering value. This allows the user to see if the IPI value that is going to be selected makes sense according to the other representations. Due to the specificity of each representation, this synchronous display of the bar might change in some situations. See below for these caveats.
Signal and spectrogram
The signal and spectrogram plots are linked, meaning that they share the same label. The labeling of these plots alternate between two modes: the labeling of P1 and the labeling of P2. P1 labeling is indicated with a dashed vertical bar that follows the cursor. In P2 labeling mode, the vertical bar is a full line. Clicking will annotate the corresponding pulse, and then leave a bar behind the signal with the corresponding line style (dashed or full).
Concerning the vertical bars, the synchronous display is only possible once a first labeling of P1 has been done. If the plots are in the P1 mode, then the synchronous display will be deactivated while hovering above the signal or the spectrogram, but will work while hovering over the other two (One might consider the other representation to label P2 instead of the IPI). Hovering over a negative IPI value will still work for the autocorrelation and the cepstrum since they are symmetric in time.
Note that since the clicks are centered with a local maximum detection, P1 should always be at 10 ms. Thus, a future update might disable the selection of P1 or add a toggle button.
Autocorrelation and Cesptrum
These two plots work in the same way. Clicking on them will set the IPI to the clicked location. The automatic IPI set is the maximum in a range of 0.15 ms.
Assigning an individual label
Next to the reset button is the case to give the currently selected group of plots (red, green, or blue) a click label.
Clicking on it will swap its state from grey (inactive) to green (active).
Upon activation, the click label will be reset to an empty string ''.
While the case remains active (green), any keyboard presses will be added to the current label.
Reset
Clicking on Reset current will set all the annotable values of the current click to nan.
Other functionalities
resizing
In order to have a responsive GUI, the automatic resizing of the plots have been deactivated
You can click on resize plot to compute one resize for the current layout (takes ~1sec).
Listening to the sound
By clicking on Play current segment, the sound of the current signal segment will be played.
Note that while the sound is played, the interface becomes irresponsive.
As stated above, the audio player used by pydub requires additional libraries.
If you're using this script on a remote server, the sound might be play over there or not work depending on the method you use to access said server.
Change the spectrogram parameters
The four buttons on the left of the groups of plots can be used to change the spectrograms' settings.
The two top buttons are used to increase/decrease the maximum frequency of the spectrogram. Frequency bins that are above the nyquist frequency are will be displayed entirely white.
The two bottom buttons are used to change the size of the FFT window used by the spectrogram. The number of bins indicated by each button will be the size of this window after clicking on the corresponding button.
keyboard shortcut
Pressing 0,1 or 2 on the keyboard will change the currently selected group of plots to the corresponding value. Note that these shortcuts are disabled while the individual label is active (in green).
Output
Once done, the script will save the label in a HDF file (whose name depends on the options used).
This file contains a database with one line per annotated click.
The column are p1_pos, ipi_sig, ipi_corr_man, ipi_corr_auto, ipi_ceps_man, ipi_ceps_auto, ind_number.
Note that the individual number is currently not implemented and will only contain NaN.
The current behavior of the scripts also saves unannotated clicks. This might be prone to change.
Linked project
IPI_bombyx.py is a script for annotating the IPI of the BOMBYX database.