DYOC-DF

DYOC: Draw Your Own Contours

DYOC is an AI tool that is built on YOLOv8 and ResNet18 and uses transfer learning to produce fast semi-automatic annotation of acoustic vocalisation contours in spectrograms.

DYOC is trained and tested on anotated whistles of short-beaked common dolphin (Delphinus delphis) from the DOLPHINFREE dataset.

Description of repository

.
│   # Files
├── README.md
├── requirements.txt
├── DYOC_preprocessing_data.py 
├── TRAIN_DYOC.py
├── predict_DYOC.py
│
│   # Folders
├── All_annotations                     # Initial annotations (for training)
├── Audio_data
│   ├── wavefiles                       # Hidden (~ GB of data)
│   └── data_availability_statement.md
├── Analyses
│   ├── stats.ipynb                     # ANALYSES of whistles features
│   ├── data_preprocessed               # RESULTS of feature extraction
│   └── ...
├── CSV_Data                            # DOLPHINFREE observation data
├── predicted_contours                  # Final results of contour prediction
│   ├── pred_parameters.json
│   └── contour files
├── preprocessed_data                   # Outputs of preprocessing_data.py
├── Resources                           # Parameters and images
├── Training_results                    # Outputs of TRAIN_DYOC.py
│   ├── 20240404                        # 8-fold tests
│   └── final_for_predictions_20240417  # Full training for predictions
└── utils                               # Functions and classes module

Setup

Requirements

• Python >=3.9.18
• Packages in requirements.txt
• Developped with Ubuntu 20.04.6 LTS
  • tested on Win10 (with conda env)
  • should work on any OS though

Recommended

It is preferable to work in a virtual environment :

$ python -m venv DYOC-ENV
$ source DYOC-ENV/bin/activate
$ pip install -r requirement.txt

Also make sure that you have :

• a CUDA-capable device with CUDA installed (see nvidia.com)
• pytorch installed with cuda dependancies (see pytorch.org)

Get started

Data exploration

See files in Training_results/20240404 for DYOC's training results (8-fold). See notebook in Analyses/stats.ipynb for in-depth analyses of whistles contours.

How to make predictions

DYOC can be used with no changes to make predictions on audio recordings of Delphinus delphis made with the Ocean Sonics icListen HF hydrophone at a sampling rate ≥64 kHz (DOLPHINFREE materials). Just run the following command :

$ python predict_DYOC.py -wav YOUR_AUDIO_FOLDER

For other species and/or devices, some parameters might need to be adjusted to match your specific use. Please modify the following files to match your needs (in Resources folder) :

• preprocessing_parameters.json
  • $ python DYOC_preprocessing.py -h for help on variables
• training_parameters.json
  • $ python TRAIN_DYOC.py -h for help on variables

You can also run $ python predict_DYOC.py -h for more details.

Training on new data

Custom annotations can be used to train the model to be able to annotate other whistle contours. Audio files must all be in one folder (preferably *.wav files as others were not tested). Annotations must be in PyAVA format (see PyAVA repo).

For training, audio data must be converted to images that the model can use. Run $ python DYOC_preprocessing.py -h for preprocessing options and $ python TRAIN_DYOC.py -h for training options.

Contact

Feel free to contact me by email if you have questions or tips.

Related to

Experiment and results are presented in details in: Lehnhoff, L., et al (in prep).