Plotbee is a library to process, manage and visualize Beepose detections.
For the kernel running the notebooks/scripts, install with conda/mamba:
mamba env create --file requirements.yml -n events
python -m ipykernel install --name 'events' --display-name "events" --user
If you run jupyterlab from the same environment, install it also:
mamba install jupyterlab
else, if jupyterlab environment is separate from the kernel environment, make sure to install:
mamba install ipympl
and that versions are compatible.
git clone --recurse-submodules https://github.com/jachansantiago/plotbee.git
cd plotbee
pip install . # or pip install .[tags] to install with apriltag
The skeleton sub-command converts beepose (--format beepose
) and SLEAP(--format sleap
) detections into plotbee format. beepose
is the default method. Note that if the video is given at this step, no need to input it at the next steps; meanwhile, the video location does not change.
pb skeleton --file merged_C02_170622120000_detections.json --video C02_170622120000.mp4 --format beepose
This command produces an output_file = skeleton_merged_C02_170622120000_detections.json
.
The pollen sub-command performs pollen detection at the detection level. Model's JSON and weights are required to perform pollen detection. Pollen detection uses a parallel implementation that creates temporary files at 'pollen_temp' directory. The default amount of workers is 4. Use CUDA_VISIBLE_DEVICES
to restrict the usage of GPU devices.
pb pollen --file data/C02_170622120000_skeleton.json --video data/C02_170622120000.mp4 \
--model /home/irodriguez/JANELIA/src/BeeLab/2l_model_2020_angle_auto_compensated1.json \
--weights /home/irodriguez/JANELIA/src/BeeLab/2l_model_2020_angle_auto_compensated1.h5 --workers 8
This command produces an output_file = data/pollen_merged_C02_170622120000_detections.json
.
Benchmark for one hour video.
Workers | Time | GPU Memory | Image Size |
---|---|---|---|
4 | ~50 min | 10.4GB | 450x375 |
8 | ~30 min | 20.8GB | 450x375 |
8 | ~7 min | 20.8GB | 90x90 |
Tags sub-command can compute or merge tag detections. Use --compute
to perform the tag detection with AprilTags. Note that tag detection requires images from the video. Be sure that the skeleton file contains the right localization of the video. Alternatively, the video location can be modified with --video
.
pb tags --file data/C02_170622120000_skeleton.json --video data/C02_170622120000.mp4 --compute
Use --merge
and --tags_file
to combine previously computed detections into a plotbee format video. Note that pb tags
requires one of this options --compute
or --merge
to run.
pb tags --file data/C02_170622120000_skeleton.json --tags_file Tags-C02_170622120000.json --merge
Both commands produces an output_file = data/tags_C02_170622120000_detections.json
.
Tracking just requires the --method
. The method can be hungarian
(default) or sort
.
pb tracking --file data/C02_170622120000_skeleton.json --method hungarian
This command can produces an output_file = data/hungarian_C02_170622120000_detections.json
or data/sort_C02_170622120000_detections.json
.
Use pb pipeline
to perform the whole pipeline at once. To active each step in the pipeline use --skeleton
, --tags
, --pollen
, --tracking
. Note that the required parameters for each step need to be input as is shown above. Start with --skeleton
is not required.
pb pipeline --skeleton --file merged_C02_170622120000_detections.json --video C02_170622120000 --method beepose \
--pollen --model_json 2l_model_2020_angle_auto_compensated1.json \
--weights 2l_model_2020_angle_auto_compensated1.h5 \
--tags --tags_file Tags-C02_170622120000.json --method merge \
--tracking --method hungarian
pb export
create pollen and tag dataset from videos in plotbee format. The options --pollen
and --tags
(not implemented yet) are mutually exclusive. Image dimensions --width
and --height
are required fields. A fixed --size
dataset is supported and returns a balanced dataset. size//2
images with the highest pollen scores and size//2
images with the lowest pollen scores. If --size
is not provided the whole video will be exported. --output_folder
is also required.
pb export --pollen --output_folder pollen_data --file test_cli/pollen_tags_skeleton_merged_C02_170628120000_detections.json --width 375 --height 450 --size 200
Use --coco
to export plotbee video format into COCO format. --width
and --height
specifies the bounding box dimmension for the COCO keypoint annotation protocol. Use --images
to activate image extraction. The image extraction can take a while to process a one hour video.
pb export --coco --file skeleton_merged_C02_170628120000_detections.json --output_folder coco --width 300 --height 450 --images
Use --analysis
to export events for behaviour analysis. This method perform track classification for events and produce a csv file with:
frame
, track_id
, pollen_score
,tag_id
, track_event
, track_tag_id
, track_pollen_score
and track_shape
.
pb export --analysis --file hungarian_pollen_tags_skeleton_merged_C02_170628120000_detections.json
This command produces an output_file = analysis_hungarian_pollen_tags_skeleton_merged_C02_170628120000_detections.csv
.
Try our demo notebook.
For bees annotations try our body annotator notebook.
To download sample data run the following command.
./download_data
If arrow keys used in widgets also generate cell change, disable them in the Jupyterlab "Settings/Settings Editor/JSON Settings Editor/User Preferences". For instance:
"shortcuts": [
{
"args": {},
"command": "notebook:move-cursor-down",
"keys": [
"ArrowDown"
],
"selector": ".jp-Notebook.jp-mod-commandMode:not(.jp-mod-readWrite) :focus"
,"disabled": true }
...