Food Recognition Benchmark 2022

🍕 Food Recognition Benchmark: Data Exploration & Baseline

🍕 Food Recognition Benchmark¶

Credits: This notebook is fork of notebook created by @shubhamai for previous iteration of the challenge. You can find the original notebook here.

Problem Statement¶

Detecting & Segmenting various kinds of food from an image. For ex. Someone got into new restaurent and get a food that he has never seen, well our DL model is in rescue, so our DL model will help indentifying which food it is from the class our model is being trained on!

Dataset¶

We will be using data from Food Recognition Challenge - A benchmark for image-based food recognition challange which is running since 2020.

https://www.aicrowd.com/challenges/food-recognition-benchmark-2022#datasets

We have a total of 39k training images with 3k validation set and 4k public-testing set. All the images are RGB and annotations exist in MS-COCO format.

Evaluation¶

The evaluation metrics is IOU aka. Intersection Over Union ( more about that later ).

The actualy metric is computed by averaging over all the precision and recall values for IOU which greater than 0.5.

https://www.aicrowd.com/challenges/food-recognition-challenge#evaluation-criteria

What does this notebook contains?¶

Setting our Workspace 💼
Data Exploration 🧐
- Reading Dataset
- Data Visualisations
Image Visulisation 🖼️
- Reading Images
Creating our Dataset 🔨
- Fixing the Dataset
- Creating our dataset
Creating our Model 🏭
- Creating R-CNN Model
- Setting up hyperparameters
Training the Model 🚂
- Setting up Tensorboard
- Start Training!
Evaluating the model 🧪
- Evaluating our Model
Testing the Model 💯
- Testing the Model
Submitting our predictions 📝
Generate More Data + Some tips & tricks 💡

Setting our Workspace 💼¶

In this section we will be downloading our dataset, unzipping it & downliading detectron2 library and importing all libraries that we will be using

Downloading & Unzipping our Dataset¶

In [1]:

# Login to AIcrowd
!pip install aicrowd-cli > /dev/null
!aicrowd login

ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
google-colab 1.0.0 requires requests~=2.23.0, but you have requests 2.26.0 which is incompatible.
datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.
Please login here: https://api.aicrowd.com/auth/NnwCEee07iHXtuxG0ByOQmZge6Jb-VfXvENxWAjJhls
/usr/bin/xdg-open: 851: /usr/bin/xdg-open: www-browser: not found
/usr/bin/xdg-open: 851: /usr/bin/xdg-open: links2: not found
/usr/bin/xdg-open: 851: /usr/bin/xdg-open: elinks: not found
/usr/bin/xdg-open: 851: /usr/bin/xdg-open: links: not found
/usr/bin/xdg-open: 851: /usr/bin/xdg-open: lynx: not found
/usr/bin/xdg-open: 851: /usr/bin/xdg-open: w3m: not found
xdg-open: no method available for opening 'https://api.aicrowd.com/auth/NnwCEee07iHXtuxG0ByOQmZge6Jb-VfXvENxWAjJhls'
API Key valid
Saved API Key successfully!

In [2]:

# List dataset for this challenge
!aicrowd dataset list -c food-recognition-benchmark-2022

# Download dataset
!aicrowd dataset download -c food-recognition-benchmark-2022

                          Datasets for challenge #962                           
┌───┬────────────────────────────────┬────────────────────────────────┬────────┐
│ # │ Title                          │ Description                    │   Size │
├───┼────────────────────────────────┼────────────────────────────────┼────────┤
│ 0 │ public_validation_set_2.0.tar… │ Validation Dataset (contains   │    59M │
│   │                                │ 1000 images and 498            │        │
│   │                                │ categories, with annotations)  │        │
│ 1 │ public_test_release_2.0.tar.gz │ [Public] Testing Dataset       │   197M │
│   │                                │ (contains 3000 images and 498  │        │
│   │                                │ categories, without            │        │
│   │                                │ annotations)                   │        │
│ 2 │ public_training_set_release_2… │ Training Dataset (contains     │ 2.14GB │
│   │                                │ 39962 images and 498           │        │
│   │                                │ categories)                    │        │
└───┴────────────────────────────────┴────────────────────────────────┴────────┘
public_validation_set_2.0.tar.gz: 100% 62.4M/62.4M [00:05<00:00, 11.6MB/s]
public_test_release_2.0.tar.gz: 100% 207M/207M [00:12<00:00, 16.2MB/s]
public_training_set_release_2.0.tar.gz: 100% 2.30G/2.30G [02:38<00:00, 14.5MB/s]

In [3]:

# Create data directory
!mkdir -p data/ data/train data/val data/test
!cp *test* data/test && cd data/test && echo "Extracting test dataset" && tar -xvf *test* > /dev/null
!cp *val* data/val && cd data/val && echo "Extracting val dataset" &&  tar -xvf *val* > /dev/null
!cp *train* data/train && cd data/train && echo "Extracting train dataset" &&  tar -xvf *train* > /dev/null

Extracting test dataset
Extracting val dataset
Extracting train dataset

So, the data directory is something like this:

Importing Necessary Libraries¶

In [4]:

# Making sure that we are using GPUs
!nvidia-smi

Tue Dec 21 06:43:52 2021       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 495.44       Driver Version: 460.32.03    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla K80           Off  | 00000000:00:04.0 Off |                    0 |
| N/A   33C    P8    27W / 149W |      0MiB / 11441MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

In [1]:

# Colab has cuda 11.1 pre-installed nowadays, downgrading to 1.9 for detectron2
!pip install -U torch==1.9.0+cu111 torchvision==0.10.0+cu111 -f https://download.pytorch.org/whl/torch_stable.html
!pip install cython pyyaml==5.1
!pip install -U pycocotools
import torch, torchvision
print(torch.__version__, torch.cuda.is_available())
!gcc --version

WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages)
WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages)
Looking in links: https://download.pytorch.org/whl/torch_stable.html
Requirement already satisfied: torch==1.9.0+cu111 in /usr/local/lib/python3.7/dist-packages (1.9.0+cu111)
Requirement already satisfied: torchvision==0.10.0+cu111 in /usr/local/lib/python3.7/dist-packages (0.10.0+cu111)
Requirement already satisfied: typing-extensions in /usr/local/lib/python3.7/dist-packages (from torch==1.9.0+cu111) (3.10.0.2)
Requirement already satisfied: pillow>=5.3.0 in /usr/local/lib/python3.7/dist-packages (from torchvision==0.10.0+cu111) (7.1.2)
Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from torchvision==0.10.0+cu111) (1.19.5)
WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages)
WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages)
WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages)
Requirement already satisfied: cython in /usr/local/lib/python3.7/dist-packages (0.29.24)
Requirement already satisfied: pyyaml==5.1 in /usr/local/lib/python3.7/dist-packages (5.1)
WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages)
WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages)
WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages)
Requirement already satisfied: pycocotools in /usr/local/lib/python3.7/dist-packages (2.0.3)
Requirement already satisfied: setuptools>=18.0 in /usr/local/lib/python3.7/dist-packages (from pycocotools) (57.4.0)
Requirement already satisfied: cython>=0.27.3 in /usr/local/lib/python3.7/dist-packages (from pycocotools) (0.29.24)
Requirement already satisfied: matplotlib>=2.1.0 in /usr/local/lib/python3.7/dist-packages (from pycocotools) (3.2.2)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.1.0->pycocotools) (3.0.6)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.1.0->pycocotools) (0.11.0)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.1.0->pycocotools) (2.8.2)
Requirement already satisfied: numpy>=1.11 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.1.0->pycocotools) (1.19.5)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib>=2.1.0->pycocotools) (1.3.2)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib>=2.1.0->pycocotools) (1.15.0)
WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages)
1.9.0+cu111 True
gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

In [2]:

!nvidia-smi

Tue Dec 21 07:20:35 2021       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 495.44       Driver Version: 460.32.03    CUDA Version: 11.2     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla K80           Off  | 00000000:00:04.0 Off |                    0 |
| N/A   33C    P8    27W / 149W |      3MiB / 11441MiB |      0%      Default |
|                               |                      |                  N/A |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

In [3]:

# install detectron2:
!pip install -U detectron2==0.6+cu111 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/index.html

WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages)
WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages)
Looking in links: https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/index.html
Collecting detectron2==0.6+cu111
  Downloading https://dl.fbaipublicfiles.com/detectron2/wheels/cu111/torch1.9/detectron2-0.6%2Bcu111-cp37-cp37m-linux_x86_64.whl (6.9 MB)
     |████████████████████████████████| 6.9 MB 811 kB/s 
Requirement already satisfied: tqdm>4.29.0 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (4.62.3)
Requirement already satisfied: pycocotools>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (2.0.3)
Requirement already satisfied: Pillow>=7.1 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (7.1.2)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (3.2.2)
Requirement already satisfied: tensorboard in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (2.7.0)
Requirement already satisfied: fvcore<0.1.6,>=0.1.5 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (0.1.5.post20211023)
Requirement already satisfied: iopath<0.1.10,>=0.1.7 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (0.1.9)
Requirement already satisfied: yacs>=0.1.8 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (0.1.8)
Requirement already satisfied: black==21.4b2 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (21.4b2)
Requirement already satisfied: pydot in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (1.3.0)
Requirement already satisfied: hydra-core>=1.1 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (1.1.1)
Requirement already satisfied: tabulate in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (0.8.9)
Requirement already satisfied: cloudpickle in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (1.3.0)
Requirement already satisfied: termcolor>=1.1 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (1.1.0)
Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (0.16.0)
Requirement already satisfied: omegaconf>=2.1 in /usr/local/lib/python3.7/dist-packages (from detectron2==0.6+cu111) (2.1.1)
Requirement already satisfied: mypy-extensions>=0.4.3 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2==0.6+cu111) (0.4.3)
Requirement already satisfied: typed-ast>=1.4.2 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2==0.6+cu111) (1.5.1)
Requirement already satisfied: regex>=2020.1.8 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2==0.6+cu111) (2021.11.10)
Requirement already satisfied: pathspec<1,>=0.8.1 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2==0.6+cu111) (0.9.0)
Requirement already satisfied: typing-extensions>=3.7.4 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2==0.6+cu111) (3.10.0.2)
Requirement already satisfied: click>=7.1.2 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2==0.6+cu111) (7.1.2)
Requirement already satisfied: toml>=0.10.1 in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2==0.6+cu111) (0.10.2)
Requirement already satisfied: appdirs in /usr/local/lib/python3.7/dist-packages (from black==21.4b2->detectron2==0.6+cu111) (1.4.4)
Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from fvcore<0.1.6,>=0.1.5->detectron2==0.6+cu111) (1.19.5)
Requirement already satisfied: pyyaml>=5.1 in /usr/local/lib/python3.7/dist-packages (from fvcore<0.1.6,>=0.1.5->detectron2==0.6+cu111) (5.1)
Requirement already satisfied: importlib-resources in /usr/local/lib/python3.7/dist-packages (from hydra-core>=1.1->detectron2==0.6+cu111) (5.4.0)
Requirement already satisfied: antlr4-python3-runtime==4.8 in /usr/local/lib/python3.7/dist-packages (from hydra-core>=1.1->detectron2==0.6+cu111) (4.8)
Requirement already satisfied: portalocker in /usr/local/lib/python3.7/dist-packages (from iopath<0.1.10,>=0.1.7->detectron2==0.6+cu111) (2.3.2)
Requirement already satisfied: setuptools>=18.0 in /usr/local/lib/python3.7/dist-packages (from pycocotools>=2.0.2->detectron2==0.6+cu111) (57.4.0)
Requirement already satisfied: cython>=0.27.3 in /usr/local/lib/python3.7/dist-packages (from pycocotools>=2.0.2->detectron2==0.6+cu111) (0.29.24)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->detectron2==0.6+cu111) (3.0.6)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->detectron2==0.6+cu111) (0.11.0)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->detectron2==0.6+cu111) (2.8.2)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->detectron2==0.6+cu111) (1.3.2)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib->detectron2==0.6+cu111) (1.15.0)
Requirement already satisfied: zipp>=3.1.0 in /usr/local/lib/python3.7/dist-packages (from importlib-resources->hydra-core>=1.1->detectron2==0.6+cu111) (3.6.0)
Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (1.0.1)
Requirement already satisfied: absl-py>=0.4 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (0.12.0)
Requirement already satisfied: wheel>=0.26 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (0.37.0)
Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (2.26.0)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (0.4.6)
Requirement already satisfied: tensorboard-data-server<0.7.0,>=0.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (0.6.1)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (1.8.0)
Requirement already satisfied: google-auth<3,>=1.6.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (1.35.0)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (3.3.6)
Requirement already satisfied: grpcio>=1.24.3 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (1.42.0)
Requirement already satisfied: protobuf>=3.6.0 in /usr/local/lib/python3.7/dist-packages (from tensorboard->detectron2==0.6+cu111) (3.17.3)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard->detectron2==0.6+cu111) (4.2.4)
Requirement already satisfied: rsa<5,>=3.1.4 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard->detectron2==0.6+cu111) (4.8)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.7/dist-packages (from google-auth<3,>=1.6.3->tensorboard->detectron2==0.6+cu111) (0.2.8)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.7/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard->detectron2==0.6+cu111) (1.3.0)
Requirement already satisfied: importlib-metadata>=4.4 in /usr/local/lib/python3.7/dist-packages (from markdown>=2.6.8->tensorboard->detectron2==0.6+cu111) (4.8.2)
Requirement already satisfied: pyasn1<0.5.0,>=0.4.6 in /usr/local/lib/python3.7/dist-packages (from pyasn1-modules>=0.2.1->google-auth<3,>=1.6.3->tensorboard->detectron2==0.6+cu111) (0.4.8)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard->detectron2==0.6+cu111) (2.10)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard->detectron2==0.6+cu111) (2021.10.8)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard->detectron2==0.6+cu111) (2.0.8)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.21.0->tensorboard->detectron2==0.6+cu111) (1.24.3)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.7/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard->detectron2==0.6+cu111) (3.1.1)
WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages)
Installing collected packages: detectron2
  Attempting uninstall: detectron2
    WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages)
    Found existing installation: detectron2 0.6+cu101
    Uninstalling detectron2-0.6+cu101:
      Successfully uninstalled detectron2-0.6+cu101
WARNING: Ignoring invalid distribution -orch (/usr/local/lib/python3.7/dist-packages)
Successfully installed detectron2-0.6+cu111

In [4]:

# You may need to restart your runtime prior to this, to let your installation take effect
# Some basic setup:
# Setup detectron2 logger
import detectron2
from detectron2.utils.logger import setup_logger
setup_logger()

# import some common libraries
import numpy as np
import pandas as pd
import cv2
import json
from tqdm.notebook import tqdm

# import some common detectron2 utilities
from detectron2 import model_zoo
from detectron2.engine import DefaultPredictor
from detectron2.config import get_cfg
from detectron2.utils.visualizer import Visualizer
from detectron2.data import MetadataCatalog
from detectron2.utils.visualizer import ColorMode
from detectron2.data.datasets import register_coco_instances
from detectron2.engine import DefaultTrainer
from detectron2.evaluation import COCOEvaluator, inference_on_dataset
from detectron2.data import build_detection_test_loader

# For reading annotations file
from pycocotools.coco import COCO

# utilities
from pprint import pprint # For beautiful print!
from collections import OrderedDict
import os 

# For data visualisation
import matplotlib.pyplot as plt
import plotly.graph_objects as go
import plotly.express as px
from google.colab.patches import cv2_imshow

Data Exploration 🧐¶

In this section we are going to read our dataset & doing some data visualisations

Reading Data¶

In [5]:

# Reading annotations.json
TRAIN_ANNOTATIONS_PATH = "data/train/annotations.json"
TRAIN_IMAGE_DIRECTIORY = "data/train/images/"

VAL_ANNOTATIONS_PATH = "data/val/annotations.json"
VAL_IMAGE_DIRECTIORY = "data/val/images/"

train_coco = COCO(TRAIN_ANNOTATIONS_PATH)

loading annotations into memory...
Done (t=8.87s)
creating index...
index created!

In [6]:

# Reading the annotation files
with open(TRAIN_ANNOTATIONS_PATH) as f:
  train_annotations_data = json.load(f)

with open(VAL_ANNOTATIONS_PATH) as f:
  val_annotations_data = json.load(f)

In [7]:

train_annotations_data['annotations'][0]

Out[7]:

{'area': 5059.0,
 'bbox': [39.5, 39.5, 167.0, 92.0],
 'category_id': 1352,
 'id': 184135,
 'image_id': 131094,
 'iscrowd': 0,
 'segmentation': [[115.0,
   206.5,
   98.0,
   204.5,
   74.5,
   182.0,
   65.0,
   167.5,
   47.5,
   156.0,
   39.5,
   137.0,
   39.5,
   130.0,
   51.0,
   118.5,
   62.00000000000001,
   112.5,
   76.0,
   113.5,
   121.5,
   151.0,
   130.5,
   169.0,
   131.5,
   185.0,
   128.5,
   195.0]]}

Data Format 🔍¶

Our COCO data format is something like this -

"info": {...},
"categories": [...],
"images": [...],
"annotations": [...],

In which categories is like this

[
  {'id': 2578,
  'name': 'water',
  'name_readable': 'Water',
  'supercategory': 'food'},
  {'id': 1157,
  'name': 'pear',
  'name_readable': 'Pear',
  'supercategory': 'food'},
  ...
  {'id': 1190,
  'name': 'peach',
  'name_readable': 'Peach',
  'supercategory': 'food'}
]

Info is empty ( not sure why )

images is like this

[
  {'file_name': '065537.jpg', 
  'height': 464, 
  'id': 65537, 
  'width': 464},
  {'file_name': '065539.jpg', 
  'height': 464, 
  'id': 65539, 
  'width': 464},
 ...
  {'file_name': '069900.jpg', 
  'height': 391, 
  'id': 69900, 
  'width': 392},
]

Annotations is like this

{'area': 44320.0,
 'bbox': [86.5, 127.49999999999999, 286.0, 170.0],
 'category_id': 2578,
 'id': 102434,
 'image_id': 65537,
 'iscrowd': 0,
 'segmentation': [[235.99999999999997,
   372.5,
   169.0,
   372.5,
   ...
   368.5,
   264.0,
   371.5]]}

In [9]:

# Reading all classes
category_ids = train_coco.loadCats(train_coco.getCatIds())
category_names = [_["name_readable"] for _ in category_ids]

print("## Categories\n-", "\n- ".join(category_names))

## Categories
- Bread, wholemeal
- Jam
- Water
- Bread, sourdough
- Banana
- Soft cheese
- Ham, raw
- Hard cheese
- Cottage cheese
- Bread, half white
- Coffee, with caffeine
- Fruit salad
- Pancakes
- Tea
- Salmon, smoked
- Avocado
- Spring onion / scallion
- Ristretto, with caffeine
- Ham
- Egg
- Bacon, frying
- Chips, french fries
- Juice, apple
- Chicken
- Tomato, raw 
- Broccoli
- Shrimp, boiled
- Beetroot, steamed, without addition of salt
- Carrot, raw
- Chickpeas
- French salad dressing
- Pasta, Hörnli
- Sauce, cream
- Meat balls
- Pasta
- Tomato sauce
- Cheese
- Pear
- Cashew nut
- Almonds
- Lentils
- Mixed vegetables
- Peanut butter
- Apple
- Blueberries
- Cucumber
- Cocoa powder
- Greek Yaourt, yahourt, yogourt ou yoghourt
- Maple syrup (Concentrate)
- Buckwheat, grain peeled
- Butter
- Herbal tea
- Mayonnaise
- Soup, vegetable
- Wine, red
- Wine, white
- Green bean, steamed, without addition of salt
- Sausage
- Pizza, Margherita, baked
- Salami
- Mushroom
- (bread, meat substitute, lettuce, sauce)
- Tart
- Tea, verveine
- Rice
- White coffee, with caffeine
- Linseeds
- Sunflower seeds
- Ham, cooked
- Bell pepper, red, raw 
- Zucchini
- Green asparagus
- Tartar sauce
- Lye pretzel (soft)
- Cucumber, pickled 
- Curry, vegetarian
- Yaourt, yahourt, yogourt ou yoghourt, natural
- Soup of lentils, Dahl (Dhal)
- Soup, cream of vegetables
- Balsamic vinegar
- Salmon
- Salt cake (vegetables, filled)
- Bacon
- Orange
- Pasta, noodles
- Cream
- Cake, chocolate
- Pasta, spaghetti
- Black olives
- Parmesan
- Spaetzle
- Salad, lambs' ear
- Salad, leaf / salad, green
- Potatoes steamed
- White cabbage
- Halloumi
- Beetroot, raw
- Bread, grain
- Applesauce, unsweetened, canned
- Cheese for raclette
- Mushrooms
- Bread, white
- Curds, natural, with at most 10% fidm
- Bagel (without filling)
- Quiche, with cheese, baked, with puff pastry
- Soup, potato
- Bouillon, vegetable
- Beef, sirloin steak
- Taboulé, prepared, with couscous
- Eggplant
- Bread
- Turnover with meat (small meat pie, empanadas)
- Mungbean sprouts
- Mozzarella
- Pasta, penne
- Lasagne, vegetable, prepared
- Mandarine
- Kiwi
- French beans
- Tartar (meat)
- Spring roll (fried)
- Pork, chop
- Caprese salad (Tomato Mozzarella)
- Leaf spinach
- Roll of half-white or white flour, with large void
- Pasta, ravioli, stuffing
- Omelette, plain
- Tuna
- Dark chocolate
- Sauce (savoury)
- Dried raisins
- Ice tea
- Kaki
- Macaroon
- Smoothie
- Crêpe, plain
- Chicken nuggets
- Chili con carne, prepared
- Veggie burger
- Cream spinach
- Cod
- Chinese cabbage
- Hamburger (Bread, meat, ketchup)
- Soup, pumpkin
- Sushi
- Chestnuts
- Coffee, decaffeinated
- Sauce, soya
- Balsamic salad dressing
- Pasta, twist
- Bolognaise sauce
- Leek
- Fajita (bread only)
- Potato-gnocchi
- Beef, cut into stripes (only meat)
- Rice noodles/vermicelli
- Tea, ginger
- Tea, green
- Bread, whole wheat
- Onion
- Garlic
- Hummus
- Pizza, with vegetables, baked
- Beer
- Glucose drink 50g
- Chicken, wing
- Ratatouille
- Peanut
- High protein pasta (made of lentils, peas, ...)
- Cauliflower
- Quiche, with spinach, baked, with cake dough
- Green olives
- Brazil nut
- Eggplant caviar
- Bread, pita
- Pasta, wholemeal
- Sauce, pesto
- Oil
- Couscous
- Sauce, roast
- Prosecco
- Crackers
- Bread, toast
- Shrimp / prawn (small)
- Panna cotta
- Romanesco
- Water with lemon juice
- Espresso, with caffeine
- Egg, scrambled, prepared
- Juice, orange
- Ice cubes
- Braided white loaf
- Emmental cheese
- Croissant, wholegrain
- Hazelnut-chocolate spread(Nutella, Ovomaltine, Caotina)
- Tomme
- Water, mineral
- Hazelnut
- Bacon, raw
- Bread, nut
- Black Forest Tart
- Soup, Miso
- Peach
- Figs
- Beef, filet
- Mustard, Dijon
- Rice, Basmati
- Mashed potatoes, prepared, with full fat milk, with butter
- Dumplings
- Pumpkin
- Swiss chard
- Red cabbage
- Spinach, raw
- Naan (indien bread)
- Chicken curry (cream/coconut milk. curry spices/paste))
- Crunch Müesli
- Biscuits
- Bread, French (white flour)
- Meatloaf
- Fresh cheese
- Honey
- Vegetable mix, peas and carrots
- Parsley
- Brownie
- Dairy ice cream
- Tea, black
- Carrot cake
- Fish fingers (breaded)
- Salad dressing
- Dried meat
- Chicken, breast
- Mixed salad (chopped without sauce)
- Feta
- Praline
- Tea, peppermint
- Walnut
- Potato salad, with mayonnaise yogurt dressing
- Kebab in pita bread
- Kolhrabi
- Alfa sprouts
- Brussel sprouts
- Bacon, cooking
- Gruyère
- Bulgur
- Grapes
- Pork, escalope
- Chocolate egg, small
- Cappuccino
- Zucchini, stewed, without addition of fat, without addition of salt
- Crisp bread, Wasa
- Bread, black
- Perch fillets (lake)
- Rosti
- Mango
- Sandwich (ham, cheese and butter)
- Müesli
- Spinach, steamed, without addition of salt
- Fish
- Risotto, without cheese, cooked
- Milk Chocolate with hazelnuts
- Cake (oblong)
- Crisps
- Pork
- Pomegranate
- Sweet corn, canned
- Flakes, oat
- Greek salad
- Cantonese fried rice
- Sesame seeds
- Bouillon
- Baked potato
- Fennel
- Meat
- Bread, olive
- Croutons
- Philadelphia
- Mushroom, (average), stewed, without addition of fat, without addition of salt
- Bell pepper, red, stewed, without addition of fat, without addition of salt
- White chocolate
- Mixed nuts
- Breadcrumbs (unspiced)
- Fondue
- Sauce, mushroom
- Tea, spice
- Strawberries
- Tea, rooibos
- Pie, plum, baked, with cake dough
- Potatoes au gratin, dauphinois, prepared
- Capers
- Vegetables
- Bread, wholemeal toast
- Red radish
- Fruit tart
- Beans, kidney
- Sauerkraut
- Mustard
- Country fries
- Ketchup
- Pasta, linguini, parpadelle, Tagliatelle
- Chicken, cut into stripes (only meat)
- Cookies
- Sun-dried tomatoe
- Bread, Ticino
- Semi-hard cheese
- Margarine
- Porridge, prepared, with partially skimmed milk
- Soya drink (soy milk)
- Juice, multifruit
- Popcorn salted
- Chocolate, filled
- Milk chocolate
- Bread, fruit
- Mix of dried fruits and nuts
- Corn
- Tête de Moine
- Dates
- Pistachio
- Celery
- White radish
- Oat milk
- Cream cheese
- Bread, rye
- Witloof chicory
- Apple crumble
- Goat cheese (soft)
- Grapefruit, pomelo
- Risotto, with mushrooms, cooked
- Blue mould cheese
- Biscuit with Butter
- Guacamole
- Pecan nut
- Tofu
- Cordon bleu, from pork schnitzel, fried
- Paprika chips
- Quinoa
- Kefir drink
- M&M's
- Salad, rocket
- Bread, spelt
- Pizza, with ham, with mushrooms, baked
- Fruit coulis
- Plums
- Beef, minced (only meat)
- Pizza, with ham, baked
- Pineapple
- Soup, tomato
- Cheddar
- Tea, fruit
- Rice, Jasmin
- Seeds
- Focaccia
- Milk
- Coleslaw (chopped without sauce)
- Pastry, flaky
- Curd
- Savoury puff pastry stick
- Sweet potato
- Chicken, leg
- Croissant
- Sour cream
- Ham, turkey
- Processed cheese
- Fruit compotes
- Cheesecake
- Pasta, tortelloni, stuffing
- Sauce, cocktail
- Croissant with chocolate filling
- Pumpkin seeds
- Artichoke
- Champagne
- Grissini
- Sweets / candies
- Brie
- Wienerli (Swiss sausage)
- Syrup (diluted, ready to drink)
- Apple pie
- White bread with butter, eggs and milk
- Savoury puff pastry
- Anchovies
- Tuna, in oil, drained
- Lemon pie
- Meat terrine, paté
- Coriander
- Falafel (balls)
- Berries
- Latte macchiato, with caffeine
- Faux-mage Cashew, vegan chers
- Beans, white
- Sugar Melon
- Mixed seeds
- Hamburger
- Hamburger bun
- Oil & vinegar salad dressing
- Soya Yaourt, yahourt, yogourt ou yoghourt
- Chocolate milk, chocolate drink
- Celeriac
- Chocolate mousse
- Cenovis, yeast spread
- Thickened cream (> 35%)
- Meringue
- Lamb, chop
- Shrimp / prawn (large)
- Beef
- Lemon
- Croque monsieur
- Chives
- Chocolate cookies
- Birchermüesli, prepared, no sugar added
- Fish crunchies (battered)
- Muffin
- Savoy cabbage, steamed, without addition of salt
- Pine nuts
- Chorizo
- Chia grains
- Frying sausage
- French pizza from Alsace, baked
- Chocolate
- Cooked sausage
- Grits, polenta, maize flour
- Gummi bears, fruit jellies, Jelly babies with fruit essence
- Wine, rosé
- Coca Cola
- Raspberries
- Roll with pieces of chocolate
- Goat, (average), raw
- Lemon Cake
- Coconut milk
- Rice, wild
- Gluten-free bread
- Pearl onions
- Buckwheat pancake
- Bread, 5-grain
- Light beer
- Sugar, glazing
- Tzatziki
- Butter, herb
- Ham croissant
- Corn crisps
- Lentils green (du Puy, du Berry)
- Cocktail
- Rice, whole-grain
- Veal sausage
- Cervelat
- Sorbet
- Aperitif, with alcohol, apérol, Spritz
- Dips
- Corn Flakes
- Peas
- Tiramisu
- Apricots
- Cake, marble
- Lamb
- Lasagne, meat, prepared
- Coca Cola Zero
- Cake, salted
- Dough (puff pastry, shortcrust, bread, pizza dough)
- Rice waffels
- Sekt
- Brioche
- Vegetable au gratin, baked
- Mango dried
- Processed meat, Charcuterie
- Mousse
- Sauce, sweet & sour
- Basil
- Butter, spread, puree almond
- Pie, apricot, baked, with cake dough
- Rusk, wholemeal
- Beef, roast
- Vanille cream, cooked, Custard, Crème dessert
- Pasta in conch form
- Nuts
- Sauce, carbonara
- Fig, dried
- Pasta in butterfly form, farfalle
- Minced meat
- Carrot, steamed, without addition of salt
- Ebly
- Damson plum
- Shoots
- Bouquet garni
- Coconut
- Banana cake
- Waffle
- Apricot, dried
- Sauce, curry
- Watermelon, fresh
- Sauce, sweet-salted (asian)
- Pork, roast
- Blackberry
- Smoked cooked sausage of pork and beef meat sausag
- bean seeds
- Italian salad dressing
- White asparagus
- Pie, rhubarb, baked, with cake dough
- Tomato, stewed, without addition of fat, without addition of salt
- Cherries
- Nectarine

In [10]:

# Getting all categoriy with respective to their total images
no_images_per_category = {}

for n, i in enumerate(train_coco.getCatIds()):
  imgIds = train_coco.getImgIds(catIds=i)
  label = category_names[n]
  no_images_per_category[label] = len(imgIds)

img_info = pd.DataFrame(train_coco.loadImgs(train_coco.getImgIds()))
no_images_per_category = OrderedDict(sorted(no_images_per_category.items(), key=lambda x: -1*x[1]))

# Top 30 categories, based on number of images
i = 0
for k, v in no_images_per_category.items():
  print(k, v)
  i += 1
  if i > 30:
    break

Water 2928
Salad, leaf / salad, green 2002
Bread, white 1891
Tomato, raw  1865
Butter 1601
Carrot, raw 1482
Bread, wholemeal 1452
Coffee, with caffeine 1406
Rice 1024
Egg 1015
Mixed vegetables 892
Apple 892
Jam 797
Cucumber 742
Wine, red 728
Banana 654
Cheese 646
Potatoes steamed 644
Bell pepper, red, raw  549
Hard cheese 547
Espresso, with caffeine 547
Tea 516
Bread, whole wheat 504
Mixed salad (chopped without sauce) 498
Avocado 480
White coffee, with caffeine 470
Tomato sauce 466
Wine, white 430
Broccoli 421
Strawberries 412
Pasta, spaghetti 398

Data Visualisations¶

In [15]:

fig = go.Figure([go.Bar(x=list(no_images_per_category.keys())[:50], y=list(no_images_per_category.values())[:50])])
fig.update_layout(
    title="No of Image per class",)

fig.show()

fig = go.Figure([go.Bar(x=list(no_images_per_category.keys())[50:200], y=list(no_images_per_category.values())[50:200])])
fig.update_layout(
    title="No of Image per class",)

fig.show()

fig = go.Figure([go.Bar(x=list(no_images_per_category.keys())[200:], y=list(no_images_per_category.values())[200:])])
fig.update_layout(
    title="No of Image per class",)

fig.show()

In [16]:

pprint(f"Average number of image per class : { sum(list(no_images_per_category.values())) / len(list(no_images_per_category.values())) }")
pprint(f"Highest number of image per class is : { list(no_images_per_category.keys())[0]} of { list(no_images_per_category.values())[0] }")
pprint(f"Lowest number of image per class is : Veggie Burger of { sorted(list(no_images_per_category.values()))[0] }")

'Average number of image per class : 141.359437751004'
'Highest number of image per class is : Water of 2928'
'Lowest number of image per class is : Veggie Burger of 12'

In [17]:

fig = go.Figure(data=[go.Pie(labels=list(no_images_per_category.keys()), values=list(no_images_per_category.values()), 
                             hole=.3, textposition='inside', )], )
fig.update_layout(
    title="No of Image per class ( In pie )",)
fig.show()

In [18]:

fig = go.Figure()
fig.add_trace(go.Histogram(x=img_info['height']))
fig.add_trace(go.Histogram(x=img_info['width']))

# Overlay both histograms
fig.update_layout(barmode='stack', title="Histogram of Image width & height",)


fig.show()

Image Visulisation 🖼️¶

In this section we are going to do imaghe visualisations!

In [19]:

print(img_info)

           id   file_name  width  height
0      131094  131094.jpg    480     480
1      131097  131097.jpg    391     390
2      131098  131098.jpg    391     390
3      131100  131100.jpg    391     390
4      131101  131101.jpg    391     390
...       ...         ...    ...     ...
39957  131017  131017.jpg    480     480
39958  131021  131021.jpg    464     464
39959  131053  131053.jpg    391     390
39960  131066  131066.jpg    464     464
39961  131071  131071.jpg    464     464

[39962 rows x 4 columns]

In [20]:

len(train_annotations_data['annotations'][n]['segmentation']), len(train_annotations_data['annotations'][n]['bbox'])

Out[20]:

(1, 4)

In [11]:

img_no = 7

annIds = train_coco.getAnnIds(imgIds=train_annotations_data['images'][img_no]['id'])
anns = train_coco.loadAnns(annIds)

# load and render the image
plt.imshow(plt.imread(TRAIN_IMAGE_DIRECTIORY+train_annotations_data['images'][img_no]['file_name']))
plt.axis('off')
# Render annotations on top of the image
train_coco.showAnns(anns)

In [22]:

w, h = 15, 15 # Setting width and height of every image
rows, cols = 5, 5 # Setting the number of image rows & cols

fig = plt.figure(figsize=(15, 15)) # Making the figure with size 

plt.title("Images") 
plt.axis('off')

# Going thought every cell in rows and cols
for i in range(1, cols * rows+1):
  annIds = train_coco.getAnnIds(imgIds=img_info['id'][i])
  anns = train_coco.loadAnns(annIds)

  fig.add_subplot(rows, cols, i)

  # Show the image

  img = plt.imread(TRAIN_IMAGE_DIRECTIORY+img_info['file_name'][i])
  for i in anns:
    [x,y,w,h] = i['bbox']
    cv2.rectangle(img, (int(x), int(y)), (int(x+h), int(y+w)), (255,0,0), 2)
  plt.imshow(img)

  # Render annotations on top of the image
  train_coco.showAnns(anns)

  # Setting the axis off
  plt.axis("off")

# Showing the figure
plt.show()

Creating our Dataset 🔨¶

In this section we are goind to fix out dataset first ( because there is some issues with dataset ( size mismatch ) & creating our dataset to put into the model

Fixing the Data¶

In [23]:

np.array(train_annotations_data['annotations'][n]['segmentation']).shape , np.array(train_annotations_data['annotations'][n]['bbox']).shape

Out[23]:

((1, 38), (4,))

In [12]:

# Function for taking a annotation & directiory of images and returning new annoation json with fixed image size info
def fix_data(annotations, directiory):
  for n, i in enumerate(tqdm((annotations['images']))):
   
      img = cv2.imread(directiory+i["file_name"])
 
      if img.shape[0] != i['height']:
          annotations['images'][n]['height'] = img.shape[0]
          print(i["file_name"])
          print(annotations['images'][n], img.shape)

      if img.shape[1] != i['width']:
          annotations['images'][n]['width'] = img.shape[1]
          print(i["file_name"])
          print(annotations['images'][n], img.shape)

  return annotations

train_annotations_data = fix_data(train_annotations_data, TRAIN_IMAGE_DIRECTIORY)

with open('data/train/new_ann.json', 'w') as f:
    json.dump(train_annotations_data, f)

008618.jpg
{'id': 8618, 'file_name': '008618.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008618.jpg
{'id': 8618, 'file_name': '008618.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008619.jpg
{'id': 8619, 'file_name': '008619.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008619.jpg
{'id': 8619, 'file_name': '008619.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008620.jpg
{'id': 8620, 'file_name': '008620.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008620.jpg
{'id': 8620, 'file_name': '008620.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008621.jpg
{'id': 8621, 'file_name': '008621.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008621.jpg
{'id': 8621, 'file_name': '008621.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008626.jpg
{'id': 8626, 'file_name': '008626.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008626.jpg
{'id': 8626, 'file_name': '008626.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008627.jpg
{'id': 8627, 'file_name': '008627.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008627.jpg
{'id': 8627, 'file_name': '008627.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008628.jpg
{'id': 8628, 'file_name': '008628.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008628.jpg
{'id': 8628, 'file_name': '008628.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008817.jpg
{'id': 8817, 'file_name': '008817.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008817.jpg
{'id': 8817, 'file_name': '008817.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008864.jpg
{'id': 8864, 'file_name': '008864.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008864.jpg
{'id': 8864, 'file_name': '008864.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008869.jpg
{'id': 8869, 'file_name': '008869.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008869.jpg
{'id': 8869, 'file_name': '008869.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008919.jpg
{'id': 8919, 'file_name': '008919.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008919.jpg
{'id': 8919, 'file_name': '008919.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
008934.jpg
{'id': 8934, 'file_name': '008934.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
008934.jpg
{'id': 8934, 'file_name': '008934.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
011967.jpg
{'id': 11967, 'file_name': '011967.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
011967.jpg
{'id': 11967, 'file_name': '011967.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
012045.jpg
{'id': 12045, 'file_name': '012045.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
012045.jpg
{'id': 12045, 'file_name': '012045.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
013989.jpg
{'id': 13989, 'file_name': '013989.jpg', 'width': 4032, 'height': 4032} (4032, 3024, 3)
013989.jpg
{'id': 13989, 'file_name': '013989.jpg', 'width': 3024, 'height': 4032} (4032, 3024, 3)
017312.jpg
{'id': 17312, 'file_name': '017312.jpg', 'width': 3264, 'height': 3264} (3264, 2448, 3)
017312.jpg
{'id': 17312, 'file_name': '017312.jpg', 'width': 2448, 'height': 3264} (3264, 2448, 3)
021923.jpg
{'id': 21923, 'file_name': '021923.jpg', 'width': 4128, 'height': 4128} (4128, 2322, 3)
021923.jpg
{'id': 21923, 'file_name': '021923.jpg', 'width': 2322, 'height': 4128} (4128, 2322, 3)
023295.jpg
{'id': 23295, 'file_name': '023295.jpg', 'width': 4128, 'height': 4128} (4128, 2322, 3)
023295.jpg
{'id': 23295, 'file_name': '023295.jpg', 'width': 2322, 'height': 4128} (4128, 2322, 3)
023296.jpg
{'id': 23296, 'file_name': '023296.jpg', 'width': 4128, 'height': 4128} (4128, 2322, 3)
023296.jpg
{'id': 23296, 'file_name': '023296.jpg', 'width': 2322, 'height': 4128} (4128, 2322, 3)
023975.jpg
{'id': 23975, 'file_name': '023975.jpg', 'width': 4128, 'height': 4128} (4128, 2322, 3)
023975.jpg
{'id': 23975, 'file_name': '023975.jpg', 'width': 2322, 'height': 4128} (4128, 2322, 3)
023976.jpg
{'id': 23976, 'file_name': '023976.jpg', 'width': 4128, 'height': 4128} (4128, 2322, 3)
023976.jpg
{'id': 23976, 'file_name': '023976.jpg', 'width': 2322, 'height': 4128} (4128, 2322, 3)
049396.jpg
{'id': 49396, 'file_name': '049396.jpg', 'width': 4608, 'height': 4608} (4608, 3456, 3)
049396.jpg
{'id': 49396, 'file_name': '049396.jpg', 'width': 3456, 'height': 4608} (4608, 3456, 3)
050167.jpg
{'id': 50167, 'file_name': '050167.jpg', 'width': 4032, 'height': 4032} (4032, 1960, 3)
050167.jpg
{'id': 50167, 'file_name': '050167.jpg', 'width': 1960, 'height': 4032} (4032, 1960, 3)
053873.jpg
{'id': 53873, 'file_name': '053873.jpg', 'width': 4608, 'height': 4608} (4608, 3456, 3)
053873.jpg
{'id': 53873, 'file_name': '053873.jpg', 'width': 3456, 'height': 4608} (4608, 3456, 3)
053875.jpg
{'id': 53875, 'file_name': '053875.jpg', 'width': 4608, 'height': 4608} (4608, 3456, 3)
053875.jpg
{'id': 53875, 'file_name': '053875.jpg', 'width': 3456, 'height': 4608} (4608, 3456, 3)
053879.jpg
{'id': 53879, 'file_name': '053879.jpg', 'width': 4608, 'height': 4608} (4608, 3456, 3)
053879.jpg
{'id': 53879, 'file_name': '053879.jpg', 'width': 3456, 'height': 4608} (4608, 3456, 3)
056091.jpg
{'id': 56091, 'file_name': '056091.jpg', 'width': 4608, 'height': 4608} (4608, 3456, 3)
056091.jpg
{'id': 56091, 'file_name': '056091.jpg', 'width': 3456, 'height': 4608} (4608, 3456, 3)

In [13]:

def fix_data_val(annotations, directiory):
  for n, i in enumerate(tqdm((annotations['images']))):
      img = cv2.imread(directiory+i["file_name"])
      if img.shape[0] != i['height']:
          print(n)
          annotations['images'][n]['height'] = img.shape[0]
          print(i["file_name"])
          print(annotations['images'][n], img.shape)

      if img.shape[1] != i['width']:
          annotations['images'][n]['width'] = img.shape[1]
          print(i["file_name"])
          print(annotations['images'][n], img.shape)

  return annotations

val_annotations_data = fix_data_val(val_annotations_data, VAL_IMAGE_DIRECTIORY)

with open('data/val/new_ann.json', 'w') as f:
    json.dump(val_annotations_data, f)

In [14]:

dict_addres = val_annotations_data['images'][748]
plt.imread(VAL_IMAGE_DIRECTIORY+dict_addres['file_name']).shape[:2], (dict_addres['height'], dict_addres['width'])

Out[14]:

((918, 919), (918, 919))

Loading Dataset¶

In [15]:

train_annotations_path = 'data/train/new_ann.json'
train_images_path = 'data/train/images'

val_annotations_path = 'data/val/new_ann.json'
val_images_path = 'data/val/images'

In [16]:

register_coco_instances("training_dataset", {},train_annotations_path, train_images_path)
register_coco_instances("validation_dataset", {},val_annotations_path, VAL_IMAGE_DIRECTIORY)

Creating our Model 🏭¶

We are going to make an Faster R-CNN Model using Detectron2, and setting up hyperpamaters to train our model

Creating R-CNN Model¶

In [17]:

cfg = get_cfg()
# Check the model zoo and use any of the models ( from detectron2 github repo)
cfg.merge_from_file(model_zoo.get_config_file("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml"))
cfg.DATASETS.TRAIN = ("training_dataset",)
cfg.DATASETS.TEST = ()

cfg.DATALOADER.NUM_WORKERS = 2
# Loading pre trained weights
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml")

Setting up hyperparameters¶

In [18]:

# No. of Batchs
cfg.SOLVER.IMS_PER_BATCH = 10

# Learning Rate: 
cfg.SOLVER.BASE_LR = 0.0025

# No of Interations
cfg.SOLVER.MAX_ITER = 50001

# Images per batch (Batch Size) 
cfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 128   

# No of Categories(Classes) present
cfg.MODEL.ROI_HEADS.NUM_CLASSES = 498

cfg.OUTPUT_DIR = "logs/"
os.makedirs(cfg.OUTPUT_DIR, exist_ok=True)

In [25]:

trainer = DefaultTrainer(cfg) 
trainer.resume_or_load(resume=False)

[12/21 07:39:41 d2.engine.defaults]: Model:
GeneralizedRCNN(
  (backbone): FPN(
    (fpn_lateral2): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1))
    (fpn_output2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (fpn_lateral3): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1))
    (fpn_output3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (fpn_lateral4): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1))
    (fpn_output4): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (fpn_lateral5): Conv2d(2048, 256, kernel_size=(1, 1), stride=(1, 1))
    (fpn_output5): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (top_block): LastLevelMaxPool()
    (bottom_up): ResNet(
      (stem): BasicStem(
        (conv1): Conv2d(
          3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False
          (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
        )
      )
      (res2): Sequential(
        (0): BottleneckBlock(
          (shortcut): Conv2d(
            64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv1): Conv2d(
            64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
          )
          (conv2): Conv2d(
            64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
          )
          (conv3): Conv2d(
            64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
        )
        (1): BottleneckBlock(
          (conv1): Conv2d(
            256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
          )
          (conv2): Conv2d(
            64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
          )
          (conv3): Conv2d(
            64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
        )
        (2): BottleneckBlock(
          (conv1): Conv2d(
            256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
          )
          (conv2): Conv2d(
            64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
          )
          (conv3): Conv2d(
            64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
        )
      )
      (res3): Sequential(
        (0): BottleneckBlock(
          (shortcut): Conv2d(
            256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
          (conv1): Conv2d(
            256, 128, kernel_size=(1, 1), stride=(2, 2), bias=False
            (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
          )
          (conv2): Conv2d(
            128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
          )
          (conv3): Conv2d(
            128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
        )
        (1): BottleneckBlock(
          (conv1): Conv2d(
            512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
          )
          (conv2): Conv2d(
            128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
          )
          (conv3): Conv2d(
            128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
        )
        (2): BottleneckBlock(
          (conv1): Conv2d(
            512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
          )
          (conv2): Conv2d(
            128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
          )
          (conv3): Conv2d(
            128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
        )
        (3): BottleneckBlock(
          (conv1): Conv2d(
            512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
          )
          (conv2): Conv2d(
            128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
          )
          (conv3): Conv2d(
            128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
        )
      )
      (res4): Sequential(
        (0): BottleneckBlock(
          (shortcut): Conv2d(
            512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False
            (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
          )
          (conv1): Conv2d(
            512, 256, kernel_size=(1, 1), stride=(2, 2), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv2): Conv2d(
            256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv3): Conv2d(
            256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
          )
        )
        (1): BottleneckBlock(
          (conv1): Conv2d(
            1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv2): Conv2d(
            256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv3): Conv2d(
            256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
          )
        )
        (2): BottleneckBlock(
          (conv1): Conv2d(
            1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv2): Conv2d(
            256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv3): Conv2d(
            256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
          )
        )
        (3): BottleneckBlock(
          (conv1): Conv2d(
            1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv2): Conv2d(
            256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv3): Conv2d(
            256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
          )
        )
        (4): BottleneckBlock(
          (conv1): Conv2d(
            1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv2): Conv2d(
            256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv3): Conv2d(
            256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
          )
        )
        (5): BottleneckBlock(
          (conv1): Conv2d(
            1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv2): Conv2d(
            256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv3): Conv2d(
            256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
          )
        )
      )
      (res5): Sequential(
        (0): BottleneckBlock(
          (shortcut): Conv2d(
            1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False
            (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
          )
          (conv1): Conv2d(
            1024, 512, kernel_size=(1, 1), stride=(2, 2), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
          (conv2): Conv2d(
            512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
          (conv3): Conv2d(
            512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
          )
        )
        (1): BottleneckBlock(
          (conv1): Conv2d(
            2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
          (conv2): Conv2d(
            512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
          (conv3): Conv2d(
            512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
          )
        )
        (2): BottleneckBlock(
          (conv1): Conv2d(
            2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
          (conv2): Conv2d(
            512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
          (conv3): Conv2d(
            512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
          )
        )
      )
    )
  )
  (proposal_generator): RPN(
    (rpn_head): StandardRPNHead(
      (conv): Conv2d(
        256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)
        (activation): ReLU()
      )
      (objectness_logits): Conv2d(256, 3, kernel_size=(1, 1), stride=(1, 1))
      (anchor_deltas): Conv2d(256, 12, kernel_size=(1, 1), stride=(1, 1))
    )
    (anchor_generator): DefaultAnchorGenerator(
      (cell_anchors): BufferList()
    )
  )
  (roi_heads): StandardROIHeads(
    (box_pooler): ROIPooler(
      (level_poolers): ModuleList(
        (0): ROIAlign(output_size=(7, 7), spatial_scale=0.25, sampling_ratio=0, aligned=True)
        (1): ROIAlign(output_size=(7, 7), spatial_scale=0.125, sampling_ratio=0, aligned=True)
        (2): ROIAlign(output_size=(7, 7), spatial_scale=0.0625, sampling_ratio=0, aligned=True)
        (3): ROIAlign(output_size=(7, 7), spatial_scale=0.03125, sampling_ratio=0, aligned=True)
      )
    )
    (box_head): FastRCNNConvFCHead(
      (flatten): Flatten(start_dim=1, end_dim=-1)
      (fc1): Linear(in_features=12544, out_features=1024, bias=True)
      (fc_relu1): ReLU()
      (fc2): Linear(in_features=1024, out_features=1024, bias=True)
      (fc_relu2): ReLU()
    )
    (box_predictor): FastRCNNOutputLayers(
      (cls_score): Linear(in_features=1024, out_features=499, bias=True)
      (bbox_pred): Linear(in_features=1024, out_features=1992, bias=True)
    )
    (mask_pooler): ROIPooler(
      (level_poolers): ModuleList(
        (0): ROIAlign(output_size=(14, 14), spatial_scale=0.25, sampling_ratio=0, aligned=True)
        (1): ROIAlign(output_size=(14, 14), spatial_scale=0.125, sampling_ratio=0, aligned=True)
        (2): ROIAlign(output_size=(14, 14), spatial_scale=0.0625, sampling_ratio=0, aligned=True)
        (3): ROIAlign(output_size=(14, 14), spatial_scale=0.03125, sampling_ratio=0, aligned=True)
      )
    )
    (mask_head): MaskRCNNConvUpsampleHead(
      (mask_fcn1): Conv2d(
        256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)
        (activation): ReLU()
      )
      (mask_fcn2): Conv2d(
        256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)
        (activation): ReLU()
      )
      (mask_fcn3): Conv2d(
        256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)
        (activation): ReLU()
      )
      (mask_fcn4): Conv2d(
        256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1)
        (activation): ReLU()
      )
      (deconv): ConvTranspose2d(256, 256, kernel_size=(2, 2), stride=(2, 2))
      (deconv_relu): ReLU()
      (predictor): Conv2d(256, 498, kernel_size=(1, 1), stride=(1, 1))
    )
  )
)
[12/21 07:39:45 d2.data.datasets.coco]: Loading data/train/new_ann.json takes 4.09 seconds.
WARNING [12/21 07:39:45 d2.data.datasets.coco]: 
Category ids in annotations are not in [1, #categories]! We'll apply a mapping for you.

[12/21 07:39:46 d2.data.datasets.coco]: Loaded 39962 images in COCO format from data/train/new_ann.json
[12/21 07:39:47 d2.data.build]: Removed 0 images with no usable annotations. 39962 images left.
[12/21 07:39:50 d2.data.dataset_mapper]: [DatasetMapper] Augmentations used in training: [ResizeShortestEdge(short_edge_length=(640, 672, 704, 736, 768, 800), max_size=1333, sample_style='choice'), RandomFlip()]
[12/21 07:39:50 d2.data.build]: Using training sampler TrainingSampler
[12/21 07:39:50 d2.data.common]: Serializing 39962 elements to byte tensors and concatenating them all ...
[12/21 07:39:51 d2.data.common]: Serialized dataset takes 117.68 MiB
WARNING [12/21 07:39:51 d2.solver.build]: SOLVER.STEPS contains values larger than SOLVER.MAX_ITER. These values will be ignored.

Skip loading parameter 'roi_heads.box_predictor.cls_score.weight' to the model due to incompatible shapes: (81, 1024) in the checkpoint but (499, 1024) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.cls_score.bias' to the model due to incompatible shapes: (81,) in the checkpoint but (499,) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.bbox_pred.weight' to the model due to incompatible shapes: (320, 1024) in the checkpoint but (1992, 1024) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.bbox_pred.bias' to the model due to incompatible shapes: (320,) in the checkpoint but (1992,) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.mask_head.predictor.weight' to the model due to incompatible shapes: (80, 256, 1, 1) in the checkpoint but (498, 256, 1, 1) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.mask_head.predictor.bias' to the model due to incompatible shapes: (80,) in the checkpoint but (498,) in the model! You might want to double check if this is expected.
Some model parameters or buffers are not found in the checkpoint:
roi_heads.box_predictor.bbox_pred.{bias, weight}
roi_heads.box_predictor.cls_score.{bias, weight}
roi_heads.mask_head.predictor.{bias, weight}

Training the Model 🚂¶

Setting up Tensorboard & finally training our model!

Setting up Tensorboard¶

In [20]:

%load_ext tensorboard

In [21]:

#%tensorboard --logdir logs

Start Training!¶

In [ ]:

trainer.train()

Evaluating the model 🧪¶

After training is done, we will evaluate our model to see how it's performing in the unseen images!

In [ ]:

cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.01

cfg.MODEL.WEIGHTS = os.path.join(cfg.OUTPUT_DIR, "model_final.pth")

evaluator = COCOEvaluator("validation_dataset", cfg, False, output_dir=cfg.OUTPUT_DIR)
val_loader = build_detection_test_loader(cfg, "validation_dataset")
valResults = inference_on_dataset(trainer.model, val_loader, evaluator)

WARNING [12/20 22:27:53 d2.data.datasets.coco]: 
Category ids in annotations are not in [1, #categories]! We'll apply a mapping for you.

[12/20 22:27:53 d2.data.datasets.coco]: Loaded 1000 images in COCO format from data/val/new_ann.json
[12/20 22:27:53 d2.data.build]: Distribution of instances among all 498 categories:
|   category    | #instances   |   category    | #instances   |   category    | #instances   |
|:-------------:|:-------------|:-------------:|:-------------|:-------------:|:-------------|
| beetroot-st.. | 5            | carrot-stea.. | 2            | mushroom-av.. | 2            |
| savoy-cabba.. | 0            | zucchini-st.. | 0            | vanille-cre.. | 0            |
| shrimp-boiled | 3            | spinach-ste.. | 2            |  spinach-raw  | 2            |
| green-bean-.. | 1            | sweet-corn-.. | 2            | watermelon-.. | 5            |
| pizza-with-.. | 1            | pizza-with-.. | 1            | pizza-with-.. | 2            |
| bell-pepper.. | 1            | applesauce-.. | 1            | quiche-with.. | 2            |
| quiche-with.. | 1            | risotto-wit.. | 0            | potato-sala.. | 1            |
| wienerli-sw.. | 5            | curds-natur.. | 2            | tuna-in-oil.. | 0            |
| tomato-stew.. | 2            | sweet-potato  | 3            | country-fries | 3            |
| potato-gnoc.. | 1            | potatoes-st.. | 8            | chips-frenc.. | 9            |
|     rosti     | 2            | vegetable-m.. | 1            | coleslaw-ch.. | 0            |
|  vegetables   | 7            | mixed-veget.. | 13           |  ratatouille  | 3            |
| mixed-salad.. | 12           | leaf-spinach  | 8            | witloof-chi.. | 2            |
| salad-rocket  | 4            | salad-leaf-.. | 62           | salad-lambs.. | 6            |
|   artichoke   | 0            |   eggplant    | 7            |    avocado    | 9            |
| french-beans  | 8            | cucumber-pi.. | 2            |   cucumber    | 26           |
|    capers     | 1            |    pumpkin    | 3            | bell-pepper.. | 19           |
|  tomato-raw   | 60           |   zucchini    | 9            |  red-radish   | 8            |
| beetroot-raw  | 3            | white-radish  | 0            |  carrot-raw   | 27           |
|   celeriac    | 0            |  cauliflower  | 7            |   broccoli    | 10           |
| chinese-cab.. | 4            |   kolhrabi    | 3            |   romanesco   | 2            |
| brussel-spr.. | 2            |  red-cabbage  | 1            |  sauerkraut   | 2            |
| white-cabbage | 0            |   mushroom    | 6            |   mushrooms   | 4            |
|     peas      | 2            |     corn      | 3            | spring-onio.. | 2            |
|    garlic     | 1            |     leek      | 5            | pearl-onions  | 0            |
|     onion     | 13           |    fennel     | 3            |  swiss-chard  | 3            |
| mungbean-sp.. | 0            | green-aspar.. | 2            | white-aspar.. | 0            |
|    shoots     | 1            | alfa-sprouts  | 0            |    celery     | 2            |
|  bean-seeds   | 0            | beans-kidney  | 3            |  beans-white  | 1            |
|   chickpeas   | 10           |    lentils    | 2            |   pineapple   | 3            |
|     apple     | 18           |  pomegranate  | 3            |   apricots    | 2            |
|    banana     | 7            |    berries    | 2            |     pear      | 5            |
|  blackberry   | 0            |     dates     | 3            | strawberries  | 12           |
|     figs      | 3            |  fruit-salad  | 1            | grapefruit-.. | 3            |
|  blueberries  | 13           |  raspberries  | 7            |     kaki      | 2            |
|   cherries    | 3            |     kiwi      | 6            |   mandarine   | 5            |
|     mango     | 1            |  sugar-melon  | 4            |   nectarine   | 3            |
|    orange     | 9            |     peach     | 3            |     plums     | 7            |
|    grapes     | 10           | dried-raisins | 1            |     lemon     | 2            |
|  damson-plum  | 1            | peanut-butter | 3            |   chestnuts   | 4            |
|     seeds     | 1            | pumpkin-seeds | 1            |   pine-nuts   | 2            |
| sunflower-s.. | 2            |  mixed-seeds  | 0            |    almonds    | 4            |
|     nuts      | 0            |    walnut     | 4            |  cashew-nut   | 3            |
|    peanut     | 2            |   hazelnut    | 4            |    coconut    | 1            |
|  brazil-nut   | 2            |   pecan-nut   | 1            |  mixed-nuts   | 7            |
|   pistachio   | 2            |   linseeds    | 1            | sesame-seeds  | 2            |
| green-olives  | 2            | black-olives  | 8            |     milk      | 6            |
|  kefir-drink  | 1            | coconut-milk  | 3            | soya-drink-.. | 1            |
| soya-yaourt.. | 3            | cottage-che.. | 4            |     curd      | 0            |
| blue-mould-.. | 1            |     brie      | 1            |    cheddar    | 1            |
| emmental-ch.. | 4            |     feta      | 8            |    fondue     | 5            |
| fresh-cheese  | 6            |    gruyere    | 8            | semi-hard-c.. | 2            |
|   halloumi    | 3            |  hard-cheese  | 12           |    cheese     | 17           |
|  mozzarella   | 8            |   parmesan    | 11           | philadelphia  | 4            |
| cheese-for-.. | 1            | cream-cheese  | 1            | processed-c.. | 1            |
| tete-de-moine | 3            |     tomme     | 1            |  soft-cheese  | 6            |
|    mousse     | 0            |  panna-cotta  | 0            |   tiramisu    | 1            |
|     cream     | 5            |  sour-cream   | 3            | thickened-c.. | 0            |
| dairy-ice-c.. | 4            |    sorbet     | 1            |  flakes-oat   | 1            |
| rice-noodle.. | 1            |    bulgur     | 2            |   couscous    | 2            |
|     ebly      | 1            | grits-polen.. | 5            |    quinoa     | 3            |
|     rice      | 24           | rice-basmati  | 3            |  rice-jasmin  | 0            |
| rice-whole-.. | 2            |   rice-wild   | 1            |   spaetzle    | 2            |
|     pasta     | 13           | pasta-hornli  | 1            | pasta-in-bu.. | 1            |
| pasta-lingu.. | 4            | pasta-in-co.. | 1            | pasta-noodles | 2            |
|  pasta-penne  | 6            | pasta-ravio.. | 2            | pasta-spagh.. | 5            |
|  pasta-twist  | 3            | pasta-torte.. | 0            | pasta-whole.. | 1            |
| bagel-witho.. | 1            | bread-frenc.. | 2            |     bread     | 12           |
| bread-5-grain | 0            |  bread-spelt  | 1            |  bread-fruit  | 2            |
| bread-half-.. | 1            |  bread-grain  | 8            |   bread-nut   | 0            |
|  bread-olive  | 2            |  bread-pita   | 2            |   bread-rye   | 2            |
| bread-whole.. | 23           | bread-sourd.. | 3            |  bread-black  | 2            |
| bread-ticino  | 1            |  bread-toast  | 10           | bread-whole.. | 4            |
| bread-whole.. | 36           |  bread-white  | 39           |    brioche    | 2            |
| roll-of-hal.. | 2            | hamburger-bun | 2            | roll-with-p.. | 2            |
| white-bread.. | 6            |   focaccia    | 2            |   croissant   | 7            |
| croissant-w.. | 1            | lye-pretzel.. | 3            | braided-whi.. | 13           |
| crisp-bread.. | 0            | breadcrumbs.. | 1            | rice-waffels  | 3            |
|   grissini    | 1            | rusk-wholem.. | 1            |  corn-flakes  | 0            |
| crunch-muesli | 3            |    muesli     | 4            | dough-puff-.. | 0            |
| pastry-flaky  | 0            |     meat      | 3            |  minced-meat  | 0            |
|     beef      | 2            |  beef-roast   | 1            | beef-sirloi.. | 2            |
|  beef-filet   | 0            | beef-minced.. | 1            | beef-cut-in.. | 3            |
|     pork      | 1            |  pork-roast   | 1            |   pork-chop   | 0            |
| pork-escalope | 0            |     lamb      | 0            |   lamb-chop   | 1            |
|    chicken    | 6            | chicken-bre.. | 3            | chicken-wing  | 0            |
| chicken-cut.. | 2            |  chicken-leg  | 1            | frying-saus.. | 1            |
|   cervelat    | 1            | chicken-nug.. | 2            |    chorizo    | 1            |
|   meatloaf    | 2            |   hamburger   | 4            |  dried-meat   | 3            |
| veal-sausage  | 1            | processed-m.. | 0            |    salami     | 6            |
| cooked-saus.. | 0            |      ham      | 5            |  ham-cooked   | 5            |
|    ham-raw    | 6            |  ham-turkey   | 1            | smoked-cook.. | 0            |
|     bacon     | 0            | bacon-frying  | 3            | bacon-cooking | 3            |
|   bacon-raw   | 1            | meat-terrin.. | 2            |    sausage    | 2            |
| veggie-burger | 2            |     tofu      | 6            |     fish      | 1            |
|      cod      | 1            |    salmon     | 7            |   anchovies   | 2            |
|     tuna      | 0            | shrimp-praw.. | 0            | shrimp-praw.. | 3            |
| fish-crunch.. | 0            | fish-finger.. | 3            |      egg      | 21           |
|      oil      | 1            |    butter     | 41           |  butter-herb  | 1            |
|   margarine   | 1            |    praline    | 1            |      jam      | 21           |
|     honey     | 5            | sugar-glazing | 0            | maple-syrup.. | 1            |
| dark-chocol.. | 8            | milk-chocol.. | 4            | white-choco.. | 0            |
|   chocolate   | 2            | chocolate-f.. | 0            | cocoa-powder  | 0            |
| hazelnut-ch.. | 5            |     m-m-s     | 4            | chocolate-e.. | 1            |
| sweets-cand.. | 1            | gummi-bears.. | 0            |   apple-pie   | 1            |
|    brownie    | 0            |  cake-oblong  | 3            |  lemon-cake   | 0            |
|  crepe-plain  | 6            |  fruit-tart   | 1            |  cake-marble  | 1            |
| cake-chocol.. | 2            |    muffin     | 4            | omelette-pl.. | 1            |
|  carrot-cake  | 3            | black-fores.. | 3            |     tart      | 1            |
|    waffle     | 0            | croissant-w.. | 1            |    cookies    | 2            |
|   biscuits    | 3            |   macaroon    | 2            |   meringue    | 3            |
| biscuit-wit.. | 1            | chocolate-c.. | 2            |  juice-apple  | 4            |
| juice-multi.. | 1            | juice-orange  | 0            |   smoothie    | 0            |
|   coca-cola   | 0            | coca-cola-z.. | 4            |    ice-tea    | 0            |
| syrup-dilut.. | 7            |      tea      | 7            |  cappuccino   | 5            |
| espresso-wi.. | 14           | coffee-with.. | 27           | coffee-deca.. | 2            |
| latte-macch.. | 1            | white-coffe.. | 17           | ristretto-w.. | 5            |
|   tea-green   | 2            |   tea-black   | 0            | tea-verveine  | 0            |
|   tea-fruit   | 0            |   tea-spice   | 0            |  tea-ginger   | 0            |
|  herbal-tea   | 4            | tea-pepperm.. | 2            |  tea-rooibos  | 1            |
|   ice-cubes   | 0            |     water     | 77           | water-mineral | 5            |
| aperitif-wi.. | 0            |   cocktail    | 1            |   champagne   | 1            |
|   prosecco    | 1            |     sekt      | 1            |   wine-rose   | 3            |
|   wine-red    | 8            |  wine-white   | 16           |     beer      | 5            |
|  light-beer   | 2            | sauce-savoury | 1            |  sauce-roast  | 1            |
| sauce-carbo.. | 0            | sauce-cockt.. | 1            |  sauce-curry  | 1            |
|  sauce-pesto  | 2            | sauce-mushr.. | 2            |  sauce-cream  | 7            |
| sauce-sweet.. | 1            |    ketchup    | 7            | bolognaise-.. | 2            |
| tomato-sauce  | 4            |     dips      | 0            | salad-dress.. | 3            |
| balsamic-sa.. | 4            | french-sala.. | 9            | italian-sal.. | 2            |
| oil-vinegar.. | 1            |   guacamole   | 3            |  mayonnaise   | 2            |
| tartar-sauce  | 1            |   tzatziki    | 3            |     basil     | 2            |
|   coriander   | 1            | bouquet-garni | 2            |    parsley    | 1            |
|    chives     | 3            | cenovis-yea.. | 0            |  sauce-soya   | 1            |
|    mustard    | 5            | mustard-dijon | 0            | balsamic-vi.. | 0            |
| soup-vegeta.. | 3            | soup-cream-.. | 2            |  soup-potato  | 1            |
| soup-pumpkin  | 3            |   soup-miso   | 1            |  soup-tomato  | 2            |
|   bouillon    | 2            | bouillon-ve.. | 1            | falafel-balls | 3            |
| savoury-puf.. | 0            | savoury-puf.. | 2            |  corn-crisps  | 0            |
|   crackers    | 1            |   croutons    | 1            |    crisps     | 3            |
| popcorn-sal.. | 3            | croque-mons.. | 6            | spring-roll.. | 2            |
| ham-croissant | 0            | salt-cake-v.. | 0            | hamburger-b.. | 4            |
| cordon-bleu.. | 2            | lasagne-mea.. | 3            | mashed-pota.. | 4            |
| pizza-margh.. | 4            | sandwich-ha.. | 0            |     sushi     | 5            |
| kebab-in-pi.. | 2            |   pancakes    | 0            |    hummus     | 13           |
|  greek-salad  | 1            |   dumplings   | 0            | apricot-dried | 1            |
| chocolate-m.. | 0            |  cheesecake   | 1            | caprese-sal.. | 1            |
| chili-con-c.. | 1            | taboule-pre.. | 0            | perch-fille.. | 0            |
| risotto-wit.. | 2            | salmon-smoked | 11           | pie-apricot.. | 2            |
| eggplant-ca.. | 1            | apple-crumble | 0            | egg-scrambl.. | 6            |
|   oat-milk    | 1            |   lemon-pie   | 0            | glucose-dri.. | 5            |
| goat-averag.. | 0            | pie-rhubarb.. | 0            | chicken-cur.. | 0            |
| pie-plum-ba.. | 2            | potatoes-au.. | 3            | buckwheat-g.. | 0            |
| birchermues.. | 7            | fajita-brea.. | 2            |  mango-dried  | 2            |
| lentils-gre.. | 5            | naan-indien.. | 0            | butter-spre.. | 2            |
| chocolate-m.. | 0            | water-with-.. | 5            | sun-dried-t.. | 1            |
| gluten-free.. | 1            | fruit-coulis  | 1            | greek-yaour.. | 3            |
|  cake-salted  | 2            | soup-of-len.. | 0            |   fig-dried   | 5            |
| turnover-wi.. | 1            | lasagne-veg.. | 2            | sauce-sweet.. | 1            |
| french-pizz.. | 3            | fruit-compo.. | 0            | vegetable-a.. | 2            |
| porridge-pr.. | 0            | curry-veget.. | 0            | bread-meat-.. | 2            |
|  tartar-meat  | 1            |  chia-grains  | 0            | faux-mage-c.. | 4            |
| milk-chocol.. | 4            | yaourt-yaho.. | 8            | paprika-chips | 0            |
|  banana-cake  | 3            | cream-spinach | 0            | cantonese-f.. | 1            |
| goat-cheese.. | 1            | buckwheat-p.. | 1            |  meat-balls   | 2            |
| high-protei.. | 1            | mix-of-drie.. | 1            | baked-potato  | 7            |
|               |              |               |              |               |              |
|     total     | 1830         |               |              |               |              |
[12/20 22:27:53 d2.data.common]: Serializing 1000 elements to byte tensors and concatenating them all ...
[12/20 22:27:53 d2.data.common]: Serialized dataset takes 1.58 MiB
[12/20 22:27:53 d2.evaluation.evaluator]: Start inference on 1000 images
[12/20 22:27:56 d2.evaluation.evaluator]: Inference done 11/1000. 0.2827 s / img. ETA=0:04:48
[12/20 22:28:01 d2.evaluation.evaluator]: Inference done 28/1000. 0.2851 s / img. ETA=0:04:45
[12/20 22:28:06 d2.evaluation.evaluator]: Inference done 45/1000. 0.2858 s / img. ETA=0:04:40
[12/20 22:28:11 d2.evaluation.evaluator]: Inference done 62/1000. 0.2868 s / img. ETA=0:04:37
[12/20 22:28:16 d2.evaluation.evaluator]: Inference done 79/1000. 0.2871 s / img. ETA=0:04:32
[12/20 22:28:21 d2.evaluation.evaluator]: Inference done 96/1000. 0.2869 s / img. ETA=0:04:27
[12/20 22:28:26 d2.evaluation.evaluator]: Inference done 113/1000. 0.2875 s / img. ETA=0:04:23
[12/20 22:28:32 d2.evaluation.evaluator]: Inference done 130/1000. 0.2877 s / img. ETA=0:04:18
[12/20 22:28:37 d2.evaluation.evaluator]: Inference done 147/1000. 0.2880 s / img. ETA=0:04:13
[12/20 22:28:42 d2.evaluation.evaluator]: Inference done 164/1000. 0.2883 s / img. ETA=0:04:08
[12/20 22:28:47 d2.evaluation.evaluator]: Inference done 181/1000. 0.2882 s / img. ETA=0:04:03
[12/20 22:28:52 d2.evaluation.evaluator]: Inference done 198/1000. 0.2883 s / img. ETA=0:03:58
[12/20 22:28:57 d2.evaluation.evaluator]: Inference done 215/1000. 0.2884 s / img. ETA=0:03:53
[12/20 22:29:02 d2.evaluation.evaluator]: Inference done 232/1000. 0.2888 s / img. ETA=0:03:49
[12/20 22:29:07 d2.evaluation.evaluator]: Inference done 249/1000. 0.2891 s / img. ETA=0:03:44
[12/20 22:29:12 d2.evaluation.evaluator]: Inference done 266/1000. 0.2891 s / img. ETA=0:03:39
[12/20 22:29:17 d2.evaluation.evaluator]: Inference done 283/1000. 0.2891 s / img. ETA=0:03:34
[12/20 22:29:23 d2.evaluation.evaluator]: Inference done 300/1000. 0.2890 s / img. ETA=0:03:29
[12/20 22:29:28 d2.evaluation.evaluator]: Inference done 317/1000. 0.2890 s / img. ETA=0:03:24
[12/20 22:29:33 d2.evaluation.evaluator]: Inference done 334/1000. 0.2890 s / img. ETA=0:03:18
[12/20 22:29:38 d2.evaluation.evaluator]: Inference done 351/1000. 0.2893 s / img. ETA=0:03:13
[12/20 22:29:43 d2.evaluation.evaluator]: Inference done 368/1000. 0.2898 s / img. ETA=0:03:09
[12/20 22:29:48 d2.evaluation.evaluator]: Inference done 385/1000. 0.2897 s / img. ETA=0:03:04
[12/20 22:29:53 d2.evaluation.evaluator]: Inference done 401/1000. 0.2906 s / img. ETA=0:02:59
[12/20 22:29:58 d2.evaluation.evaluator]: Inference done 418/1000. 0.2904 s / img. ETA=0:02:54
[12/20 22:30:04 d2.evaluation.evaluator]: Inference done 435/1000. 0.2905 s / img. ETA=0:02:49
[12/20 22:30:09 d2.evaluation.evaluator]: Inference done 451/1000. 0.2909 s / img. ETA=0:02:45
[12/20 22:30:14 d2.evaluation.evaluator]: Inference done 468/1000. 0.2910 s / img. ETA=0:02:40
[12/20 22:30:19 d2.evaluation.evaluator]: Inference done 485/1000. 0.2909 s / img. ETA=0:02:35
[12/20 22:30:24 d2.evaluation.evaluator]: Inference done 502/1000. 0.2909 s / img. ETA=0:02:30
[12/20 22:30:29 d2.evaluation.evaluator]: Inference done 519/1000. 0.2909 s / img. ETA=0:02:25
[12/20 22:30:34 d2.evaluation.evaluator]: Inference done 536/1000. 0.2907 s / img. ETA=0:02:19
[12/20 22:30:40 d2.evaluation.evaluator]: Inference done 553/1000. 0.2905 s / img. ETA=0:02:14
[12/20 22:30:45 d2.evaluation.evaluator]: Inference done 570/1000. 0.2905 s / img. ETA=0:02:09
[12/20 22:30:50 d2.evaluation.evaluator]: Inference done 587/1000. 0.2906 s / img. ETA=0:02:04
[12/20 22:30:55 d2.evaluation.evaluator]: Inference done 604/1000. 0.2906 s / img. ETA=0:01:59
[12/20 22:31:00 d2.evaluation.evaluator]: Inference done 621/1000. 0.2906 s / img. ETA=0:01:54
[12/20 22:31:05 d2.evaluation.evaluator]: Inference done 638/1000. 0.2906 s / img. ETA=0:01:49
[12/20 22:31:10 d2.evaluation.evaluator]: Inference done 655/1000. 0.2905 s / img. ETA=0:01:44
[12/20 22:31:16 d2.evaluation.evaluator]: Inference done 672/1000. 0.2904 s / img. ETA=0:01:38
[12/20 22:31:21 d2.evaluation.evaluator]: Inference done 689/1000. 0.2904 s / img. ETA=0:01:33
[12/20 22:31:26 d2.evaluation.evaluator]: Inference done 706/1000. 0.2903 s / img. ETA=0:01:28
[12/20 22:31:31 d2.evaluation.evaluator]: Inference done 723/1000. 0.2902 s / img. ETA=0:01:23
[12/20 22:31:36 d2.evaluation.evaluator]: Inference done 740/1000. 0.2902 s / img. ETA=0:01:18
[12/20 22:31:41 d2.evaluation.evaluator]: Inference done 757/1000. 0.2902 s / img. ETA=0:01:13
[12/20 22:31:46 d2.evaluation.evaluator]: Inference done 774/1000. 0.2901 s / img. ETA=0:01:08
[12/20 22:31:52 d2.evaluation.evaluator]: Inference done 792/1000. 0.2900 s / img. ETA=0:01:02
[12/20 22:31:57 d2.evaluation.evaluator]: Inference done 808/1000. 0.2902 s / img. ETA=0:00:57
[12/20 22:32:02 d2.evaluation.evaluator]: Inference done 825/1000. 0.2901 s / img. ETA=0:00:52
[12/20 22:32:07 d2.evaluation.evaluator]: Inference done 842/1000. 0.2901 s / img. ETA=0:00:47
[12/20 22:32:12 d2.evaluation.evaluator]: Inference done 859/1000. 0.2900 s / img. ETA=0:00:42
[12/20 22:32:17 d2.evaluation.evaluator]: Inference done 876/1000. 0.2900 s / img. ETA=0:00:37
[12/20 22:32:22 d2.evaluation.evaluator]: Inference done 893/1000. 0.2901 s / img. ETA=0:00:32
[12/20 22:32:27 d2.evaluation.evaluator]: Inference done 910/1000. 0.2900 s / img. ETA=0:00:27
[12/20 22:32:32 d2.evaluation.evaluator]: Inference done 926/1000. 0.2899 s / img. ETA=0:00:22
[12/20 22:32:37 d2.evaluation.evaluator]: Inference done 943/1000. 0.2898 s / img. ETA=0:00:17
[12/20 22:32:43 d2.evaluation.evaluator]: Inference done 960/1000. 0.2897 s / img. ETA=0:00:12
[12/20 22:32:48 d2.evaluation.evaluator]: Inference done 977/1000. 0.2897 s / img. ETA=0:00:06
[12/20 22:32:53 d2.evaluation.evaluator]: Inference done 994/1000. 0.2896 s / img. ETA=0:00:01
[12/20 22:32:55 d2.evaluation.evaluator]: Total inference time: 0:05:00.808809 (0.302320 s / img per device, on 1 devices)
[12/20 22:32:55 d2.evaluation.evaluator]: Total inference pure compute time: 0:04:48 (0.289574 s / img per device, on 1 devices)
[12/20 22:32:55 d2.evaluation.coco_evaluation]: Preparing results for COCO format ...
[12/20 22:32:55 d2.evaluation.coco_evaluation]: Saving results to /content/logs/coco_instances_results.json
[12/20 22:32:55 d2.evaluation.coco_evaluation]: Evaluating predictions ...
Loading and preparing results...
DONE (t=0.01s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=9.96s).
Accumulating evaluation results...
DONE (t=2.67s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.003
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.008
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.002
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.015
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.003
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.007
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.009
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.009
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.015
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.009
[12/20 22:33:08 d2.evaluation.coco_evaluation]: Evaluation results for bbox: 
|  AP   |  AP50  |  AP75  |  APs  |  APm  |  APl  |
|:-----:|:------:|:------:|:-----:|:-----:|:-----:|
| 0.301 | 0.825  | 0.164  | 0.000 | 1.515 | 0.318 |
[12/20 22:33:08 d2.evaluation.coco_evaluation]: Per-category bbox AP: 
| category                                                                | AP     | category                                                         | AP     | category                                                                 | AP     |
|:------------------------------------------------------------------------|:-------|:-----------------------------------------------------------------|:-------|:-------------------------------------------------------------------------|:-------|
| beetroot-steamed-without-addition-of-salt                               | 0.000  | carrot-steamed-without-addition-of-salt                          | 0.000  | mushroom-average-stewed-without-addition-of-fat-without-addition-of-salt | 0.000  |
| savoy-cabbage-steamed-without-addition-of-salt                          | nan    | zucchini-stewed-without-addition-of-fat-without-addition-of-salt | nan    | vanille-cream-cooked-custard-creme-dessert                               | nan    |
| shrimp-boiled                                                           | 0.000  | spinach-steamed-without-addition-of-salt                         | 0.000  | spinach-raw                                                              | 0.000  |
| green-bean-steamed-without-addition-of-salt                             | 0.000  | sweet-corn-canned                                                | 0.000  | watermelon-fresh                                                         | 0.000  |
| pizza-with-ham-with-mushrooms-baked                                     | 0.000  | pizza-with-ham-baked                                             | 0.000  | pizza-with-vegetables-baked                                              | 0.000  |
| bell-pepper-red-stewed-without-addition-of-fat-without-addition-of-salt | 0.000  | applesauce-unsweetened-canned                                    | 0.000  | quiche-with-cheese-baked-with-puff-pastry                                | 0.000  |
| quiche-with-spinach-baked-with-cake-dough                               | 0.000  | risotto-with-mushrooms-cooked                                    | nan    | potato-salad-with-mayonnaise-yogurt-dressing                             | 0.000  |
| wienerli-swiss-sausage                                                  | 0.000  | curds-natural-with-at-most-10-fidm                               | 0.000  | tuna-in-oil-drained                                                      | nan    |
| tomato-stewed-without-addition-of-fat-without-addition-of-salt          | 0.000  | sweet-potato                                                     | 0.000  | country-fries                                                            | 0.000  |
| potato-gnocchi                                                          | 0.000  | potatoes-steamed                                                 | 0.000  | chips-french-fries                                                       | 0.000  |
| rosti                                                                   | 0.000  | vegetable-mix-peas-and-carrots                                   | 0.000  | coleslaw-chopped-without-sauce                                           | nan    |
| vegetables                                                              | 0.000  | mixed-vegetables                                                 | 0.000  | ratatouille                                                              | 0.000  |
| mixed-salad-chopped-without-sauce                                       | 6.172  | leaf-spinach                                                     | 0.000  | witloof-chicory                                                          | 0.000  |
| salad-rocket                                                            | 0.000  | salad-leaf-salad-green                                           | 21.202 | salad-lambs-ear                                                          | 0.000  |
| artichoke                                                               | nan    | eggplant                                                         | 0.000  | avocado                                                                  | 0.000  |
| french-beans                                                            | 0.000  | cucumber-pickled                                                 | 0.000  | cucumber                                                                 | 0.000  |
| capers                                                                  | 0.000  | pumpkin                                                          | 0.000  | bell-pepper-red-raw                                                      | 0.000  |
| tomato-raw                                                              | 0.000  | zucchini                                                         | 0.000  | red-radish                                                               | 0.000  |
| beetroot-raw                                                            | 0.000  | white-radish                                                     | nan    | carrot-raw                                                               | 1.015  |
| celeriac                                                                | nan    | cauliflower                                                      | 0.000  | broccoli                                                                 | 0.000  |
| chinese-cabbage                                                         | 0.000  | kolhrabi                                                         | 0.000  | romanesco                                                                | 0.000  |
| brussel-sprouts                                                         | 0.000  | red-cabbage                                                      | 0.000  | sauerkraut                                                               | 0.000  |
| white-cabbage                                                           | nan    | mushroom                                                         | 0.000  | mushrooms                                                                | 0.000  |
| peas                                                                    | 0.000  | corn                                                             | 0.000  | spring-onion-scallion                                                    | 0.000  |
| garlic                                                                  | 0.000  | leek                                                             | 0.000  | pearl-onions                                                             | nan    |
| onion                                                                   | 0.000  | fennel                                                           | 0.000  | swiss-chard                                                              | 0.000  |
| mungbean-sprouts                                                        | nan    | green-asparagus                                                  | 0.000  | white-asparagus                                                          | nan    |
| shoots                                                                  | 0.000  | alfa-sprouts                                                     | nan    | celery                                                                   | 0.000  |
| bean-seeds                                                              | nan    | beans-kidney                                                     | 0.000  | beans-white                                                              | 0.000  |
| chickpeas                                                               | 0.000  | lentils                                                          | 0.000  | pineapple                                                                | 0.000  |
| apple                                                                   | 10.218 | pomegranate                                                      | 0.000  | apricots                                                                 | 0.000  |
| banana                                                                  | 13.809 | berries                                                          | 0.000  | pear                                                                     | 0.000  |
| blackberry                                                              | nan    | dates                                                            | 0.000  | strawberries                                                             | 0.000  |
| figs                                                                    | 0.000  | fruit-salad                                                      | 0.000  | grapefruit-pomelo                                                        | 0.000  |
| blueberries                                                             | 0.000  | raspberries                                                      | 0.000  | kaki                                                                     | 0.000  |
| cherries                                                                | 0.000  | kiwi                                                             | 0.000  | mandarine                                                                | 24.480 |
| mango                                                                   | 0.000  | sugar-melon                                                      | 0.000  | nectarine                                                                | 0.000  |
| orange                                                                  | 0.000  | peach                                                            | 0.000  | plums                                                                    | 0.000  |
| grapes                                                                  | 0.000  | dried-raisins                                                    | 0.000  | lemon                                                                    | 0.000  |
| damson-plum                                                             | 0.000  | peanut-butter                                                    | 0.000  | chestnuts                                                                | 0.000  |
| seeds                                                                   | 0.000  | pumpkin-seeds                                                    | 0.000  | pine-nuts                                                                | 0.000  |
| sunflower-seeds                                                         | 0.000  | mixed-seeds                                                      | nan    | almonds                                                                  | 0.000  |
| nuts                                                                    | nan    | walnut                                                           | 0.000  | cashew-nut                                                               | 0.000  |
| peanut                                                                  | 0.000  | hazelnut                                                         | 0.000  | coconut                                                                  | 0.000  |
| brazil-nut                                                              | 0.000  | pecan-nut                                                        | 0.000  | mixed-nuts                                                               | 0.000  |
| pistachio                                                               | 0.000  | linseeds                                                         | 0.000  | sesame-seeds                                                             | 0.000  |
| green-olives                                                            | 0.000  | black-olives                                                     | 0.000  | milk                                                                     | 0.000  |
| kefir-drink                                                             | 0.000  | coconut-milk                                                     | 0.000  | soya-drink-soy-milk                                                      | 0.000  |
| soya-yaourt-yahourt-yogourt-ou-yoghourt                                 | 0.000  | cottage-cheese                                                   | 0.000  | curd                                                                     | nan    |
| blue-mould-cheese                                                       | 0.000  | brie                                                             | 0.000  | cheddar                                                                  | 0.000  |
| emmental-cheese                                                         | 0.000  | feta                                                             | 0.000  | fondue                                                                   | 0.000  |
| fresh-cheese                                                            | 0.000  | gruyere                                                          | 0.000  | semi-hard-cheese                                                         | 0.000  |
| halloumi                                                                | 0.000  | hard-cheese                                                      | 0.000  | cheese                                                                   | 0.000  |
| mozzarella                                                              | 0.000  | parmesan                                                         | 0.000  | philadelphia                                                             | 0.000  |
| cheese-for-raclette                                                     | 0.000  | cream-cheese                                                     | 0.000  | processed-cheese                                                         | 0.000  |
| tete-de-moine                                                           | 0.000  | tomme                                                            | 0.000  | soft-cheese                                                              | 0.000  |
| mousse                                                                  | nan    | panna-cotta                                                      | nan    | tiramisu                                                                 | 0.000  |
| cream                                                                   | 0.000  | sour-cream                                                       | 0.000  | thickened-cream-35                                                       | nan    |
| dairy-ice-cream                                                         | 0.000  | sorbet                                                           | 0.000  | flakes-oat                                                               | 0.000  |
| rice-noodles-vermicelli                                                 | 0.000  | bulgur                                                           | 0.000  | couscous                                                                 | 0.000  |
| ebly                                                                    | 0.000  | grits-polenta-maize-flour                                        | 0.000  | quinoa                                                                   | 0.000  |
| rice                                                                    | 0.000  | rice-basmati                                                     | 0.000  | rice-jasmin                                                              | nan    |
| rice-whole-grain                                                        | 0.000  | rice-wild                                                        | 0.000  | spaetzle                                                                 | 0.000  |
| pasta                                                                   | 0.000  | pasta-hornli                                                     | 0.000  | pasta-in-butterfly-form-farfalle                                         | 0.000  |
| pasta-linguini-parpadelle-tagliatelle                                   | 0.000  | pasta-in-conch-form                                              | 0.000  | pasta-noodles                                                            | 0.000  |
| pasta-penne                                                             | 0.000  | pasta-ravioli-stuffing                                           | 0.000  | pasta-spaghetti                                                          | 0.000  |
| pasta-twist                                                             | 0.000  | pasta-tortelloni-stuffing                                        | nan    | pasta-wholemeal                                                          | 0.000  |
| bagel-without-filling                                                   | 0.000  | bread-french-white-flour                                         | 0.000  | bread                                                                    | 0.000  |
| bread-5-grain                                                           | nan    | bread-spelt                                                      | 0.000  | bread-fruit                                                              | 0.000  |
| bread-half-white                                                        | 0.000  | bread-grain                                                      | 0.000  | bread-nut                                                                | nan    |
| bread-olive                                                             | 0.000  | bread-pita                                                       | 0.000  | bread-rye                                                                | 0.000  |
| bread-whole-wheat                                                       | 0.000  | bread-sourdough                                                  | 0.000  | bread-black                                                              | 0.000  |
| bread-ticino                                                            | 0.000  | bread-toast                                                      | 0.000  | bread-wholemeal-toast                                                    | 0.000  |
| bread-wholemeal                                                         | 0.545  | bread-white                                                      | 5.438  | brioche                                                                  | 0.000  |
| roll-of-half-white-or-white-flour-with-large-void                       | 0.000  | hamburger-bun                                                    | 0.000  | roll-with-pieces-of-chocolate                                            | 0.000  |
| white-bread-with-butter-eggs-and-milk                                   | 0.000  | focaccia                                                         | 0.000  | croissant                                                                | 0.000  |
| croissant-wholegrain                                                    | 0.000  | lye-pretzel-soft                                                 | 0.000  | braided-white-loaf                                                       | 0.000  |
| crisp-bread-wasa                                                        | nan    | breadcrumbs-unspiced                                             | 0.000  | rice-waffels                                                             | 0.000  |
| grissini                                                                | 0.000  | rusk-wholemeal                                                   | 0.000  | corn-flakes                                                              | nan    |
| crunch-muesli                                                           | 0.000  | muesli                                                           | 0.000  | dough-puff-pastry-shortcrust-bread-pizza-dough                           | nan    |
| pastry-flaky                                                            | nan    | meat                                                             | 0.000  | minced-meat                                                              | nan    |
| beef                                                                    | 0.000  | beef-roast                                                       | 0.000  | beef-sirloin-steak                                                       | 0.000  |
| beef-filet                                                              | nan    | beef-minced-only-meat                                            | 0.000  | beef-cut-into-stripes-only-meat                                          | 0.000  |
| pork                                                                    | 0.000  | pork-roast                                                       | 0.000  | pork-chop                                                                | nan    |
| pork-escalope                                                           | nan    | lamb                                                             | nan    | lamb-chop                                                                | 0.000  |
| chicken                                                                 | 0.000  | chicken-breast                                                   | 0.000  | chicken-wing                                                             | nan    |
| chicken-cut-into-stripes-only-meat                                      | 0.000  | chicken-leg                                                      | 0.000  | frying-sausage                                                           | 0.000  |
| cervelat                                                                | 0.000  | chicken-nuggets                                                  | 0.000  | chorizo                                                                  | 0.000  |
| meatloaf                                                                | 0.000  | hamburger                                                        | 0.000  | dried-meat                                                               | 0.000  |
| veal-sausage                                                            | 0.000  | processed-meat-charcuterie                                       | nan    | salami                                                                   | 0.000  |
| cooked-sausage                                                          | nan    | ham                                                              | 0.000  | ham-cooked                                                               | 0.000  |
| ham-raw                                                                 | 0.000  | ham-turkey                                                       | 0.000  | smoked-cooked-sausage-of-pork-and-beef-meat-sausag                       | nan    |
| bacon                                                                   | nan    | bacon-frying                                                     | 0.000  | bacon-cooking                                                            | 0.000  |
| bacon-raw                                                               | 0.000  | meat-terrine-pate                                                | 0.000  | sausage                                                                  | 0.000  |
| veggie-burger                                                           | 0.000  | tofu                                                             | 0.000  | fish                                                                     | 0.000  |
| cod                                                                     | 0.000  | salmon                                                           | 0.000  | anchovies                                                                | 0.000  |
| tuna                                                                    | nan    | shrimp-prawn-small                                               | nan    | shrimp-prawn-large                                                       | 0.000  |
| fish-crunchies-battered                                                 | nan    | fish-fingers-breaded                                             | 0.000  | egg                                                                      | 0.988  |
| oil                                                                     | 0.000  | butter                                                           | 0.642  | butter-herb                                                              | 0.000  |
| margarine                                                               | 0.000  | praline                                                          | 0.000  | jam                                                                      | 0.000  |
| honey                                                                   | 0.000  | sugar-glazing                                                    | nan    | maple-syrup-concentrate                                                  | 0.000  |
| dark-chocolate                                                          | 0.000  | milk-chocolate                                                   | 0.000  | white-chocolate                                                          | nan    |
| chocolate                                                               | 0.000  | chocolate-filled                                                 | nan    | cocoa-powder                                                             | nan    |
| hazelnut-chocolate-spread-nutella-ovomaltine-caotina                    | 0.000  | m-m-s                                                            | 0.000  | chocolate-egg-small                                                      | 0.000  |
| sweets-candies                                                          | 0.000  | gummi-bears-fruit-jellies-jelly-babies-with-fruit-essence        | nan    | apple-pie                                                                | 0.000  |
| brownie                                                                 | nan    | cake-oblong                                                      | 0.000  | lemon-cake                                                               | nan    |
| crepe-plain                                                             | 0.000  | fruit-tart                                                       | 0.000  | cake-marble                                                              | 0.000  |
| cake-chocolate                                                          | 0.000  | muffin                                                           | 0.000  | omelette-plain                                                           | 0.000  |
| carrot-cake                                                             | 0.000  | black-forest-tart                                                | 0.000  | tart                                                                     | 0.000  |
| waffle                                                                  | nan    | croissant-with-chocolate-filling                                 | 0.000  | cookies                                                                  | 0.000  |
| biscuits                                                                | 0.000  | macaroon                                                         | 0.000  | meringue                                                                 | 0.000  |
| biscuit-with-butter                                                     | 0.000  | chocolate-cookies                                                | 0.000  | juice-apple                                                              | 0.000  |
| juice-multifruit                                                        | 0.000  | juice-orange                                                     | nan    | smoothie                                                                 | nan    |
| coca-cola                                                               | nan    | coca-cola-zero                                                   | 0.000  | ice-tea                                                                  | nan    |
| syrup-diluted-ready-to-drink                                            | 0.000  | tea                                                              | 0.000  | cappuccino                                                               | 0.000  |
| espresso-with-caffeine                                                  | 0.769  | coffee-with-caffeine                                             | 5.884  | coffee-decaffeinated                                                     | 0.000  |
| latte-macchiato-with-caffeine                                           | 0.000  | white-coffee-with-caffeine                                       | 0.932  | ristretto-with-caffeine                                                  | 0.000  |
| tea-green                                                               | 0.000  | tea-black                                                        | nan    | tea-verveine                                                             | nan    |
| tea-fruit                                                               | nan    | tea-spice                                                        | nan    | tea-ginger                                                               | nan    |
| herbal-tea                                                              | 0.000  | tea-peppermint                                                   | 0.000  | tea-rooibos                                                              | 0.000  |
| ice-cubes                                                               | nan    | water                                                            | 22.046 | water-mineral                                                            | 0.000  |
| aperitif-with-alcohol-aperol-spritz                                     | nan    | cocktail                                                         | 0.000  | champagne                                                                | 0.000  |
| prosecco                                                                | 0.000  | sekt                                                             | 0.000  | wine-rose                                                                | 0.000  |
| wine-red                                                                | 2.094  | wine-white                                                       | 5.914  | beer                                                                     | 0.000  |
| light-beer                                                              | 0.000  | sauce-savoury                                                    | 0.000  | sauce-roast                                                              | 0.000  |
| sauce-carbonara                                                         | nan    | sauce-cocktail                                                   | 0.000  | sauce-curry                                                              | 0.000  |
| sauce-pesto                                                             | 0.000  | sauce-mushroom                                                   | 0.000  | sauce-cream                                                              | 0.000  |
| sauce-sweet-sour                                                        | 0.000  | ketchup                                                          | 0.000  | bolognaise-sauce                                                         | 0.000  |
| tomato-sauce                                                            | 0.000  | dips                                                             | nan    | salad-dressing                                                           | 0.000  |
| balsamic-salad-dressing                                                 | 0.000  | french-salad-dressing                                            | 0.000  | italian-salad-dressing                                                   | 0.000  |
| oil-vinegar-salad-dressing                                              | 0.000  | guacamole                                                        | 0.000  | mayonnaise                                                               | 0.000  |
| tartar-sauce                                                            | 0.000  | tzatziki                                                         | 0.000  | basil                                                                    | 0.000  |
| coriander                                                               | 0.000  | bouquet-garni                                                    | 0.000  | parsley                                                                  | 0.000  |
| chives                                                                  | 0.000  | cenovis-yeast-spread                                             | nan    | sauce-soya                                                               | 0.000  |
| mustard                                                                 | 0.000  | mustard-dijon                                                    | nan    | balsamic-vinegar                                                         | nan    |
| soup-vegetable                                                          | 0.000  | soup-cream-of-vegetables                                         | 0.000  | soup-potato                                                              | 0.000  |
| soup-pumpkin                                                            | 0.000  | soup-miso                                                        | 0.000  | soup-tomato                                                              | 0.000  |
| bouillon                                                                | 0.000  | bouillon-vegetable                                               | 0.000  | falafel-balls                                                            | 0.000  |
| savoury-puff-pastry                                                     | nan    | savoury-puff-pastry-stick                                        | 0.000  | corn-crisps                                                              | nan    |
| crackers                                                                | 0.000  | croutons                                                         | 0.000  | crisps                                                                   | 0.000  |
| popcorn-salted                                                          | 0.000  | croque-monsieur                                                  | 0.000  | spring-roll-fried                                                        | 0.000  |
| ham-croissant                                                           | nan    | salt-cake-vegetables-filled                                      | nan    | hamburger-bread-meat-ketchup                                             | 0.000  |
| cordon-bleu-from-pork-schnitzel-fried                                   | 0.000  | lasagne-meat-prepared                                            | 0.000  | mashed-potatoes-prepared-with-full-fat-milk-with-butter                  | 0.000  |
| pizza-margherita-baked                                                  | 0.000  | sandwich-ham-cheese-and-butter                                   | nan    | sushi                                                                    | 0.000  |
| kebab-in-pita-bread                                                     | 0.000  | pancakes                                                         | nan    | hummus                                                                   | 0.000  |
| greek-salad                                                             | 0.000  | dumplings                                                        | nan    | apricot-dried                                                            | 0.000  |
| chocolate-mousse                                                        | nan    | cheesecake                                                       | 0.000  | caprese-salad-tomato-mozzarella                                          | 0.000  |
| chili-con-carne-prepared                                                | 0.000  | taboule-prepared-with-couscous                                   | nan    | perch-fillets-lake                                                       | nan    |
| risotto-without-cheese-cooked                                           | 0.000  | salmon-smoked                                                    | 0.000  | pie-apricot-baked-with-cake-dough                                        | 0.000  |
| eggplant-caviar                                                         | 0.000  | apple-crumble                                                    | nan    | egg-scrambled-prepared                                                   | 0.000  |
| oat-milk                                                                | 0.000  | lemon-pie                                                        | nan    | glucose-drink-50g                                                        | 0.000  |
| goat-average-raw                                                        | nan    | pie-rhubarb-baked-with-cake-dough                                | nan    | chicken-curry-cream-coconut-milk-curry-spices-paste                      | nan    |
| pie-plum-baked-with-cake-dough                                          | 0.000  | potatoes-au-gratin-dauphinois-prepared                           | 0.000  | buckwheat-grain-peeled                                                   | nan    |
| birchermuesli-prepared-no-sugar-added                                   | 0.000  | fajita-bread-only                                                | 0.000  | mango-dried                                                              | 0.000  |
| lentils-green-du-puy-du-berry                                           | 0.000  | naan-indien-bread                                                | nan    | butter-spread-puree-almond                                               | 0.000  |
| chocolate-milk-chocolate-drink                                          | nan    | water-with-lemon-juice                                           | 0.000  | sun-dried-tomatoe                                                        | 0.000  |
| gluten-free-bread                                                       | 0.000  | fruit-coulis                                                     | 0.000  | greek-yaourt-yahourt-yogourt-ou-yoghourt                                 | 0.000  |
| cake-salted                                                             | 0.000  | soup-of-lentils-dahl-dhal                                        | nan    | fig-dried                                                                | 0.000  |
| turnover-with-meat-small-meat-pie-empanadas                             | 0.000  | lasagne-vegetable-prepared                                       | 0.000  | sauce-sweet-salted-asian                                                 | 0.000  |
| french-pizza-from-alsace-baked                                          | 0.000  | fruit-compotes                                                   | nan    | vegetable-au-gratin-baked                                                | 0.000  |
| porridge-prepared-with-partially-skimmed-milk                           | nan    | curry-vegetarian                                                 | nan    | bread-meat-substitute-lettuce-sauce                                      | 0.000  |
| tartar-meat                                                             | 0.000  | chia-grains                                                      | nan    | faux-mage-cashew-vegan-chers                                             | 0.000  |
| milk-chocolate-with-hazelnuts                                           | 0.000  | yaourt-yahourt-yogourt-ou-yoghourt-natural                       | 0.000  | paprika-chips                                                            | nan    |
| banana-cake                                                             | 0.000  | cream-spinach                                                    | nan    | cantonese-fried-rice                                                     | 0.000  |
| goat-cheese-soft                                                        | 0.000  | buckwheat-pancake                                                | 0.000  | meat-balls                                                               | 0.000  |
| high-protein-pasta-made-of-lentils-peas                                 | 0.000  | mix-of-dried-fruits-and-nuts                                     | 0.000  | baked-potato                                                             | 0.000  |
Loading and preparing results...
DONE (t=0.16s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *segm*
DONE (t=9.27s).
Accumulating evaluation results...
DONE (t=2.70s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.006
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.010
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.007
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.002
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.006
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.014
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.016
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.016
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.010
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.017
[12/20 22:33:21 d2.evaluation.coco_evaluation]: Evaluation results for segm: 
|  AP   |  AP50  |  AP75  |  APs  |  APm  |  APl  |
|:-----:|:------:|:------:|:-----:|:-----:|:-----:|
| 0.600 | 1.017  | 0.663  | 0.000 | 0.195 | 0.642 |
[12/20 22:33:21 d2.evaluation.coco_evaluation]: Per-category segm AP: 
| category                                                                | AP     | category                                                         | AP     | category                                                                 | AP     |
|:------------------------------------------------------------------------|:-------|:-----------------------------------------------------------------|:-------|:-------------------------------------------------------------------------|:-------|
| beetroot-steamed-without-addition-of-salt                               | 0.000  | carrot-steamed-without-addition-of-salt                          | 0.000  | mushroom-average-stewed-without-addition-of-fat-without-addition-of-salt | 0.000  |
| savoy-cabbage-steamed-without-addition-of-salt                          | nan    | zucchini-stewed-without-addition-of-fat-without-addition-of-salt | nan    | vanille-cream-cooked-custard-creme-dessert                               | nan    |
| shrimp-boiled                                                           | 0.000  | spinach-steamed-without-addition-of-salt                         | 0.000  | spinach-raw                                                              | 0.000  |
| green-bean-steamed-without-addition-of-salt                             | 0.000  | sweet-corn-canned                                                | 0.000  | watermelon-fresh                                                         | 0.000  |
| pizza-with-ham-with-mushrooms-baked                                     | 0.000  | pizza-with-ham-baked                                             | 0.000  | pizza-with-vegetables-baked                                              | 0.000  |
| bell-pepper-red-stewed-without-addition-of-fat-without-addition-of-salt | 0.000  | applesauce-unsweetened-canned                                    | 0.000  | quiche-with-cheese-baked-with-puff-pastry                                | 0.000  |
| quiche-with-spinach-baked-with-cake-dough                               | 0.000  | risotto-with-mushrooms-cooked                                    | nan    | potato-salad-with-mayonnaise-yogurt-dressing                             | 0.000  |
| wienerli-swiss-sausage                                                  | 0.000  | curds-natural-with-at-most-10-fidm                               | 0.000  | tuna-in-oil-drained                                                      | nan    |
| tomato-stewed-without-addition-of-fat-without-addition-of-salt          | 0.000  | sweet-potato                                                     | 0.000  | country-fries                                                            | 0.000  |
| potato-gnocchi                                                          | 0.000  | potatoes-steamed                                                 | 0.000  | chips-french-fries                                                       | 0.000  |
| rosti                                                                   | 0.000  | vegetable-mix-peas-and-carrots                                   | 0.000  | coleslaw-chopped-without-sauce                                           | nan    |
| vegetables                                                              | 0.000  | mixed-vegetables                                                 | 0.000  | ratatouille                                                              | 0.000  |
| mixed-salad-chopped-without-sauce                                       | 4.664  | leaf-spinach                                                     | 0.000  | witloof-chicory                                                          | 0.000  |
| salad-rocket                                                            | 0.000  | salad-leaf-salad-green                                           | 24.273 | salad-lambs-ear                                                          | 0.000  |
| artichoke                                                               | nan    | eggplant                                                         | 0.000  | avocado                                                                  | 0.000  |
| french-beans                                                            | 0.000  | cucumber-pickled                                                 | 0.000  | cucumber                                                                 | 0.000  |
| capers                                                                  | 0.000  | pumpkin                                                          | 0.000  | bell-pepper-red-raw                                                      | 0.000  |
| tomato-raw                                                              | 0.000  | zucchini                                                         | 0.000  | red-radish                                                               | 0.000  |
| beetroot-raw                                                            | 0.000  | white-radish                                                     | nan    | carrot-raw                                                               | 4.958  |
| celeriac                                                                | nan    | cauliflower                                                      | 0.000  | broccoli                                                                 | 0.000  |
| chinese-cabbage                                                         | 0.000  | kolhrabi                                                         | 0.000  | romanesco                                                                | 0.000  |
| brussel-sprouts                                                         | 0.000  | red-cabbage                                                      | 0.000  | sauerkraut                                                               | 0.000  |
| white-cabbage                                                           | nan    | mushroom                                                         | 0.000  | mushrooms                                                                | 0.000  |
| peas                                                                    | 0.000  | corn                                                             | 0.000  | spring-onion-scallion                                                    | 0.000  |
| garlic                                                                  | 0.000  | leek                                                             | 0.000  | pearl-onions                                                             | nan    |
| onion                                                                   | 0.000  | fennel                                                           | 0.000  | swiss-chard                                                              | 0.000  |
| mungbean-sprouts                                                        | nan    | green-asparagus                                                  | 0.000  | white-asparagus                                                          | nan    |
| shoots                                                                  | 0.000  | alfa-sprouts                                                     | nan    | celery                                                                   | 0.000  |
| bean-seeds                                                              | nan    | beans-kidney                                                     | 0.000  | beans-white                                                              | 0.000  |
| chickpeas                                                               | 0.000  | lentils                                                          | 0.000  | pineapple                                                                | 0.000  |
| apple                                                                   | 24.733 | pomegranate                                                      | 0.000  | apricots                                                                 | 0.000  |
| banana                                                                  | 38.713 | berries                                                          | 0.000  | pear                                                                     | 0.000  |
| blackberry                                                              | nan    | dates                                                            | 0.000  | strawberries                                                             | 0.000  |
| figs                                                                    | 0.000  | fruit-salad                                                      | 0.000  | grapefruit-pomelo                                                        | 0.000  |
| blueberries                                                             | 0.000  | raspberries                                                      | 0.000  | kaki                                                                     | 0.000  |
| cherries                                                                | 0.000  | kiwi                                                             | 0.000  | mandarine                                                                | 35.804 |
| mango                                                                   | 0.000  | sugar-melon                                                      | 0.000  | nectarine                                                                | 0.000  |
| orange                                                                  | 0.000  | peach                                                            | 0.000  | plums                                                                    | 0.000  |
| grapes                                                                  | 0.000  | dried-raisins                                                    | 0.000  | lemon                                                                    | 0.000  |
| damson-plum                                                             | 0.000  | peanut-butter                                                    | 0.000  | chestnuts                                                                | 0.000  |
| seeds                                                                   | 0.000  | pumpkin-seeds                                                    | 0.000  | pine-nuts                                                                | 0.000  |
| sunflower-seeds                                                         | 0.000  | mixed-seeds                                                      | nan    | almonds                                                                  | 0.000  |
| nuts                                                                    | nan    | walnut                                                           | 0.000  | cashew-nut                                                               | 0.000  |
| peanut                                                                  | 0.000  | hazelnut                                                         | 0.000  | coconut                                                                  | 0.000  |
| brazil-nut                                                              | 0.000  | pecan-nut                                                        | 0.000  | mixed-nuts                                                               | 0.000  |
| pistachio                                                               | 0.000  | linseeds                                                         | 0.000  | sesame-seeds                                                             | 0.000  |
| green-olives                                                            | 0.000  | black-olives                                                     | 0.000  | milk                                                                     | 0.000  |
| kefir-drink                                                             | 0.000  | coconut-milk                                                     | 0.000  | soya-drink-soy-milk                                                      | 0.000  |
| soya-yaourt-yahourt-yogourt-ou-yoghourt                                 | 0.000  | cottage-cheese                                                   | 0.000  | curd                                                                     | nan    |
| blue-mould-cheese                                                       | 0.000  | brie                                                             | 0.000  | cheddar                                                                  | 0.000  |
| emmental-cheese                                                         | 0.000  | feta                                                             | 0.000  | fondue                                                                   | 0.000  |
| fresh-cheese                                                            | 0.000  | gruyere                                                          | 0.000  | semi-hard-cheese                                                         | 0.000  |
| halloumi                                                                | 0.000  | hard-cheese                                                      | 0.000  | cheese                                                                   | 0.000  |
| mozzarella                                                              | 0.000  | parmesan                                                         | 0.000  | philadelphia                                                             | 0.000  |
| cheese-for-raclette                                                     | 0.000  | cream-cheese                                                     | 0.000  | processed-cheese                                                         | 0.000  |
| tete-de-moine                                                           | 0.000  | tomme                                                            | 0.000  | soft-cheese                                                              | 0.000  |
| mousse                                                                  | nan    | panna-cotta                                                      | nan    | tiramisu                                                                 | 0.000  |
| cream                                                                   | 0.000  | sour-cream                                                       | 0.000  | thickened-cream-35                                                       | nan    |
| dairy-ice-cream                                                         | 0.000  | sorbet                                                           | 0.000  | flakes-oat                                                               | 0.000  |
| rice-noodles-vermicelli                                                 | 0.000  | bulgur                                                           | 0.000  | couscous                                                                 | 0.000  |
| ebly                                                                    | 0.000  | grits-polenta-maize-flour                                        | 0.000  | quinoa                                                                   | 0.000  |
| rice                                                                    | 0.000  | rice-basmati                                                     | 0.000  | rice-jasmin                                                              | nan    |
| rice-whole-grain                                                        | 0.000  | rice-wild                                                        | 0.000  | spaetzle                                                                 | 0.000  |
| pasta                                                                   | 0.000  | pasta-hornli                                                     | 0.000  | pasta-in-butterfly-form-farfalle                                         | 0.000  |
| pasta-linguini-parpadelle-tagliatelle                                   | 0.000  | pasta-in-conch-form                                              | 0.000  | pasta-noodles                                                            | 0.000  |
| pasta-penne                                                             | 0.000  | pasta-ravioli-stuffing                                           | 0.000  | pasta-spaghetti                                                          | 0.000  |
| pasta-twist                                                             | 0.000  | pasta-tortelloni-stuffing                                        | nan    | pasta-wholemeal                                                          | 0.000  |
| bagel-without-filling                                                   | 0.000  | bread-french-white-flour                                         | 0.000  | bread                                                                    | 0.000  |
| bread-5-grain                                                           | nan    | bread-spelt                                                      | 0.000  | bread-fruit                                                              | 0.000  |
| bread-half-white                                                        | 0.000  | bread-grain                                                      | 0.000  | bread-nut                                                                | nan    |
| bread-olive                                                             | 0.000  | bread-pita                                                       | 0.000  | bread-rye                                                                | 0.000  |
| bread-whole-wheat                                                       | 0.354  | bread-sourdough                                                  | 0.000  | bread-black                                                              | 0.000  |
| bread-ticino                                                            | 0.000  | bread-toast                                                      | 0.000  | bread-wholemeal-toast                                                    | 0.000  |
| bread-wholemeal                                                         | 3.144  | bread-white                                                      | 15.266 | brioche                                                                  | 0.000  |
| roll-of-half-white-or-white-flour-with-large-void                       | 0.000  | hamburger-bun                                                    | 0.000  | roll-with-pieces-of-chocolate                                            | 0.000  |
| white-bread-with-butter-eggs-and-milk                                   | 0.000  | focaccia                                                         | 0.000  | croissant                                                                | 0.000  |
| croissant-wholegrain                                                    | 0.000  | lye-pretzel-soft                                                 | 0.000  | braided-white-loaf                                                       | 0.000  |
| crisp-bread-wasa                                                        | nan    | breadcrumbs-unspiced                                             | 0.000  | rice-waffels                                                             | 0.000  |
| grissini                                                                | 0.000  | rusk-wholemeal                                                   | 0.000  | corn-flakes                                                              | nan    |
| crunch-muesli                                                           | 0.000  | muesli                                                           | 0.000  | dough-puff-pastry-shortcrust-bread-pizza-dough                           | nan    |
| pastry-flaky                                                            | nan    | meat                                                             | 0.000  | minced-meat                                                              | nan    |
| beef                                                                    | 0.000  | beef-roast                                                       | 0.000  | beef-sirloin-steak                                                       | 0.000  |
| beef-filet                                                              | nan    | beef-minced-only-meat                                            | 0.000  | beef-cut-into-stripes-only-meat                                          | 0.000  |
| pork                                                                    | 0.000  | pork-roast                                                       | 0.000  | pork-chop                                                                | nan    |
| pork-escalope                                                           | nan    | lamb                                                             | nan    | lamb-chop                                                                | 0.000  |
| chicken                                                                 | 0.000  | chicken-breast                                                   | 0.000  | chicken-wing                                                             | nan    |
| chicken-cut-into-stripes-only-meat                                      | 0.000  | chicken-leg                                                      | 0.000  | frying-sausage                                                           | 0.000  |
| cervelat                                                                | 0.000  | chicken-nuggets                                                  | 0.000  | chorizo                                                                  | 0.000  |
| meatloaf                                                                | 0.000  | hamburger                                                        | 0.000  | dried-meat                                                               | 0.000  |
| veal-sausage                                                            | 0.000  | processed-meat-charcuterie                                       | nan    | salami                                                                   | 0.000  |
| cooked-sausage                                                          | nan    | ham                                                              | 0.000  | ham-cooked                                                               | 0.000  |
| ham-raw                                                                 | 0.000  | ham-turkey                                                       | 0.000  | smoked-cooked-sausage-of-pork-and-beef-meat-sausag                       | nan    |
| bacon                                                                   | nan    | bacon-frying                                                     | 0.000  | bacon-cooking                                                            | 0.000  |
| bacon-raw                                                               | 0.000  | meat-terrine-pate                                                | 0.000  | sausage                                                                  | 0.000  |
| veggie-burger                                                           | 0.000  | tofu                                                             | 0.000  | fish                                                                     | 0.000  |
| cod                                                                     | 0.000  | salmon                                                           | 0.000  | anchovies                                                                | 0.000  |
| tuna                                                                    | nan    | shrimp-prawn-small                                               | nan    | shrimp-prawn-large                                                       | 0.000  |
| fish-crunchies-battered                                                 | nan    | fish-fingers-breaded                                             | 0.000  | egg                                                                      | 4.055  |
| oil                                                                     | 0.000  | butter                                                           | 0.770  | butter-herb                                                              | 0.000  |
| margarine                                                               | 0.000  | praline                                                          | 0.000  | jam                                                                      | 0.000  |
| honey                                                                   | 0.000  | sugar-glazing                                                    | nan    | maple-syrup-concentrate                                                  | 0.000  |
| dark-chocolate                                                          | 0.000  | milk-chocolate                                                   | 0.000  | white-chocolate                                                          | nan    |
| chocolate                                                               | 0.000  | chocolate-filled                                                 | nan    | cocoa-powder                                                             | nan    |
| hazelnut-chocolate-spread-nutella-ovomaltine-caotina                    | 0.000  | m-m-s                                                            | 0.000  | chocolate-egg-small                                                      | 0.000  |
| sweets-candies                                                          | 0.000  | gummi-bears-fruit-jellies-jelly-babies-with-fruit-essence        | nan    | apple-pie                                                                | 0.000  |
| brownie                                                                 | nan    | cake-oblong                                                      | 0.000  | lemon-cake                                                               | nan    |
| crepe-plain                                                             | 0.000  | fruit-tart                                                       | 0.000  | cake-marble                                                              | 0.000  |
| cake-chocolate                                                          | 0.000  | muffin                                                           | 0.000  | omelette-plain                                                           | 0.000  |
| carrot-cake                                                             | 0.000  | black-forest-tart                                                | 0.000  | tart                                                                     | 0.000  |
| waffle                                                                  | nan    | croissant-with-chocolate-filling                                 | 0.000  | cookies                                                                  | 0.000  |
| biscuits                                                                | 0.000  | macaroon                                                         | 0.000  | meringue                                                                 | 0.000  |
| biscuit-with-butter                                                     | 0.000  | chocolate-cookies                                                | 0.000  | juice-apple                                                              | 0.000  |
| juice-multifruit                                                        | 0.000  | juice-orange                                                     | nan    | smoothie                                                                 | nan    |
| coca-cola                                                               | nan    | coca-cola-zero                                                   | 0.000  | ice-tea                                                                  | nan    |
| syrup-diluted-ready-to-drink                                            | 0.000  | tea                                                              | 0.000  | cappuccino                                                               | 0.000  |
| espresso-with-caffeine                                                  | 5.758  | coffee-with-caffeine                                             | 12.123 | coffee-decaffeinated                                                     | 0.000  |
| latte-macchiato-with-caffeine                                           | 0.000  | white-coffee-with-caffeine                                       | 7.710  | ristretto-with-caffeine                                                  | 0.000  |
| tea-green                                                               | 0.000  | tea-black                                                        | nan    | tea-verveine                                                             | nan    |
| tea-fruit                                                               | nan    | tea-spice                                                        | nan    | tea-ginger                                                               | nan    |
| herbal-tea                                                              | 0.000  | tea-peppermint                                                   | 0.000  | tea-rooibos                                                              | 0.000  |
| ice-cubes                                                               | nan    | water                                                            | 26.454 | water-mineral                                                            | 0.000  |
| aperitif-with-alcohol-aperol-spritz                                     | nan    | cocktail                                                         | 0.000  | champagne                                                                | 0.000  |
| prosecco                                                                | 0.000  | sekt                                                             | 0.000  | wine-rose                                                                | 0.000  |
| wine-red                                                                | 18.478 | wine-white                                                       | 16.528 | beer                                                                     | 0.000  |
| light-beer                                                              | 0.000  | sauce-savoury                                                    | 0.000  | sauce-roast                                                              | 0.000  |
| sauce-carbonara                                                         | nan    | sauce-cocktail                                                   | 0.000  | sauce-curry                                                              | 0.000  |
| sauce-pesto                                                             | 0.000  | sauce-mushroom                                                   | 0.000  | sauce-cream                                                              | 0.000  |
| sauce-sweet-sour                                                        | 0.000  | ketchup                                                          | 0.000  | bolognaise-sauce                                                         | 0.000  |
| tomato-sauce                                                            | 0.000  | dips                                                             | nan    | salad-dressing                                                           | 0.000  |
| balsamic-salad-dressing                                                 | 0.000  | french-salad-dressing                                            | 0.000  | italian-salad-dressing                                                   | 0.000  |
| oil-vinegar-salad-dressing                                              | 0.000  | guacamole                                                        | 0.000  | mayonnaise                                                               | 0.000  |
| tartar-sauce                                                            | 0.000  | tzatziki                                                         | 0.000  | basil                                                                    | 0.000  |
| coriander                                                               | 0.000  | bouquet-garni                                                    | 0.000  | parsley                                                                  | 0.000  |
| chives                                                                  | 0.000  | cenovis-yeast-spread                                             | nan    | sauce-soya                                                               | 0.000  |
| mustard                                                                 | 0.000  | mustard-dijon                                                    | nan    | balsamic-vinegar                                                         | nan    |
| soup-vegetable                                                          | 0.000  | soup-cream-of-vegetables                                         | 0.000  | soup-potato                                                              | 0.000  |
| soup-pumpkin                                                            | 0.000  | soup-miso                                                        | 0.000  | soup-tomato                                                              | 0.000  |
| bouillon                                                                | 0.000  | bouillon-vegetable                                               | 0.000  | falafel-balls                                                            | 0.000  |
| savoury-puff-pastry                                                     | nan    | savoury-puff-pastry-stick                                        | 0.000  | corn-crisps                                                              | nan    |
| crackers                                                                | 0.000  | croutons                                                         | 0.000  | crisps                                                                   | 0.000  |
| popcorn-salted                                                          | 0.000  | croque-monsieur                                                  | 0.000  | spring-roll-fried                                                        | 0.000  |
| ham-croissant                                                           | nan    | salt-cake-vegetables-filled                                      | nan    | hamburger-bread-meat-ketchup                                             | 0.000  |
| cordon-bleu-from-pork-schnitzel-fried                                   | 0.000  | lasagne-meat-prepared                                            | 0.000  | mashed-potatoes-prepared-with-full-fat-milk-with-butter                  | 0.000  |
| pizza-margherita-baked                                                  | 0.000  | sandwich-ham-cheese-and-butter                                   | nan    | sushi                                                                    | 0.000  |
| kebab-in-pita-bread                                                     | 0.000  | pancakes                                                         | nan    | hummus                                                                   | 0.000  |
| greek-salad                                                             | 0.000  | dumplings                                                        | nan    | apricot-dried                                                            | 0.000  |
| chocolate-mousse                                                        | nan    | cheesecake                                                       | 0.000  | caprese-salad-tomato-mozzarella                                          | 0.000  |
| chili-con-carne-prepared                                                | 0.000  | taboule-prepared-with-couscous                                   | nan    | perch-fillets-lake                                                       | nan    |
| risotto-without-cheese-cooked                                           | 0.000  | salmon-smoked                                                    | 0.000  | pie-apricot-baked-with-cake-dough                                        | 0.000  |
| eggplant-caviar                                                         | 0.000  | apple-crumble                                                    | nan    | egg-scrambled-prepared                                                   | 0.000  |
| oat-milk                                                                | 0.000  | lemon-pie                                                        | nan    | glucose-drink-50g                                                        | 0.000  |
| goat-average-raw                                                        | nan    | pie-rhubarb-baked-with-cake-dough                                | nan    | chicken-curry-cream-coconut-milk-curry-spices-paste                      | nan    |
| pie-plum-baked-with-cake-dough                                          | 0.000  | potatoes-au-gratin-dauphinois-prepared                           | 0.000  | buckwheat-grain-peeled                                                   | nan    |
| birchermuesli-prepared-no-sugar-added                                   | 0.000  | fajita-bread-only                                                | 0.000  | mango-dried                                                              | 0.000  |
| lentils-green-du-puy-du-berry                                           | 0.000  | naan-indien-bread                                                | nan    | butter-spread-puree-almond                                               | 0.000  |
| chocolate-milk-chocolate-drink                                          | nan    | water-with-lemon-juice                                           | 0.000  | sun-dried-tomatoe                                                        | 0.000  |
| gluten-free-bread                                                       | 0.000  | fruit-coulis                                                     | 0.000  | greek-yaourt-yahourt-yogourt-ou-yoghourt                                 | 0.000  |
| cake-salted                                                             | 0.000  | soup-of-lentils-dahl-dhal                                        | nan    | fig-dried                                                                | 0.000  |
| turnover-with-meat-small-meat-pie-empanadas                             | 0.000  | lasagne-vegetable-prepared                                       | 0.000  | sauce-sweet-salted-asian                                                 | 0.000  |
| french-pizza-from-alsace-baked                                          | 0.000  | fruit-compotes                                                   | nan    | vegetable-au-gratin-baked                                                | 0.000  |
| porridge-prepared-with-partially-skimmed-milk                           | nan    | curry-vegetarian                                                 | nan    | bread-meat-substitute-lettuce-sauce                                      | 0.000  |
| tartar-meat                                                             | 0.000  | chia-grains                                                      | nan    | faux-mage-cashew-vegan-chers                                             | 0.000  |
| milk-chocolate-with-hazelnuts                                           | 0.000  | yaourt-yahourt-yogourt-ou-yoghourt-natural                       | 0.000  | paprika-chips                                                            | nan    |
| banana-cake                                                             | 0.000  | cream-spinach                                                    | nan    | cantonese-fried-rice                                                     | 0.000  |
| goat-cheese-soft                                                        | 0.000  | buckwheat-pancake                                                | 0.000  | meat-balls                                                               | 0.000  |
| high-protein-pasta-made-of-lentils-peas                                 | 0.000  | mix-of-dried-fruits-and-nuts                                     | 0.000  | baked-potato                                                             | 0.000  |

In [ ]:

cfg.MODEL.WEIGHTS = os.path.join(cfg.OUTPUT_DIR, "model_final.pth")
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.01
cfg.MODEL.ROI_HEADS.NUM_CLASSES = 498

cfg.DATASETS.TEST = ("validation_dataset", )
predictor = DefaultPredictor(cfg)

In [ ]:

val_metadata = MetadataCatalog.get("val_dataset")

im = cv2.imread("data/val/images/085220.jpg")

outputs = predictor(im)

v = Visualizer(im[:, :, ::-1],
                   metadata=val_metadata, 
                   scale=2,
                   instance_mode=ColorMode.IMAGE_BW
    )

out = v.draw_instance_predictions(outputs["instances"].to("cpu"))
cv2_imshow(out.get_image()[:, :, ::-1])

In [ ]:

#!bash <(curl -sL https://gitlab.aicrowd.com/jyotish/food-recognition-challenge-detectron2-baseline/raw/master/utils/submit-colab.sh)

Data Argumentation¶

In [ ]:

!pip install --upgrade fastai

Requirement already satisfied: fastai in /usr/local/lib/python3.7/dist-packages (2.5.3)
Requirement already satisfied: pandas in /usr/local/lib/python3.7/dist-packages (from fastai) (1.1.5)
Requirement already satisfied: torch<1.11,>=1.7.0 in /usr/local/lib/python3.7/dist-packages (from fastai) (1.10.1)
Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from fastai) (1.4.1)
Requirement already satisfied: torchvision>=0.8.2 in /usr/local/lib/python3.7/dist-packages (from fastai) (0.11.2)
Requirement already satisfied: pillow>6.0.0 in /usr/local/lib/python3.7/dist-packages (from fastai) (7.1.2)
Requirement already satisfied: pip in /usr/local/lib/python3.7/dist-packages (from fastai) (21.1.3)
Requirement already satisfied: pyyaml in /usr/local/lib/python3.7/dist-packages (from fastai) (5.1)
Requirement already satisfied: spacy<4 in /usr/local/lib/python3.7/dist-packages (from fastai) (2.2.4)
Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (from fastai) (1.0.1)
Requirement already satisfied: fastprogress>=0.2.4 in /usr/local/lib/python3.7/dist-packages (from fastai) (1.0.0)
Requirement already satisfied: requests in /usr/local/lib/python3.7/dist-packages (from fastai) (2.26.0)
Requirement already satisfied: matplotlib in /usr/local/lib/python3.7/dist-packages (from fastai) (3.2.2)
Requirement already satisfied: fastdownload<2,>=0.0.5 in /usr/local/lib/python3.7/dist-packages (from fastai) (0.0.5)
Requirement already satisfied: fastcore<1.4,>=1.3.22 in /usr/local/lib/python3.7/dist-packages (from fastai) (1.3.27)
Requirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from fastai) (21.3)
Requirement already satisfied: numpy in /usr/local/lib/python3.7/dist-packages (from fastprogress>=0.2.4->fastai) (1.19.5)
Requirement already satisfied: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (1.0.6)
Requirement already satisfied: blis<0.5.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (0.4.1)
Requirement already satisfied: wasabi<1.1.0,>=0.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (0.8.2)
Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (57.4.0)
Requirement already satisfied: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (2.0.6)
Requirement already satisfied: tqdm<5.0.0,>=4.38.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (4.62.3)
Requirement already satisfied: catalogue<1.1.0,>=0.0.7 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (1.0.0)
Requirement already satisfied: thinc==7.4.0 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (7.4.0)
Requirement already satisfied: plac<1.2.0,>=0.9.6 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (1.1.3)
Requirement already satisfied: srsly<1.1.0,>=1.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (1.0.5)
Requirement already satisfied: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.7/dist-packages (from spacy<4->fastai) (3.0.6)
Requirement already satisfied: importlib-metadata>=0.20 in /usr/local/lib/python3.7/dist-packages (from catalogue<1.1.0,>=0.0.7->spacy<4->fastai) (4.8.2)
Requirement already satisfied: typing-extensions>=3.6.4 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=0.20->catalogue<1.1.0,>=0.0.7->spacy<4->fastai) (3.10.0.2)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.7/dist-packages (from importlib-metadata>=0.20->catalogue<1.1.0,>=0.0.7->spacy<4->fastai) (3.6.0)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.7/dist-packages (from requests->fastai) (2.0.8)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests->fastai) (2.10)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests->fastai) (1.24.3)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests->fastai) (2021.10.8)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fastai) (3.0.6)
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fastai) (2.8.2)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fastai) (0.11.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.7/dist-packages (from matplotlib->fastai) (1.3.2)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.1->matplotlib->fastai) (1.15.0)
Requirement already satisfied: pytz>=2017.2 in /usr/local/lib/python3.7/dist-packages (from pandas->fastai) (2018.9)
Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->fastai) (3.0.0)
Requirement already satisfied: joblib>=0.11 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->fastai) (1.1.0)

In [ ]:

from fastai.vision.core import *
from fastai.vision.utils import *
from fastai.vision.augment import *
from fastai.data.core import *
from fastai.data.transforms import *

In [ ]:

images, lbl_bbox = get_annotations('data/train/annotations.json')\

In [ ]:

idx=14
coco_fn,bbox = 'data/train/images/'+images[idx],lbl_bbox[idx]

def _coco_bb(x):  return TensorBBox.create(bbox[0])
def _coco_lbl(x): return bbox[1]

In [ ]:

coco_dsrc = Datasets([coco_fn]*10, [PILImage.create, [_coco_bb,], [_coco_lbl, MultiCategorize(add_na=True)]], n_inp=1)
coco_tdl = TfmdDL(coco_dsrc, bs=9, after_item=[BBoxLabeler(), PointScaler(), ToTensor(), Resize(256)],
                  after_batch=[IntToFloatTensor(), *aug_transforms()])

coco_tdl.show_batch(max_n=9)

Content

5021

Show Comments

Comments

You must login before you can post a comment.