Food Recognition Challenge

🍕 Food Recognition Challenge : Data Exploration & Baseline

So, in this project, we are building a Deep Learning Model which is capable to detect various food using Detectron2, coco, along with other libraries such as Weights & Biases for recording our experimentations, and a bunch of other things

Problem¶

Detecting & Segmenting various kinds of food from an image. For ex. Someone got into new restaurent and get a food that he has never seen, well our DL model is in rescue, so our DL model will help indentifying which food it is from the class our model is being trained on!

Data¶

We will be using data from Food Recognition Challenge - A benchmark for image-based food recognition challange which was launched on March 9, 2020 and ended on May 26, 2020.

https://www.aicrowd.com/challenges/food-recognition-challenge#datasets

We have a total of 24120 RGB images with 2053 validation, all in MS-COCO format and test set for now is same as validation ( debug mode ).

Evaluation¶

The evaluation metrics is IOU aka. Intersection Over Union ( more about that later ).

The actualy metric is computed by averaging over all the precision and recall values for IOU which greater than 0.5.

https://www.aicrowd.com/challenges/food-recognition-challenge#evaluation-criteria

Table of Content¶

Setting our Workspace 💼
- Downloading & Unzipping our Dataset
- Downloading & Importing Necessary Libraries

Data Exploration 🧐
- Reading our Dataset
- Data Visualisations
Image Visulisation 🖼️
- Reading Images
Creating our Dataset 🔨
- Fixing the Dataset
- Creating our dataset
Creating our Model 🏭
- Creating R-CNN Model
- Setting up hyperparameters
Training the Model 🚂
- Setting up Tensorboard
- Start Training!
Evaluating the model 🧪
- Evaluating our Model
Testing the Model 💯
- Testing the Model
Submitting our predictions 📝
Generate More Data + Some tips & tricks 💡

In [ ]:

Setting our Workspace 💼¶

In this section we will be downloading our dataset, unzipping it & downliading detectron2 library and importing all libraries that we will be using

Downloading & Unzipping our Dataset¶

In [ ]:

# Downloading Training Dataset
!wget -q https://datasets.aicrowd.com/default/aicrowd-public-datasets/food-recognition-challenge/v0.4/train-v0.4.tar.gz -O train.zip

# Downloading Validation Dataset
!wget -q https://datasets.aicrowd.com/default/aicrowd-public-datasets/food-recognition-challenge/v0.4/val-v0.4.tar.gz -O val.zip

In [ ]:

# Unzipping Training Dataset
!unzip train.zip > /dev/null

In [ ]:

# Unzipping Validation Dataset
!unzip val.zip > /dev/null

So, the data structure is something like this

content
|
└─── sample_data
|
└─── Train 
│   │   annotations.json
│   └───images
│       │   012170.jpg
│       │   012030.jpg
│       │   ...
│   
└─── val
│   │   annotations.json
│   └───images
│       │   011397.jpg
│       │   012340.jpg
│       │   ...
|    train.zip
|    val.zip

Importing Necessary Libraries¶

In [1]:

# Making sure that we are using GPUs
!nvidia-smi

Fri Feb  5 05:10:23 2021       
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 460.39       Driver Version: 418.67       CUDA Version: 10.1     |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|                               |                      |               MIG M. |
|===============================+======================+======================|
|   0  Tesla P100-PCIE...  Off  | 00000000:00:04.0 Off |                    0 |
| N/A   36C    P0    26W / 250W |      0MiB / 16280MiB |      0%      Default |
|                               |                      |                 ERR! |
+-------------------------------+----------------------+----------------------+
                                                                               
+-----------------------------------------------------------------------------+
| Processes:                                                                  |
|  GPU   GI   CI        PID   Type   Process name                  GPU Memory |
|        ID   ID                                                   Usage      |
|=============================================================================|
|  No running processes found                                                 |
+-----------------------------------------------------------------------------+

In [ ]:

# install dependencies: (use cu101 because colab has CUDA 10.1)
!pip install -U torch==1.5 torchvision==0.6 -f https://download.pytorch.org/whl/cu101/torch_stable.html 
!pip install cython pyyaml==5.1
!pip install -U 'git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI'
import torch, torchvision
print(torch.__version__, torch.cuda.is_available())
!gcc --version

Looking in links: https://download.pytorch.org/whl/cu101/torch_stable.html
Collecting torch==1.5
  Downloading https://download.pytorch.org/whl/cu101/torch-1.5.0%2Bcu101-cp36-cp36m-linux_x86_64.whl (703.8MB)
     |████████████████████████████████| 703.8MB 24kB/s 
Collecting torchvision==0.6
  Downloading https://download.pytorch.org/whl/cu101/torchvision-0.6.0%2Bcu101-cp36-cp36m-linux_x86_64.whl (6.6MB)
     |████████████████████████████████| 6.6MB 26.7MB/s 
Requirement already satisfied, skipping upgrade: numpy in /usr/local/lib/python3.6/dist-packages (from torch==1.5) (1.19.5)
Requirement already satisfied, skipping upgrade: future in /usr/local/lib/python3.6/dist-packages (from torch==1.5) (0.16.0)
Requirement already satisfied, skipping upgrade: pillow>=4.1.1 in /usr/local/lib/python3.6/dist-packages (from torchvision==0.6) (7.0.0)
Installing collected packages: torch, torchvision
  Found existing installation: torch 1.7.0+cu101
    Uninstalling torch-1.7.0+cu101:
      Successfully uninstalled torch-1.7.0+cu101
  Found existing installation: torchvision 0.8.1+cu101
    Uninstalling torchvision-0.8.1+cu101:
      Successfully uninstalled torchvision-0.8.1+cu101
Successfully installed torch-1.5.0+cu101 torchvision-0.6.0+cu101
Requirement already satisfied: cython in /usr/local/lib/python3.6/dist-packages (0.29.21)
Collecting pyyaml==5.1
  Downloading https://files.pythonhosted.org/packages/9f/2c/9417b5c774792634834e730932745bc09a7d36754ca00acf1ccd1ac2594d/PyYAML-5.1.tar.gz (274kB)
     |████████████████████████████████| 276kB 8.6MB/s 
Building wheels for collected packages: pyyaml
  Building wheel for pyyaml (setup.py) ... done
  Created wheel for pyyaml: filename=PyYAML-5.1-cp36-cp36m-linux_x86_64.whl size=44075 sha256=d5c1880e50d90a9cdc7e61d91bf4def3e289689b69a2f756cd8fe7f977133c5d
  Stored in directory: /root/.cache/pip/wheels/ad/56/bc/1522f864feb2a358ea6f1a92b4798d69ac783a28e80567a18b
Successfully built pyyaml
Installing collected packages: pyyaml
  Found existing installation: PyYAML 3.13
    Uninstalling PyYAML-3.13:
      Successfully uninstalled PyYAML-3.13
Successfully installed pyyaml-5.1
Collecting git+https://github.com/cocodataset/cocoapi.git#subdirectory=PythonAPI
  Cloning https://github.com/cocodataset/cocoapi.git to /tmp/pip-req-build-ck15ryyp
  Running command git clone -q https://github.com/cocodataset/cocoapi.git /tmp/pip-req-build-ck15ryyp
Requirement already satisfied, skipping upgrade: setuptools>=18.0 in /usr/local/lib/python3.6/dist-packages (from pycocotools==2.0) (53.0.0)
Requirement already satisfied, skipping upgrade: cython>=0.27.3 in /usr/local/lib/python3.6/dist-packages (from pycocotools==2.0) (0.29.21)
Requirement already satisfied, skipping upgrade: matplotlib>=2.1.0 in /usr/local/lib/python3.6/dist-packages (from pycocotools==2.0) (3.2.2)
Requirement already satisfied, skipping upgrade: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib>=2.1.0->pycocotools==2.0) (2.4.7)
Requirement already satisfied, skipping upgrade: numpy>=1.11 in /usr/local/lib/python3.6/dist-packages (from matplotlib>=2.1.0->pycocotools==2.0) (1.19.5)
Requirement already satisfied, skipping upgrade: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib>=2.1.0->pycocotools==2.0) (0.10.0)
Requirement already satisfied, skipping upgrade: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib>=2.1.0->pycocotools==2.0) (2.8.1)
Requirement already satisfied, skipping upgrade: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib>=2.1.0->pycocotools==2.0) (1.3.1)
Requirement already satisfied, skipping upgrade: six in /usr/local/lib/python3.6/dist-packages (from cycler>=0.10->matplotlib>=2.1.0->pycocotools==2.0) (1.15.0)
Building wheels for collected packages: pycocotools
  Building wheel for pycocotools (setup.py) ... done
  Created wheel for pycocotools: filename=pycocotools-2.0-cp36-cp36m-linux_x86_64.whl size=265567 sha256=cbe08ba616ad4b301ac531aa3a7d8feda31eb5da93c91abfb36305afa400828a
  Stored in directory: /tmp/pip-ephem-wheel-cache-g1nh7ukv/wheels/90/51/41/646daf401c3bc408ff10de34ec76587a9b3ebfac8d21ca5c3a
Successfully built pycocotools
Installing collected packages: pycocotools
  Found existing installation: pycocotools 2.0.2
    Uninstalling pycocotools-2.0.2:
      Successfully uninstalled pycocotools-2.0.2
Successfully installed pycocotools-2.0
1.5.0+cu101 True
gcc (Ubuntu 7.5.0-3ubuntu1~18.04) 7.5.0
Copyright (C) 2017 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

In [ ]:

# install detectron2:
!pip install detectron2==0.1.2 -f https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/index.html

Looking in links: https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/index.html
Collecting detectron2==0.1.2
  Downloading https://dl.fbaipublicfiles.com/detectron2/wheels/cu101/torch1.5/detectron2-0.1.2%2Bcu101-cp36-cp36m-linux_x86_64.whl (6.2MB)
     |████████████████████████████████| 6.2MB 4.6MB/s 
Collecting yacs>=0.1.6
  Downloading https://files.pythonhosted.org/packages/38/4f/fe9a4d472aa867878ce3bb7efb16654c5d63672b86dc0e6e953a67018433/yacs-0.1.8-py3-none-any.whl
Requirement already satisfied: Pillow in /usr/local/lib/python3.6/dist-packages (from detectron2==0.1.2) (7.0.0)
Collecting fvcore
  Downloading https://files.pythonhosted.org/packages/ed/38/e425d90ddd07e3d23a84b49d636df76a41f645e62fd6dc944b5769c8ab34/fvcore-0.1.3.post20210204.tar.gz
Requirement already satisfied: matplotlib in /usr/local/lib/python3.6/dist-packages (from detectron2==0.1.2) (3.2.2)
Requirement already satisfied: cloudpickle in /usr/local/lib/python3.6/dist-packages (from detectron2==0.1.2) (1.3.0)
Requirement already satisfied: tqdm>4.29.0 in /usr/local/lib/python3.6/dist-packages (from detectron2==0.1.2) (4.41.1)
Requirement already satisfied: tensorboard in /usr/local/lib/python3.6/dist-packages (from detectron2==0.1.2) (2.4.1)
Requirement already satisfied: pydot in /usr/local/lib/python3.6/dist-packages (from detectron2==0.1.2) (1.3.0)
Requirement already satisfied: future in /usr/local/lib/python3.6/dist-packages (from detectron2==0.1.2) (0.16.0)
Collecting mock
  Downloading https://files.pythonhosted.org/packages/5c/03/b7e605db4a57c0f6fba744b11ef3ddf4ddebcada35022927a2b5fc623fdf/mock-4.0.3-py3-none-any.whl
Requirement already satisfied: tabulate in /usr/local/lib/python3.6/dist-packages (from detectron2==0.1.2) (0.8.7)
Requirement already satisfied: termcolor>=1.1 in /usr/local/lib/python3.6/dist-packages (from detectron2==0.1.2) (1.1.0)
Requirement already satisfied: PyYAML in /usr/local/lib/python3.6/dist-packages (from yacs>=0.1.6->detectron2==0.1.2) (5.1)
Requirement already satisfied: numpy in /usr/local/lib/python3.6/dist-packages (from fvcore->detectron2==0.1.2) (1.19.5)
Collecting iopath>=0.1.2
  Downloading https://files.pythonhosted.org/packages/f2/c8/1830019bcecf26e76c3fdc36e2a6fa454388a233894dcf6f5eb00a881468/iopath-0.1.3.tar.gz
Requirement already satisfied: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->detectron2==0.1.2) (2.8.1)
Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib->detectron2==0.1.2) (0.10.0)
Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->detectron2==0.1.2) (1.3.1)
Requirement already satisfied: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->detectron2==0.1.2) (2.4.7)
Requirement already satisfied: absl-py>=0.4 in /usr/local/lib/python3.6/dist-packages (from tensorboard->detectron2==0.1.2) (0.10.0)
Requirement already satisfied: werkzeug>=0.11.15 in /usr/local/lib/python3.6/dist-packages (from tensorboard->detectron2==0.1.2) (1.0.1)
Requirement already satisfied: google-auth<2,>=1.6.3 in /usr/local/lib/python3.6/dist-packages (from tensorboard->detectron2==0.1.2) (1.24.0)
Requirement already satisfied: protobuf>=3.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard->detectron2==0.1.2) (3.12.4)
Requirement already satisfied: setuptools>=41.0.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard->detectron2==0.1.2) (53.0.0)
Requirement already satisfied: six>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard->detectron2==0.1.2) (1.15.0)
Requirement already satisfied: grpcio>=1.24.3 in /usr/local/lib/python3.6/dist-packages (from tensorboard->detectron2==0.1.2) (1.32.0)
Requirement already satisfied: requests<3,>=2.21.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard->detectron2==0.1.2) (2.23.0)
Requirement already satisfied: markdown>=2.6.8 in /usr/local/lib/python3.6/dist-packages (from tensorboard->detectron2==0.1.2) (3.3.3)
Requirement already satisfied: tensorboard-plugin-wit>=1.6.0 in /usr/local/lib/python3.6/dist-packages (from tensorboard->detectron2==0.1.2) (1.8.0)
Requirement already satisfied: wheel>=0.26; python_version >= "3" in /usr/local/lib/python3.6/dist-packages (from tensorboard->detectron2==0.1.2) (0.36.2)
Requirement already satisfied: google-auth-oauthlib<0.5,>=0.4.1 in /usr/local/lib/python3.6/dist-packages (from tensorboard->detectron2==0.1.2) (0.4.2)
Collecting portalocker
  Downloading https://files.pythonhosted.org/packages/82/22/e684c9e2e59b561dbe36538852e81849122c666c423448e3a5c99362c228/portalocker-2.2.1-py2.py3-none-any.whl
Requirement already satisfied: rsa<5,>=3.1.4; python_version >= "3.6" in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard->detectron2==0.1.2) (4.7)
Requirement already satisfied: pyasn1-modules>=0.2.1 in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard->detectron2==0.1.2) (0.2.8)
Requirement already satisfied: cachetools<5.0,>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from google-auth<2,>=1.6.3->tensorboard->detectron2==0.1.2) (4.2.1)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard->detectron2==0.1.2) (3.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard->detectron2==0.1.2) (2020.12.5)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard->detectron2==0.1.2) (2.10)
Requirement already satisfied: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.21.0->tensorboard->detectron2==0.1.2) (1.24.3)
Requirement already satisfied: importlib-metadata; python_version < "3.8" in /usr/local/lib/python3.6/dist-packages (from markdown>=2.6.8->tensorboard->detectron2==0.1.2) (3.4.0)
Requirement already satisfied: requests-oauthlib>=0.7.0 in /usr/local/lib/python3.6/dist-packages (from google-auth-oauthlib<0.5,>=0.4.1->tensorboard->detectron2==0.1.2) (1.3.0)
Requirement already satisfied: pyasn1>=0.1.3 in /usr/local/lib/python3.6/dist-packages (from rsa<5,>=3.1.4; python_version >= "3.6"->google-auth<2,>=1.6.3->tensorboard->detectron2==0.1.2) (0.4.8)
Requirement already satisfied: zipp>=0.5 in /usr/local/lib/python3.6/dist-packages (from importlib-metadata; python_version < "3.8"->markdown>=2.6.8->tensorboard->detectron2==0.1.2) (3.4.0)
Requirement already satisfied: typing-extensions>=3.6.4; python_version < "3.8" in /usr/local/lib/python3.6/dist-packages (from importlib-metadata; python_version < "3.8"->markdown>=2.6.8->tensorboard->detectron2==0.1.2) (3.7.4.3)
Requirement already satisfied: oauthlib>=3.0.0 in /usr/local/lib/python3.6/dist-packages (from requests-oauthlib>=0.7.0->google-auth-oauthlib<0.5,>=0.4.1->tensorboard->detectron2==0.1.2) (3.1.0)
Building wheels for collected packages: fvcore, iopath
  Building wheel for fvcore (setup.py) ... done
  Created wheel for fvcore: filename=fvcore-0.1.3.post20210204-cp36-none-any.whl size=44945 sha256=27670c0261008ace4e3cb172a66fc686fd57d97d2ab64ed5db8864c4aad9c1a8
  Stored in directory: /root/.cache/pip/wheels/7a/d5/3d/1393f94e0a0c6345f674387da5bd382c9aba98a23371a3804e
  Building wheel for iopath (setup.py) ... done
  Created wheel for iopath: filename=iopath-0.1.3-cp36-none-any.whl size=11169 sha256=bc4be0e595dfbaf354f3442d36c2eee174d58159f10e0ce78a7ae4f49c5d28bf
  Stored in directory: /root/.cache/pip/wheels/a9/1d/55/94a55e032409ac7617f9cbb88a0fa2cf4e7208806c29730804
Successfully built fvcore iopath
Installing collected packages: yacs, portalocker, iopath, fvcore, mock, detectron2
Successfully installed detectron2-0.1.2+cu101 fvcore-0.1.3.post20210204 iopath-0.1.3 mock-4.0.3 portalocker-2.2.1 yacs-0.1.8

In [ ]:

# You may need to restart your runtime prior to this, to let your installation take effect
# Some basic setup:
# Setup detectron2 logger
import detectron2
from detectron2.utils.logger import setup_logger
setup_logger()

# import some common libraries
import numpy as np
import pandas as pd
import cv2
import json
from tqdm.notebook import tqdm

# import some common detectron2 utilities
from detectron2 import model_zoo
from detectron2.engine import DefaultPredictor
from detectron2.config import get_cfg
from detectron2.utils.visualizer import Visualizer
from detectron2.data import MetadataCatalog
from detectron2.utils.visualizer import ColorMode
from detectron2.data.datasets import register_coco_instances
from detectron2.engine import DefaultTrainer
from detectron2.evaluation import COCOEvaluator, inference_on_dataset
from detectron2.data import build_detection_test_loader

# For reading annotations file
from pycocotools.coco import COCO

# utilities
from pprint import pprint # For beautiful print!
import os 

# For data visualisation
import matplotlib.pyplot as plt
import plotly.graph_objects as go
import plotly.express as px
from google.colab.patches import cv2_imshow

** fvcore version of PathManager will be deprecated soon. **
** Please migrate to the version in iopath repo. **
https://github.com/facebookresearch/iopath

In [ ]:

Data Exploration 🧐¶

In this section we are going to read our dataset & doing some data visualisations

Reading Data¶

In [ ]:

# Reading annotations.json

TRAIN_ANNOTATIONS_PATH = "/content/drive/MyDrive/aicrowd/annotations/training.json"
TRAIN_IMAGE_DIRECTIORY = "/content/train/images/"

VAL_IMAGE_DIRECTIORY = "/content/drive/MyDrive/aicrowd/annotations/validation.json"
VAL_ANNOTATIONS_PATH = "/content/val/annotations.json"

train_coco = COCO(TRAIN_ANNOTATIONS_PATH)

loading annotations into memory...
Done (t=4.92s)
creating index...
index created!

In [ ]:

# Reading the annotation files
with open(TRAIN_ANNOTATIONS_PATH) as f:
  train_annotations_data = json.load(f)

with open(VAL_ANNOTATIONS_PATH) as f:
  val_annotations_data = json.load(f)

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-6-e838d88172ff> in <module>()
      3   train_annotations_data = json.load(f)
      4 
----> 5 with open(VAL_ANNOTATIONS_PATH) as f:
      6   val_annotations_data = json.load(f)

FileNotFoundError: [Errno 2] No such file or directory: '/content/val/annotations.json'

In [ ]:

train_annotations_data['annotations'][0]

Data Format:

Our COCO data format is something like this -

"info": {...},
"categories": [...],
"images": [...],
"annotations": [...],

In which categories is like this

[
  {'id': 2578,
  'name': 'water',
  'name_readable': 'Water',
  'supercategory': 'food'},
  {'id': 1157,
  'name': 'pear',
  'name_readable': 'Pear',
  'supercategory': 'food'},
  ...
  {'id': 1190,
  'name': 'peach',
  'name_readable': 'Peach',
  'supercategory': 'food'}
]

Info is empty ( not sure why )

images is like this

[
  {'file_name': '065537.jpg', 
  'height': 464, 
  'id': 65537, 
  'width': 464},
  {'file_name': '065539.jpg', 
  'height': 464, 
  'id': 65539, 
  'width': 464},
 ...
  {'file_name': '069900.jpg', 
  'height': 391, 
  'id': 69900, 
  'width': 392},
]

Annotations is like this

{'area': 44320.0,
 'bbox': [86.5, 127.49999999999999, 286.0, 170.0],
 'category_id': 2578,
 'id': 102434,
 'image_id': 65537,
 'iscrowd': 0,
 'segmentation': [[235.99999999999997,
   372.5,
   169.0,
   372.5,
   ...
   368.5,
   264.0,
   371.5]]}

In [ ]:

# Reading all classes

category_ids = train_coco.loadCats(train_coco.getCatIds())

category_names = [_["name_readable"] for _ in category_ids]

pprint(", ".join(category_names))

('Water, Pear, Egg, Grapes, Butter, Bread, white, Jam, Bread, whole wheat, '
 'Apple, Tea, green, White coffee, with caffeine, Tea, black, Mixed salad '
 '(chopped without sauce), Cheese, Tomato sauce, Pasta, spaghetti, Carrot, '
 'Onion, Beef, cut into stripes (only meat), Rice noodles/vermicelli, Salad, '
 'leaf / salad, green, Bread, grain, Espresso, with caffeine, Banana, Mixed '
 'vegetables, Bread, wholemeal, Savoury puff pastry, Wine, white, Dried meat, '
 'Fresh cheese, Red radish, Hard cheese, Ham, raw, Bread, fruit, Oil & vinegar '
 'salad dressing, Tomato, Cauliflower, Potato-gnocchi, Wine, red, Sauce, '
 'cream, Pasta, linguini, parpadelle, Tagliatelle, French beans, Almonds, Dark '
 'chocolate, Mandarine, Semi-hard cheese, Croissant, Sushi, Berries, Biscuits, '
 'Thickened cream (> 35%), Corn, Celeriac, Alfa sprouts, Chickpeas, Leaf '
 'spinach, Rice, Chocolate cookies, Pineapple, Tart, Coffee, with caffeine, '
 'Focaccia, Pizza, with vegetables, baked, Soup, vegetable, Bread, toast, '
 'Potatoes steamed, Spaetzle, Frying sausage, Lasagne, meat, prepared, Boisson '
 'au glucose 50g, MÃ¼esli, Peanut butter, Chips, french fries, Mushroom, '
 'Ratatouille, Veggie burger, Country fries, Yaourt, yahourt, yogourt ou '
 'yoghourt, natural, Hummus, Fish, Beer, Peanut, Pizza, Margherita, baked, '
 'Pickle, Ham, cooked, Cake, chocolate, Bread, French (white flour), Sauce, '
 'mushroom, Rice, Basmati, Soup of lentils, Dahl (Dhal) , Pumpkin, Witloof '
 'chicory, Vegetable au gratin, baked, Balsamic salad dressing, Pasta, penne, '
 'Tea, peppermint, Soup, pumpkin, Quiche, with cheese, baked, with puff '
 'pastry, Mango, Green bean, steamed, without addition of salt, Cucumber, '
 'Bread, half white, Pasta, Beef, filet, Pasta, twist, Pasta, wholemeal, '
 'Walnut, Soft cheese, Salmon, smoked, Sweet pepper, Sauce, soya, Chicken, '
 'breast, Rice, whole-grain, Bread, nut, Green olives, Roll of half-white or '
 'white flour, with large void, Parmesan, Cappuccino, Flakes, oat, Mayonnaise, '
 'Chicken, Cheese for raclette, Orange, Goat cheese (soft), Tuna, Tomme, Apple '
 'pie, Rosti, Broccoli, Beans, kidney, White cabbage, Ketchup, Salt cake '
 '(vegetables, filled) , Pistachio, Feta, Salmon, Avocado, Sauce, pesto, '
 'Salad, rocket, Pizza, with ham, baked, GruyÃ¨re, Ristretto, with caffeine, '
 'Risotto, without cheese, cooked, Crunch MÃ¼esli, Braided white loaf, Peas, '
 'Chicken curry (cream/coconut milk. curry spices/paste)), Bolognaise sauce, '
 'Bacon, frying, Salami, Lentils, Mushrooms, Mashed potatoes, prepared, with '
 'full fat milk, with butter, Fennel, Chocolate mousse, Corn crisps, Sweet '
 'potato, BirchermÃ¼esli, prepared, no sugar added, Beetroot, steamed, without '
 'addition of salt, Sauce (savoury), Leek, Milk, Tea, Fruit salad , Bread, '
 "rye, Salad, lambs' ear, Potatoes au gratin, dauphinois, prepared, Red "
 'cabbage, Praline, Bread, black, Black olives, Mozzarella, Bacon, cooking, '
 'Pomegranate, Hamburger (Bread, meat, ketchup), Curry, vegetarian, Honey, '
 'Juice, orange, Cookies, Mixed nuts, Breadcrumbs (unspiced), Chicken, leg, '
 'Raspberries, Beef, sirloin steak, Salad dressing, Shrimp / prawn (large), '
 'Sour cream, Greek salad, Sauce, roast, Zucchini, Greek Yaourt, yahourt, '
 'yogourt ou yoghourt, Cashew nut, Meat terrine, patÃ©, Chicken, cut into '
 'stripes (only meat), Couscous, Bread, wholemeal toast, CrÃªpe, plain, Bread, '
 '5-grain, Tofu, Water, mineral, Ham croissant, Juice, apple, Falafel (balls), '
 'Egg, scrambled, prepared, Brioche, Bread, pita, Pasta, HÃ¶rnli, Blue mould '
 'cheese, Vegetable mix, peas and carrots, Quinoa, Crisps, Beef, Butter, '
 'spread, puree almond, Beef, minced (only meat), Hazelnut-chocolate '
 'spread(Nutella, Ovomaltine, Caotina), Chocolate, Nectarine, Ice tea , '
 'Applesauce, unsweetened, canned, Syrup (diluted, ready to drink), Sugar '
 'Melon , Bread, sourdough, Rusk, wholemeal, Gluten-free bread, Shrimp / prawn '
 '(small), French salad dressing, Pancakes, Milk chocolate, Pork, Dairy ice '
 'cream, Guacamole, Sausage, Herbal tea, Fruit coulis, Water with lemon juice, '
 'Brownie, Lemon, Veal sausage, Dates, Roll with pieces of chocolate, '
 'TaboulÃ©, prepared, with couscous, Croissant with chocolate filling, '
 'Eggplant, Sesame seeds, Cottage cheese, Fruit tart, Cream cheese, Tea, '
 'verveine, Tiramisu, Grits, polenta, maize flour, Pasta, noodles, Artichoke, '
 'Blueberries, Mixed seeds, Caprese salad (Tomato Mozzarella), Omelette, '
 'plain, Hazelnut, Kiwi, Dried raisins, Kolhrabi, Plums, Beetroot, raw, Cream, '
 'Fajita (bread only), Apricots, Kefir drink, Bread, Strawberries, Wine, '
 'rosÃ©, Watermelon, fresh, Green asparagus, White asparagus, Peach')

In [ ]:

category_names

Out[ ]:

['Water',
 'Pear',
 'Egg',
 'Grapes',
 'Butter',
 'Bread, white',
 'Jam',
 'Bread, whole wheat',
 'Apple',
 'Tea, green',
 'White coffee, with caffeine',
 'Tea, black',
 'Mixed salad (chopped without sauce)',
 'Cheese',
 'Tomato sauce',
 'Pasta, spaghetti',
 'Carrot',
 'Onion',
 'Beef, cut into stripes (only meat)',
 'Rice noodles/vermicelli',
 'Salad, leaf / salad, green',
 'Bread, grain',
 'Espresso, with caffeine',
 'Banana',
 'Mixed vegetables',
 'Bread, wholemeal',
 'Savoury puff pastry',
 'Wine, white',
 'Dried meat',
 'Fresh cheese',
 'Red radish',
 'Hard cheese',
 'Ham, raw',
 'Bread, fruit',
 'Oil & vinegar salad dressing',
 'Tomato',
 'Cauliflower',
 'Potato-gnocchi',
 'Wine, red',
 'Sauce, cream',
 'Pasta, linguini, parpadelle, Tagliatelle',
 'French beans',
 'Almonds',
 'Dark chocolate',
 'Mandarine',
 'Semi-hard cheese',
 'Croissant',
 'Sushi',
 'Berries',
 'Biscuits',
 'Thickened cream (> 35%)',
 'Corn',
 'Celeriac',
 'Alfa sprouts',
 'Chickpeas',
 'Leaf spinach',
 'Rice',
 'Chocolate cookies',
 'Pineapple',
 'Tart',
 'Coffee, with caffeine',
 'Focaccia',
 'Pizza, with vegetables, baked',
 'Soup, vegetable',
 'Bread, toast',
 'Potatoes steamed',
 'Spaetzle',
 'Frying sausage',
 'Lasagne, meat, prepared',
 'Boisson au glucose 50g',
 'MÃ¼esli',
 'Peanut butter',
 'Chips, french fries',
 'Mushroom',
 'Ratatouille',
 'Veggie burger',
 'Country fries',
 'Yaourt, yahourt, yogourt ou yoghourt, natural',
 'Hummus',
 'Fish',
 'Beer',
 'Peanut',
 'Pizza, Margherita, baked',
 'Pickle',
 'Ham, cooked',
 'Cake, chocolate',
 'Bread, French (white flour)',
 'Sauce, mushroom',
 'Rice, Basmati',
 'Soup of lentils, Dahl (Dhal) ',
 'Pumpkin',
 'Witloof chicory',
 'Vegetable au gratin, baked',
 'Balsamic salad dressing',
 'Pasta, penne',
 'Tea, peppermint',
 'Soup, pumpkin',
 'Quiche, with cheese, baked, with puff pastry',
 'Mango',
 'Green bean, steamed, without addition of salt',
 'Cucumber',
 'Bread, half white',
 'Pasta',
 'Beef, filet',
 'Pasta, twist',
 'Pasta, wholemeal',
 'Walnut',
 'Soft cheese',
 'Salmon, smoked',
 'Sweet pepper',
 'Sauce, soya',
 'Chicken, breast',
 'Rice, whole-grain',
 'Bread, nut',
 'Green olives',
 'Roll of half-white or white flour, with large void',
 'Parmesan',
 'Cappuccino',
 'Flakes, oat',
 'Mayonnaise',
 'Chicken',
 'Cheese for raclette',
 'Orange',
 'Goat cheese (soft)',
 'Tuna',
 'Tomme',
 'Apple pie',
 'Rosti',
 'Broccoli',
 'Beans, kidney',
 'White cabbage',
 'Ketchup',
 'Salt cake (vegetables, filled) ',
 'Pistachio',
 'Feta',
 'Salmon',
 'Avocado',
 'Sauce, pesto',
 'Salad, rocket',
 'Pizza, with ham, baked',
 'GruyÃ¨re',
 'Ristretto, with caffeine',
 'Risotto, without cheese, cooked',
 'Crunch MÃ¼esli',
 'Braided white loaf',
 'Peas',
 'Chicken curry (cream/coconut milk. curry spices/paste))',
 'Bolognaise sauce',
 'Bacon, frying',
 'Salami',
 'Lentils',
 'Mushrooms',
 'Mashed potatoes, prepared, with full fat milk, with butter',
 'Fennel',
 'Chocolate mousse',
 'Corn crisps',
 'Sweet potato',
 'BirchermÃ¼esli, prepared, no sugar added',
 'Beetroot, steamed, without addition of salt',
 'Sauce (savoury)',
 'Leek',
 'Milk',
 'Tea',
 'Fruit salad ',
 'Bread, rye',
 "Salad, lambs' ear",
 'Potatoes au gratin, dauphinois, prepared',
 'Red cabbage',
 'Praline',
 'Bread, black',
 'Black olives',
 'Mozzarella',
 'Bacon, cooking',
 'Pomegranate',
 'Hamburger (Bread, meat, ketchup)',
 'Curry, vegetarian',
 'Honey',
 'Juice, orange',
 'Cookies',
 'Mixed nuts',
 'Breadcrumbs (unspiced)',
 'Chicken, leg',
 'Raspberries',
 'Beef, sirloin steak',
 'Salad dressing',
 'Shrimp / prawn (large)',
 'Sour cream',
 'Greek salad',
 'Sauce, roast',
 'Zucchini',
 'Greek Yaourt, yahourt, yogourt ou yoghourt',
 'Cashew nut',
 'Meat terrine, patÃ©',
 'Chicken, cut into stripes (only meat)',
 'Couscous',
 'Bread, wholemeal toast',
 'CrÃªpe, plain',
 'Bread, 5-grain',
 'Tofu',
 'Water, mineral',
 'Ham croissant',
 'Juice, apple',
 'Falafel (balls)',
 'Egg, scrambled, prepared',
 'Brioche',
 'Bread, pita',
 'Pasta, HÃ¶rnli',
 'Blue mould cheese',
 'Vegetable mix, peas and carrots',
 'Quinoa',
 'Crisps',
 'Beef',
 'Butter, spread, puree almond',
 'Beef, minced (only meat)',
 'Hazelnut-chocolate spread(Nutella, Ovomaltine, Caotina)',
 'Chocolate',
 'Nectarine',
 'Ice tea ',
 'Applesauce, unsweetened, canned',
 'Syrup (diluted, ready to drink)',
 'Sugar Melon ',
 'Bread, sourdough',
 'Rusk, wholemeal',
 'Gluten-free bread',
 'Shrimp / prawn (small)',
 'French salad dressing',
 'Pancakes',
 'Milk chocolate',
 'Pork',
 'Dairy ice cream',
 'Guacamole',
 'Sausage',
 'Herbal tea',
 'Fruit coulis',
 'Water with lemon juice',
 'Brownie',
 'Lemon',
 'Veal sausage',
 'Dates',
 'Roll with pieces of chocolate',
 'TaboulÃ©, prepared, with couscous',
 'Croissant with chocolate filling',
 'Eggplant',
 'Sesame seeds',
 'Cottage cheese',
 'Fruit tart',
 'Cream cheese',
 'Tea, verveine',
 'Tiramisu',
 'Grits, polenta, maize flour',
 'Pasta, noodles',
 'Artichoke',
 'Blueberries',
 'Mixed seeds',
 'Caprese salad (Tomato Mozzarella)',
 'Omelette, plain',
 'Hazelnut',
 'Kiwi',
 'Dried raisins',
 'Kolhrabi',
 'Plums',
 'Beetroot, raw',
 'Cream',
 'Fajita (bread only)',
 'Apricots',
 'Kefir drink',
 'Bread',
 'Strawberries',
 'Wine, rosÃ©',
 'Watermelon, fresh',
 'Green asparagus',
 'White asparagus',
 'Peach']

In [ ]:

# Getting all categoriy with respective to their total images

no_images_per_category = {}

for n, i in enumerate(train_coco.getCatIds()):
  imgIds = train_coco.getImgIds(catIds=i)
  label = category_names[n]
  no_images_per_category[label] = len(imgIds)

img_info = pd.DataFrame(train_coco.loadImgs(train_coco.getImgIds()))

no_images_per_category

Out[ ]:

{'Alfa sprouts': 40,
 'Almonds': 159,
 'Apple': 504,
 'Apple pie': 104,
 'Applesauce, unsweetened, canned': 39,
 'Apricots': 91,
 'Artichoke': 43,
 'Avocado': 296,
 'Bacon, cooking': 47,
 'Bacon, frying': 127,
 'Balsamic salad dressing': 117,
 'Banana': 412,
 'Beans, kidney': 40,
 'Beef': 85,
 'Beef, cut into stripes (only meat)': 41,
 'Beef, filet': 51,
 'Beef, minced (only meat)': 65,
 'Beef, sirloin steak': 41,
 'Beer': 158,
 'Beetroot, raw': 45,
 'Beetroot, steamed, without addition of salt': 91,
 'Berries': 64,
 'BirchermÃ¼esli, prepared, no sugar added': 96,
 'Biscuits': 134,
 'Black olives': 132,
 'Blue mould cheese': 68,
 'Blueberries': 159,
 'Boisson au glucose 50g': 171,
 'Bolognaise sauce': 87,
 'Braided white loaf': 194,
 'Bread': 63,
 'Bread, 5-grain': 48,
 'Bread, French (white flour)': 121,
 'Bread, black': 54,
 'Bread, fruit': 48,
 'Bread, grain': 102,
 'Bread, half white': 76,
 'Bread, nut': 57,
 'Bread, pita': 38,
 'Bread, rye': 47,
 'Bread, sourdough': 124,
 'Bread, toast': 75,
 'Bread, white': 1273,
 'Bread, whole wheat': 223,
 'Bread, wholemeal': 901,
 'Bread, wholemeal toast': 65,
 'Breadcrumbs (unspiced)': 56,
 'Brioche': 45,
 'Broccoli': 261,
 'Brownie': 46,
 'Butter': 1008,
 'Butter, spread, puree almond': 45,
 'Cake, chocolate': 147,
 'Cappuccino': 139,
 'Caprese salad (Tomato Mozzarella)': 85,
 'Carrot': 893,
 'Cashew nut': 75,
 'Cauliflower': 116,
 'Celeriac': 53,
 'Cheese': 404,
 'Cheese for raclette': 77,
 'Chicken': 280,
 'Chicken curry (cream/coconut milk. curry spices/paste))': 80,
 'Chicken, breast': 91,
 'Chicken, cut into stripes (only meat)': 72,
 'Chicken, leg': 57,
 'Chickpeas': 92,
 'Chips, french fries': 238,
 'Chocolate': 82,
 'Chocolate cookies': 40,
 'Chocolate mousse': 53,
 'Coffee, with caffeine': 876,
 'Cookies': 58,
 'Corn': 130,
 'Corn crisps': 46,
 'Cottage cheese': 69,
 'Country fries': 52,
 'Couscous': 80,
 'Cream': 42,
 'Cream cheese': 52,
 'Crisps': 74,
 'Croissant': 144,
 'Croissant with chocolate filling': 66,
 'Crunch MÃ¼esli': 46,
 'CrÃªpe, plain': 103,
 'Cucumber': 382,
 'Curry, vegetarian': 91,
 'Dairy ice cream': 152,
 'Dark chocolate': 213,
 'Dates': 40,
 'Dried meat': 140,
 'Dried raisins': 44,
 'Egg': 626,
 'Egg, scrambled, prepared': 84,
 'Eggplant': 138,
 'Espresso, with caffeine': 391,
 'Fajita (bread only)': 56,
 'Falafel (balls)': 36,
 'Fennel': 134,
 'Feta': 107,
 'Fish': 119,
 'Flakes, oat': 48,
 'Focaccia': 40,
 'French beans': 160,
 'French salad dressing': 94,
 'Fresh cheese': 52,
 'Fruit coulis': 41,
 'Fruit salad ': 97,
 'Fruit tart': 49,
 'Frying sausage': 38,
 'Gluten-free bread': 68,
 'Goat cheese (soft)': 55,
 'Grapes': 94,
 'Greek Yaourt, yahourt, yogourt ou yoghourt': 39,
 'Greek salad': 43,
 'Green asparagus': 131,
 'Green bean, steamed, without addition of salt': 40,
 'Green olives': 88,
 'Grits, polenta, maize flour': 56,
 'GruyÃ¨re': 183,
 'Guacamole': 84,
 'Ham croissant': 44,
 'Ham, cooked': 151,
 'Ham, raw': 161,
 'Hamburger (Bread, meat, ketchup)': 101,
 'Hard cheese': 315,
 'Hazelnut': 39,
 'Hazelnut-chocolate spread(Nutella, Ovomaltine, Caotina)': 49,
 'Herbal tea': 126,
 'Honey': 230,
 'Hummus': 120,
 'Ice tea ': 44,
 'Jam': 502,
 'Juice, apple': 43,
 'Juice, orange': 164,
 'Kefir drink': 44,
 'Ketchup': 87,
 'Kiwi': 120,
 'Kolhrabi': 51,
 'Lasagne, meat, prepared': 54,
 'Leaf spinach': 194,
 'Leek': 65,
 'Lemon': 74,
 'Lentils': 83,
 'Mandarine': 138,
 'Mango': 64,
 'Mashed potatoes, prepared, with full fat milk, with butter': 96,
 'Mayonnaise': 198,
 'Meat terrine, patÃ©': 44,
 'Milk': 106,
 'Milk chocolate': 71,
 'Mixed nuts': 163,
 'Mixed salad (chopped without sauce)': 374,
 'Mixed seeds': 47,
 'Mixed vegetables': 624,
 'Mozzarella': 139,
 'Mushroom': 72,
 'Mushrooms': 118,
 'MÃ¼esli': 75,
 'Nectarine': 69,
 'Oil & vinegar salad dressing': 50,
 'Omelette, plain': 65,
 'Onion': 144,
 'Orange': 139,
 'Pancakes': 49,
 'Parmesan': 200,
 'Pasta': 136,
 'Pasta, HÃ¶rnli': 61,
 'Pasta, linguini, parpadelle, Tagliatelle': 75,
 'Pasta, noodles': 49,
 'Pasta, penne': 131,
 'Pasta, spaghetti': 256,
 'Pasta, twist': 100,
 'Pasta, wholemeal': 85,
 'Peach': 53,
 'Peanut': 68,
 'Peanut butter': 54,
 'Pear': 151,
 'Peas': 73,
 'Pickle': 138,
 'Pineapple': 54,
 'Pistachio': 65,
 'Pizza, Margherita, baked': 202,
 'Pizza, with ham, baked': 42,
 'Pizza, with vegetables, baked': 56,
 'Plums': 44,
 'Pomegranate': 72,
 'Pork': 69,
 'Potato-gnocchi': 58,
 'Potatoes au gratin, dauphinois, prepared': 49,
 'Potatoes steamed': 450,
 'Praline': 76,
 'Pumpkin': 48,
 'Quiche, with cheese, baked, with puff pastry': 51,
 'Quinoa': 120,
 'Raspberries': 116,
 'Ratatouille': 92,
 'Red cabbage': 79,
 'Red radish': 138,
 'Rice': 659,
 'Rice noodles/vermicelli': 50,
 'Rice, Basmati': 67,
 'Rice, whole-grain': 42,
 'Risotto, without cheese, cooked': 111,
 'Ristretto, with caffeine': 82,
 'Roll of half-white or white flour, with large void': 54,
 'Roll with pieces of chocolate': 53,
 'Rosti': 49,
 'Rusk, wholemeal': 38,
 'Salad dressing': 78,
 "Salad, lambs' ear": 101,
 'Salad, leaf / salad, green': 1189,
 'Salad, rocket': 130,
 'Salami': 173,
 'Salmon': 177,
 'Salmon, smoked': 162,
 'Salt cake (vegetables, filled) ': 67,
 'Sauce (savoury)': 129,
 'Sauce, cream': 131,
 'Sauce, mushroom': 39,
 'Sauce, pesto': 77,
 'Sauce, roast': 76,
 'Sauce, soya': 39,
 'Sausage': 80,
 'Savoury puff pastry': 49,
 'Semi-hard cheese': 101,
 'Sesame seeds': 43,
 'Shrimp / prawn (large)': 63,
 'Shrimp / prawn (small)': 58,
 'Soft cheese': 180,
 'Soup of lentils, Dahl (Dhal) ': 44,
 'Soup, pumpkin': 78,
 'Soup, vegetable': 96,
 'Sour cream': 55,
 'Spaetzle': 60,
 'Strawberries': 244,
 'Sugar Melon ': 102,
 'Sushi': 56,
 'Sweet pepper': 275,
 'Sweet potato': 121,
 'Syrup (diluted, ready to drink)': 51,
 'TaboulÃ©, prepared, with couscous': 50,
 'Tart': 98,
 'Tea': 353,
 'Tea, black': 109,
 'Tea, green': 140,
 'Tea, peppermint': 39,
 'Tea, verveine': 40,
 'Thickened cream (> 35%)': 81,
 'Tiramisu': 59,
 'Tofu': 120,
 'Tomato': 1069,
 'Tomato sauce': 290,
 'Tomme': 52,
 'Tuna': 96,
 'Veal sausage': 44,
 'Vegetable au gratin, baked': 52,
 'Vegetable mix, peas and carrots': 42,
 'Veggie burger': 35,
 'Walnut': 98,
 'Water': 1835,
 'Water with lemon juice': 87,
 'Water, mineral': 185,
 'Watermelon, fresh': 51,
 'White asparagus': 56,
 'White cabbage': 51,
 'White coffee, with caffeine': 274,
 'Wine, red': 545,
 'Wine, rosÃ©': 67,
 'Wine, white': 333,
 'Witloof chicory': 98,
 'Yaourt, yahourt, yogourt ou yoghourt, natural': 139,
 'Zucchini': 231}

In [ ]:

pd.DataFrame(no_images_per_category.items()).sort_values(1).iloc[::-1][0][:30].tolist()

Out[ ]:

['Water',
 'Bread, white',
 'Salad, leaf / salad, green',
 'Tomato',
 'Butter',
 'Bread, wholemeal',
 'Carrot',
 'Coffee, with caffeine',
 'Rice',
 'Egg',
 'Mixed vegetables',
 'Wine, red',
 'Apple',
 'Jam',
 'Potatoes steamed',
 'Banana',
 'Cheese',
 'Espresso, with caffeine',
 'Cucumber',
 'Mixed salad (chopped without sauce)',
 'Tea',
 'Wine, white',
 'Hard cheese',
 'Avocado',
 'Tomato sauce',
 'Chicken',
 'Sweet pepper',
 'White coffee, with caffeine',
 'Broccoli',
 'Pasta, spaghetti']

Data Visualisations¶

In [ ]:

fig = go.Figure([go.Bar(x=list(no_images_per_category.keys()), y=list(no_images_per_category.values()))])
fig.update_layout(
    title="No of Image per class",)
fig.show()

In [ ]:

pprint(f"Average number of image per class : { sum(list(no_images_per_category.values())) / len(list(no_images_per_category.values())) }")
pprint(f"Highest number of image per class is : { list(no_images_per_category.keys())[0]} of { list(no_images_per_category.values())[0] }")
pprint(f"Lowest number of image per class is : Veggie Burger of { sorted(list(no_images_per_category.values()))[0] }")

'Average number of image per class : 143.6153846153846'
'Highest number of image per class is : Water of 1835'
'Lowest number of image per class is : Veggie Burger of 35'

In [ ]:

fig = go.Figure(data=[go.Pie(labels=list(no_images_per_category.keys()), values=list(no_images_per_category.values()), 
                             hole=.3, textposition='inside', )], )
fig.update_layout(
    title="No of Image per class ( In pie )",)
fig.show()

In [ ]:

fig = go.Figure()
fig.add_trace(go.Histogram(x=img_info['height']))
fig.add_trace(go.Histogram(x=img_info['width']))

# Overlay both histograms
fig.update_layout(barmode='stack', title="Histogram of Image width & height",)


fig.show()

Image Visulisation 🖼️¶

In this section we are going to do imaghe visualisations!

In [ ]:

img_info

Out[ ]:

	id	file_name	width	height
0	65537	065537.jpg	464	464
1	65539	065539.jpg	464	464
2	65561	065561.jpg	464	464
3	65574	065574.jpg	831	830
4	65577	065577.jpg	480	480
...	...	...	...	...
24115	65500	065500.jpg	628	628
24116	65514	065514.jpg	1037	1036
24117	65516	065516.jpg	480	480
24118	65523	065523.jpg	480	480
24119	65524	065524.jpg	464	464

24120 rows × 4 columns

In [ ]:

len(train_annotations_data['annotations'][n]['segmentation']), len(train_annotations_data['annotations'][n]['bbox'])

Out[ ]:

(2, 4)

In [ ]:

for n, i in enumerate(tqdm((train_annotations_data['annotations']))):

  # if np.array(train_annotations_data['annotations'][n]['segmentation']).shape[0] != np.array(train_annotations_data['annotations'][n]['bbox']).shape[0]:

    # print(n)
  if np.array(train_annotations_data['annotations'][n]['segmentation']).shape[0] != 1:
    print(n)

  else:
    pass

/usr/local/lib/python3.6/dist-packages/ipykernel_launcher.py:6: VisibleDeprecationWarning:

Creating an ndarray from ragged nested sequences (which is a list-or-tuple of lists-or-tuples-or ndarrays with different lengths or shapes) is deprecated. If you meant to do this, you must specify 'dtype=object' when creating the ndarray

In [ ]:

img_no = 4

annIds = train_coco.getAnnIds(imgIds=train_annotations_data['annotations'][img_no]['image_id'])
anns = train_coco.loadAnns(annIds)

# load and render the image

plt.imshow(plt.imread(TRAIN_IMAGE_DIRECTIORY+train_annotations_data['images'][img_no]['file_name']))
plt.axis('off')
# Render annotations on top of the image
train_coco.showAnns(anns)

In [ ]:

w, h = 12, 12 # Setting width and height of every image
rows, cols = 5, 5 # Setting the number of image rows & cols

fig = plt.figure(figsize=(20, 14)) # Making the figure with size 

plt.title("Images") 
plt.axis('off')

# Going thought every cell in rows and cols
for i in range(1, cols * rows+1):

  annIds = train_coco.getAnnIds(imgIds=img_info['id'][i])
  anns = train_coco.loadAnns(annIds)

  fig.add_subplot(rows, cols, i)

  # Show the image

  img = plt.imread(TRAIN_IMAGE_DIRECTIORY+img_info['file_name'][i])

  for i in anns:
    [x,y,w,h] = i['bbox']
    cv2.rectangle(img, (int(x), int(y)), (int(x+h), int(y+w)), (255,0,0), 5)

  plt.imshow(img)

  # Render annotations on top of the image
  train_coco.showAnns(anns)


  # Setting the axis off
  plt.axis("off")

# Showing the figure
plt.show()

Creating our Dataset 🔨¶

In this section we are goind to fix out dataset first ( because there is some issues with dataset ( size mismatch ) & creating our dataset to put into the model

Fixing the Data¶

In [ ]:

np.array(train_annotations_data['annotations'][n]['segmentation']).shape , np.array(train_annotations_data['annotations'][n]['bbox']).shape

Out[ ]:

((1, 22), (4,))

In [ ]:

# Function for taking a annotation & directiory of images and returning new annoation json with fixed image size info
def fix_data(annotations, directiory):
  for n, i in enumerate(tqdm((annotations['images']))):
   
      img = cv2.imread(directiory+i["file_name"])
 

      if img.shape[0] != i['height']:
          annotations['images'][n]['height'] = img.shape[0]
          print(i["file_name"])
          print(annotations['images'][n], img.shape)

      if img.shape[1] != i['width']:
          annotations['images'][n]['width'] = img.shape[1]
          print(i["file_name"])
          print(annotations['images'][n], img.shape)

  return annotations

    
train_annotations_data = fix_data(train_annotations_data, TRAIN_IMAGE_DIRECTIORY)

  
with open('/content/train/new_ann.json', 'w') as f:
    json.dump(train_annotations_data, f)

In [ ]:

def fix_data_val(annotations, directiory):
  for n, i in enumerate(tqdm((annotations['images']))):
   
      img = cv2.imread(directiory+i["file_name"])
      
 

      if img.shape[0] != i['height']:
          print(n)
          annotations['images'][n]['height'] = img.shape[0]
          print(i["file_name"])
          print(annotations['images'][n], img.shape)

      if img.shape[1] != i['width']:
          annotations['images'][n]['width'] = img.shape[1]
          print(i["file_name"])
          print(annotations['images'][n], img.shape)

  return annotations

val_annotations_data = fix_data_val(val_annotations_data, VAL_IMAGE_DIRECTIORY)

with open('/content/val/new_ann.json', 'w') as f:
    json.dump(val_annotations_data, f)

---------------------------------------------------------------------------
AttributeError                            Traceback (most recent call last)
<ipython-input-95-2c84961ef844> in <module>()
     19   return annotations
     20 
---> 21 val_annotations_data = fix_data_val(val_annotations_data, VAL_IMAGE_DIRECTIORY)
     22 
     23 with open('/content/val/new_ann.json', 'w') as f:

<ipython-input-95-2c84961ef844> in fix_data_val(annotations, directiory)
      6 
      7 
----> 8       if img.shape[0] != i['height']:
      9           print(n)
     10           annotations['images'][n]['height'] = img.shape[0]

AttributeError: 'NoneType' object has no attribute 'shape'

In [ ]:

dict_addres = val_annotations_data['images'][748]
plt.imread(VAL_IMAGE_DIRECTIORY+dict_addres['file_name']).shape[:2], (dict_addres['height'], dict_addres['width'])

Out[ ]:

((3456, 4608), (4608, 3456))

In [ ]:

for n, i in enumerate(val_annotations_data['images']):
  if i['file_name'] == '053879.jpg':
    print(n)
    print("yes")

748
yes

Creating our Dataset¶

In [ ]:

train_annotations_path = '/content/train/new_ann.json'
train_images_path = '/content/train/images'

val_annotations_path = '/content/val/new_ann.json'
val_images_path = '/content/val/images'

In [ ]:

register_coco_instances("training_dataset", {},train_annotations_path, train_images_path)

register_coco_instances("validation_dataset", {},val_annotations_path, VAL_IMAGE_DIRECTIORY)

Creating our Model 🏭¶

We are going to make an Faster R-CNN Model using Detectron2, and setting up hyperpamaters to train our model

Creating R-CNN Model¶

In [ ]:

cfg = get_cfg()
# Check the model zoo and use any of the models ( from detectron2 github repo)
cfg.merge_from_file(model_zoo.get_config_file("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml"))
cfg.DATASETS.TRAIN = ("training_dataset",)
cfg.DATASETS.TEST = ()

cfg.DATALOADER.NUM_WORKERS = 2
# Loading pre trained weights
cfg.MODEL.WEIGHTS = model_zoo.get_checkpoint_url("COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml")

Setting up hyperparameters¶

In [ ]:

# No. of Batchs
cfg.SOLVER.IMS_PER_BATCH = 2

# Learning Rate: 
cfg.SOLVER.BASE_LR = 0.00025 

# No of Interations
cfg.SOLVER.MAX_ITER = 2000

# Images per batch (Batch Size) 
cfg.MODEL.ROI_HEADS.BATCH_SIZE_PER_IMAGE = 128   

# No of Categories(Classes) present
cfg.MODEL.ROI_HEADS.NUM_CLASSES = 273

cfg.OUTPUT_DIR = "/content/logs/"
os.makedirs(cfg.OUTPUT_DIR, exist_ok=True)

In [ ]:

trainer = DefaultTrainer(cfg) 
trainer.resume_or_load(resume=False)

[11/08 07:20:20 d2.engine.defaults]: Model:
GeneralizedRCNN(
  (backbone): FPN(
    (fpn_lateral2): Conv2d(256, 256, kernel_size=(1, 1), stride=(1, 1))
    (fpn_output2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (fpn_lateral3): Conv2d(512, 256, kernel_size=(1, 1), stride=(1, 1))
    (fpn_output3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (fpn_lateral4): Conv2d(1024, 256, kernel_size=(1, 1), stride=(1, 1))
    (fpn_output4): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (fpn_lateral5): Conv2d(2048, 256, kernel_size=(1, 1), stride=(1, 1))
    (fpn_output5): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
    (top_block): LastLevelMaxPool()
    (bottom_up): ResNet(
      (stem): BasicStem(
        (conv1): Conv2d(
          3, 64, kernel_size=(7, 7), stride=(2, 2), padding=(3, 3), bias=False
          (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
        )
      )
      (res2): Sequential(
        (0): BottleneckBlock(
          (shortcut): Conv2d(
            64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv1): Conv2d(
            64, 64, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
          )
          (conv2): Conv2d(
            64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
          )
          (conv3): Conv2d(
            64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
        )
        (1): BottleneckBlock(
          (conv1): Conv2d(
            256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
          )
          (conv2): Conv2d(
            64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
          )
          (conv3): Conv2d(
            64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
        )
        (2): BottleneckBlock(
          (conv1): Conv2d(
            256, 64, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
          )
          (conv2): Conv2d(
            64, 64, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=64, eps=1e-05)
          )
          (conv3): Conv2d(
            64, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
        )
      )
      (res3): Sequential(
        (0): BottleneckBlock(
          (shortcut): Conv2d(
            256, 512, kernel_size=(1, 1), stride=(2, 2), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
          (conv1): Conv2d(
            256, 128, kernel_size=(1, 1), stride=(2, 2), bias=False
            (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
          )
          (conv2): Conv2d(
            128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
          )
          (conv3): Conv2d(
            128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
        )
        (1): BottleneckBlock(
          (conv1): Conv2d(
            512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
          )
          (conv2): Conv2d(
            128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
          )
          (conv3): Conv2d(
            128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
        )
        (2): BottleneckBlock(
          (conv1): Conv2d(
            512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
          )
          (conv2): Conv2d(
            128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
          )
          (conv3): Conv2d(
            128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
        )
        (3): BottleneckBlock(
          (conv1): Conv2d(
            512, 128, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
          )
          (conv2): Conv2d(
            128, 128, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=128, eps=1e-05)
          )
          (conv3): Conv2d(
            128, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
        )
      )
      (res4): Sequential(
        (0): BottleneckBlock(
          (shortcut): Conv2d(
            512, 1024, kernel_size=(1, 1), stride=(2, 2), bias=False
            (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
          )
          (conv1): Conv2d(
            512, 256, kernel_size=(1, 1), stride=(2, 2), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv2): Conv2d(
            256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv3): Conv2d(
            256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
          )
        )
        (1): BottleneckBlock(
          (conv1): Conv2d(
            1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv2): Conv2d(
            256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv3): Conv2d(
            256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
          )
        )
        (2): BottleneckBlock(
          (conv1): Conv2d(
            1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv2): Conv2d(
            256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv3): Conv2d(
            256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
          )
        )
        (3): BottleneckBlock(
          (conv1): Conv2d(
            1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv2): Conv2d(
            256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv3): Conv2d(
            256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
          )
        )
        (4): BottleneckBlock(
          (conv1): Conv2d(
            1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv2): Conv2d(
            256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv3): Conv2d(
            256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
          )
        )
        (5): BottleneckBlock(
          (conv1): Conv2d(
            1024, 256, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv2): Conv2d(
            256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=256, eps=1e-05)
          )
          (conv3): Conv2d(
            256, 1024, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=1024, eps=1e-05)
          )
        )
      )
      (res5): Sequential(
        (0): BottleneckBlock(
          (shortcut): Conv2d(
            1024, 2048, kernel_size=(1, 1), stride=(2, 2), bias=False
            (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
          )
          (conv1): Conv2d(
            1024, 512, kernel_size=(1, 1), stride=(2, 2), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
          (conv2): Conv2d(
            512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
          (conv3): Conv2d(
            512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
          )
        )
        (1): BottleneckBlock(
          (conv1): Conv2d(
            2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
          (conv2): Conv2d(
            512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
          (conv3): Conv2d(
            512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
          )
        )
        (2): BottleneckBlock(
          (conv1): Conv2d(
            2048, 512, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
          (conv2): Conv2d(
            512, 512, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=512, eps=1e-05)
          )
          (conv3): Conv2d(
            512, 2048, kernel_size=(1, 1), stride=(1, 1), bias=False
            (norm): FrozenBatchNorm2d(num_features=2048, eps=1e-05)
          )
        )
      )
    )
  )
  (proposal_generator): RPN(
    (anchor_generator): DefaultAnchorGenerator(
      (cell_anchors): BufferList()
    )
    (rpn_head): StandardRPNHead(
      (conv): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (objectness_logits): Conv2d(256, 3, kernel_size=(1, 1), stride=(1, 1))
      (anchor_deltas): Conv2d(256, 12, kernel_size=(1, 1), stride=(1, 1))
    )
  )
  (roi_heads): StandardROIHeads(
    (box_pooler): ROIPooler(
      (level_poolers): ModuleList(
        (0): ROIAlign(output_size=(7, 7), spatial_scale=0.25, sampling_ratio=0, aligned=True)
        (1): ROIAlign(output_size=(7, 7), spatial_scale=0.125, sampling_ratio=0, aligned=True)
        (2): ROIAlign(output_size=(7, 7), spatial_scale=0.0625, sampling_ratio=0, aligned=True)
        (3): ROIAlign(output_size=(7, 7), spatial_scale=0.03125, sampling_ratio=0, aligned=True)
      )
    )
    (box_head): FastRCNNConvFCHead(
      (fc1): Linear(in_features=12544, out_features=1024, bias=True)
      (fc2): Linear(in_features=1024, out_features=1024, bias=True)
    )
    (box_predictor): FastRCNNOutputLayers(
      (cls_score): Linear(in_features=1024, out_features=274, bias=True)
      (bbox_pred): Linear(in_features=1024, out_features=1092, bias=True)
    )
    (mask_pooler): ROIPooler(
      (level_poolers): ModuleList(
        (0): ROIAlign(output_size=(14, 14), spatial_scale=0.25, sampling_ratio=0, aligned=True)
        (1): ROIAlign(output_size=(14, 14), spatial_scale=0.125, sampling_ratio=0, aligned=True)
        (2): ROIAlign(output_size=(14, 14), spatial_scale=0.0625, sampling_ratio=0, aligned=True)
        (3): ROIAlign(output_size=(14, 14), spatial_scale=0.03125, sampling_ratio=0, aligned=True)
      )
    )
    (mask_head): MaskRCNNConvUpsampleHead(
      (mask_fcn1): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (mask_fcn2): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (mask_fcn3): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (mask_fcn4): Conv2d(256, 256, kernel_size=(3, 3), stride=(1, 1), padding=(1, 1))
      (deconv): ConvTranspose2d(256, 256, kernel_size=(2, 2), stride=(2, 2))
      (predictor): Conv2d(256, 273, kernel_size=(1, 1), stride=(1, 1))
    )
  )
)
[11/08 07:20:22 d2.data.datasets.coco]: Loading /content/train/new_ann.json takes 1.51 seconds.
WARNING [11/08 07:20:22 d2.data.datasets.coco]: 
Category ids in annotations are not in [1, #categories]! We'll apply a mapping for you.

[11/08 07:20:22 d2.data.datasets.coco]: Loaded 24120 images in COCO format from /content/train/new_ann.json
[11/08 07:20:22 d2.data.build]: Removed 0 images with no usable annotations. 24120 images left.
[11/08 07:20:23 d2.data.build]: Distribution of instances among all 273 categories:
|   category    | #instances   |   category    | #instances   |   category    | #instances   |
|:-------------:|:-------------|:-------------:|:-------------|:-------------:|:-------------|
| beetroot-st.. | 91           | green-bean-.. | 40           | watermelon-.. | 51           |
| pizza-with-.. | 42           | pizza-with-.. | 56           | applesauce-.. | 40           |
| quiche-with.. | 52           | sweet-potato  | 121          | country-fries | 52           |
| potato-gnoc.. | 58           | potatoes-st.. | 452          | chips-frenc.. | 238          |
|     rosti     | 49           | vegetable-m.. | 42           | mixed-veget.. | 627          |
|  ratatouille  | 92           | mixed-salad.. | 374          | leaf-spinach  | 194          |
| witloof-chi.. | 98           | salad-rocket  | 130          | salad-leaf-.. | 1190         |
| salad-lambs.. | 101          |   artichoke   | 43           |   eggplant    | 138          |
|    avocado    | 298          | french-beans  | 160          |    pickle     | 138          |
|   cucumber    | 383          |    pumpkin    | 48           | sweet-pepper  | 279          |
|    tomato     | 1072         |   zucchini    | 233          |  red-radish   | 138          |
| beetroot-raw  | 45           |    carrot     | 898          |   celeriac    | 53           |
|  cauliflower  | 116          |   broccoli    | 262          |   kolhrabi    | 51           |
|  red-cabbage  | 79           | white-cabbage | 51           |   mushroom    | 72           |
|   mushrooms   | 118          |     peas      | 73           |     corn      | 130          |
|     leek      | 65           |     onion     | 146          |    fennel     | 134          |
| green-aspar.. | 132          | white-aspar.. | 56           | alfa-sprouts  | 40           |
| beans-kidney  | 40           |   chickpeas   | 92           |    lentils    | 83           |
|   pineapple   | 54           |     apple     | 504          |  pomegranate  | 72           |
|   apricots    | 94           |    banana     | 412          |    berries    | 64           |
|     pear      | 151          |     dates     | 42           | strawberries  | 245          |
|  fruit-salad  | 97           |  blueberries  | 159          |  raspberries  | 116          |
|     kiwi      | 121          |   mandarine   | 138          |     mango     | 64           |
|  sugar-melon  | 102          |   nectarine   | 69           |    orange     | 139          |
|     peach     | 54           |     plums     | 45           |    grapes     | 96           |
| dried-raisins | 45           |     lemon     | 74           | peanut-butter | 54           |
|  mixed-seeds  | 47           |    almonds    | 159          |    walnut     | 98           |
|  cashew-nut   | 75           |    peanut     | 68           |   hazelnut    | 39           |
|  mixed-nuts   | 163          |   pistachio   | 65           | sesame-seeds  | 43           |
| green-olives  | 88           | black-olives  | 132          |     milk      | 106          |
|  kefir-drink  | 44           | cottage-che.. | 69           | blue-mould-.. | 69           |
|     feta      | 107          | fresh-cheese  | 52           |   gruya-re    | 183          |
| semi-hard-c.. | 101          |  hard-cheese  | 319          |    cheese     | 405          |
|  mozzarella   | 139          |   parmesan    | 201          | cheese-for-.. | 77           |
| cream-cheese  | 52           |     tomme     | 52           |  soft-cheese  | 181          |
|   tiramisu    | 59           |     cream     | 42           |  sour-cream   | 55           |
| thickened-c.. | 81           | dairy-ice-c.. | 154          |  flakes-oat   | 48           |
| rice-noodle.. | 50           |   couscous    | 80           | grits-polen.. | 57           |
|    quinoa     | 121          |     rice      | 659          | rice-basmati  | 67           |
| rice-whole-.. | 42           |   spaetzle    | 60           |     pasta     | 136          |
| pasta-haprnli | 61           | pasta-lingu.. | 75           | pasta-noodles | 49           |
|  pasta-penne  | 132          | pasta-spagh.. | 256          |  pasta-twist  | 100          |
| pasta-whole.. | 85           | bread-frenc.. | 123          |     bread     | 63           |
| bread-5-grain | 48           |  bread-fruit  | 49           | bread-half-.. | 76           |
|  bread-grain  | 104          |   bread-nut   | 57           |  bread-pita   | 39           |
|   bread-rye   | 47           | bread-whole.. | 228          | bread-sourd.. | 125          |
|  bread-black  | 56           |  bread-toast  | 75           | bread-whole.. | 66           |
| bread-whole.. | 905          |  bread-white  | 1276         |    brioche    | 46           |
| roll-of-hal.. | 54           | roll-with-p.. | 53           |   focaccia    | 41           |
|   croissant   | 144          | braided-whi.. | 194          | breadcrumbs.. | 56           |
| rusk-wholem.. | 41           | crunch-ma1-.. | 46           |   ma1-4esli   | 75           |
|     beef      | 85           | beef-sirloi.. | 41           |  beef-filet   | 51           |
| beef-minced.. | 65           | beef-cut-in.. | 41           |     pork      | 69           |
|    chicken    | 280          | chicken-bre.. | 91           | chicken-cut.. | 74           |
|  chicken-leg  | 57           | frying-saus.. | 38           |  dried-meat   | 140          |
| veal-sausage  | 44           |    salami     | 173          |  ham-cooked   | 151          |
|    ham-raw    | 161          | bacon-frying  | 127          | bacon-cooking | 48           |
| meat-terrin.. | 45           |    sausage    | 81           | veggie-burger | 36           |
|     tofu      | 121          |     fish      | 119          |    salmon     | 177          |
|     tuna      | 96           | shrimp-praw.. | 58           | shrimp-praw.. | 63           |
|      egg      | 632          |    butter     | 1010         |    praline    | 77           |
|      jam      | 505          |     honey     | 230          | dark-chocol.. | 213          |
| milk-chocol.. | 71           |   chocolate   | 82           | hazelnut-ch.. | 49           |
|   apple-pie   | 104          |    brownie    | 46           | craape-plain  | 103          |
|  fruit-tart   | 49           | cake-chocol.. | 147          | omelette-pl.. | 65           |
|     tart      | 99           | croissant-w.. | 66           |    cookies    | 58           |
|   biscuits    | 135          | chocolate-c.. | 40           |  juice-apple  | 43           |
| juice-orange  | 164          |    ice-tea    | 44           | syrup-dilut.. | 51           |
|      tea      | 353          |  cappuccino   | 139          | espresso-wi.. | 391          |
| coffee-with.. | 876          | white-coffe.. | 274          | ristretto-w.. | 82           |
|   tea-green   | 140          |   tea-black   | 109          | tea-verveine  | 40           |
|  herbal-tea   | 126          | tea-pepperm.. | 39           |     water     | 1837         |
| water-mineral | 185          |  wine-rosa-c  | 67           |   wine-red    | 546          |
|  wine-white   | 333          |     beer      | 158          | sauce-savoury | 129          |
|  sauce-roast  | 76           |  sauce-pesto  | 77           | sauce-mushr.. | 39           |
|  sauce-cream  | 132          |    ketchup    | 87           | bolognaise-.. | 87           |
| tomato-sauce  | 292          | salad-dress.. | 78           | balsamic-sa.. | 117          |
| french-sala.. | 95           | oil-vinegar.. | 50           |   guacamole   | 85           |
|  mayonnaise   | 198          |  sauce-soya   | 39           | soup-vegeta.. | 96           |
| soup-pumpkin  | 78           | falafel-balls | 37           | savoury-puf.. | 50           |
|  corn-crisps  | 46           |    crisps     | 74           | ham-croissant | 44           |
| salt-cake-v.. | 67           | hamburger-b.. | 103          | lasagne-mea.. | 54           |
| mashed-pota.. | 96           | pizza-margh.. | 206          |     sushi     | 59           |
|   pancakes    | 49           |    hummus     | 120          |  greek-salad  | 43           |
| chocolate-m.. | 53           | caprese-sal.. | 85           | taboula-c-p.. | 50           |
| risotto-wit.. | 111          | salmon-smoked | 162          | egg-scrambl.. | 84           |
| boisson-au-.. | 171          | chicken-cur.. | 80           | potatoes-au.. | 49           |
| bircherma1-.. | 96           | fajita-brea.. | 56           | butter-spre.. | 45           |
| water-with-.. | 87           | gluten-free.. | 68           | fruit-coulis  | 41           |
| greek-yaour.. | 39           | soup-of-len.. | 44           | vegetable-a.. | 52           |
| curry-veget.. | 91           | yaourt-yaho.. | 139          | goat-cheese.. | 55           |
|               |              |               |              |               |              |
|     total     | 39328        |               |              |               |              |
[11/08 07:20:23 d2.data.common]: Serializing 24120 elements to byte tensors and concatenating them all ...
[11/08 07:20:23 d2.data.common]: Serialized dataset takes 70.18 MiB
[11/08 07:20:23 d2.data.detection_utils]: TransformGens used in training: [ResizeShortestEdge(short_edge_length=(640, 672, 704, 736, 768, 800), max_size=1333, sample_style='choice'), RandomFlip()]
[11/08 07:20:23 d2.data.build]: Using training sampler TrainingSampler

model_final_f10217.pkl: 178MB [00:01, 94.1MB/s]                           
Skip loading parameter 'roi_heads.box_predictor.cls_score.weight' to the model due to incompatible shapes: (81, 1024) in the checkpoint but (274, 1024) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.cls_score.bias' to the model due to incompatible shapes: (81,) in the checkpoint but (274,) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.bbox_pred.weight' to the model due to incompatible shapes: (320, 1024) in the checkpoint but (1092, 1024) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.box_predictor.bbox_pred.bias' to the model due to incompatible shapes: (320,) in the checkpoint but (1092,) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.mask_head.predictor.weight' to the model due to incompatible shapes: (80, 256, 1, 1) in the checkpoint but (273, 256, 1, 1) in the model! You might want to double check if this is expected.
Skip loading parameter 'roi_heads.mask_head.predictor.bias' to the model due to incompatible shapes: (80,) in the checkpoint but (273,) in the model! You might want to double check if this is expected.

Training the Model 🚂¶

Setting up Tensorboard & finally training our model!

Setting up Tensorboard¶

In [ ]:

!pip install --upgrade git+git://github.com/wandb/client.git

Collecting git+git://github.com/wandb/client.git
  Cloning git://github.com/wandb/client.git to /tmp/pip-req-build-coaxvwht
  Running command git clone -q git://github.com/wandb/client.git /tmp/pip-req-build-coaxvwht
  Installing build dependencies ... done
  Getting requirements to build wheel ... done
    Preparing wheel metadata ... done
Collecting shortuuid>=0.5.0
  Downloading https://files.pythonhosted.org/packages/25/a6/2ecc1daa6a304e7f1b216f0896b26156b78e7c38e1211e9b798b4716c53d/shortuuid-1.0.1-py3-none-any.whl
Collecting GitPython>=1.0.0
  Downloading https://files.pythonhosted.org/packages/c0/d7/b2b0672e0331567157adf9281f41ee731c412ee518ca5e6552c27fa73c91/GitPython-3.1.9-py3-none-any.whl (159kB)
     |████████████████████████████████| 163kB 12.2MB/s 
Requirement already satisfied, skipping upgrade: protobuf>=3.12.0 in /usr/local/lib/python3.6/dist-packages (from wandb==0.10.7.dev1) (3.12.4)
Requirement already satisfied, skipping upgrade: python-dateutil>=2.6.1 in /usr/local/lib/python3.6/dist-packages (from wandb==0.10.7.dev1) (2.8.1)
Collecting docker-pycreds>=0.4.0
  Downloading https://files.pythonhosted.org/packages/f5/e8/f6bd1eee09314e7e6dee49cbe2c5e22314ccdb38db16c9fc72d2fa80d054/docker_pycreds-0.4.0-py2.py3-none-any.whl
Requirement already satisfied, skipping upgrade: promise<3,>=2.0 in /usr/local/lib/python3.6/dist-packages (from wandb==0.10.7.dev1) (2.3)
Requirement already satisfied, skipping upgrade: six>=1.13.0 in /usr/local/lib/python3.6/dist-packages (from wandb==0.10.7.dev1) (1.15.0)
Collecting sentry-sdk>=0.4.0
  Downloading https://files.pythonhosted.org/packages/1f/08/5eb320799e3085ccc66ec0fc3360421302803f3b784f74959564dbc6cdc9/sentry_sdk-0.19.0-py2.py3-none-any.whl (120kB)
     |████████████████████████████████| 122kB 30.3MB/s 
Collecting subprocess32>=3.5.3
  Downloading https://files.pythonhosted.org/packages/32/c8/564be4d12629b912ea431f1a50eb8b3b9d00f1a0b1ceff17f266be190007/subprocess32-3.5.4.tar.gz (97kB)
     |████████████████████████████████| 102kB 11.9MB/s 
Collecting configparser>=3.8.1
  Downloading https://files.pythonhosted.org/packages/08/b2/ef713e0e67f6e7ec7d59aea3ee78d05b39c15930057e724cc6d362a8c3bb/configparser-5.0.1-py3-none-any.whl
Requirement already satisfied, skipping upgrade: PyYAML in /usr/local/lib/python3.6/dist-packages (from wandb==0.10.7.dev1) (5.1)
Requirement already satisfied, skipping upgrade: requests<3,>=2.0.0 in /usr/local/lib/python3.6/dist-packages (from wandb==0.10.7.dev1) (2.23.0)
Requirement already satisfied, skipping upgrade: psutil>=5.0.0 in /usr/local/lib/python3.6/dist-packages (from wandb==0.10.7.dev1) (5.4.8)
Requirement already satisfied, skipping upgrade: Click>=7.0 in /usr/local/lib/python3.6/dist-packages (from wandb==0.10.7.dev1) (7.1.2)
Collecting watchdog>=0.8.3
  Downloading https://files.pythonhosted.org/packages/0e/06/121302598a4fc01aca942d937f4a2c33430b7181137b35758913a8db10ad/watchdog-0.10.3.tar.gz (94kB)
     |████████████████████████████████| 102kB 12.0MB/s 
Collecting gitdb<5,>=4.0.1
  Downloading https://files.pythonhosted.org/packages/48/11/d1800bca0a3bae820b84b7d813ad1eff15a48a64caea9c823fc8c1b119e8/gitdb-4.0.5-py3-none-any.whl (63kB)
     |████████████████████████████████| 71kB 7.6MB/s 
Requirement already satisfied, skipping upgrade: setuptools in /usr/local/lib/python3.6/dist-packages (from protobuf>=3.12.0->wandb==0.10.7.dev1) (50.3.0)
Requirement already satisfied, skipping upgrade: urllib3>=1.10.0 in /usr/local/lib/python3.6/dist-packages (from sentry-sdk>=0.4.0->wandb==0.10.7.dev1) (1.24.3)
Requirement already satisfied, skipping upgrade: certifi in /usr/local/lib/python3.6/dist-packages (from sentry-sdk>=0.4.0->wandb==0.10.7.dev1) (2020.6.20)
Requirement already satisfied, skipping upgrade: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.0.0->wandb==0.10.7.dev1) (3.0.4)
Requirement already satisfied, skipping upgrade: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests<3,>=2.0.0->wandb==0.10.7.dev1) (2.10)
Collecting pathtools>=0.1.1
  Downloading https://files.pythonhosted.org/packages/e7/7f/470d6fcdf23f9f3518f6b0b76be9df16dcc8630ad409947f8be2eb0ed13a/pathtools-0.1.2.tar.gz
Collecting smmap<4,>=3.0.1
  Downloading https://files.pythonhosted.org/packages/b0/9a/4d409a6234eb940e6a78dfdfc66156e7522262f5f2fecca07dc55915952d/smmap-3.0.4-py2.py3-none-any.whl
Building wheels for collected packages: wandb
  Building wheel for wandb (PEP 517) ... done
  Created wheel for wandb: filename=wandb-0.10.7.dev1-cp36-none-any.whl size=1693091 sha256=3222c8a75719b9aa12ceb6adced4c065c1111c0d8098a7cc6230f65173b46b03
  Stored in directory: /tmp/pip-ephem-wheel-cache-s8m9f4gs/wheels/f5/bb/45/4b2ff6f79b35baf639a4eaf2673c5df97fa21335bf82cc7bfa
Successfully built wandb
Building wheels for collected packages: subprocess32, watchdog, pathtools
  Building wheel for subprocess32 (setup.py) ... done
  Created wheel for subprocess32: filename=subprocess32-3.5.4-cp36-none-any.whl size=6489 sha256=797ae5ded101c517d2dd89cd2e3835d82137e9153a07043ce1f862c595fbf2d4
  Stored in directory: /root/.cache/pip/wheels/68/39/1a/5e402bdfdf004af1786c8b853fd92f8c4a04f22aad179654d1
  Building wheel for watchdog (setup.py) ... done
  Created wheel for watchdog: filename=watchdog-0.10.3-cp36-none-any.whl size=73873 sha256=39f3a578f4c871a3c54ea53c2951051acfd3b91639c243847cd543ebd4d7560e
  Stored in directory: /root/.cache/pip/wheels/a8/1d/38/2c19bb311f67cc7b4d07a2ec5ea36ab1a0a0ea50db994a5bc7
  Building wheel for pathtools (setup.py) ... done
  Created wheel for pathtools: filename=pathtools-0.1.2-cp36-none-any.whl size=8785 sha256=07700793fd010b8da19b70de0c6006c38fb9e111783c482816162da157e26346
  Stored in directory: /root/.cache/pip/wheels/0b/04/79/c3b0c3a0266a3cb4376da31e5bfe8bba0c489246968a68e843
Successfully built subprocess32 watchdog pathtools
Installing collected packages: shortuuid, smmap, gitdb, GitPython, docker-pycreds, sentry-sdk, subprocess32, configparser, pathtools, watchdog, wandb
Successfully installed GitPython-3.1.9 configparser-5.0.1 docker-pycreds-0.4.0 gitdb-4.0.5 pathtools-0.1.2 sentry-sdk-0.19.0 shortuuid-1.0.1 smmap-3.0.4 subprocess32-3.5.4 wandb-0.10.7.dev1 watchdog-0.10.3

In [ ]:

import wandb
wandb.init(project='food detection', sync_tensorboard=True)

wandb: Appending key for api.wandb.ai to your netrc file: /root/.netrc
wandb: wandb version 0.10.7 is available!  To upgrade, please run:
wandb:  $ pip install wandb --upgrade

Tracking run with wandb version 0.10.7.dev1
Syncing run helpful-sun-12 to Weights & Biases (Documentation).
Project page: https://wandb.ai/shubhamai/food%20detection
Run page: https://wandb.ai/shubhamai/food%20detection/runs/1kuucz5k
Run data is saved locally in wandb/run-20201017_060820-1kuucz5k

Out[ ]:

Run(1kuucz5k)

In [ ]:

%load_ext tensorboard

In [ ]:

%tensorboard --logdir logs

Output hidden; open in https://colab.research.google.com to view.

Start Training!¶

In [ ]:

trainer.train()

[11/08 07:20:34 d2.engine.train_loop]: Starting training from iteration 0
[11/08 07:20:41 d2.utils.events]:  eta: 0:10:28  iter: 19  total_loss: 6.803  loss_cls: 5.567  loss_box_reg: 0.527  loss_mask: 0.692  loss_rpn_cls: 0.026  loss_rpn_loc: 0.016  time: 0.3200  data_time: 0.0174  lr: 0.000005  max_mem: 1926M
[11/08 07:20:48 d2.utils.events]:  eta: 0:10:25  iter: 39  total_loss: 6.706  loss_cls: 5.464  loss_box_reg: 0.521  loss_mask: 0.693  loss_rpn_cls: 0.008  loss_rpn_loc: 0.012  time: 0.3217  data_time: 0.0074  lr: 0.000010  max_mem: 2090M
[11/08 07:20:54 d2.utils.events]:  eta: 0:10:24  iter: 59  total_loss: 6.288  loss_cls: 4.979  loss_box_reg: 0.626  loss_mask: 0.693  loss_rpn_cls: 0.023  loss_rpn_loc: 0.021  time: 0.3261  data_time: 0.0117  lr: 0.000015  max_mem: 2090M
[11/08 07:21:01 d2.utils.events]:  eta: 0:10:20  iter: 79  total_loss: 5.606  loss_cls: 4.338  loss_box_reg: 0.560  loss_mask: 0.692  loss_rpn_cls: 0.034  loss_rpn_loc: 0.014  time: 0.3271  data_time: 0.0073  lr: 0.000020  max_mem: 2090M
[11/08 07:21:08 d2.utils.events]:  eta: 0:10:27  iter: 99  total_loss: 4.539  loss_cls: 3.214  loss_box_reg: 0.667  loss_mask: 0.692  loss_rpn_cls: 0.011  loss_rpn_loc: 0.010  time: 0.3297  data_time: 0.0080  lr: 0.000025  max_mem: 2090M
[11/08 07:21:15 d2.utils.events]:  eta: 0:10:27  iter: 119  total_loss: 3.083  loss_cls: 1.640  loss_box_reg: 0.653  loss_mask: 0.690  loss_rpn_cls: 0.004  loss_rpn_loc: 0.012  time: 0.3325  data_time: 0.0075  lr: 0.000030  max_mem: 2090M
[11/08 07:21:22 d2.utils.events]:  eta: 0:10:24  iter: 139  total_loss: 2.526  loss_cls: 1.218  loss_box_reg: 0.590  loss_mask: 0.690  loss_rpn_cls: 0.007  loss_rpn_loc: 0.017  time: 0.3344  data_time: 0.0088  lr: 0.000035  max_mem: 2191M
[11/08 07:21:28 d2.utils.events]:  eta: 0:10:17  iter: 159  total_loss: 2.498  loss_cls: 1.169  loss_box_reg: 0.650  loss_mask: 0.691  loss_rpn_cls: 0.009  loss_rpn_loc: 0.015  time: 0.3344  data_time: 0.0074  lr: 0.000040  max_mem: 2191M
[11/08 07:21:35 d2.utils.events]:  eta: 0:10:14  iter: 179  total_loss: 2.692  loss_cls: 1.268  loss_box_reg: 0.695  loss_mask: 0.688  loss_rpn_cls: 0.014  loss_rpn_loc: 0.013  time: 0.3349  data_time: 0.0062  lr: 0.000045  max_mem: 2191M
[11/08 07:21:42 d2.utils.events]:  eta: 0:10:11  iter: 199  total_loss: 2.448  loss_cls: 1.124  loss_box_reg: 0.600  loss_mask: 0.690  loss_rpn_cls: 0.029  loss_rpn_loc: 0.017  time: 0.3380  data_time: 0.0198  lr: 0.000050  max_mem: 2191M
[11/08 07:21:49 d2.utils.events]:  eta: 0:10:05  iter: 219  total_loss: 2.777  loss_cls: 1.323  loss_box_reg: 0.725  loss_mask: 0.689  loss_rpn_cls: 0.024  loss_rpn_loc: 0.017  time: 0.3389  data_time: 0.0069  lr: 0.000055  max_mem: 2191M
[11/08 07:21:56 d2.utils.events]:  eta: 0:10:01  iter: 239  total_loss: 2.624  loss_cls: 1.256  loss_box_reg: 0.707  loss_mask: 0.685  loss_rpn_cls: 0.008  loss_rpn_loc: 0.017  time: 0.3404  data_time: 0.0063  lr: 0.000060  max_mem: 2351M
[11/08 07:22:03 d2.utils.events]:  eta: 0:09:55  iter: 259  total_loss: 2.322  loss_cls: 1.014  loss_box_reg: 0.534  loss_mask: 0.685  loss_rpn_cls: 0.011  loss_rpn_loc: 0.010  time: 0.3410  data_time: 0.0176  lr: 0.000065  max_mem: 2351M
[11/08 07:22:11 d2.utils.events]:  eta: 0:09:51  iter: 279  total_loss: 2.551  loss_cls: 1.178  loss_box_reg: 0.628  loss_mask: 0.684  loss_rpn_cls: 0.016  loss_rpn_loc: 0.017  time: 0.3421  data_time: 0.0088  lr: 0.000070  max_mem: 2351M
[11/08 07:22:18 d2.utils.events]:  eta: 0:09:45  iter: 299  total_loss: 2.351  loss_cls: 1.034  loss_box_reg: 0.566  loss_mask: 0.687  loss_rpn_cls: 0.007  loss_rpn_loc: 0.009  time: 0.3426  data_time: 0.0063  lr: 0.000075  max_mem: 2351M
[11/08 07:22:24 d2.utils.events]:  eta: 0:09:36  iter: 319  total_loss: 2.496  loss_cls: 1.144  loss_box_reg: 0.617  loss_mask: 0.677  loss_rpn_cls: 0.009  loss_rpn_loc: 0.010  time: 0.3413  data_time: 0.0087  lr: 0.000080  max_mem: 2351M
[11/08 07:22:31 d2.utils.events]:  eta: 0:09:30  iter: 339  total_loss: 2.349  loss_cls: 1.029  loss_box_reg: 0.611  loss_mask: 0.678  loss_rpn_cls: 0.003  loss_rpn_loc: 0.009  time: 0.3415  data_time: 0.0078  lr: 0.000085  max_mem: 2351M
[11/08 07:22:38 d2.utils.events]:  eta: 0:09:23  iter: 359  total_loss: 2.250  loss_cls: 1.026  loss_box_reg: 0.571  loss_mask: 0.679  loss_rpn_cls: 0.009  loss_rpn_loc: 0.011  time: 0.3418  data_time: 0.0060  lr: 0.000090  max_mem: 2351M

/usr/local/lib/python3.6/dist-packages/PIL/TiffImagePlugin.py:590: UserWarning:

Metadata Warning, tag 282 had too many entries: 2, expected 1

/usr/local/lib/python3.6/dist-packages/PIL/TiffImagePlugin.py:590: UserWarning:

Metadata Warning, tag 283 had too many entries: 2, expected 1

[11/08 07:22:46 d2.utils.events]:  eta: 0:09:18  iter: 379  total_loss: 2.436  loss_cls: 1.109  loss_box_reg: 0.648  loss_mask: 0.675  loss_rpn_cls: 0.010  loss_rpn_loc: 0.014  time: 0.3448  data_time: 0.0291  lr: 0.000095  max_mem: 2723M
[11/08 07:22:53 d2.utils.events]:  eta: 0:09:12  iter: 399  total_loss: 2.420  loss_cls: 1.106  loss_box_reg: 0.596  loss_mask: 0.673  loss_rpn_cls: 0.014  loss_rpn_loc: 0.013  time: 0.3452  data_time: 0.0086  lr: 0.000100  max_mem: 2723M
[11/08 07:23:00 d2.utils.events]:  eta: 0:09:05  iter: 419  total_loss: 2.638  loss_cls: 1.189  loss_box_reg: 0.760  loss_mask: 0.674  loss_rpn_cls: 0.005  loss_rpn_loc: 0.011  time: 0.3454  data_time: 0.0075  lr: 0.000105  max_mem: 2723M
[11/08 07:23:07 d2.utils.events]:  eta: 0:09:00  iter: 439  total_loss: 2.604  loss_cls: 1.206  loss_box_reg: 0.669  loss_mask: 0.676  loss_rpn_cls: 0.007  loss_rpn_loc: 0.012  time: 0.3458  data_time: 0.0078  lr: 0.000110  max_mem: 2723M
[11/08 07:23:14 d2.utils.events]:  eta: 0:08:54  iter: 459  total_loss: 2.164  loss_cls: 0.941  loss_box_reg: 0.596  loss_mask: 0.668  loss_rpn_cls: 0.010  loss_rpn_loc: 0.012  time: 0.3468  data_time: 0.0064  lr: 0.000115  max_mem: 2723M
[11/08 07:23:22 d2.utils.events]:  eta: 0:08:48  iter: 479  total_loss: 2.608  loss_cls: 1.188  loss_box_reg: 0.663  loss_mask: 0.659  loss_rpn_cls: 0.008  loss_rpn_loc: 0.014  time: 0.3476  data_time: 0.0094  lr: 0.000120  max_mem: 2723M
[11/08 07:23:29 d2.utils.events]:  eta: 0:08:42  iter: 499  total_loss: 2.343  loss_cls: 1.026  loss_box_reg: 0.586  loss_mask: 0.659  loss_rpn_cls: 0.013  loss_rpn_loc: 0.011  time: 0.3486  data_time: 0.0193  lr: 0.000125  max_mem: 2723M
[11/08 07:23:36 d2.utils.events]:  eta: 0:08:36  iter: 519  total_loss: 2.317  loss_cls: 1.060  loss_box_reg: 0.577  loss_mask: 0.673  loss_rpn_cls: 0.008  loss_rpn_loc: 0.011  time: 0.3489  data_time: 0.0070  lr: 0.000130  max_mem: 2723M
[11/08 07:23:43 d2.utils.events]:  eta: 0:08:29  iter: 539  total_loss: 2.468  loss_cls: 1.075  loss_box_reg: 0.678  loss_mask: 0.671  loss_rpn_cls: 0.010  loss_rpn_loc: 0.010  time: 0.3489  data_time: 0.0085  lr: 0.000135  max_mem: 2723M
[11/08 07:23:50 d2.utils.events]:  eta: 0:08:22  iter: 559  total_loss: 2.124  loss_cls: 0.958  loss_box_reg: 0.532  loss_mask: 0.671  loss_rpn_cls: 0.012  loss_rpn_loc: 0.011  time: 0.3490  data_time: 0.0091  lr: 0.000140  max_mem: 2723M
[11/08 07:23:58 d2.utils.events]:  eta: 0:08:16  iter: 579  total_loss: 2.376  loss_cls: 1.076  loss_box_reg: 0.622  loss_mask: 0.669  loss_rpn_cls: 0.008  loss_rpn_loc: 0.010  time: 0.3498  data_time: 0.0147  lr: 0.000145  max_mem: 2723M
[11/08 07:24:05 d2.utils.events]:  eta: 0:08:10  iter: 599  total_loss: 2.478  loss_cls: 1.110  loss_box_reg: 0.690  loss_mask: 0.648  loss_rpn_cls: 0.008  loss_rpn_loc: 0.009  time: 0.3502  data_time: 0.0070  lr: 0.000150  max_mem: 2723M
[11/08 07:24:12 d2.utils.events]:  eta: 0:08:03  iter: 619  total_loss: 2.403  loss_cls: 1.109  loss_box_reg: 0.595  loss_mask: 0.640  loss_rpn_cls: 0.008  loss_rpn_loc: 0.010  time: 0.3504  data_time: 0.0054  lr: 0.000155  max_mem: 2723M
[11/08 07:24:20 d2.utils.events]:  eta: 0:07:56  iter: 639  total_loss: 2.153  loss_cls: 0.914  loss_box_reg: 0.554  loss_mask: 0.647  loss_rpn_cls: 0.007  loss_rpn_loc: 0.010  time: 0.3510  data_time: 0.0164  lr: 0.000160  max_mem: 2723M
[11/08 07:24:27 d2.utils.events]:  eta: 0:07:50  iter: 659  total_loss: 2.515  loss_cls: 1.182  loss_box_reg: 0.683  loss_mask: 0.601  loss_rpn_cls: 0.005  loss_rpn_loc: 0.009  time: 0.3513  data_time: 0.0081  lr: 0.000165  max_mem: 2723M
[11/08 07:24:34 d2.utils.events]:  eta: 0:07:43  iter: 679  total_loss: 1.995  loss_cls: 0.738  loss_box_reg: 0.505  loss_mask: 0.612  loss_rpn_cls: 0.008  loss_rpn_loc: 0.010  time: 0.3521  data_time: 0.0068  lr: 0.000170  max_mem: 2723M
[11/08 07:24:42 d2.utils.events]:  eta: 0:07:36  iter: 699  total_loss: 2.145  loss_cls: 0.879  loss_box_reg: 0.582  loss_mask: 0.538  loss_rpn_cls: 0.012  loss_rpn_loc: 0.011  time: 0.3522  data_time: 0.0064  lr: 0.000175  max_mem: 2723M

/usr/local/lib/python3.6/dist-packages/PIL/TiffImagePlugin.py:590: UserWarning:

Metadata Warning, tag 282 had too many entries: 2, expected 1

/usr/local/lib/python3.6/dist-packages/PIL/TiffImagePlugin.py:590: UserWarning:

Metadata Warning, tag 283 had too many entries: 2, expected 1

[11/08 07:24:49 d2.utils.events]:  eta: 0:07:29  iter: 719  total_loss: 2.147  loss_cls: 0.979  loss_box_reg: 0.628  loss_mask: 0.555  loss_rpn_cls: 0.004  loss_rpn_loc: 0.009  time: 0.3525  data_time: 0.0179  lr: 0.000180  max_mem: 2723M
[11/08 07:24:57 d2.utils.events]:  eta: 0:07:23  iter: 739  total_loss: 2.420  loss_cls: 1.120  loss_box_reg: 0.669  loss_mask: 0.596  loss_rpn_cls: 0.013  loss_rpn_loc: 0.010  time: 0.3534  data_time: 0.0262  lr: 0.000185  max_mem: 2723M
[11/08 07:25:04 d2.utils.events]:  eta: 0:07:16  iter: 759  total_loss: 2.407  loss_cls: 1.078  loss_box_reg: 0.640  loss_mask: 0.604  loss_rpn_cls: 0.009  loss_rpn_loc: 0.010  time: 0.3539  data_time: 0.0150  lr: 0.000190  max_mem: 2723M
[11/08 07:25:12 d2.utils.events]:  eta: 0:07:09  iter: 779  total_loss: 2.255  loss_cls: 0.998  loss_box_reg: 0.616  loss_mask: 0.568  loss_rpn_cls: 0.005  loss_rpn_loc: 0.009  time: 0.3545  data_time: 0.0063  lr: 0.000195  max_mem: 2723M
[11/08 07:25:19 d2.utils.events]:  eta: 0:07:03  iter: 799  total_loss: 2.313  loss_cls: 1.040  loss_box_reg: 0.663  loss_mask: 0.605  loss_rpn_cls: 0.006  loss_rpn_loc: 0.010  time: 0.3553  data_time: 0.0066  lr: 0.000200  max_mem: 2723M
[11/08 07:25:27 d2.utils.events]:  eta: 0:06:57  iter: 819  total_loss: 2.268  loss_cls: 1.023  loss_box_reg: 0.643  loss_mask: 0.541  loss_rpn_cls: 0.013  loss_rpn_loc: 0.009  time: 0.3558  data_time: 0.0084  lr: 0.000205  max_mem: 2723M
[11/08 07:25:34 d2.utils.events]:  eta: 0:06:50  iter: 839  total_loss: 2.234  loss_cls: 1.022  loss_box_reg: 0.635  loss_mask: 0.554  loss_rpn_cls: 0.005  loss_rpn_loc: 0.012  time: 0.3562  data_time: 0.0065  lr: 0.000210  max_mem: 2723M
[11/08 07:25:42 d2.utils.events]:  eta: 0:06:44  iter: 859  total_loss: 2.262  loss_cls: 1.017  loss_box_reg: 0.642  loss_mask: 0.499  loss_rpn_cls: 0.014  loss_rpn_loc: 0.009  time: 0.3564  data_time: 0.0056  lr: 0.000215  max_mem: 2723M
[11/08 07:25:49 d2.utils.events]:  eta: 0:06:37  iter: 879  total_loss: 2.353  loss_cls: 1.117  loss_box_reg: 0.695  loss_mask: 0.577  loss_rpn_cls: 0.007  loss_rpn_loc: 0.008  time: 0.3565  data_time: 0.0061  lr: 0.000220  max_mem: 2723M
[11/08 07:25:56 d2.utils.events]:  eta: 0:06:30  iter: 899  total_loss: 1.785  loss_cls: 0.722  loss_box_reg: 0.591  loss_mask: 0.430  loss_rpn_cls: 0.011  loss_rpn_loc: 0.007  time: 0.3565  data_time: 0.0063  lr: 0.000225  max_mem: 2723M
[11/08 07:26:04 d2.utils.events]:  eta: 0:06:23  iter: 919  total_loss: 2.241  loss_cls: 1.001  loss_box_reg: 0.655  loss_mask: 0.523  loss_rpn_cls: 0.008  loss_rpn_loc: 0.006  time: 0.3570  data_time: 0.0169  lr: 0.000230  max_mem: 2723M
[11/08 07:26:11 d2.utils.events]:  eta: 0:06:16  iter: 939  total_loss: 2.241  loss_cls: 1.050  loss_box_reg: 0.679  loss_mask: 0.463  loss_rpn_cls: 0.007  loss_rpn_loc: 0.008  time: 0.3568  data_time: 0.0071  lr: 0.000235  max_mem: 2723M
[11/08 07:26:18 d2.utils.events]:  eta: 0:06:09  iter: 959  total_loss: 1.777  loss_cls: 0.778  loss_box_reg: 0.575  loss_mask: 0.446  loss_rpn_cls: 0.006  loss_rpn_loc: 0.007  time: 0.3569  data_time: 0.0077  lr: 0.000240  max_mem: 2723M
[11/08 07:26:25 d2.utils.events]:  eta: 0:06:02  iter: 979  total_loss: 2.094  loss_cls: 0.937  loss_box_reg: 0.661  loss_mask: 0.476  loss_rpn_cls: 0.007  loss_rpn_loc: 0.008  time: 0.3571  data_time: 0.0056  lr: 0.000245  max_mem: 2723M
[11/08 07:26:32 d2.utils.events]:  eta: 0:05:55  iter: 999  total_loss: 1.920  loss_cls: 0.885  loss_box_reg: 0.570  loss_mask: 0.527  loss_rpn_cls: 0.004  loss_rpn_loc: 0.009  time: 0.3572  data_time: 0.0065  lr: 0.000250  max_mem: 2723M
[11/08 07:26:40 d2.utils.events]:  eta: 0:05:49  iter: 1019  total_loss: 1.690  loss_cls: 0.789  loss_box_reg: 0.531  loss_mask: 0.457  loss_rpn_cls: 0.009  loss_rpn_loc: 0.008  time: 0.3575  data_time: 0.0074  lr: 0.000250  max_mem: 2723M
[11/08 07:26:47 d2.utils.events]:  eta: 0:05:43  iter: 1039  total_loss: 2.275  loss_cls: 1.064  loss_box_reg: 0.676  loss_mask: 0.456  loss_rpn_cls: 0.007  loss_rpn_loc: 0.008  time: 0.3577  data_time: 0.0060  lr: 0.000250  max_mem: 2723M
[11/08 07:26:55 d2.utils.events]:  eta: 0:05:37  iter: 1059  total_loss: 2.176  loss_cls: 1.083  loss_box_reg: 0.677  loss_mask: 0.467  loss_rpn_cls: 0.011  loss_rpn_loc: 0.011  time: 0.3584  data_time: 0.0190  lr: 0.000250  max_mem: 2723M
[11/08 07:27:03 d2.utils.events]:  eta: 0:05:31  iter: 1079  total_loss: 2.361  loss_cls: 1.121  loss_box_reg: 0.660  loss_mask: 0.503  loss_rpn_cls: 0.010  loss_rpn_loc: 0.012  time: 0.3588  data_time: 0.0064  lr: 0.000250  max_mem: 2723M
[11/08 07:27:10 d2.utils.events]:  eta: 0:05:25  iter: 1099  total_loss: 2.100  loss_cls: 0.999  loss_box_reg: 0.579  loss_mask: 0.451  loss_rpn_cls: 0.007  loss_rpn_loc: 0.007  time: 0.3590  data_time: 0.0065  lr: 0.000250  max_mem: 2723M
[11/08 07:27:18 d2.utils.events]:  eta: 0:05:18  iter: 1119  total_loss: 1.989  loss_cls: 0.845  loss_box_reg: 0.641  loss_mask: 0.381  loss_rpn_cls: 0.006  loss_rpn_loc: 0.007  time: 0.3593  data_time: 0.0065  lr: 0.000250  max_mem: 2723M
[11/08 07:27:25 d2.utils.events]:  eta: 0:05:11  iter: 1139  total_loss: 1.992  loss_cls: 0.948  loss_box_reg: 0.597  loss_mask: 0.495  loss_rpn_cls: 0.010  loss_rpn_loc: 0.009  time: 0.3595  data_time: 0.0097  lr: 0.000250  max_mem: 2723M
[11/08 07:27:33 d2.utils.events]:  eta: 0:05:04  iter: 1159  total_loss: 2.172  loss_cls: 1.043  loss_box_reg: 0.650  loss_mask: 0.439  loss_rpn_cls: 0.006  loss_rpn_loc: 0.008  time: 0.3597  data_time: 0.0274  lr: 0.000250  max_mem: 2723M
[11/08 07:27:40 d2.utils.events]:  eta: 0:04:57  iter: 1179  total_loss: 1.856  loss_cls: 0.861  loss_box_reg: 0.611  loss_mask: 0.362  loss_rpn_cls: 0.005  loss_rpn_loc: 0.006  time: 0.3597  data_time: 0.0067  lr: 0.000250  max_mem: 2723M
[11/08 07:27:47 d2.utils.events]:  eta: 0:04:50  iter: 1199  total_loss: 2.023  loss_cls: 0.946  loss_box_reg: 0.639  loss_mask: 0.378  loss_rpn_cls: 0.008  loss_rpn_loc: 0.010  time: 0.3599  data_time: 0.0077  lr: 0.000250  max_mem: 2723M
[11/08 07:27:55 d2.utils.events]:  eta: 0:04:43  iter: 1219  total_loss: 2.055  loss_cls: 1.035  loss_box_reg: 0.679  loss_mask: 0.426  loss_rpn_cls: 0.005  loss_rpn_loc: 0.008  time: 0.3600  data_time: 0.0071  lr: 0.000250  max_mem: 2723M
[11/08 07:28:02 d2.utils.events]:  eta: 0:04:36  iter: 1239  total_loss: 2.244  loss_cls: 0.992  loss_box_reg: 0.705  loss_mask: 0.479  loss_rpn_cls: 0.004  loss_rpn_loc: 0.012  time: 0.3602  data_time: 0.0055  lr: 0.000250  max_mem: 2723M
[11/08 07:28:09 d2.utils.events]:  eta: 0:04:29  iter: 1259  total_loss: 2.120  loss_cls: 1.072  loss_box_reg: 0.645  loss_mask: 0.457  loss_rpn_cls: 0.006  loss_rpn_loc: 0.009  time: 0.3602  data_time: 0.0077  lr: 0.000250  max_mem: 2723M
[11/08 07:28:17 d2.utils.events]:  eta: 0:04:22  iter: 1279  total_loss: 2.253  loss_cls: 1.069  loss_box_reg: 0.667  loss_mask: 0.425  loss_rpn_cls: 0.009  loss_rpn_loc: 0.008  time: 0.3603  data_time: 0.0067  lr: 0.000250  max_mem: 2723M
[11/08 07:28:24 d2.utils.events]:  eta: 0:04:15  iter: 1299  total_loss: 2.022  loss_cls: 0.971  loss_box_reg: 0.623  loss_mask: 0.371  loss_rpn_cls: 0.007  loss_rpn_loc: 0.008  time: 0.3604  data_time: 0.0070  lr: 0.000250  max_mem: 2723M
[11/08 07:28:31 d2.utils.events]:  eta: 0:04:09  iter: 1319  total_loss: 1.972  loss_cls: 0.872  loss_box_reg: 0.603  loss_mask: 0.453  loss_rpn_cls: 0.007  loss_rpn_loc: 0.010  time: 0.3606  data_time: 0.0071  lr: 0.000250  max_mem: 2723M
[11/08 07:28:39 d2.utils.events]:  eta: 0:04:02  iter: 1339  total_loss: 1.894  loss_cls: 0.846  loss_box_reg: 0.661  loss_mask: 0.381  loss_rpn_cls: 0.011  loss_rpn_loc: 0.007  time: 0.3607  data_time: 0.0053  lr: 0.000250  max_mem: 2723M
[11/08 07:28:46 d2.utils.events]:  eta: 0:03:55  iter: 1359  total_loss: 2.301  loss_cls: 1.086  loss_box_reg: 0.688  loss_mask: 0.499  loss_rpn_cls: 0.010  loss_rpn_loc: 0.015  time: 0.3609  data_time: 0.0085  lr: 0.000250  max_mem: 2723M
[11/08 07:28:54 d2.utils.events]:  eta: 0:03:48  iter: 1379  total_loss: 2.246  loss_cls: 1.055  loss_box_reg: 0.637  loss_mask: 0.429  loss_rpn_cls: 0.007  loss_rpn_loc: 0.008  time: 0.3611  data_time: 0.0052  lr: 0.000250  max_mem: 2723M
[11/08 07:29:01 d2.utils.events]:  eta: 0:03:41  iter: 1399  total_loss: 2.276  loss_cls: 1.058  loss_box_reg: 0.748  loss_mask: 0.421  loss_rpn_cls: 0.008  loss_rpn_loc: 0.013  time: 0.3613  data_time: 0.0067  lr: 0.000250  max_mem: 2723M
[11/08 07:29:09 d2.utils.events]:  eta: 0:03:33  iter: 1419  total_loss: 2.130  loss_cls: 1.083  loss_box_reg: 0.696  loss_mask: 0.385  loss_rpn_cls: 0.005  loss_rpn_loc: 0.008  time: 0.3614  data_time: 0.0062  lr: 0.000250  max_mem: 2723M
[11/08 07:29:16 d2.utils.events]:  eta: 0:03:26  iter: 1439  total_loss: 2.001  loss_cls: 0.989  loss_box_reg: 0.638  loss_mask: 0.315  loss_rpn_cls: 0.004  loss_rpn_loc: 0.008  time: 0.3616  data_time: 0.0143  lr: 0.000250  max_mem: 2723M
[11/08 07:29:24 d2.utils.events]:  eta: 0:03:19  iter: 1459  total_loss: 1.886  loss_cls: 0.925  loss_box_reg: 0.689  loss_mask: 0.328  loss_rpn_cls: 0.003  loss_rpn_loc: 0.006  time: 0.3616  data_time: 0.0069  lr: 0.000250  max_mem: 2723M
[11/08 07:29:31 d2.utils.events]:  eta: 0:03:12  iter: 1479  total_loss: 2.038  loss_cls: 1.071  loss_box_reg: 0.666  loss_mask: 0.349  loss_rpn_cls: 0.011  loss_rpn_loc: 0.008  time: 0.3619  data_time: 0.0089  lr: 0.000250  max_mem: 2723M
[11/08 07:29:39 d2.utils.events]:  eta: 0:03:04  iter: 1499  total_loss: 2.131  loss_cls: 0.901  loss_box_reg: 0.613  loss_mask: 0.449  loss_rpn_cls: 0.013  loss_rpn_loc: 0.009  time: 0.3620  data_time: 0.0157  lr: 0.000250  max_mem: 2723M
[11/08 07:29:46 d2.utils.events]:  eta: 0:02:57  iter: 1519  total_loss: 2.084  loss_cls: 0.989  loss_box_reg: 0.650  loss_mask: 0.445  loss_rpn_cls: 0.009  loss_rpn_loc: 0.006  time: 0.3622  data_time: 0.0194  lr: 0.000250  max_mem: 2723M
[11/08 07:29:53 d2.utils.events]:  eta: 0:02:50  iter: 1539  total_loss: 2.102  loss_cls: 1.059  loss_box_reg: 0.667  loss_mask: 0.389  loss_rpn_cls: 0.005  loss_rpn_loc: 0.005  time: 0.3622  data_time: 0.0071  lr: 0.000250  max_mem: 2723M
[11/08 07:30:01 d2.utils.events]:  eta: 0:02:42  iter: 1559  total_loss: 2.174  loss_cls: 1.049  loss_box_reg: 0.734  loss_mask: 0.407  loss_rpn_cls: 0.009  loss_rpn_loc: 0.010  time: 0.3624  data_time: 0.0085  lr: 0.000250  max_mem: 2723M
[11/08 07:30:08 d2.utils.events]:  eta: 0:02:35  iter: 1579  total_loss: 2.138  loss_cls: 0.999  loss_box_reg: 0.666  loss_mask: 0.380  loss_rpn_cls: 0.009  loss_rpn_loc: 0.008  time: 0.3626  data_time: 0.0065  lr: 0.000250  max_mem: 2723M
[11/08 07:30:16 d2.utils.events]:  eta: 0:02:28  iter: 1599  total_loss: 2.154  loss_cls: 1.093  loss_box_reg: 0.727  loss_mask: 0.375  loss_rpn_cls: 0.004  loss_rpn_loc: 0.006  time: 0.3626  data_time: 0.0160  lr: 0.000250  max_mem: 2723M
[11/08 07:30:23 d2.utils.events]:  eta: 0:02:20  iter: 1619  total_loss: 2.177  loss_cls: 0.944  loss_box_reg: 0.664  loss_mask: 0.434  loss_rpn_cls: 0.007  loss_rpn_loc: 0.008  time: 0.3627  data_time: 0.0081  lr: 0.000250  max_mem: 2723M
[11/08 07:30:31 d2.utils.events]:  eta: 0:02:13  iter: 1639  total_loss: 1.974  loss_cls: 0.903  loss_box_reg: 0.652  loss_mask: 0.281  loss_rpn_cls: 0.010  loss_rpn_loc: 0.009  time: 0.3628  data_time: 0.0097  lr: 0.000250  max_mem: 2723M
[11/08 07:30:38 d2.utils.events]:  eta: 0:02:06  iter: 1659  total_loss: 2.110  loss_cls: 1.015  loss_box_reg: 0.719  loss_mask: 0.359  loss_rpn_cls: 0.010  loss_rpn_loc: 0.011  time: 0.3630  data_time: 0.0181  lr: 0.000250  max_mem: 2723M
[11/08 07:30:45 d2.utils.events]:  eta: 0:01:58  iter: 1679  total_loss: 2.212  loss_cls: 1.013  loss_box_reg: 0.654  loss_mask: 0.343  loss_rpn_cls: 0.010  loss_rpn_loc: 0.010  time: 0.3630  data_time: 0.0095  lr: 0.000250  max_mem: 2723M
[11/08 07:30:53 d2.utils.events]:  eta: 0:01:51  iter: 1699  total_loss: 1.787  loss_cls: 0.833  loss_box_reg: 0.561  loss_mask: 0.299  loss_rpn_cls: 0.007  loss_rpn_loc: 0.012  time: 0.3630  data_time: 0.0050  lr: 0.000250  max_mem: 2723M
[11/08 07:31:00 d2.utils.events]:  eta: 0:01:43  iter: 1719  total_loss: 2.136  loss_cls: 1.048  loss_box_reg: 0.685  loss_mask: 0.322  loss_rpn_cls: 0.008  loss_rpn_loc: 0.006  time: 0.3631  data_time: 0.0072  lr: 0.000250  max_mem: 2723M
[11/08 07:31:08 d2.utils.events]:  eta: 0:01:36  iter: 1739  total_loss: 2.008  loss_cls: 0.941  loss_box_reg: 0.610  loss_mask: 0.396  loss_rpn_cls: 0.009  loss_rpn_loc: 0.009  time: 0.3632  data_time: 0.0071  lr: 0.000250  max_mem: 2723M
[11/08 07:31:15 d2.utils.events]:  eta: 0:01:29  iter: 1759  total_loss: 1.915  loss_cls: 1.006  loss_box_reg: 0.649  loss_mask: 0.257  loss_rpn_cls: 0.008  loss_rpn_loc: 0.006  time: 0.3632  data_time: 0.0059  lr: 0.000250  max_mem: 2723M
[11/08 07:31:22 d2.utils.events]:  eta: 0:01:21  iter: 1779  total_loss: 1.681  loss_cls: 0.764  loss_box_reg: 0.559  loss_mask: 0.329  loss_rpn_cls: 0.006  loss_rpn_loc: 0.008  time: 0.3633  data_time: 0.0057  lr: 0.000250  max_mem: 2723M
[11/08 07:31:30 d2.utils.events]:  eta: 0:01:14  iter: 1799  total_loss: 1.931  loss_cls: 0.909  loss_box_reg: 0.605  loss_mask: 0.333  loss_rpn_cls: 0.006  loss_rpn_loc: 0.008  time: 0.3633  data_time: 0.0080  lr: 0.000250  max_mem: 2723M
[11/08 07:31:37 d2.utils.events]:  eta: 0:01:06  iter: 1819  total_loss: 2.299  loss_cls: 1.019  loss_box_reg: 0.654  loss_mask: 0.436  loss_rpn_cls: 0.005  loss_rpn_loc: 0.010  time: 0.3633  data_time: 0.0059  lr: 0.000250  max_mem: 2723M
[11/08 07:31:45 d2.utils.events]:  eta: 0:00:59  iter: 1839  total_loss: 2.160  loss_cls: 1.048  loss_box_reg: 0.697  loss_mask: 0.341  loss_rpn_cls: 0.008  loss_rpn_loc: 0.007  time: 0.3636  data_time: 0.0078  lr: 0.000250  max_mem: 2723M
[11/08 07:31:52 d2.utils.events]:  eta: 0:00:52  iter: 1859  total_loss: 2.017  loss_cls: 0.948  loss_box_reg: 0.682  loss_mask: 0.369  loss_rpn_cls: 0.007  loss_rpn_loc: 0.010  time: 0.3637  data_time: 0.0072  lr: 0.000250  max_mem: 2723M
[11/08 07:32:00 d2.utils.events]:  eta: 0:00:44  iter: 1879  total_loss: 2.056  loss_cls: 0.852  loss_box_reg: 0.638  loss_mask: 0.497  loss_rpn_cls: 0.006  loss_rpn_loc: 0.006  time: 0.3637  data_time: 0.0074  lr: 0.000250  max_mem: 2723M
[11/08 07:32:07 d2.utils.events]:  eta: 0:00:37  iter: 1899  total_loss: 1.862  loss_cls: 0.991  loss_box_reg: 0.677  loss_mask: 0.304  loss_rpn_cls: 0.008  loss_rpn_loc: 0.008  time: 0.3639  data_time: 0.0082  lr: 0.000250  max_mem: 2723M
[11/08 07:32:15 d2.utils.events]:  eta: 0:00:29  iter: 1919  total_loss: 2.036  loss_cls: 0.975  loss_box_reg: 0.632  loss_mask: 0.312  loss_rpn_cls: 0.007  loss_rpn_loc: 0.007  time: 0.3640  data_time: 0.0062  lr: 0.000250  max_mem: 2723M
[11/08 07:32:22 d2.utils.events]:  eta: 0:00:22  iter: 1939  total_loss: 2.010  loss_cls: 0.938  loss_box_reg: 0.592  loss_mask: 0.423  loss_rpn_cls: 0.006  loss_rpn_loc: 0.008  time: 0.3641  data_time: 0.0077  lr: 0.000250  max_mem: 2723M
[11/08 07:32:30 d2.utils.events]:  eta: 0:00:15  iter: 1959  total_loss: 2.156  loss_cls: 1.077  loss_box_reg: 0.717  loss_mask: 0.333  loss_rpn_cls: 0.012  loss_rpn_loc: 0.010  time: 0.3642  data_time: 0.0066  lr: 0.000250  max_mem: 2723M
[11/08 07:32:37 d2.utils.events]:  eta: 0:00:07  iter: 1979  total_loss: 2.120  loss_cls: 1.032  loss_box_reg: 0.720  loss_mask: 0.423  loss_rpn_cls: 0.009  loss_rpn_loc: 0.009  time: 0.3643  data_time: 0.0065  lr: 0.000250  max_mem: 2723M
[11/08 07:32:46 d2.utils.events]:  eta: 0:00:00  iter: 1999  total_loss: 2.235  loss_cls: 0.940  loss_box_reg: 0.633  loss_mask: 0.410  loss_rpn_cls: 0.010  loss_rpn_loc: 0.012  time: 0.3643  data_time: 0.0071  lr: 0.000250  max_mem: 2723M
[11/08 07:32:46 d2.engine.hooks]: Overall training speed: 1997 iterations in 0:12:07 (0.3645 s / it)
[11/08 07:32:46 d2.engine.hooks]: Total training time: 0:12:10 (0:00:03 on hooks)

Evaluating the model 🧪¶

After training is done, we will evaluate our model to see how it's performing in the unseen images!

In [ ]:

cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.01

cfg.MODEL.WEIGHTS = os.path.join(cfg.OUTPUT_DIR, "model_final.pth")

evaluator = COCOEvaluator("validation_dataset", cfg, False, output_dir=cfg.OUTPUT_DIR)
val_loader = build_detection_test_loader(cfg, "validation_dataset")
valResults = inference_on_dataset(trainer.model, val_loader, evaluator)

WARNING [10/17 07:39:37 d2.data.datasets.coco]: 
Category ids in annotations are not in [1, #categories]! We'll apply a mapping for you.

[10/17 07:39:37 d2.data.datasets.coco]: Loaded 1269 images in COCO format from /content/val/new_ann.json
[10/17 07:39:37 d2.data.common]: Serializing 1269 elements to byte tensors and concatenating them all ...
[10/17 07:39:37 d2.data.common]: Serialized dataset takes 3.59 MiB
[10/17 07:39:37 d2.evaluation.evaluator]: Start inference on 1269 images
[10/17 07:39:38 d2.evaluation.evaluator]: Inference done 11/1269. 0.0565 s / img. ETA=0:01:16
[10/17 07:39:43 d2.evaluation.evaluator]: Inference done 90/1269. 0.0572 s / img. ETA=0:01:15
[10/17 07:39:48 d2.evaluation.evaluator]: Inference done 164/1269. 0.0579 s / img. ETA=0:01:12
[10/17 07:39:53 d2.evaluation.evaluator]: Inference done 237/1269. 0.0580 s / img. ETA=0:01:09
[10/17 07:39:58 d2.evaluation.evaluator]: Inference done 304/1269. 0.0581 s / img. ETA=0:01:06
[10/17 07:40:04 d2.evaluation.evaluator]: Inference done 377/1269. 0.0582 s / img. ETA=0:01:03
[10/17 07:40:09 d2.evaluation.evaluator]: Inference done 448/1269. 0.0581 s / img. ETA=0:00:58
[10/17 07:40:16 d2.evaluation.evaluator]: Inference done 524/1269. 0.0583 s / img. ETA=0:00:55

/usr/local/lib/python3.6/dist-packages/PIL/TiffImagePlugin.py:590: UserWarning:

Metadata Warning, tag 282 had too many entries: 2, expected 1

/usr/local/lib/python3.6/dist-packages/PIL/TiffImagePlugin.py:590: UserWarning:

Metadata Warning, tag 283 had too many entries: 2, expected 1

/usr/local/lib/python3.6/dist-packages/PIL/TiffImagePlugin.py:590: UserWarning:

Metadata Warning, tag 34853 had too many entries: 9, expected 1

[10/17 07:40:21 d2.evaluation.evaluator]: Inference done 579/1269. 0.0584 s / img. ETA=0:00:52
[10/17 07:40:26 d2.evaluation.evaluator]: Inference done 652/1269. 0.0585 s / img. ETA=0:00:46
[10/17 07:40:31 d2.evaluation.evaluator]: Inference done 727/1269. 0.0585 s / img. ETA=0:00:40
[10/17 07:40:36 d2.evaluation.evaluator]: Inference done 802/1269. 0.0585 s / img. ETA=0:00:34
[10/17 07:40:41 d2.evaluation.evaluator]: Inference done 880/1269. 0.0585 s / img. ETA=0:00:28
[10/17 07:40:46 d2.evaluation.evaluator]: Inference done 954/1269. 0.0585 s / img. ETA=0:00:22
[10/17 07:40:51 d2.evaluation.evaluator]: Inference done 1034/1269. 0.0584 s / img. ETA=0:00:16
[10/17 07:40:56 d2.evaluation.evaluator]: Inference done 1107/1269. 0.0585 s / img. ETA=0:00:11
[10/17 07:41:01 d2.evaluation.evaluator]: Inference done 1185/1269. 0.0584 s / img. ETA=0:00:05
[10/17 07:41:06 d2.evaluation.evaluator]: Inference done 1268/1269. 0.0583 s / img. ETA=0:00:00
[10/17 07:41:07 d2.evaluation.evaluator]: Total inference time: 0:01:29.038839 (0.070442 s / img per device, on 1 devices)
[10/17 07:41:07 d2.evaluation.evaluator]: Total inference pure compute time: 0:01:13 (0.058350 s / img per device, on 1 devices)
[10/17 07:41:07 d2.evaluation.coco_evaluation]: Preparing results for COCO format ...
[10/17 07:41:07 d2.evaluation.coco_evaluation]: Saving results to /content/logs/coco_instances_results.json
[10/17 07:41:07 d2.evaluation.coco_evaluation]: Evaluating predictions ...
Loading and preparing results...
DONE (t=0.01s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *bbox*
DONE (t=4.62s).
Accumulating evaluation results...
DONE (t=1.18s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.002
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.001
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.001
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.002
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.001
[10/17 07:41:13 d2.evaluation.coco_evaluation]: Evaluation results for bbox: 
|  AP   |  AP50  |  AP75  |  APs  |  APm  |  APl  |
|:-----:|:------:|:------:|:-----:|:-----:|:-----:|
| 0.010 | 0.043  | 0.000  | 0.000 | 0.222 | 0.010 |
[10/17 07:41:13 d2.evaluation.coco_evaluation]: Per-category bbox AP: 
| category                                                | AP    | category                                            | AP    | category                                             | AP    |
|:--------------------------------------------------------|:------|:----------------------------------------------------|:------|:-----------------------------------------------------|:------|
| beetroot-steamed-without-addition-of-salt               | 0.000 | green-bean-steamed-without-addition-of-salt         | 0.000 | watermelon-fresh                                     | 0.000 |
| pizza-with-ham-baked                                    | 0.000 | pizza-with-vegetables-baked                         | 0.000 | applesauce-unsweetened-canned                        | 0.000 |
| quiche-with-cheese-baked-with-puff-pastry               | 0.000 | sweet-potato                                        | 0.000 | country-fries                                        | 0.000 |
| potato-gnocchi                                          | 0.000 | potatoes-steamed                                    | 0.000 | chips-french-fries                                   | 0.000 |
| rosti                                                   | 0.000 | vegetable-mix-peas-and-carrots                      | 0.000 | mixed-vegetables                                     | 0.000 |
| ratatouille                                             | 0.000 | mixed-salad-chopped-without-sauce                   | 0.000 | leaf-spinach                                         | 0.000 |
| witloof-chicory                                         | 0.000 | salad-rocket                                        | 0.000 | salad-leaf-salad-green                               | 0.000 |
| salad-lambs-ear                                         | 0.000 | artichoke                                           | 0.000 | eggplant                                             | 0.000 |
| avocado                                                 | 0.000 | french-beans                                        | 0.000 | pickle                                               | 0.000 |
| cucumber                                                | 0.000 | pumpkin                                             | 0.000 | sweet-pepper                                         | 0.000 |
| tomato                                                  | 0.000 | zucchini                                            | 0.000 | red-radish                                           | 0.000 |
| beetroot-raw                                            | 0.000 | carrot                                              | 0.000 | celeriac                                             | 0.000 |
| cauliflower                                             | 0.000 | broccoli                                            | 0.000 | kolhrabi                                             | 0.000 |
| red-cabbage                                             | 0.000 | white-cabbage                                       | 0.000 | mushroom                                             | 0.000 |
| mushrooms                                               | 0.000 | peas                                                | 0.000 | corn                                                 | 0.000 |
| leek                                                    | 0.000 | onion                                               | 0.000 | fennel                                               | 0.000 |
| green-asparagus                                         | 0.000 | white-asparagus                                     | nan   | alfa-sprouts                                         | 0.000 |
| beans-kidney                                            | 0.000 | chickpeas                                           | 0.000 | lentils                                              | 0.000 |
| pineapple                                               | 0.000 | apple                                               | 0.000 | pomegranate                                          | 0.000 |
| apricots                                                | 0.000 | banana                                              | 0.000 | berries                                              | 0.000 |
| pear                                                    | 0.000 | dates                                               | 0.000 | strawberries                                         | 0.000 |
| fruit-salad                                             | 0.000 | blueberries                                         | 0.000 | raspberries                                          | nan   |
| kiwi                                                    | 0.000 | mandarine                                           | 0.000 | mango                                                | 0.000 |
| sugar-melon                                             | 0.000 | nectarine                                           | 0.000 | orange                                               | 0.000 |
| peach                                                   | 0.000 | plums                                               | 0.000 | grapes                                               | 0.000 |
| dried-raisins                                           | 0.000 | lemon                                               | 0.000 | peanut-butter                                        | 0.000 |
| mixed-seeds                                             | 0.000 | almonds                                             | 0.000 | walnut                                               | 0.000 |
| cashew-nut                                              | 0.000 | peanut                                              | 0.000 | hazelnut                                             | 0.000 |
| mixed-nuts                                              | 0.000 | pistachio                                           | 0.000 | sesame-seeds                                         | 0.000 |
| green-olives                                            | 0.000 | black-olives                                        | 0.000 | milk                                                 | 0.000 |
| kefir-drink                                             | 0.000 | cottage-cheese                                      | 0.000 | blue-mould-cheese                                    | 0.000 |
| feta                                                    | 0.000 | fresh-cheese                                        | 0.000 | gruya-re                                             | 0.000 |
| semi-hard-cheese                                        | 0.000 | hard-cheese                                         | 0.000 | cheese                                               | 0.000 |
| mozzarella                                              | 0.000 | parmesan                                            | 0.000 | cheese-for-raclette                                  | 0.000 |
| cream-cheese                                            | 0.000 | tomme                                               | 0.000 | soft-cheese                                          | 0.000 |
| tiramisu                                                | nan   | cream                                               | nan   | sour-cream                                           | 0.000 |
| thickened-cream-35                                      | 0.000 | dairy-ice-cream                                     | 0.000 | flakes-oat                                           | 0.000 |
| rice-noodles-vermicelli                                 | 0.000 | couscous                                            | 0.000 | grits-polenta-maize-flour                            | 0.000 |
| quinoa                                                  | 0.000 | rice                                                | 0.000 | rice-basmati                                         | 0.000 |
| rice-whole-grain                                        | 0.000 | spaetzle                                            | nan   | pasta                                                | 0.000 |
| pasta-haprnli                                           | 0.000 | pasta-linguini-parpadelle-tagliatelle               | 0.000 | pasta-noodles                                        | 0.000 |
| pasta-penne                                             | 0.000 | pasta-spaghetti                                     | 0.000 | pasta-twist                                          | 0.000 |
| pasta-wholemeal                                         | 0.000 | bread-french-white-flour                            | 0.000 | bread                                                | 0.000 |
| bread-5-grain                                           | 0.000 | bread-fruit                                         | 0.000 | bread-half-white                                     | 0.000 |
| bread-grain                                             | 0.000 | bread-nut                                           | 0.000 | bread-pita                                           | 0.000 |
| bread-rye                                               | 0.000 | bread-whole-wheat                                   | 0.000 | bread-sourdough                                      | 0.000 |
| bread-black                                             | 0.000 | bread-toast                                         | 0.000 | bread-wholemeal-toast                                | 0.000 |
| bread-wholemeal                                         | 0.000 | bread-white                                         | 0.000 | brioche                                              | 0.000 |
| roll-of-half-white-or-white-flour-with-large-void       | 0.000 | roll-with-pieces-of-chocolate                       | 0.000 | focaccia                                             | 0.000 |
| croissant                                               | 0.000 | braided-white-loaf                                  | 0.000 | breadcrumbs-unspiced                                 | 0.000 |
| rusk-wholemeal                                          | 0.000 | crunch-ma1-4esli                                    | 0.000 | ma1-4esli                                            | 0.000 |
| beef                                                    | 0.000 | beef-sirloin-steak                                  | 0.000 | beef-filet                                           | 0.000 |
| beef-minced-only-meat                                   | 0.000 | beef-cut-into-stripes-only-meat                     | 0.000 | pork                                                 | 0.000 |
| chicken                                                 | 0.000 | chicken-breast                                      | 0.000 | chicken-cut-into-stripes-only-meat                   | 0.000 |
| chicken-leg                                             | 0.000 | frying-sausage                                      | 0.000 | dried-meat                                           | 0.000 |
| veal-sausage                                            | 0.000 | salami                                              | 0.000 | ham-cooked                                           | 0.000 |
| ham-raw                                                 | 0.000 | bacon-frying                                        | 0.000 | bacon-cooking                                        | 0.000 |
| meat-terrine-pata-c                                     | 0.000 | sausage                                             | 0.000 | veggie-burger                                        | 0.000 |
| tofu                                                    | 0.000 | fish                                                | 0.000 | salmon                                               | 0.000 |
| tuna                                                    | 0.000 | shrimp-prawn-small                                  | 0.000 | shrimp-prawn-large                                   | 0.000 |
| egg                                                     | 0.000 | butter                                              | 0.000 | praline                                              | 0.000 |
| jam                                                     | 0.000 | honey                                               | 0.000 | dark-chocolate                                       | 0.000 |
| milk-chocolate                                          | 0.000 | chocolate                                           | 0.000 | hazelnut-chocolate-spread-nutella-ovomaltine-caotina | nan   |
| apple-pie                                               | 0.000 | brownie                                             | 0.000 | craape-plain                                         | 0.000 |
| fruit-tart                                              | 0.000 | cake-chocolate                                      | 0.000 | omelette-plain                                       | nan   |
| tart                                                    | 0.000 | croissant-with-chocolate-filling                    | 0.000 | cookies                                              | 0.000 |
| biscuits                                                | 0.000 | chocolate-cookies                                   | 0.000 | juice-apple                                          | 0.000 |
| juice-orange                                            | 0.000 | ice-tea                                             | 0.000 | syrup-diluted-ready-to-drink                         | 0.000 |
| tea                                                     | 0.000 | cappuccino                                          | 0.000 | espresso-with-caffeine                               | 0.000 |
| coffee-with-caffeine                                    | 0.000 | white-coffee-with-caffeine                          | 0.000 | ristretto-with-caffeine                              | 0.000 |
| tea-green                                               | 0.000 | tea-black                                           | 0.000 | tea-verveine                                         | 0.000 |
| herbal-tea                                              | 0.000 | tea-peppermint                                      | 0.000 | water                                                | 2.566 |
| water-mineral                                           | 0.000 | wine-rosa-c                                         | 0.000 | wine-red                                             | 0.000 |
| wine-white                                              | 0.000 | beer                                                | 0.000 | sauce-savoury                                        | 0.000 |
| sauce-roast                                             | 0.000 | sauce-pesto                                         | 0.000 | sauce-mushroom                                       | 0.000 |
| sauce-cream                                             | 0.000 | ketchup                                             | 0.000 | bolognaise-sauce                                     | 0.000 |
| tomato-sauce                                            | 0.000 | salad-dressing                                      | 0.000 | balsamic-salad-dressing                              | 0.000 |
| french-salad-dressing                                   | 0.000 | oil-vinegar-salad-dressing                          | 0.000 | guacamole                                            | 0.000 |
| mayonnaise                                              | 0.000 | sauce-soya                                          | 0.000 | soup-vegetable                                       | 0.000 |
| soup-pumpkin                                            | 0.000 | falafel-balls                                       | 0.000 | savoury-puff-pastry                                  | 0.000 |
| corn-crisps                                             | 0.000 | crisps                                              | 0.000 | ham-croissant                                        | 0.000 |
| salt-cake-vegetables-filled                             | 0.000 | hamburger-bread-meat-ketchup                        | 0.000 | lasagne-meat-prepared                                | 0.000 |
| mashed-potatoes-prepared-with-full-fat-milk-with-butter | 0.000 | pizza-margherita-baked                              | 0.000 | sushi                                                | 0.000 |
| pancakes                                                | 0.000 | hummus                                              | 0.000 | greek-salad                                          | 0.000 |
| chocolate-mousse                                        | nan   | caprese-salad-tomato-mozzarella                     | 0.000 | taboula-c-prepared-with-couscous                     | 0.000 |
| risotto-without-cheese-cooked                           | 0.000 | salmon-smoked                                       | 0.000 | egg-scrambled-prepared                               | 0.000 |
| boisson-au-glucose-50g                                  | 0.000 | chicken-curry-cream-coconut-milk-curry-spices-paste | 0.000 | potatoes-au-gratin-dauphinois-prepared               | 0.000 |
| bircherma1-4esli-prepared-no-sugar-added                | 0.000 | fajita-bread-only                                   | 0.000 | butter-spread-puree-almond                           | nan   |
| water-with-lemon-juice                                  | 0.000 | gluten-free-bread                                   | 0.000 | fruit-coulis                                         | 0.000 |
| greek-yaourt-yahourt-yogourt-ou-yoghourt                | 0.000 | soup-of-lentils-dahl-dhal                           | 0.000 | vegetable-au-gratin-baked                            | 0.000 |
| curry-vegetarian                                        | 0.000 | yaourt-yahourt-yogourt-ou-yoghourt-natural          | 0.000 | goat-cheese-soft                                     | 0.000 |
Loading and preparing results...
DONE (t=0.13s)
creating index...
index created!
Running per image evaluation...
Evaluate annotation type *segm*
DONE (t=4.33s).
Accumulating evaluation results...
DONE (t=1.21s).
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.50      | area=   all | maxDets=100 ] = 0.001
 Average Precision  (AP) @[ IoU=0.75      | area=   all | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000
 Average Precision  (AP) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=  1 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets= 10 ] = 0.002
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=   all | maxDets=100 ] = 0.002
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= small | maxDets=100 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area=medium | maxDets=100 ] = 0.000
 Average Recall     (AR) @[ IoU=0.50:0.95 | area= large | maxDets=100 ] = 0.002
[10/17 07:41:18 d2.evaluation.coco_evaluation]: Evaluation results for segm: 
|  AP   |  AP50  |  AP75  |  APs  |  APm  |  APl  |
|:-----:|:------:|:------:|:-----:|:-----:|:-----:|
| 0.020 | 0.072  | 0.006  | 0.000 | 0.000 | 0.021 |
[10/17 07:41:18 d2.evaluation.coco_evaluation]: Per-category segm AP: 
| category                                                | AP    | category                                            | AP    | category                                             | AP    |
|:--------------------------------------------------------|:------|:----------------------------------------------------|:------|:-----------------------------------------------------|:------|
| beetroot-steamed-without-addition-of-salt               | 0.000 | green-bean-steamed-without-addition-of-salt         | 0.000 | watermelon-fresh                                     | 0.000 |
| pizza-with-ham-baked                                    | 0.000 | pizza-with-vegetables-baked                         | 0.000 | applesauce-unsweetened-canned                        | 0.000 |
| quiche-with-cheese-baked-with-puff-pastry               | 0.000 | sweet-potato                                        | 0.000 | country-fries                                        | 0.000 |
| potato-gnocchi                                          | 0.000 | potatoes-steamed                                    | 0.000 | chips-french-fries                                   | 0.000 |
| rosti                                                   | 0.000 | vegetable-mix-peas-and-carrots                      | 0.000 | mixed-vegetables                                     | 0.000 |
| ratatouille                                             | 0.000 | mixed-salad-chopped-without-sauce                   | 0.000 | leaf-spinach                                         | 0.000 |
| witloof-chicory                                         | 0.000 | salad-rocket                                        | 0.000 | salad-leaf-salad-green                               | 0.000 |
| salad-lambs-ear                                         | 0.000 | artichoke                                           | 0.000 | eggplant                                             | 0.000 |
| avocado                                                 | 0.000 | french-beans                                        | 0.000 | pickle                                               | 0.000 |
| cucumber                                                | 0.000 | pumpkin                                             | 0.000 | sweet-pepper                                         | 0.000 |
| tomato                                                  | 0.000 | zucchini                                            | 0.000 | red-radish                                           | 0.000 |
| beetroot-raw                                            | 0.000 | carrot                                              | 0.000 | celeriac                                             | 0.000 |
| cauliflower                                             | 0.000 | broccoli                                            | 0.000 | kolhrabi                                             | 0.000 |
| red-cabbage                                             | 0.000 | white-cabbage                                       | 0.000 | mushroom                                             | 0.000 |
| mushrooms                                               | 0.000 | peas                                                | 0.000 | corn                                                 | 0.000 |
| leek                                                    | 0.000 | onion                                               | 0.000 | fennel                                               | 0.000 |
| green-asparagus                                         | 0.000 | white-asparagus                                     | nan   | alfa-sprouts                                         | 0.000 |
| beans-kidney                                            | 0.000 | chickpeas                                           | 0.000 | lentils                                              | 0.000 |
| pineapple                                               | 0.000 | apple                                               | 0.000 | pomegranate                                          | 0.000 |
| apricots                                                | 0.000 | banana                                              | 0.000 | berries                                              | 0.000 |
| pear                                                    | 0.000 | dates                                               | 0.000 | strawberries                                         | 0.000 |
| fruit-salad                                             | 0.000 | blueberries                                         | 0.000 | raspberries                                          | nan   |
| kiwi                                                    | 0.000 | mandarine                                           | 0.000 | mango                                                | 0.000 |
| sugar-melon                                             | 0.000 | nectarine                                           | 0.000 | orange                                               | 0.000 |
| peach                                                   | 0.000 | plums                                               | 0.000 | grapes                                               | 0.000 |
| dried-raisins                                           | 0.000 | lemon                                               | 0.000 | peanut-butter                                        | 0.000 |
| mixed-seeds                                             | 0.000 | almonds                                             | 0.000 | walnut                                               | 0.000 |
| cashew-nut                                              | 0.000 | peanut                                              | 0.000 | hazelnut                                             | 0.000 |
| mixed-nuts                                              | 0.000 | pistachio                                           | 0.000 | sesame-seeds                                         | 0.000 |
| green-olives                                            | 0.000 | black-olives                                        | 0.000 | milk                                                 | 0.000 |
| kefir-drink                                             | 0.000 | cottage-cheese                                      | 0.000 | blue-mould-cheese                                    | 0.000 |
| feta                                                    | 0.000 | fresh-cheese                                        | 0.000 | gruya-re                                             | 0.000 |
| semi-hard-cheese                                        | 0.000 | hard-cheese                                         | 0.000 | cheese                                               | 0.000 |
| mozzarella                                              | 0.000 | parmesan                                            | 0.000 | cheese-for-raclette                                  | 0.000 |
| cream-cheese                                            | 0.000 | tomme                                               | 0.000 | soft-cheese                                          | 0.000 |
| tiramisu                                                | nan   | cream                                               | nan   | sour-cream                                           | 0.000 |
| thickened-cream-35                                      | 0.000 | dairy-ice-cream                                     | 0.000 | flakes-oat                                           | 0.000 |
| rice-noodles-vermicelli                                 | 0.000 | couscous                                            | 0.000 | grits-polenta-maize-flour                            | 0.000 |
| quinoa                                                  | 0.000 | rice                                                | 0.000 | rice-basmati                                         | 0.000 |
| rice-whole-grain                                        | 0.000 | spaetzle                                            | nan   | pasta                                                | 0.000 |
| pasta-haprnli                                           | 0.000 | pasta-linguini-parpadelle-tagliatelle               | 0.000 | pasta-noodles                                        | 0.000 |
| pasta-penne                                             | 0.000 | pasta-spaghetti                                     | 0.000 | pasta-twist                                          | 0.000 |
| pasta-wholemeal                                         | 0.000 | bread-french-white-flour                            | 0.000 | bread                                                | 0.000 |
| bread-5-grain                                           | 0.000 | bread-fruit                                         | 0.000 | bread-half-white                                     | 0.000 |
| bread-grain                                             | 0.000 | bread-nut                                           | 0.000 | bread-pita                                           | 0.000 |
| bread-rye                                               | 0.000 | bread-whole-wheat                                   | 0.000 | bread-sourdough                                      | 0.000 |
| bread-black                                             | 0.000 | bread-toast                                         | 0.000 | bread-wholemeal-toast                                | 0.000 |
| bread-wholemeal                                         | 0.000 | bread-white                                         | 0.000 | brioche                                              | 0.000 |
| roll-of-half-white-or-white-flour-with-large-void       | 0.000 | roll-with-pieces-of-chocolate                       | 0.000 | focaccia                                             | 0.000 |
| croissant                                               | 0.000 | braided-white-loaf                                  | 0.000 | breadcrumbs-unspiced                                 | 0.000 |
| rusk-wholemeal                                          | 0.000 | crunch-ma1-4esli                                    | 0.000 | ma1-4esli                                            | 0.000 |
| beef                                                    | 0.000 | beef-sirloin-steak                                  | 0.000 | beef-filet                                           | 0.000 |
| beef-minced-only-meat                                   | 0.000 | beef-cut-into-stripes-only-meat                     | 0.000 | pork                                                 | 0.000 |
| chicken                                                 | 0.000 | chicken-breast                                      | 0.000 | chicken-cut-into-stripes-only-meat                   | 0.000 |
| chicken-leg                                             | 0.000 | frying-sausage                                      | 0.000 | dried-meat                                           | 0.000 |
| veal-sausage                                            | 0.000 | salami                                              | 0.000 | ham-cooked                                           | 0.000 |
| ham-raw                                                 | 0.000 | bacon-frying                                        | 0.000 | bacon-cooking                                        | 0.000 |
| meat-terrine-pata-c                                     | 0.000 | sausage                                             | 0.000 | veggie-burger                                        | 0.000 |
| tofu                                                    | 0.000 | fish                                                | 0.000 | salmon                                               | 0.000 |
| tuna                                                    | 0.000 | shrimp-prawn-small                                  | 0.000 | shrimp-prawn-large                                   | 0.000 |
| egg                                                     | 0.000 | butter                                              | 0.000 | praline                                              | 0.000 |
| jam                                                     | 0.000 | honey                                               | 0.000 | dark-chocolate                                       | 0.000 |
| milk-chocolate                                          | 0.000 | chocolate                                           | 0.000 | hazelnut-chocolate-spread-nutella-ovomaltine-caotina | nan   |
| apple-pie                                               | 0.000 | brownie                                             | 0.000 | craape-plain                                         | 0.000 |
| fruit-tart                                              | 0.000 | cake-chocolate                                      | 0.000 | omelette-plain                                       | nan   |
| tart                                                    | 0.000 | croissant-with-chocolate-filling                    | 0.000 | cookies                                              | 0.000 |
| biscuits                                                | 0.000 | chocolate-cookies                                   | 0.000 | juice-apple                                          | 0.000 |
| juice-orange                                            | 0.000 | ice-tea                                             | 0.000 | syrup-diluted-ready-to-drink                         | 0.000 |
| tea                                                     | 0.000 | cappuccino                                          | 0.000 | espresso-with-caffeine                               | 0.000 |
| coffee-with-caffeine                                    | 0.000 | white-coffee-with-caffeine                          | 0.000 | ristretto-with-caffeine                              | 0.000 |
| tea-green                                               | 0.000 | tea-black                                           | 0.000 | tea-verveine                                         | 0.000 |
| herbal-tea                                              | 0.000 | tea-peppermint                                      | 0.000 | water                                                | 5.288 |
| water-mineral                                           | 0.000 | wine-rosa-c                                         | 0.000 | wine-red                                             | 0.000 |
| wine-white                                              | 0.000 | beer                                                | 0.000 | sauce-savoury                                        | 0.000 |
| sauce-roast                                             | 0.000 | sauce-pesto                                         | 0.000 | sauce-mushroom                                       | 0.000 |
| sauce-cream                                             | 0.000 | ketchup                                             | 0.000 | bolognaise-sauce                                     | 0.000 |
| tomato-sauce                                            | 0.000 | salad-dressing                                      | 0.000 | balsamic-salad-dressing                              | 0.000 |
| french-salad-dressing                                   | 0.000 | oil-vinegar-salad-dressing                          | 0.000 | guacamole                                            | 0.000 |
| mayonnaise                                              | 0.000 | sauce-soya                                          | 0.000 | soup-vegetable                                       | 0.000 |
| soup-pumpkin                                            | 0.000 | falafel-balls                                       | 0.000 | savoury-puff-pastry                                  | 0.000 |
| corn-crisps                                             | 0.000 | crisps                                              | 0.000 | ham-croissant                                        | 0.000 |
| salt-cake-vegetables-filled                             | 0.000 | hamburger-bread-meat-ketchup                        | 0.000 | lasagne-meat-prepared                                | 0.000 |
| mashed-potatoes-prepared-with-full-fat-milk-with-butter | 0.000 | pizza-margherita-baked                              | 0.000 | sushi                                                | 0.000 |
| pancakes                                                | 0.000 | hummus                                              | 0.000 | greek-salad                                          | 0.000 |
| chocolate-mousse                                        | nan   | caprese-salad-tomato-mozzarella                     | 0.000 | taboula-c-prepared-with-couscous                     | 0.000 |
| risotto-without-cheese-cooked                           | 0.000 | salmon-smoked                                       | 0.000 | egg-scrambled-prepared                               | 0.000 |
| boisson-au-glucose-50g                                  | 0.000 | chicken-curry-cream-coconut-milk-curry-spices-paste | 0.000 | potatoes-au-gratin-dauphinois-prepared               | 0.000 |
| bircherma1-4esli-prepared-no-sugar-added                | 0.000 | fajita-bread-only                                   | 0.000 | butter-spread-puree-almond                           | nan   |
| water-with-lemon-juice                                  | 0.000 | gluten-free-bread                                   | 0.000 | fruit-coulis                                         | 0.000 |
| greek-yaourt-yahourt-yogourt-ou-yoghourt                | 0.000 | soup-of-lentils-dahl-dhal                           | 0.000 | vegetable-au-gratin-baked                            | 0.000 |
| curry-vegetarian                                        | 0.000 | yaourt-yahourt-yogourt-ou-yoghourt-natural          | 0.000 | goat-cheese-soft                                     | 0.000 |

In [ ]:

cfg.MODEL.WEIGHTS = os.path.join(cfg.OUTPUT_DIR, "model_final.pth")
cfg.MODEL.ROI_HEADS.SCORE_THRESH_TEST = 0.01
cfg.MODEL.ROI_HEADS.NUM_CLASSES = 273

cfg.DATASETS.TEST = ("validation_dataset", )
predictor = DefaultPredictor(cfg)

In [ ]:

val_metadata = MetadataCatalog.get("val_dataset")

im = cv2.imread("val/images/084035.jpg")

outputs = predictor(im)

v = Visualizer(im[:, :, ::-1],
                   metadata=val_metadata, 
                   scale=2, 
               

                   instance_mode=ColorMode.IMAGE_BW
    )

out = v.draw_instance_predictions(outputs["instances"].to("cpu"))
cv2_imshow(out.get_image()[:, :, ::-1])

Output hidden; open in https://colab.research.google.com to view.

In [ ]:

trainer.resume_or_load(resume=True)

Making Submission on AICrowd¶

Step 0 : Fork the baseline to make your own changes to it. Go to settings and make the repo private.

Step 1 : Setting up SSH to login locally to Gitlab

Run ssh-keygen -t ecdsa -b 521
Run cat ~./ssh/id_ecdsa.pub and copy the output
Go to Gitlab SSH Keys and then paste the output inside the key and use whaever title you like.

Step 2: Clone Repo & Add Models & Push Changes

Run git clone git@gitlab.aicrowd.com:[Your Username]/food_recognition_detectron2_baseline.git
Put your model inside the data directioary and then run git-lfs track "data/"
Run git stage . then git commit -m " adding model"
Run git push origin master

Step 3. Create Submission

Go to the repo and then tags and then New Tag.
In the tag name, use submission-v1, ( Everytime you make a new submission, just increase the no. like - submission-v2, submission-v3 )
A new issue will be created with showing the process. Enjoy!

If you do not have SSH Keys, Check this link

Add your SSH Keys to your GitLab account by following the instructions here

In [ ]:

aicrowd_submission = {
    "author": "Shubham",
    "username": "Shubhamai",
    "description": "detectron2 trial 2",
    "model_path": "logs/model_final.pth",
    "model_type": "model_zoo",
    "model_config_file": "COCO-InstanceSegmentation/mask_rcnn_R_50_FPN_3x.yaml",
    "detectron_model_config": {
      "ROI_HEADS": {
        "SCORE_THRESH_TEST": 0.3,
        "NUM_CLASSES": 273
      }
    }
}

aicrowd_submission["description"] = aicrowd_submission["description"].replace(" ", "-")
with open("aicrowd.json", "w") as fp:
  json.dump(aicrowd_submission, fp)

Submit to AIcrowd¶

Note: We will create an SSH key on your google drive. This key will be used to identify you on gitlab.aicrowd.com.

In [ ]:

from pycocotools.coco import COCO
import json

coco_api = COCO(TRAIN_ANNOTATIONS_PATH)

category_ids = sorted(coco_api.getCatIds())
categories = coco_api.loadCats(category_ids)

class_to_category = { int(class_id): int(category_id) for class_id, category_id in enumerate(category_ids) }

with open("class_to_category.json", "w") as fp:
  json.dump(class_to_category, fp)

loading annotations into memory...
Done (t=1.67s)
creating index...
index created!

In [ ]:

!bash <(curl -sL https://gitlab.aicrowd.com/jyotish/food-recognition-challenge-detectron2-baseline/raw/master/utils/submit-colab.sh)

4/6AGMvrF0BpkxiyuEgRUOBTD_5Z02OyP0lpURVjWWWOEYFMIxFdKa3Gg
4/6AGMvrF0BpkxiyuEgRUOBTD_5Z02OyP0lpURVjWWWOEYFMIxFdKa3Gg
4/6AGMvrF0BpkxiyuEgRUOBTD_5Z02OyP0lpURVjWWWOEYFMIxFdKa3Gg
4/6AGMvrF0BpkxiyuEgRUOBTD_5Z02OyP0lpURVjWWWOEYFMIxFdKa3Gg
4/6AGMvrF0BpkxiyuEgRUOBTD_5Z02OyP0lpURVjWWWOEYFMIxFdKa3Gg
Preparing the environment for submission 💪

WARNING: apt does not have a stable CLI interface. Use with caution in scripts.

🔍 Checking for SSH key...
Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly&response_type=code

Enter your authorization code:
Mounted at /content/drive
Verified! ✅

⚙️ Setting up git... 
Identity added: /content/drive/My Drive/aicrowd/id_ecdsa (/content/drive/My Drive/aicrowd/id_ecdsa)

⚙️ Cloning the git repository... 
Updated git hooks.
Git LFS initialized.
Git LFS: (2 of 2 files) 691.75 MB / 691.75 MB

💾️ Copying your files to the git repo... 
"*.pth" already supported
[master 5849bfc] detectron2-trial-2
 2 files changed, 3 insertions(+), 3 deletions(-)
Git LFS: (1 of 1 files) 345.88 MB / 345.88 MB
Git LFS: (0 of 0 files, 1 skipped) 0 B / 0 B, 345.88 MB skippedCounting objects: 5, done.
Delta compression using up to 2 threads.
Compressing objects: 100% (5/5), done.
Writing objects: 100% (5/5), 580 bytes | 580.00 KiB/s, done.
Total 5 (delta 3), reused 0 (delta 0)
To gitlab.aicrowd.com:Shubhamai/food-recognition-challenge-starter-kit.git
   773e57a..bdcd841  master -> master
Counting objects: 1, done.
Writing objects: 100% (1/1), 178 bytes | 178.00 KiB/s, done.
Total 1 (delta 0), reused 0 (delta 0)
To gitlab.aicrowd.com:Shubhamai/food-recognition-challenge-starter-kit.git
 * [new tag]         submission-detectron2-trial-2 -> submission-detectron2-trial-2


======================================


🎊 Solution submitted successfully! 🙌
Track the progress of your submission at https://gitlab.aicrowd.com/Shubhamai/food-recognition-challenge-starter-kit/issues

Generate More Data + Some tips & tricks 💡¶

CLoDSA seems a really great choice to generate more dataset. Fortunately there is also a really cool colab notebook on how to generate more dataset using CLoDSA

There is also an ongoing youtube series series from Immersive Limit on how to generate more dataset using blender ! Blender for AI Devs

And also this --

Data Argumentation¶

In [ ]:

!pip install --upgrade fastai

Collecting fastai
  Downloading https://files.pythonhosted.org/packages/98/2e/d4dcc69f67b4557c8543a4c65d3e136b1929b01136b227ceb986e2596825/fastai-2.0.15-py3-none-any.whl (185kB)
     |████████████████████████████████| 194kB 4.5MB/s 
Requirement already satisfied, skipping upgrade: scikit-learn in /usr/local/lib/python3.6/dist-packages (from fastai) (0.22.2.post1)
Requirement already satisfied, skipping upgrade: matplotlib in /usr/local/lib/python3.6/dist-packages (from fastai) (3.2.2)
Requirement already satisfied, skipping upgrade: requests in /usr/local/lib/python3.6/dist-packages (from fastai) (2.23.0)
Requirement already satisfied, skipping upgrade: fastprogress>=0.2.4 in /usr/local/lib/python3.6/dist-packages (from fastai) (1.0.0)
Requirement already satisfied, skipping upgrade: pillow in /usr/local/lib/python3.6/dist-packages (from fastai) (7.0.0)
Collecting torch>=1.6.0
  Downloading https://files.pythonhosted.org/packages/38/53/914885a93a44b96c0dd1c36f36ff10afe341f091230aad68f7228d61db1e/torch-1.6.0-cp36-cp36m-manylinux1_x86_64.whl (748.8MB)
     |████████████████████████████████| 748.8MB 18kB/s 
Requirement already satisfied, skipping upgrade: packaging in /usr/local/lib/python3.6/dist-packages (from fastai) (20.4)
Collecting fastcore>=1.0.5
  Downloading https://files.pythonhosted.org/packages/8e/4f/dc306a98a16a2c2c83d04636387e2945e19f8693ca8bce11bc147107d3bb/fastcore-1.0.20-py3-none-any.whl (41kB)
     |████████████████████████████████| 51kB 7.7MB/s 
Requirement already satisfied, skipping upgrade: pandas in /usr/local/lib/python3.6/dist-packages (from fastai) (1.1.2)
Requirement already satisfied, skipping upgrade: pyyaml in /usr/local/lib/python3.6/dist-packages (from fastai) (5.1)
Requirement already satisfied, skipping upgrade: scipy in /usr/local/lib/python3.6/dist-packages (from fastai) (1.4.1)
Requirement already satisfied, skipping upgrade: pip in /usr/local/lib/python3.6/dist-packages (from fastai) (19.3.1)
Collecting torchvision>=0.7
  Downloading https://files.pythonhosted.org/packages/8e/dc/4a939cfbd38398f4765f712576df21425241020bfccc200af76d19088533/torchvision-0.7.0-cp36-cp36m-manylinux1_x86_64.whl (5.9MB)
     |████████████████████████████████| 5.9MB 49.7MB/s 
Requirement already satisfied, skipping upgrade: spacy in /usr/local/lib/python3.6/dist-packages (from fastai) (2.2.4)
Requirement already satisfied, skipping upgrade: joblib>=0.11 in /usr/local/lib/python3.6/dist-packages (from scikit-learn->fastai) (0.16.0)
Requirement already satisfied, skipping upgrade: numpy>=1.11.0 in /usr/local/lib/python3.6/dist-packages (from scikit-learn->fastai) (1.18.5)
Requirement already satisfied, skipping upgrade: kiwisolver>=1.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->fastai) (1.2.0)
Requirement already satisfied, skipping upgrade: cycler>=0.10 in /usr/local/lib/python3.6/dist-packages (from matplotlib->fastai) (0.10.0)
Requirement already satisfied, skipping upgrade: python-dateutil>=2.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->fastai) (2.8.1)
Requirement already satisfied, skipping upgrade: pyparsing!=2.0.4,!=2.1.2,!=2.1.6,>=2.0.1 in /usr/local/lib/python3.6/dist-packages (from matplotlib->fastai) (2.4.7)
Requirement already satisfied, skipping upgrade: urllib3!=1.25.0,!=1.25.1,<1.26,>=1.21.1 in /usr/local/lib/python3.6/dist-packages (from requests->fastai) (1.24.3)
Requirement already satisfied, skipping upgrade: chardet<4,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from requests->fastai) (3.0.4)
Requirement already satisfied, skipping upgrade: idna<3,>=2.5 in /usr/local/lib/python3.6/dist-packages (from requests->fastai) (2.10)
Requirement already satisfied, skipping upgrade: certifi>=2017.4.17 in /usr/local/lib/python3.6/dist-packages (from requests->fastai) (2020.6.20)
Requirement already satisfied, skipping upgrade: future in /usr/local/lib/python3.6/dist-packages (from torch>=1.6.0->fastai) (0.16.0)
Requirement already satisfied, skipping upgrade: six in /usr/local/lib/python3.6/dist-packages (from packaging->fastai) (1.15.0)
Requirement already satisfied, skipping upgrade: pytz>=2017.2 in /usr/local/lib/python3.6/dist-packages (from pandas->fastai) (2018.9)
Requirement already satisfied, skipping upgrade: catalogue<1.1.0,>=0.0.7 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai) (1.0.0)
Requirement already satisfied, skipping upgrade: murmurhash<1.1.0,>=0.28.0 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai) (1.0.2)
Requirement already satisfied, skipping upgrade: srsly<1.1.0,>=1.0.2 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai) (1.0.2)
Requirement already satisfied, skipping upgrade: thinc==7.4.0 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai) (7.4.0)
Requirement already satisfied, skipping upgrade: blis<0.5.0,>=0.4.0 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai) (0.4.1)
Requirement already satisfied, skipping upgrade: tqdm<5.0.0,>=4.38.0 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai) (4.41.1)
Requirement already satisfied, skipping upgrade: wasabi<1.1.0,>=0.4.0 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai) (0.8.0)
Requirement already satisfied, skipping upgrade: preshed<3.1.0,>=3.0.2 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai) (3.0.2)
Requirement already satisfied, skipping upgrade: cymem<2.1.0,>=2.0.2 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai) (2.0.3)
Requirement already satisfied, skipping upgrade: setuptools in /usr/local/lib/python3.6/dist-packages (from spacy->fastai) (50.3.0)
Requirement already satisfied, skipping upgrade: plac<1.2.0,>=0.9.6 in /usr/local/lib/python3.6/dist-packages (from spacy->fastai) (1.1.3)
Requirement already satisfied, skipping upgrade: importlib-metadata>=0.20; python_version < "3.8" in /usr/local/lib/python3.6/dist-packages (from catalogue<1.1.0,>=0.0.7->spacy->fastai) (2.0.0)
Requirement already satisfied, skipping upgrade: zipp>=0.5 in /usr/local/lib/python3.6/dist-packages (from importlib-metadata>=0.20; python_version < "3.8"->catalogue<1.1.0,>=0.0.7->spacy->fastai) (3.2.0)
Installing collected packages: torch, fastcore, torchvision, fastai
  Found existing installation: torch 1.5.0+cu101
    Uninstalling torch-1.5.0+cu101:
      Successfully uninstalled torch-1.5.0+cu101
  Found existing installation: torchvision 0.6.0+cu101
    Uninstalling torchvision-0.6.0+cu101:
      Successfully uninstalled torchvision-0.6.0+cu101
  Found existing installation: fastai 1.0.61
    Uninstalling fastai-1.0.61:
      Successfully uninstalled fastai-1.0.61
Successfully installed fastai-2.0.15 fastcore-1.0.20 torch-1.6.0 torchvision-0.7.0

In [ ]:

from fastai.vision.core import *
from fastai.vision.utils import *
from fastai.vision.augment import *
from fastai.data.core import *
from fastai.data.transforms import *

In [ ]:

images, lbl_bbox = get_annotations('/content/train/annotations.json')
# idx=2
# coco_fn,bbox = coco/'train'/images[idx],lbl_bbox[idx]

# def _coco_bb(x):  return TensorBBox.create(bbox[0])
# def _coco_lbl(x): return bbox[1]

In [ ]:

idx=4
coco_fn,bbox = '/content/train/images/'+images[idx],lbl_bbox[idx]

def _coco_bb(x):  return TensorBBox.create(bbox[0])
def _coco_lbl(x): return bbox[1]

In [ ]:

coco_dsrc = Datasets([coco_fn]*10, [PILImage.create, [_coco_bb,], [_coco_lbl, MultiCategorize(add_na=True)]], n_inp=1)
coco_tdl = TfmdDL(coco_dsrc, bs=9, after_item=[BBoxLabeler(), PointScaler(), ToTensor(), Resize(256)],
                  after_batch=[IntToFloatTensor(), *aug_transforms()])

coco_tdl.show_batch(max_n=9)

Content

3651

Show Comments

Comments

You must login before you can post a comment.