Loading

Lidar Car Detection

[Getting Started Notebook] Lidar Car Detection

A Getting Started notebook for Car Detection using Lidar Puzzle of BlitzXI.

Shubhamaicrowd

Starter Code for Lidar Car Detection

What we are going to Learn

  • Learning about how lidar works
  • Using scikit-learn for binary classification.

Note : Create a copy of the notebook and use the copy for submission. Go to File > Save a Copy in Drive to create a new copy

Downloading Dataset

Installing aicrowd-cli

In [ ]:
!pip install aicrowd-cli
%load_ext aicrowd.magic
Collecting aicrowd-cli
  Downloading aicrowd_cli-0.1.9-py3-none-any.whl (43 kB)
     |████████████████████████████████| 43 kB 866 kB/s 
Collecting requests-toolbelt<1,>=0.9.1
  Downloading requests_toolbelt-0.9.1-py2.py3-none-any.whl (54 kB)
     |████████████████████████████████| 54 kB 2.2 MB/s 
Requirement already satisfied: click<8,>=7.1.2 in /usr/local/lib/python3.7/dist-packages (from aicrowd-cli) (7.1.2)
Collecting GitPython==3.1.18
  Downloading GitPython-3.1.18-py3-none-any.whl (170 kB)
     |████████████████████████████████| 170 kB 59.2 MB/s 
Collecting requests<3,>=2.25.1
  Downloading requests-2.26.0-py2.py3-none-any.whl (62 kB)
     |████████████████████████████████| 62 kB 861 kB/s 
Collecting rich<11,>=10.0.0
  Downloading rich-10.7.0-py3-none-any.whl (209 kB)
     |████████████████████████████████| 209 kB 52.5 MB/s 
Requirement already satisfied: toml<1,>=0.10.2 in /usr/local/lib/python3.7/dist-packages (from aicrowd-cli) (0.10.2)
Requirement already satisfied: tqdm<5,>=4.56.0 in /usr/local/lib/python3.7/dist-packages (from aicrowd-cli) (4.62.0)
Collecting gitdb<5,>=4.0.1
  Downloading gitdb-4.0.7-py3-none-any.whl (63 kB)
     |████████████████████████████████| 63 kB 1.6 MB/s 
Requirement already satisfied: typing-extensions>=3.7.4.0 in /usr/local/lib/python3.7/dist-packages (from GitPython==3.1.18->aicrowd-cli) (3.7.4.3)
Collecting smmap<5,>=3.0.1
  Downloading smmap-4.0.0-py2.py3-none-any.whl (24 kB)
Requirement already satisfied: charset-normalizer~=2.0.0 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.25.1->aicrowd-cli) (2.0.4)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.25.1->aicrowd-cli) (2021.5.30)
Requirement already satisfied: idna<4,>=2.5 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.25.1->aicrowd-cli) (2.10)
Requirement already satisfied: urllib3<1.27,>=1.21.1 in /usr/local/lib/python3.7/dist-packages (from requests<3,>=2.25.1->aicrowd-cli) (1.24.3)
Collecting colorama<0.5.0,>=0.4.0
  Downloading colorama-0.4.4-py2.py3-none-any.whl (16 kB)
Collecting commonmark<0.10.0,>=0.9.0
  Downloading commonmark-0.9.1-py2.py3-none-any.whl (51 kB)
     |████████████████████████████████| 51 kB 5.9 MB/s 
Requirement already satisfied: pygments<3.0.0,>=2.6.0 in /usr/local/lib/python3.7/dist-packages (from rich<11,>=10.0.0->aicrowd-cli) (2.6.1)
Installing collected packages: smmap, requests, gitdb, commonmark, colorama, rich, requests-toolbelt, GitPython, aicrowd-cli
  Attempting uninstall: requests
    Found existing installation: requests 2.23.0
    Uninstalling requests-2.23.0:
      Successfully uninstalled requests-2.23.0
ERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.
google-colab 1.0.0 requires requests~=2.23.0, but you have requests 2.26.0 which is incompatible.
datascience 0.10.6 requires folium==0.2.1, but you have folium 0.8.3 which is incompatible.
Successfully installed GitPython-3.1.18 aicrowd-cli-0.1.9 colorama-0.4.4 commonmark-0.9.1 gitdb-4.0.7 requests-2.26.0 requests-toolbelt-0.9.1 rich-10.7.0 smmap-4.0.0
In [ ]:
%aicrowd login
Please login here: https://api.aicrowd.com/auth/qh-1j89QrIq8pINo27vn-1ZgTNPTVt5Nrv3pLH7nkEs
API Key valid
Saved API Key successfully!
In [ ]:
!rm -rf data
!mkdir data
%aicrowd ds dl -c lidar-car-detection -o data

Importing Libraries

In [ ]:
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestRegressor
import os
import matplotlib.pyplot as plt
import plotly.graph_objects as go
import random

Reading the dataset

In [ ]:
# Reading the training dataset

train_data = np.load("/content/data/train.npz", allow_pickle=True)
train_data = train_data['train']

train_data.shape
Out[ ]:
(400, 2)

Visualizing the dataset

In this section, we will be visualizing a sample 3D lidar data

In [ ]:
# Getting a random 3D lidar sample data
INDEX = random.randint(0, train_data.shape[0])

# Getting the individual x,y and z points.
x = train_data[INDEX][0][:, 0].tolist()
y = train_data[INDEX][0][:, 1].tolist()
z = train_data[INDEX][0][:, 2].tolist()

# Label for the corrosponding sample ( no. of cars )
label  = train_data[INDEX][1]

# Generating the 3D graph
fig = go.Figure(data=[go.Scatter3d(x=x, y=y, z=z,
                                   mode='markers',
                                   marker=dict(
                                   size=1,       
                                   colorscale='Viridis',
                                   opacity=0.8))])
print("No. of cars : ", label)
fig.show()
No. of cars :  2

Can you try finding cars in this 3d data ?

Splitting the dataset

In [ ]:
# Getting the 3d points and flattening the points into 1d array ( using only 100 training samples for faster teaining )
X = train_data[:100, 0]
X = [i.flatten()    for i in X]

# labels
y = train_data[:100, 1]
In [ ]:
# Splitting the dataset into training and testing
X_train, X_val, y_train, y_val = train_test_split(X, y, test_size=0.2)

Training the model

In [ ]:
model = RandomForestRegressor(verbose=True, n_jobs=-1)
In [ ]:
model.fit(X_train, y_train)
[Parallel(n_jobs=-1)]: Using backend ThreadingBackend with 2 concurrent workers.
[Parallel(n_jobs=-1)]: Done  46 tasks      | elapsed:   41.8s
[Parallel(n_jobs=-1)]: Done 100 out of 100 | elapsed:  1.5min finished
Out[ ]:
RandomForestRegressor(bootstrap=True, ccp_alpha=0.0, criterion='mse',
                      max_depth=None, max_features='auto', max_leaf_nodes=None,
                      max_samples=None, min_impurity_decrease=0.0,
                      min_impurity_split=None, min_samples_leaf=1,
                      min_samples_split=2, min_weight_fraction_leaf=0.0,
                      n_estimators=100, n_jobs=-1, oob_score=False,
                      random_state=None, verbose=True, warm_start=False)

Validation

In [ ]:
model.score(X_val, y_val)
[Parallel(n_jobs=2)]: Using backend ThreadingBackend with 2 concurrent workers.
[Parallel(n_jobs=2)]: Done  46 tasks      | elapsed:    0.0s
[Parallel(n_jobs=2)]: Done 100 out of 100 | elapsed:    0.0s finished
Out[ ]:
0.028448863636363897

Generating the predictions

In [ ]:
# Loading the test data

test_data = np.load("/content/data/test.npz", allow_pickle=True)
test_data = test_data['test']

test_data.shape
Out[ ]:
(601,)
In [ ]:
# flattening the points into 1d array
X_test = X = [i.flatten()    for i in test_data]
In [ ]:
# Generating the predictions
predictions = model.predict(X_test)
predictions.shape
[Parallel(n_jobs=2)]: Using backend ThreadingBackend with 2 concurrent workers.
[Parallel(n_jobs=2)]: Done  46 tasks      | elapsed:    0.0s
[Parallel(n_jobs=2)]: Done 100 out of 100 | elapsed:    0.0s finished
Out[ ]:
(601,)
In [ ]:
submission = pd.DataFrame({"label":predictions})
submission
Out[ ]:
label
0 2.14
1 3.56
2 2.25
3 3.02
4 2.28
... ...
596 1.79
597 2.04
598 2.43
599 2.18
600 2.01

601 rows × 1 columns

In [ ]:
# Saving the predictions
!rm -rf assets
!mkdir assets
submission.to_csv(os.path.join("assets", "submission.csv"))

Submitting our Predictions

Note : Please save the notebook before submitting it (Ctrl + S)

In [ ]:
!aicrowd notebook submit -c lidar-car-detection -a assets --no-verify
Mounting Google Drive 💾
Your Google Drive will be mounted to access the colab notebook
Go to this URL in a browser: https://accounts.google.com/o/oauth2/auth?client_id=947318989803-6bn6qk8qdgf4n4g3pfee6491hc0brc4i.apps.googleusercontent.com&redirect_uri=urn%3aietf%3awg%3aoauth%3a2.0%3aoob&scope=email%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdocs.test%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.photos.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fpeopleapi.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fdrive.activity.readonly%20https%3a%2f%2fwww.googleapis.com%2fauth%2fexperimentsandconfigs%20https%3a%2f%2fwww.googleapis.com%2fauth%2fphotos.native&response_type=code

Enter your authorization code:
4/1AX4XfWhdK3UfTXVvL7raKdiHQ8QGpX1YJwQOZJoDT6NddhPN-_7kMyfCvyw
Mounted at /content/drive
Using notebook: /content/drive/MyDrive/Colab Notebooks/Lidar Car Prediction for submission...
Scrubbing API keys from the notebook...
Collecting notebook...
submission.zip ━━━━━━━━━━━━━━━━━━━━━━━━ 100.0%1.4/1.4 MB1.9 MB/s0:00:00
                                                  ╭─────────────────────────╮                                                  
                                                  │ Successfully submitted! │                                                  
                                                  ╰─────────────────────────╯                                                  
                                                        Important links                                                        
┌──────────────────┬──────────────────────────────────────────────────────────────────────────────────────────────────────────┐
│  This submission │ https://www.aicrowd.com/challenges/blitz-xi/problems/lidar-car-detection/submissions/153311              │
│                  │                                                                                                          │
│  All submissions │ https://www.aicrowd.com/challenges/blitz-xi/problems/lidar-car-detection/submissions?my_submissions=true │
│                  │                                                                                                          │
│      Leaderboard │ https://www.aicrowd.com/challenges/blitz-xi/problems/lidar-car-detection/leaderboards                    │
│                  │                                                                                                          │
│ Discussion forum │ https://discourse.aicrowd.com/c/blitz-xi                                                                 │
│                  │                                                                                                          │
│   Challenge page │ https://www.aicrowd.com/challenges/blitz-xi/problems/lidar-car-detection                                 │
└──────────────────┴──────────────────────────────────────────────────────────────────────────────────────────────────────────┘
In [ ]:


Comments

sean_benhur
About 3 years ago

Just a small mistake, we are using scikit-learn for regression

eric_parisot
About 3 years ago

Comment deleted by eric_parisot.

You must login before you can post a comment.

Execute