Lidar Car Detection
Keras Classification for Lidar Car Detection (counting)
Count the cars in 3D lidar data
eric_parisot
In this dataset we got 3D points cloud from lidars and the task is to train a classifier to count other cars on the road.
To make things easyer to work with, we are going to remove useless data (over the car, floor informations), flatten the 3D points to obtain 2D images (view from top), augment the quantity of data (x4) by rotating the images and then train a Keras NN classifer on it...
Let's go !
Starter Code for Lidar Car Detection
What we are going to Learn¶
- Learning about how lidar works
- data normalisation
- data augmentation
- Using Keras for binary classification.
Note : Create a copy of the notebook and use the copy for submission. Go to File > Save a Copy in Drive to create a new copy
Hi fellas !¶
In this dataset we got 3D points cloud from lidars and the task is to train a classifier to count other cars on the road. To make things easyer to work with, we are going to remove useless data (over the car, floor informations), flatten the 3D points to obtain 2D images (view from top), augment the quantity of data (x4) by rotating the images and then train a Keras NN classifer on it... Let's go !
Downloading Dataset¶
Installing aicrowd-cli
In [1]:
!pip install aicrowd-cli
%load_ext aicrowd.magic
In [2]:
%aicrowd login
In [3]:
!rm -rf data
!mkdir data
%aicrowd ds dl -c lidar-car-detection -o data
Importing Libraries¶
In [1]:
import pandas as pd
import numpy as np
from sklearn.model_selection import StratifiedKFold
import random
import os
import matplotlib.pyplot as plt
import plotly.graph_objects as go
%matplotlib notebook
import tensorflow as tf
from tensorflow.keras.layers import *
from tensorflow.keras.models import Model, load_model
from tensorflow.keras.optimizers import Adam
from tensorflow.keras.utils import to_categorical
import tensorflow.keras.backend as K
Reading the dataset¶
In [2]:
# Reading the training dataset
train_data = np.load("data/train.npz", allow_pickle=True)
test_data = np.load("data/test.npz", allow_pickle=True)
train_data = train_data['train']
test_data = test_data['test']
train_data.shape, test_data.shape
Out[2]:
Visualizing the dataset¶
In this section, we will be visualizing a sample 3D lidar data
In [3]:
# Getting a random 3D lidar sample data
INDEX = random.randint(0, train_data.shape[0]-1)
# Getting the individual x,y and z points.
x = train_data[INDEX][0][:, 0].tolist()
y = train_data[INDEX][0][:, 1].tolist()
z = train_data[INDEX][0][:, 2].tolist()
# Label for the corrosponding sample ( no. of cars )
label = train_data[INDEX][1]
# Generating the 3D graph
fig = go.Figure(data=[go.Scatter3d(x=x, y=y, z=z,
mode='markers',
marker=dict(
size=1,
colorscale='Viridis',
opacity=0.8))])
print("No. of cars : ", label)
fig.show()
Labels Distribution¶
In [4]:
print(train_data[:, 1].max())
plt.figure()
plt.hist(train_data[:, 1])
Out[4]:
Can you try finding cars in this 3d data ?
turn X and X_test to DataFrames¶
In [38]:
X = np.array([sample for sample in train_data[:, 0].tolist()])
Y = np.array(train_data[:, 1], dtype=np.int)
X_test = np.array([sample for sample in test_data.tolist()])
m,n,r = X.shape
out_arr = np.column_stack((np.repeat(np.arange(m),n), X.reshape(m*n,-1)))
df = pd.DataFrame(out_arr)
df.columns = ["idx", "x", "y", "z"]
m,n,r = X_test.shape
out_arr = np.column_stack((np.repeat(np.arange(m),n), X_test.reshape(m*n,-1)))
df_test = pd.DataFrame(out_arr)
df_test.columns = ["idx", "x", "y", "z"]
In [ ]:
In [39]:
df, df_test
Out[39]:
Normalize¶
In [15]:
df["x"] = df["x"] / 60 + 0.5
df["y"] = df["y"] / 60 + 0.5
df["z"] = df["z"] / 16 + 0.2
df_test["x"] = df_test["x"] / 60 + 0.5
df_test["y"] = df_test["y"] / 60 + 0.5
df_test["z"] = df_test["z"] / 16 + 0.2
Drop low / high z values rows¶
In [16]:
df = df.drop(df[(df["z"] > 0.25) | (df["z"] < 0.095)].index).reset_index()
df = df.drop(["z",], axis=1).reset_index()
df_test = df_test.drop(df_test[(df_test["z"] > 0.25) | (df_test["z"] < 0.095)].index).reset_index()
df_test = df_test.drop(["z",], axis=1).reset_index()
In [17]:
# Getting a random 3D lidar sample data
INDEX = random.randint(0, train_data.shape[0]-1)
# Getting the individual x,y and z points.
x = df["x"][df["idx"] == INDEX]
y = df["y"][df["idx"] == INDEX]
plt.figure()
plt.plot(x)
plt.plot(y)
# Label for the corrosponding sample ( no. of cars )
label = Y[INDEX]
# Generating the 3D graph
fig = go.Figure(data=[go.Scatter(x=x, y=y,
mode='markers',
marker=dict(
size=1,
colorscale='Viridis',
opacity=0.8))])
print("No. of cars : ", label)
fig.show()
Turn 3D points cloud to flat images¶
In [18]:
img_size = 128
images = np.zeros((int(df["idx"].max()+1), img_size, img_size, 1))
images_test = np.zeros((int(df_test["idx"].max()+1), img_size, img_size, 1))
def make_img(points, images):
for i, r in points.iterrows():
images[int(r["idx"])][int(r["x"]*img_size)][int(r["y"]*img_size)] = [1]
df.groupby("idx").apply(lambda x: make_img(x, images))
df_test.groupby("idx").apply(lambda x: make_img(x, images_test))
Out[18]:
In [19]:
plt.figure()
plt.imshow(images[INDEX], cmap="gray")
plt.show()
Data augmentation (+ 90, 180, 270 rotations)¶
In [42]:
augmented_images = np.vstack((np.rot90(images, axes=(1, 2)), images))
augmented_images = np.vstack((np.rot90(images, k=2, axes=(1, 2)), augmented_images))
augmented_images = np.vstack((np.rot90(images, k=3, axes=(1, 2)), augmented_images))
augmented_Y = np.hstack((Y, Y, Y, Y))
In [43]:
augmented_images.shape, augmented_Y.shape
Out[43]:
Splitting the dataset¶
In [47]:
# Splitting the dataset into training and testing
skf = StratifiedKFold(n_splits=5)
for train_index, test_index in skf.split(augmented_images, augmented_Y):
X_train, X_val = augmented_images[train_index], augmented_images[test_index]
Y_train, Y_val = augmented_Y[train_index], augmented_Y[test_index]
Make targets categorical¶
In [48]:
cat_Y = to_categorical(augmented_Y, num_classes=8)
Y_train = to_categorical(Y_train, num_classes=8)
Y_val = to_categorical(Y_val, num_classes=8)
#X_train, X_val, Y_train, Y_val = train_test_split(images, Y, test_size=0.2)
print(X_train.shape, X_val.shape, Y_train.shape, Y_val.shape)
Training the model¶
In [74]:
K.clear_session()
#############################################################
img_in = Input(shape=(img_size, img_size, 1), name='img_in')
x = img_in
x = Convolution2D(32, (3,3), strides=(2,2), activation="relu")(x)
x = MaxPooling2D(2)(x) # MaxPooling will preserve the "geographical" information of images
x = Dropout(0.4)(x)
x = Convolution2D(64, (3,3), strides=(2,2), activation="relu")(x)
x = MaxPooling2D(2)(x)
x = Dropout(0.3)(x)
x = Convolution2D(128, (1,1), activation="relu")(x)
x = MaxPooling2D(2)(x)
x = Dropout(0.2)(x)
x = Flatten(name='flattened')(x)
x = Dense(256, activation='relu')(x)
x = Dropout(0.2)(x)
# Output
out = Dense(8, activation='softmax')(x)
# Compile Model
model = Model(inputs=[img_in], outputs=[out])
model.compile(loss='categorical_crossentropy', optimizer=Adam(learning_rate=1e-3), metrics=['accuracy'])
model.summary()
In [75]:
#save best model if model improved
model_name = "car_lidar_detect.h5"
best_checkpoint = tf.keras.callbacks.ModelCheckpoint(model_name, monitor='val_loss', verbose=1, save_best_only=True, mode='min')
In [76]:
#h = model.fit(X_train, Y_train, validation_data=(X_val, Y_val), batch_size=32, epochs=600, callbacks=[best_checkpoint])
h = model.fit(augmented_images, cat_Y, batch_size=32, epochs=200)
In [ ]:
In [77]:
#print History graph
historydf = pd.DataFrame(h.history, index=h.epoch)
historydf.plot(ylim=(0,1))
Out[77]:
Validation¶
In [68]:
X_val.shape
Out[68]:
In [69]:
#model = load_model(model_name)
In [ ]:
score = 0
for i, d in enumerate(X_val):
val_pred = model.predict(np.array([d]))
#print(val_pred[0], Y_val[i])
if np.argmax(val_pred) == np.argmax(Y_val[i]):
score += 1
print(score, "/", len(Y_val))
Generating the predictions¶
In [ ]:
# Generating the predictions
pred = model.predict(images_test)
predictions = np.argmax(pred, axis=1)
predictions.shape
In [ ]:
submission = pd.DataFrame({"label":predictions})
submission.head(1000)
In [ ]:
# Saving the predictions
!rm -rf assets
!mkdir assets
submission.to_csv(os.path.join("assets", "submission.csv"))
In [132]:
model.save(model_name)
LeaderBoard Result : TotErrors = 14.000 MaxError = 6.000¶
Thank you !¶
Content
Comments
You must login before you can post a comment.