Lidar Car Detection
5 Fold Cross Validation Solution for Lidar Car Detection
A detailed solution for challenge Lidar Car Detection
g_mothy
5-Fold Cross Validation XGBoost Solution for Lidar Car Detection
Lidar Car Detection¶
Downloading Dataset¶
Installing aicrowd-cli
In [1]:
!pip install aicrowd-cli
%load_ext aicrowd.magic
In [5]:
%aicrowd login
In [6]:
!rm -rf data
!mkdir data
%aicrowd ds dl -c lidar-car-detection -o data
Importing Libraries¶
In [11]:
import os
import random
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
import plotly.graph_objects as go
import xgboost as xgb
import lightgbm as lgb
from sklearn.model_selection import train_test_split
from sklearn.tree import DecisionTreeRegressor,ExtraTreeRegressor
from sklearn.ensemble import RandomForestRegressor
from sklearn.ensemble import AdaBoostRegressor,BaggingRegressor
from sklearn.ensemble import ExtraTreesRegressor,GradientBoostingRegressor
from sklearn.model_selection import KFold
seed = 2020
Reading the dataset¶
In [13]:
# Reading the training dataset
data_dir = "./data"
train_data = np.load(os.path.join(data_dir,"train.npz"), allow_pickle=True)
train_data = train_data['train']
# Loading the test data
test_data = np.load(os.path.join(data_dir, "test.npz"), allow_pickle=True)
test_data = test_data['test']
train_data.shape, test_data.shape
Out[13]:
Visualizing the dataset¶
In this section, we will be visualizing a sample 3D lidar data
In [9]:
# Getting a random 3D lidar sample data
INDEX = random.randint(0, train_data.shape[0])
# Getting the individual x,y and z points.
x = train_data[INDEX][0][:, 0].tolist()
y = train_data[INDEX][0][:, 1].tolist()
z = train_data[INDEX][0][:, 2].tolist()
# Label for the corrosponding sample ( no. of cars )
label = train_data[INDEX][1]
# Generating the 3D graph
fig = go.Figure(data=[go.Scatter3d(x=x, y=y, z=z,
mode='markers',
marker=dict(
size=1,
colorscale='Viridis',
opacity=0.8))])
print("No. of cars : ", label)
fig.show()
Splitting the dataset¶
In [15]:
# Getting the 3d points and flattening the points into 1d array
X = train_data[:, 0]
X = [i.flatten() for i in X]
# labels
y = train_data[:, 1]
# flattening the points into 1d array
X_test = [i.flatten() for i in test_data]
Training the model¶
5 Fold Cross Validation
In [34]:
K = 5
kf = KFold(n_splits = K, random_state = seed, shuffle = True)
xgb_preds = []
train = np.array(X)
target_train = np.array(y)
test = np.array(X_test)
train.shape,target_train.shape,test.shape
Out[34]:
In [35]:
for train_index, test_index in kf.split(train):
train_X, valid_X = train[train_index], train[test_index]
train_y, valid_y = target_train[train_index], target_train[test_index]
xgb_params = {
"objective":"reg:squarederror",
'colsample_bytree': 0.3,
'learning_rate': 0.1,
'max_depth': 5,
'alpha': 10}
d_train = xgb.DMatrix(train_X, train_y)
d_valid = xgb.DMatrix(valid_X, valid_y)
d_test = xgb.DMatrix(test)
watchlist = [(d_train, 'train'), (d_valid, 'valid')]
model = xgb.train(xgb_params,
d_train,
5000,
watchlist,
maximize=True,
verbose_eval=50,
early_stopping_rounds=100)
xgb_pred = model.predict(d_test)
xgb_preds.append(list(xgb_pred))
In [36]:
preds=[]
for i in range(len(xgb_preds[0])):
sum=0
for j in range(K):
sum+=xgb_preds[j][i]
preds.append(sum / K)
In [45]:
submission = pd.DataFrame({"label":np.round(preds)})
submission
Out[45]:
In [47]:
# Saving the predictions
!rm -rf assets
!mkdir assets
submission.to_csv(os.path.join("assets", "submission.csv"))
In [48]:
submission['label'].value_counts()
Out[48]:
Submitting our Predictions¶
In [ ]:
In [ ]:
Content
Comments
You must login before you can post a comment.