Ever wonder if it is possible to calculate the speed of the vehicle on the road through camera videos? Yes, indeed it could be, but not so accurate. My goal is to:
- Achieve at least 10 FPS.
- Able to track all if not most of the vehicles from the camera.
- The speed should be reasonable and can be verified through the bare eye. (We don’t want the estimator to say the car is 100 km/h, but we can see the car is not even moving in the video).
- Simple, with less library and less line of codes.
So, that left with me to use the DNN module in OpenCV. And, SSD is chosen given the mAP and the FPS achieved. The model used is the pre-trained with COCO dataset.
Library used
SSD Model
In cv2.dnn.readNetFromCaffe, the prototxt and caffemodel files are needed. The prototxt can be found here and the caffemodel here.
Deploy them.
Video source
Get your sample video source. I get mine here.
Estimating speed
The basic of calculating speed is to get the distance travelled per second. But in images, we did not have any info in meter, thus, we need to convert the pixel travelled per second (px/s) to meters per second (m/s). Then from meters per second (m/s) to kilometre per hour (km/h).
To calculate the distance travelled, we get the distance of the same object in two frames, the current and previous frame. From simple mathematics equation, we can derive the distance such as,
where (x₂, y₂) and (x₁,y₁) represents the current frame’s and previous frame’s centroid of the vehicle respectively.
Next, the speed can be calculated through the equation below.
The pixel per meter can be calculated with a reference object. For example, the width of a car is 2 metre, and in the camera, it is represented by 100 pixels, then one can say the ppm is 50. But, as you can see here, the object will become smaller as it becomes further in the image. Thus, the value is just a bare estimation. In this case, we will define the region where the ppm is calculated, and the speed estimation will be done in the selected region. In code implementation,
Note that this method is not accurate.
The main part
Now, let’s begin our adventure!
- From line 2–12, we are declaring some variables to be used later.
- Line 15 marks the starting time, which is used to calculate the FPS later.
- Line 16 reads the video input
- Line 17–18 check if the video ended (having type of None) and break ‘em.
- Line 20 resize the video frame to 1280*780
- Line 21 copies the image which is to add in the annotation boxes later.
- Line 23 states the frame passed. We only takes 1 frames per 10 frames, where if the video have 30 FPS, only 3 frames are taken every second.
- Line 25 is an arbitrary list to record the car ID to be deleted later.
- Line 27–31 determine if the vehicle shall be stoped tracked since if the vehicle has gone far away, we shall stop tracking them. The ID will be added to
carIDtoDelete. - Line 33–37 delete all the vehicles that are in
carIDtoDelete. - Line 39 check if 10 frames has passed and goes on the detection.
- Line 40 calls the CV2’s DNN function. Note that the input image is scaled to 300⨉300, because the input to SSD model is 300⨉300. The confidence values are pre-defined for the optimum detection.
- Line 41 tells that the input to SSD is the image from Line 40.
- Line 42 tells the model to go forward.
- Line 45–64 stores the detected cars or buses in the array.
- Line 66–102 check if the vehicles exists with an ID, if not assigned with a new one. If the vehicle is being tracked, get the current location.
- Line 105–112 is to copy the current location and save it for later used, and the difference between current and previous frame pixel can then be used to calculate the distance travelled.
- Line 113 is to draw the rectangle around the car.
- Line 117 marks the end of time taken for the detection, which is used to calculate the FPS.
- Line 119–122 is to calculate the FPS and show them.
- Line 125–137 is then used to estimate the speed and show on the picture.
Thank you for making till the end!
Kaggle notebook is available here! (Note that some of the code is changed to show video in Jupyter Notebook)
References:
