Single Camera Object Detection for Self-Driving Vehicle: A Review

S. Herman and K. Ismail

Abstract: The development of technologies for autonomous vehicle (AV) have seen rapid achievement in the recent years. Commercial carmakers are actively embedding this system in their production and are undergoing tremendous testing in the real world traffic environment. It is one of today’s most challenging topics in the intelligent transportation system (ITS) field in term of reliability as well as accelerating the world’s transition to a sustainable future. The utilization of current sensor technology however indicates some drawbacks where the complexity is high and the cost is extremely huge. This paper reviews the recent sensor technologies and their contributions in becoming part of the autonomous self-driving vehicle system. The ultimate focus is toward reducing the sensor count to just a single camera based on the single modality model. The capability of the sensor to detect and recognize on-the-road obstacles such as overtaking vehicle, pedestrians, signboards, bicycle, road lane marker and road curvature will be discussed. Different feature extraction approach will be reviewed further with the selection of the recent Artificial Intelligent (AI) methods that are being implemented. At the end of this review, the optimal techniques of processing information from single camera system will be discussed and summarized.

Keywords:Self-driving vehicle, autonomous vehicle


Aly, M. (2014). Real time detection of lane markers in urban streets. California: California Institute of Technology (Caltech).

Audi USA (6 January 2017). Press release: Audi and NVIDIA team up to bring fully automated driving to the roads starting in 2020 accelerated with artificial intelligence. Retrieved from

Baik, S., & Greenblatt, N. (2016). A big question for self-driving car developers. Sidley Publication.

Bilbeisi, K.M., & Kesse, M. (2017). Tesla: A successful entrepreneurship strategy. Morrow, GA: Clayton State University.

Bojarski, M., Testa, D.D., Dworakowski, D., Firner, B., Flepp, B., Goyal, P., Jackel, L.D., Monfort, M., Muller, U., Zhang, J., Zhang, X., & Zhao, J. (2016). End to end learning for self-driving cars. Holmdel, NJ: NVIDIA Corporation.

Chen, C., Seff, A., Kornhauser, A., & Xiao, J. (2015). DeepDriving: Learning affordance for direct perception in autonomous driving. Computer Vision Foundation (CVF).

Fan, Q., Brown, L., & Smith, J. (2016). A closer look at Faster R-CNN for vehicle detection. Paper presented at IEEE Intelligent Vehicles Symposium (IV), Gothenburg, Sweden.

Haloi, M., & Jayagopi, D.B. (2015). Vehicle local position estimation system. Paper presented at 2014 IEEE International Conference on Vehicular Electronics and Safety (ICVES’14), Hyderabad, India.

Jeon, J., Hwang, S-H, & Moon, H. (2016). Monocular vision-based object recognition for autonomous driving in a real driving environment. Paper presented at 13th International Conference on Ubiquitous Robots and Ambient Intelligent (URAI), Xian, China.

Jiang, T., Petrovic, S., Ayyer, U., Tolani, A., & Husain, S. (2015). Self-driving cars: Disruptive or incremental. Applied Innovation Review, June 2015(1), 3-22.

Kaur, R., & Marwaha, C. (2017). A review on the performance of object detection algorithm. International Journal of Engineering and Computer Science, 6, 20572-20576.

LeCun, Y., Urs Muller, U., Ben, J., Cosatto, E., & Flepp, B. (2005). Off-road obstacle avoidance through end-to-end learning. Advances in Neural Information Processing Systems 18 – Proceedings of the 2005 Conference, 739-746.

Litman, T. (2017). Autonomous vehicle implementation predictions, implications for transport planning. Victoria, Canada: Victoria Transport Policy Institute.

McBride, J., Snorrason, M., Eaton, R., Checka, N., Reiter, A., Foil, G., & Stevens, M.R. (2006). Object detection with single-camera stereo. Proceedings Volume 6230, Unmanned Systems Technology VIII, 623002. doi: 10.1117/12.669024

Miao, X., Li, S., & Shen, H. (2012). On-board lane detection system for intelligent vehicle based on monocular vision. International Journal on Smart Sensing and Intelligent Systems, 5(4), 957- 972.

Peng, H. (2016). Connected automated vehicle, the roles of dynamics and control. The Magazine of ASME: Mechanical Engineering, Technology that Moves the World 12,138, 4-11.

Romera, E., Bergasa, L.M., & Arroy, R. (2016). Can we unify monocular detectors for autonomous driving by using the pixel-wise semantic segmentation of CNNs? Paper presented at Workshop at the IEEE Symposium on Intelligent Vehicles 2016 (IV16-WS), Gothenburg, Sweden.

Sanchez, D. (2015). Collective technologies: Autonomous vehicles (Working paper). Securing Australia’s Future (SAF), Project 05. Melbourne, VIC: Australian Council of Learn Academies (ACOLA).