Due to my heavy workload on coursework and research, I have to pause this project and expect to continue in Spring 2020. Here is a brief description of what I've finished so far and some future plans.
- Hardware: Jetson Nano or Google Coral (currently I'm using a Ubuntu laptop), Logitech Webcam, 7-inch HDMI display with touchscreen for Raspberry Pi
- Environment: Ubuntu 18.04, Python 3.7, OpenCV 4.1
Stop signs, traffic lights, pedestrians.
- LaneNet lane detection
- Towards End-to-End Lane Detection: an Instance Segmentation Approach by
- High-precision LaneNet DNN by NVIDIA
- Detect road markings and landmarks with high-precision
Forward Collision Warning
[Not started yet] We plan to make use of the TensorFlow/YOLO Object Detection to detect the vehicles in front and further predict the distance by estimating the type of the vehicle (car, bus, truck, etc.) and the size of the detection box.
- Hardware: DJI Tello Drone
- Environment: macOS 10.15, Python 3.7, OpenCV 4.1
My plan was to use some starter codes from Github to interact with the drone and jump into the computer vision tasks. However, the program crashed in my first flight and two propellers broke. So I bought a full propeller guard for better protection and now I'm working on a Python control module to interact with the Tello SDK 2.0.
- Hardware: iPhone Xs
- Enviornment: iOS 13, OpenCV 3.4, Dlib 19
- Collaboration: Tongji University, BMW Group Shanghai
To ensure a safe drive, we detect drivers' status with the frontal camera of an iPhone. Face recognition and status detection are conducted in an iOS app and status codes are sent to BMW iDrive system and head-up display for notifications.
TODO: add demo
Some Future Project Ideas
- 3D Scene Reconstruction with GAN in Minecraft
- Interpretable? Xiangqi (Chinese chess) AI