Detect-Object-From-ESP32CAM

Table of contents

Sections	Descriptions	Status
Documentations	Build idea, input, output, tasks and plan for project
Object Detection	Build model using PyTorch and deploy it onto Web-server using FastAPI
Web-server	Building a web-server to embedded model AI, monitor Camera and control system
Embedded Artificial Intelligence	Building a system has hardward and AI, in detail that system can be a robot or similar
Reference papers	Research for project

Documentations

Project Documentation

Object Detection

Datasets

🔗 Dataset: https://www.kaggle.com/datasets/sriramr/apples-bananas-oranges/data?select=original_data_set

💁 We only use apples and oranges data, so before train model, we need to pre-process dataset:

• First: We need re-oraginal data to apples directory and oranges directory.
• Second: Split data to train and test data. Actually, I want to create two folder (train and test).

🔗 Pre-processing datasest:

Training models

💁 We have pre-processed data above, now we can train some models with the data.

• In this project, I'm going to use PyTorch to train models.
• I will experiment some models such as ResNet, Efficient, MobileNet...
• See which model is compatible with our project based on criteria such as its accuracy and size.

🔗 Code:

✈️ Such as I trained model with ResNet50 module, well, I have some experiment to get a good result like below:

• 📉 This chart is descript for train and test loss/accuracy of model

  

• 🍏🍊 We can make some prediction to see what's going on

  

Evaluating models

Name model	Accuracy Testing (%)	Accuracy Predict (%)	Time Predict (s)	Size (MB)
ResNet50	98.660714	97.001250	0.203992	89
ResNet18	97.321429	99.881893	0.178116	42
MobileNetV2	98.660714	99.923056	0.170975	8

• ⏲️ Shortest predict time: MobileNetV2 with 0.17 seconds

• 📁 Smallest size file: MobileNetV2 with 8 MB

• 📉 Accuracy: The above three models have similar accuracy rates, but MobileNetV2 is trained with the fewest training iterations (5), while ResNet50 and ResNet18 are trained with 8 iterations.

Optimizing model

Conclusion

Web-server

• GET API from ESP32 ESP32 GET JSON from API - ESP32 GET request from API receive JSON
• POST API from ESP32 ESP32 POST JSON to api endpoint - ESP32 POST request to API endpoint
• PUT (update) API from ESP32 ESP32 PUT JSON to api endpoint - ESP32 POST request to API endpoint

Embedded Artificial Intelligence

Reference Papers

Paper	Link	Quote
Object Detection using ESP 32 CAM	https://papers.ssrn.com/sol3/papers.cfm?abstract_id=4152378	Mehendale, Ninad. "Object Detection using ESP 32 CAM." Available at SSRN 4152378 (2022).
Real-Time Reinforcement Learning for Vision-Based Robotics Utilizing Local and Remote Computers	https://arxiv.org/pdf/2210.02317	Wang, Yan, Gautham Vasan, and A. Rupam Mahmood. "Real-time reinforcement learning for vision-based robotics utilizing local and remote computers." 2023 IEEE International Conference on Robotics and Automation (ICRA). IEEE, 2023.
Flow-guided Semi-supervised Video Object Segmentation	https://arxiv.org/pdf/2301.10492	Zhang, Yushan, et al. "Flow-guided semi-supervised video object segmentation." arXiv preprint arXiv:2301.10492 (2023).