BuddyWatch - 2024

Key Takeaways

  • Labeling and augmenting training data
  • Creating custom deep learning models with Keras Functional API
  • Creating REST APIs with Django and Django REST Framework
  • Integrating Azure Blob Storage to the Django application
  • Using NextAuth for handling token based authentication
  • Benefits of TypeScript

BuddyWatch is a deep learning enhanced surveillance system application for detecting humans from camera feed. The user interface is built with Next.js and Tailwind. It uses the webcam of the user's device to capture images and videos, and provides functionality to send those to the server in order to be stored or processed by an object detection model.

The server is built using Django framework and Python 3.10. The server uses PostgreSQL database to store information like user data and video information such as title, creation time, and owner. Videos themself are stored in Azure Blob Storage. When the server is started, it loads pre-trained Tensorflow object detection model to memory.

The object detection model has been developed using Keras Functional API. VGG16 model was used as a base, and a new classification layer for detecting faces was built on top of it. The model was trained using a set of webcam images.

Overall, I am very happy with the outcome of this project. It was a great learning experience and I got to work with a lot of new technologies and tools. This was my first time developing with Django and overall the framework really impressed with its development speed. The Django's 'batteries included' approach was very helpful in getting the server up and running quickly. Getting same functionality in many other similar frameworks would have needed many more lines of code. Also to give credits where credits are due, Django's documentation is top notch. Propably the best I have ever seen. All the necessary information was readily available and in easy to understand format.

I decided to use Axure Blob Storage for storing the uploaded video files which may have been an overkill because just a simple media folder would more likely than not have been enough for the scope of this project. However, getting my hands dirty with cloud services such as AWS and Azure has been a long lasting goal of mine, so integrating Azure Blob Storage to the Django application was definitely worth it in the end and will surely help me in the future, seeing how important cloud providers are in today's IT market.

Next.js was chosen for the user interface because of its excellent development experience with features such as automatic routing and hot code reloading. This was also the first full project where I used TypeScript and I greatly prefer it to compared to JavaScript. It makes the code more readable and it is much easier to avoid bugs compared to JavaScript. Tailwind was used for the styling because it's very customizable and makes the styling more enjoyable tahn with just CSS. Tailwind can also be more performant and have smaller bundle size than some other UI libraries like Material UI. BuddyWatch already does a lot of slow and heavy processes, so rendering should be kept as fast as possible without sacrifizing the developer experience. Overall Next.js was a great choice for the user interface and I will definitely use it again in the future. Most hardships I had to endure were due to the NextAuth library, which I used for token-based authentication. The functionality of the library was a bit hard to understand at first and the documentation was lacking in some areas, but after a while I got the hang of it. It definitely increased my understanding of how tokens work on the client side, so in the end it was a win.