Eyes in the Back of My Head: An AI Alternative to Looking Over Your Shoulder
On November 2nd, we’ll be headed back to the office. After almost two years of the safety of working from home, I realized that soon I’m going to face a challenge I’d almost forgotten: People walking behind me as sit at my desk. How am I going to get away with watching Netflix at work now?
But that’s all good! Because I think I have a solution! I’m going to build a tool that gives me eyes in the back of my head. Aka a live detection tool that alerts me when a person is approaching me from behind. That way I’ll never be caught off guard again. And thanks to machine learning this should be a snap!
In terms of the tech stack, I’m going to keep things fairly simple for this little experiment. The front end will be built with Vue and then on the back end, I’ll have TensorflowJS doing all of the image processing.
The Front End
Because this app is hardly going to be seen, I’m going to keep everything super minimal. At its core, it’s just a camera stream, but to make it a little more fun, I’m going to set that video inside an eye. So it really feels alive!
I then went ahead and slapped a fancy logo at the top and had the first half of the project done.
The Back End
This is where things actually get interesting and where we get to start playing with machine learning. There are a few basic components to the back end: the camera feed, loading the object detection model, and alerting the user to an incoming person.
Setup the Webcam
Getting going with object detection is a pretty simple process, but the most important thing we will need is the feed. To do that I’ll create a video element that mounts the webcam when the page loads.
Loading the CNN Model
Of course, I could spend the time collecting images of people, segmenting and labeling each, and finally training the model but that takes a long time. Not only that, but why would I when there is a perfectly good model bundled with Tensorflow JS. Coco-ssd is a pre-trained model that out of the box can recognize 80 different objects. One of those being people!
Loading the model is really simple and only takes a couple lines of code.
Send Signal
With the camera and model loaded, we’re ready to move on to actually interpreting the model output. Everything is stored in a pretty simple object. Within it you get the item recognized, confidence level, and then the position of the object. Knowing that, I threw together this little function that looks for two people in frame. Whenever two people are detected it will notify me!
To actually send the notification, I ended up just using an NPM package that I found online. It was called Simple Web Notifications and for this demo it worked great!
The Final
With just a couple of hours of coding, I’ve managed to give myself eyes in the back of my head. I’ll never have to worry about someone catching me while I’m streaming the latest season of You. All of this meaning, that I’m officially ready to head back to the office!
Machine learning can sometimes seem like a black hole but the tools out there have made it really easy to pick up. Even for a novice coder like myself, I can throw together a working app super quickly. And while this demo is just basic object detection, there are a ton of other really cool features built into Tensorflow JS that you can check out here. Want to give yourself eyes in the back of your head? Check out the live demo here: https://mleyes.netlify.app/
And full code can be viewed here: https://github.com/iamkunkel/ml-eyes
