So far, I've successfully setup Agent DVR on Windows with CodeProject.AI Server, and used Facial Recognition in Agent DVR. In this article, I show how to use a custom AI model in Agent DVR and CodeProject.AI Server. More specifically, I show to use Agent DVR and CodeProject.AI Server to detect a delivery package, and take a recording when the package is detected.
This article assumes you have a camera already set up and working with Agent DVR and CodeProject.AI Server. I won't retread ground here, so if you need to do this first, please check out my article How to Setup Agent DVR and CodeProject.AI Server on Windows Using a Wyze Cam.
Getting the Custom Model
First we need a custom AI model. For this article, use a custom AI model that detects packages. The great Mike Lud, CodeProject Community Engineer is training many models and has developed a model that detects packages. So the first step is to download the package model, and put it in the custom model folder. Go to Mike Lud's GitHubM, and download package.pt. Then, copy package.pt into the custom model folder for CodeProject.AI Server, which is C:\Program Files\CodeProject\AI\modules\ObjectDetectionYolo\custom-models.
Setting Up a Custom AI Model in Agent DVR
First, open CodeProject.AI Server by putting http://localhost:32168 in your browser. In this article, we're using YOLOv5 6.2 to detect objects, so ensure that Object Detection (YOLOv5 6.2) is active. We don't need any other modules, so I switched all the others off.
Now, launch Agent DVR. Go to the Server Menu, which is in the upper-left hand corner.
This article assumes CodeProject.AI Server is already setup, so hit Settings under Configuration.
Then from the General dropdown, select AI Settings.
Then click the ... button next to Configure.
Then hit the Edit icon to update the AI Server settings.
There are two things to update here. First, the default Endpoint is /v1/vision/detection. Whenever you're doing a custom model, this needs to be modified to /v1/vision/model-name. In our case, we're using package.pt, so this needs to be /v1/vision/package.
The next thing to update is the Object Classes. Whenever adding a custom model, this list of objects needs to be updated to include the item you're deteching. In this case, "package". The default list is:
person,bicycle,car,motorcycle,airplane,bus,train,truck,boat,traffic light,
fire hydrant,stop sign,parking meter,bench,bird,cat,dog,horse,sheep,cow,elephant,
bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,skis,
snowboard,sports ball,kite,baseball bat,baseball glove,skateboard,surfboard,
tennis racket,bottle,wine glass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,orange,
broccoli,carrot,hot dog,pizza,donot,cake,chair,couch,potted plant,bed,
dining table,toilet,tv,laptop,mouse,remote,keyboard,cell phone,microwave,oven,toaster,sink,
refrigerator,book,clock,vase,scissors,teddy bear,hair dryer,toothbrush
This needs to be updated to:
person,bicycle,car,motorcycle,airplane,bus,train,truck,
boat,traffic light,fire hydrant,stop sign,parking meter,bench,bird,cat,dog,horse,sheep,
cow,elephant,bear,zebra,giraffe,backpack,umbrella,handbag,tie,suitcase,frisbee,
skis,snowboard,sports ball,kite,baseball bat,baseball glove,skateboard,surfboard,
tennis racket,bottle,wine glass,cup,fork,knife,spoon,bowl,banana,apple,sandwich,
orange,broccoli,carrot,hot dog,pizza,donot,cake,chair,couch,potted plant,bed,
dining table,toilet,tv,laptop,mouse,remote,keyboard,cell phone,microwave,oven,toaster,sink,
refrigerator,book,clock,vase,scissors,teddy bear,hair dryer,toothbrush,package
Hit OK on the next four prompts to accept the AI Server settings.
Now go to the camera settings by clicking the edit icon in the bottom left corner while on the Live Feed.
Then from the General dropdown, select Object Recognition.
Click the ... button in front of Configure.
Now that "package" has been added to the Object Classes list, there is a button for it. Typically packages are delivered by people so it's a good idea to set up the object detection for two items, both person, and package. Either type in "person,package" or simply select the person and package buttons from the list. Hit OK on this menu and the next to accept these settings.
While trying to get custom models working, I ran into a lot of problems. My mistake was that I forgot to set up an Endpoint in the AI Server settings, but along the way I implemented everything I could out of desperation to get Agent DVR to detect a package. One of the things I tried was implementing Mask. So go to camera settings, and from the General dropdown, select Alert Filter. Then, hit ... next to Configure.
Here you can add a mask. This isn't necessary for my current setup, but if you had a static camera that was pointed at your door and you only wanted to trigger when someone came up to the steps, rather than the street, this could be helpful. Left-clicking the mouse and dragging it over the camera image will specify which areas to ignore. Right-clicking the mouse will allow you to erase your mask selection.
Whether you add a mask or not, in the Find field, make sure "person,package" is listed. Hit OK twice to accept these settings.
Now we need to change the Detector settings. Go to the camera settings, and from the dropdown, select Detector.
Change the Detector type to "Simple". The Simple detector means that the AI detection will be trigged on motion. I want it to be real easy to detect the package, so I put this setting to Simple and hit OK to accept the settings.
Finally, we want someone leaving a package in the camera field to trigger an event. In another article I wrote, someone asked for how to trigger a recording on detection, so I'll demonstrate that here. Go to the camera settings, and select Actions. Then click Add. From the If dropdown, select Object Found, and on the Then dropdown, select Start Recording On. The camera should automatically be selected after this, but ensure the camera (in my case, Eye Spy) is in the Device section, then hit OK. Hit OK again to accept those actions.
Detecting a Package
All right! Let's put a package into the camera field of this bad boy and check it out! For illustration purposes, my Action will trigger Object Detection so I can demonstrate that a package was detected.
KA-BLAM! I finally did it! Package detected.
Well that's it! This process should work for other custom models too. We have now shown how setup Agent DVR to use at custom AI model, and trigger a recording when a package was detected.
If you have any questions, please feel free to ask them in the article forum below.
