How to Setup CodeProject.AI Server with Home Assistant Container

In my previous article, I set up Home Asssitant Container and used a shortcut setting up CodeProject.AI Server, by integrating Agent DVR (which I already set up with CodeProject.AI) with Home Assistant. In this article, I will do it properly, and set up Home Assistant Container to work with CodeProject.AI Server and a Wyze Cam v3. Then as a proof of concept, I'll use CodeProject.AI Server to detect a person from Home Assistant.

• Requirements
• Setting Up Home Assistant Container
• Setting Up Home Assistant Community Store
• Setting Up CodeProject.AI-HomeAssist-ObjectDetect
• Detecting a Person from Home Assistant Using CodeProject.AI Server

Requirements

This article assumes that you are using Windows (10), have Docker Desktop installed, CodeProject.AI Server installed, a Wyze Cam v3 set up with the RTSP firmware, and the docker-wyze-bridge container running and working with Home Assistant. So this article assumes you have the following:

• A Wyze Cam v3. If you need help installing a Wyze Cam v3, please see my guide to Setting up your Wyze Cam.
• Docker Desktop. If you need help setting up Docker, please see my guide to installing Installing Docker Desktop.
• CodeProject.AI Server. If you need help installing CodeProject.AI Server, please see my guide to Installing CodeProject.AI Server.
• docker-wyze-bridge. If you need help setting up docker-wyze-bridge, please see my guide to Setting Up docker-wyze-bridge with Docker Compose.

Setting Up Home Assistant Container

The first thing we need to do is create a configuration folder for Home Assistant. We're setting up Home Assistant in Docker, but we still need to access Home Assistant's configuration.yaml file.

The configuration folder will be on whatever drive you run the Docker pull on, which in my case, is the C:\ drive. So open Windows Explorer, go to the C:\ drive, right-click in the folder area, and go to New -> Folder. Input the folder name you want, I'm calling mine "haconfig".

Now go to the Start button for Windows and type "CMD" and open Command Prompt. Then type:

docker run -d --name homeassistant --privileged --restart=unless-stopped 
-e TZ=America/Edmonton -v c:\haconfig:/config -p 8123:8123 homeassistant/home-assistant:latest

where --name homeassistant is the name of your container, TZ=America/Edmonton is the time zone for Home Assistant (just enter your own time zone here), and -v c:\haconfig:/config is the path of the config folder (if you created a folder in C:\Users\Joseph\haconfig, this would be -v C:\Users\Joseph\haconfig:/config).

All right, let's log into Home Assistant. In the browser, put http://localhost:8123. Input your Name (which automatically starts entering that as your Username in lower case), then choose a password. Make sure you write this password down somewhere, you're going to need it a lot. Hit Create Account.

Next input more specific details for Home Assistant. You can give Home Assistant a name (I'm very creative and call mine "Home"), then select a Country, Language, Time Zone, and Currency. Hit Next.

Then is a screen about sharing data. I'm not interested in sharing so I click Next.

This next page is where all your IoT devices would show up if have them. Nothing shows up for me. Click Finish.

Setting Up Home Assistant Community Store

In order to install our Home Assistant Integration for CodeProject.AI Server, we first need Home Assistant Community Store (HACS). For this, we need to run the HACS installer inside our homeassistant container. Go the the Windows Start button and start typing "CMD", then select the Command Prompt. Then, type docker exec -it homeassistant bash.

docker exec -it homeassistant bash

This opens the command line for the homeassistant container. From here, type wget -O - https://install.hacs.xyz | bash -. And yes, that's an "O" as in "Octopus." Not a zero.

wget -O - https://install.hacs.xyz | bash -

The script then kindly reminds us to restart Home Assistant. Go back to the browser tab for Home Assistant. Click on Developer Tools on the left-hand side navigation menu, then under Check and Restart, click Restart and then Restart again.

Now we can go to Settings on the left-hand side navigation menu, then Devices & Services. Then hit + Add Integration.

Type in "HACS" and now HACS appears. Select it.

This window appears that asks you to acknowledge a series of items. Check all the boxes and hit Submit.

A waiting for device activation screen pops up. Make sure you are logged into GitHub, then click the link https://github.com/login/device and copy the activation key.

In the next window, paste in the activation key.

Then click Authorize hacs.

That's it! We should be all set for HACS.

Go back to the Home Assistant tab and you should see a success screen. Select the Area you want to use HACS for. My only camera is in my Office so I added an area called "Office" and hit Finish.

Now we can restart Home Assistant. Click on Developer Tools on the left-hand side navigation menu, then under Check and Restart, click Restart and then Restart again.

Setting Up CodeProject.AI-HomeAssist-ObjectDetect

Now we can finally add CodeProject.AI-HomeAssist-ObjectDetect. Click on the new HACS tab in the left-hand navigation, then click Integrations.

On the top right-hand corner, click on the three dots to bring down the integrations dropdown, and select Custom repositories.

In the Repository field, input the URL for the CodeProject.AI-HomeAssist-ObjectDetect repository, which is https://github.com/codeproject/CodeProject.AI-HomeAssist-ObjectDetect. Then from the Category dropdown, select Integration. Then click Add.

Now under the Integrations tab in HACS, a new repository appears. That's us! Click CodeProject.AI Server object custom integration.

Then, in the bottom-right, click Download to install it.

A dialog appears that kindly reminds, "Remember that you need to restart Home Assistant before changes to integrations (custom_components) are applied." Thanks Home Assistant HACS Integrations! But not just yet. Click Download to finish the download and install of the repository.

Now let's edit the Home Assistant configuration.yaml file. To do this, we'll use Visual Studio Code. I've already installed Visual Studio Code, but to install, you need to go to https://code.visualstudio.com/ and click Download for Windows. This should download a file similar to VSCodeUserSetup-x64-1.74.3.exe. Run the install file to set up Visual Studio Code.

Once Visual Studio Code is installed, the first thing to do is download the YAML extension. Within the Extensions view, type "YAML" then hit the blue Install button when the YAML extension shows up.

Now go to File, Open File, then navigate to the folder that we created earlier. For me, this is C:\haconfig. Select configuration.yaml and hit Open.

# Loads default set of integrations. Do not remove.
default_config:

# Load frontend themes from the themes folder
frontend:
  themes: !include_dir_merge_named themes

# Text to speech
tts:
  - platform: google_translate

automation: !include automations.yaml
script: !include scripts.yaml
scene: !include scenes.yaml

This is the default configuration.yaml for Home Assistant. In order to set up CodeProject.AI Server in Home Assistant, we need to add to this file (which is why we went through all the trouble of getting access to it). Here are the additions we want to add to configuration.yaml.

image_processing:
  - platform: codeproject_ai_object
    ip_address: localhost
    port: 32168
    # custom_model: mask
    # confidence: 80
    save_file_folder: /config/snapshots/
    save_file_format: png
    save_timestamped_file: True
    always_save_latest_file: True
    scale: 0.75
    # roi_x_min: 0.35
    roi_x_max: 0.8
    #roi_y_min: 0.4
    roi_y_max: 0.8
    crop_to_roi: True
    targets:
      - target: person
      - target: vehicle
        confidence: 60
      - target: car
        confidence: 40
    source:
      - entity_id: camera.local_file

A couple of notes about this. First, make sure you go to your config directory C:/haconfig for me, and add a snapshots folder. I'm not so sure this next step will work if you don't. Second, you'll note that ip_address is set to localhost. This might work for you. Personally, I had to change this to my IP address. If you do not know your IP address, go to the Start button in Windows and start to type "CMD" then select Command Prompt. Then, type ipconfig. The address you want is the IPv4 Address. Third, you need to change the entity_id to the entity for your camera. I'll show you how to do that.

Go back to your Home Assistant tab in your browser. Then go to Settings, Devices & Services, and look for your Generic Camera. Click on the entity link. For me, this says 1 entity.

Your camera Entity ID is listed under Entity ID. For me, this is camera. then some variation of my IP address. Copy this value, go back to Visual Studio Code and paste it in to configuration.yaml replacing camera.local_file.

Then go to File in Visual Studio Code and then Save.

Now we can restart Home Assistant. Click on Developer Tools on the left-hand side navigation menu, then under Check and Restart, click Restart and then Restart again.

Detecting a Person from Home Assistant Using CodeProject.AI Server

Now we're ready to detect a person with CodeProject.AI Server. So open CodeProject.AI Server by putting http://localhost:32168/ in your browser. I've previously installed a bunch of modules like Face Processing, and License Plate Reader, but for this all we really need is Object Detection (YOLOv5 6.2), so I click the ... next to all the other services, and select Stop.

Now go back to the Home Assistant tab and go to Developer Tools, then click on the Services tab. In the Service field, type "image" and then select Image processing: Scan. Then click on the Choose entity button and select codeproject_ai_object from the list. Finally, hit Call Service. A little green check mark appears, indicating the service has been called.

Now click on the States tab. If this worked, in the Attributes column, there is targets_found: summary: {} and all_objects: - person: 68.036. The first few times you call this service, the summary is blank, but the more you do it, it says something like targets_found: - person: 83.589 and summary: person: 1. It works!

Go to your configuration folder and look for the snapshots folder you created. For me, this is C:/haconfig/snapshots. In the folder, there should be a snapshot of what you just took, something like codeproject_ai_object_cameraentityname_date.png and codeproject_ai_object_cameraentityname_latest.png. See? That's me! I'm a person!

This is just the beginning. From here, we can start automations based on what we detect in our cameras. That's what Home Assistant is all about and will be the subject of my next articles. But we've taken the first step and got CodeProject.AI Server running in Home Assistant Container and detected a person with it. If you have any questions, please ask them in the article forum below.