Body Pose Detection (Python)

This application runs on Atlas 200 DK, to infer human body poses. The model used is modified from Lightweight OpenPose [Osokin, Daniil. "Real-time 2d multi-person pose estimation on CPU: Lightweight OpenPose." arXiv preprint arXiv:1811.12004 (2018)], an open-source pose detection network. This application can be run on various input formats, namely image input, video input as well as live camera input.

The pose detection network originates from [Cao, Zhe, et al. "OpenPose: realtime multi-person 2D pose estimation using Part Affinity Fields." arXiv preprint arXiv:1812.08008 (2018)], where the network provides two outputs: heatmaps and PAF maps for each frame. In the post-processing of the model, both heatmaps and PAF maps are used to obtain multi-persion pose estimation.

In this repository, the post-processing a simplied version for edge computing, where only the heatmap is used to calculate the predicted locations of the human body joints. A set of 14 detected joints are grouped, connected to form limbs and associated to a person. The diagram below represents connection scheme:

                 12                     12: head, 13: neck
                 |
                 |
           0-----13-----3
          /     / \      \
         1     /   \      4
        /     /     \      \
       2     6       9      5
             |       |
             7       10
             |       |
             8       11

The figure below shows the sample output on a single image input. The detected pose is displayed in the form of a skeleton overlay on the image.

Figure OpenPose detection result

Software Preparation

Clone or download the project repository:

mkdir -p $HOME/AscendProjects cd $HOME/AscendProjects
git clone ssh://git@rnd-gitlab-ca-g.huawei.com:2222/hispark/openpose-pythonc73.git OR wget https://rnd-gitlab-ca-g.huawei.com/hispark/openpose-pythonc73.git

Then, unzip the downloaded file:

unzip openpose-pythonC73.zip

Environment Preparation

Install required libraries for Python3 environment (OpenCV, PresentAgent and Python3env)

Please refer to https://github.com/Huawei-Ascend/samples/tree/master/common

Environment Deployment

1. Go to the root directory where the application code is located, such as: $HOME/AscendProjects/openpose-pythonC73/.

   cd $HOME/AscendProjects/openpose-pythonC73/

2. Modify the configuration file.

   Modify presenter_server_ip and presenter_view_ip in script/body_pose.conf to the current ubuntu server and atlas200dk development board network port ip, presenter _agent_ip is the ip of the network port connected to the ubuntu server on the development board.

   If you use USB connection, the USB network port ip of the development board is 192.168.1.2, and the network port ip of the virtual network card connected to the ubuntu server and the development board is 192.168.1.223, then the configuration file content is as follows:

   presenter_server_ip=192.168.1.223

   presenter_view_ip=192.168.1.223

   presenter_agent_ip=192.168.1.2

   Generally, when connecting via USB, atlas200dk_board_ip is the USB network port ip of the development board, and the default is 192.168.1.2. When connecting through a network port, atlas200dk_board_ip is the network port ip of the development board, and the default is 192.168.0.2.

3. Copy the application code to the development board.

   Navigate to the root directory where the openpose-pythonC73 application code is located, such as: AscendProjects/openpose-pythonC73, execute the following command to copy the application code to the development board. If the copy fails, please check if there is a directory HIAI_PROJECTS on the development board, and if not, create it.

   scp -r ~/AscendProjects/openpose-pythonC73 HwHiAiUser@192.168.1.2:/home/HwHiAiUser/HIAI_PROJECTS

   Enter the development board password when prompted for password. The default password of the development board is Mind@123, as shown below:

   

4. Start Presenter Server.

   NOTE: This step is required when the presenter server needs to be used for video or live display on the browser. Otherwise, skip this step.

   Execute the following command to start the Presenter Server in the background.

   bash $HOME/AscendProjects/openpose-pythonC73/script/run_presenter_server.sh &

   Log in to the Presenter Server using the prompted URL. The figure below shows that the Presenter Server has started successfully.

   Figure Home Page Display
   

   The communication between Presenter Server, Mind Studio and Atlas 200 DK is shown as below：

   Figure Examples of IP addresses
   

   NOTE:

   • The IP address used by the Atlas 200 DK developer board is 192.168.1.2 (USB connection).
   • The IP address for the communication between Presenter Server and Atlas 200 DK is the IP address of the UI Host server on the same network segment as Atlas 200 DK, for example: 192.168.1.223.
   • This example of the IP address for accessing Presenter Server through a browser is 10.10.0.1. Since Presenter Server and Mind Studio are deployed on the same server, this IP address is also the IP for accessing Mind Studio through a browser.

5. Copy acl.so to the development board. Please skip this step if it is already done in other projects.

   scp ~/Ascend/ascend-toolkit/20.0.RC1/arm64-linux_gcc7.3.0/pyACL/python/site-packages/acl/acl.so HwHiAiUser@192.168.1.2:/home/HwHiAiUser/Ascend/

   Please replace X.X.X with the actual version number of the Ascend-Toolkit development kit package
   For example: the package name of the Toolkit package is Ascend-Toolkit-20.0.RC1-x86_64-linux_gcc7.3.0.run, then the version number of this Toolkit is 20.0.RC1.

6. Log in to the development board and add environment variables. Please skip this step if it is already done in other projects.

   ssh HwHiAiUser@192.168.1.2
   vim ~/.bashrc
   Add two lines at the end
   export LD_LIBRARY_PATH=/home/HwHiAiUser/Ascend/acllib/lib64
   export PYTHONPATH=/home/HwHiAiUser/Ascend/:\${PYTHONPATH}
   

   Execute the following command to make the environment variable take effect source ~/.bashrc

Running the Application

1. Check whether the "CAMERA0" camera is used. The "CAMERA0" camera is used by default in the code. Please refer to the link below for the viewing method.
   https://support.huaweicloud.com/usermanual-A200dk_3000/atlas200dk_02_0051.html

2. Log in to the development board. Navigate to the code directory corresponding to the required input source format (image, video or live camera), and execute one of the following commands to run the application according to the input source format.

   NOTE: To execute it, you could use the default input (just run python3 main.py without parameter), or input parameters as below for input/output paths and model path as indicated in main.py. For the video input, you also have the option to save the output as a video or display to presenter server, with the value in parameter 'is_presenter_server' set to False or True, respectively .

   Image input source:

   cd ~/HIAI_PROJECTS/openpose-pythonC73/code_image
   python3 main.py --model='model/body_pose.om' --frames_input_src='tennis_player.jpg' --output_dir='outputs'

   Video input source (output video):

   cd ~/HIAI_PROJECTS/openpose-pythonC73/code_video
   python3 main.py --model='model/body_pose.om' --frames_input_src='yoga.mp4' --output_dir='outputs' --is_presenter_server=False

   Video input source (output streamed to presenter server)*:

   cd ~/HIAI_PROJECTS/openpose-pythonC73/code_video
   python3 main.py --model='model/body_pose.om' --frames_input_src='yoga.mp4' --output_dir='outputs' --is_presenter_server=True

   Live camera source:

   cd ~/HIAI_PROJECTS/openpose-pythonC73/code_live
   python3 main.py --model='model/body_pose.om'

3. If you need to view the detection results using presenter server for the live input or video source, log in to the Presenter Server website using the URL that was prompted when the Presenter Server service was started. Otherwise, skip this step.

Wait for the Presenter Agent to transmit data to the server, and click "Refresh" to refresh. When there is data, the status of the corresponding Channel turns green, as shown in the figure below.

Figure Presenter Server

Click the corresponding View Name link on the right, such as "video" in the above picture, to view the results.

Stopping Application

If the presenter server is being used for display, stop

• Stop Presenter Server

  The Presenter Server service will always be running after it is started. If you want to stop the Presenter Server service corresponding to the pose detection application, you can perform the following operations.

  Execute the following command on the command line on the server where the process of the Presenter Server service is running:

  ps -ef | grep presenter

  ascend@ubuntu:~/AscendProjects/openpose-pythonC73/script$ ps -ef | grep presenter
  ascend 9560 1342 0 02:19 pts/4  00:00:04   python3/home/ascend/AscendProjects/openpose-pythonC73.bak/script/..//present
  erserver/presenter_server.py --app openpose-pythonC73
  

  As shown above, 9650 is the process ID of the Presenter Server service corresponding to the openpose-pythonC73 application.

  If you want to stop this service, execute the following command:

  kill -9 9650