Real-time video processing on PYNQ-Z2

Support
1

Hi, I’m trying real-time video processing project on PYNQ-Z2.
There was no problem outputting the webcam input to the HDMI monitor as it is, I was also able to get a good frame.(without any processing, just print out camera input)

Drowsiness1_pr.ipynb (6.4 KB)
drowsiness Detection Code

And then I tried to run the drowsiness detection code attached above.
In the PC environment, good results were obtained, but It didn’t run properly in PYNQ.
Perhaps It’s because I had a lack of understanding of PYNQ and FPGA, so I didn’t use it well.

So my question is as follows.

  1. Can the real-time image processing function run on PYNQ as much as the performance that runs in the PC environment? (functions such as the attached code)
  2. If the answer of question 1 is yes, is it possible only by just using base overlay or does it require more ability? (For example, need a new custom overlay, knowledge of H/W and FPGA)

Since I’m a fpga beginner, the expert’s detailed or brief advice is very helpful to me.
Thank you for reading my concerns.

2

Hi @everglow,

but It didn’t run properly in PYNQ.

What do you mean by this?

The code you are providing will only run in the ARM A9 processor that is an embedded processor and not as performant as the x86 machine you may have.

There is no FPGA acceleration in the code you provide. I do not think you will be able to use the base overlay to accelerate this on the FPGA, you will need to identify the code you want to accelerate and then design the overlay for the FPGA. There are a number of accelerated vision libraries in the Vitis Libraries

Mario

3

Thank you so much, mario.

The intention was to do an image processing project with fpga acceleration, but it was all about following the flow of examples.
Now I realize it’s silly to want as much performance as a pc environment without advanced utilization like acceleration, custom overlay, etc.
It seems like learning fpga and HW first through other basic exercises.

Thank you! I think it will help me continue to study.

1 Like