Custom Video Overlay for Image Processing on PYNQ-Z2

[walsha61]

Hi, I’m trying to build a custom overlay for doing image processing on an image captured by my webcam and then display the result via hdmi. I’m using PYNQ-Z2 with 2.7 image but the Vivado I have is 2018.2 so I have to use the 2.4 PYNQ files for this.

Here is the overlay:

Here is the overlay I was trying to follow (except using the colour converter HLS instead of Sobel): pynq_cv/hls_sobel_bd.png at master · 21stars/pynq_cv · GitHub

I’m getting errors when I try to build a hdl wrapper:
ERROR: [ip:xlconcat:2.1-10] /xlconcat_0Xlconcat input pins are connected to different type of pins
ERROR: [ip:xlconcat:2.1-10] /xlconcat_0Xlconcat input pins are connected to different type of pins
CRITICAL WARNING: [BD 41-237] Bus Interface property TDATA_NUM_BYTES does not match between /color_convert_0/stream_in_24(3) and /axi_vdma_0/M_AXIS_MM2S(4)
ERROR: [BD 41-237] Bus Interface property FREQ_HZ does not match between /color_convert_0/s_axi_AXILiteS(100000000) and /axi_interconnect_0/xbar/M03_AXI(142857132)
ERROR: [BD 41-237] Bus Interface property CLK_DOMAIN does not match between /color_convert_0/s_axi_AXILiteS(base_ps7_0_0_FCLK_CLK0) and /axi_interconnect_0/xbar/M03_AXI(base_ps7_0_0_FCLK_CLK1)

For the concat errors:

• I understand that it’s connected to different types of input pins but I can’t see how this is different to the pinning in the github project overlay I linked to above.

For the Color Convert errors:

• I’m not sure what to set the number of bytes as. How should I decide this?
• How do I fix the clock domain error? How can I see what clock domain each is using?

For the Bus Interface AXI error:

• Is this a knock-on effect from other errors with the M03_AXI or do I manually have to alter the frequency? I can’t see where to edit it.

Would it be easier to take the full PYNQ overlay and just add the HLS IP to that?

Thanks in advance for any help!

[marioruiz]

Hi @walsha61,

@walsha61

Would it be easier to take the full PYNQ overlay and just add the HLS IP to that?

Yes. You can start from the base overlay and prune anything you do not need.

Mario

[walsha61]

Thanks for the reply @marioruiz. Have you come across any project examples that I could follow? I’m not fully sure how to properly integrate the HLS IP into the base overlay. Would I need to add in a second DMA? Thanks.

[marioruiz]

The color conver IP is already part of the base overlay. It is inside the video hierarchy.

If the data movement is only through stream you can integrate your IP easily on the base overlay video datapath.

You can also check this topic:

Vitis Vision Library on PYNQ

Hello everyone! I am new to PYNQ framework and I have acquired a PYNQ-Z2 board. I am interested in using PYNQ for accelerating image processing pipelines in Python, using FPGA capabilities. What is the easiest way to use OpenCV accelerated functions from Xilinx. I have seen Vitis Vision Library, but I could not create IPs using HLS to use in my design and create my overlays. I have experimented the helloworld project that uses the “resize” IP generated from that library, and I would like to re…

I do not know any tutorial that provides step by step integration of a custom IP in the video pipeline.

You can also experiment with the Composable Video Pipeline

The Composable Video Pipeline

Hello PYNQ aficionados, Edit 10 March 2022: comments closed to request support open a new post in the support category. I am pleased to announce that the PYNQ team has just released the composable (video) pipeline, this is an open source project and you can find it in GitHub PYNQ_Composable_Pipeline Note that through the blog, I will use the term “composable overlay” and “composable pipeline” interchangeably. What is a composable overlay? A composable overlay provides run time configuration…

Mario

[walsha61]

Thanks again for your help @marioruiz. I expanded the hdmi_in and hdmi_out in the PYNQ base overlay and saw the color convert IP like you said. I’m now trying to integrate a different image processing IP into the base overlay video datapath.

I found this example which integrates an edge detect IP:
https://web-pcm.cnfm.fr/wp-content/uploads/2017/04/Workbook-Digilent_ZYBO_Video_Workshop.pdf (Page 39).

The PYNQ base video datapath is a lot more complicated than this but I’m going to see if I can use this to help me integrate the IP!