Custom Video Overlay for Image Processing on PYNQ-Z2

Hi, I’m trying to build a custom overlay for doing image processing on an image captured by my webcam and then display the result via hdmi. I’m using PYNQ-Z2 with 2.7 image but the Vivado I have is 2018.2 so I have to use the 2.4 PYNQ files for this.

Here is the overlay:

Here is the overlay I was trying to follow (except using the colour converter HLS instead of Sobel): pynq_cv/hls_sobel_bd.png at master · 21stars/pynq_cv · GitHub

I’m getting errors when I try to build a hdl wrapper:
ERROR: [ip:xlconcat:2.1-10] /xlconcat_0Xlconcat input pins are connected to different type of pins
ERROR: [ip:xlconcat:2.1-10] /xlconcat_0Xlconcat input pins are connected to different type of pins
CRITICAL WARNING: [BD 41-237] Bus Interface property TDATA_NUM_BYTES does not match between /color_convert_0/stream_in_24(3) and /axi_vdma_0/M_AXIS_MM2S(4)
ERROR: [BD 41-237] Bus Interface property FREQ_HZ does not match between /color_convert_0/s_axi_AXILiteS(100000000) and /axi_interconnect_0/xbar/M03_AXI(142857132)
ERROR: [BD 41-237] Bus Interface property CLK_DOMAIN does not match between /color_convert_0/s_axi_AXILiteS(base_ps7_0_0_FCLK_CLK0) and /axi_interconnect_0/xbar/M03_AXI(base_ps7_0_0_FCLK_CLK1)

For the concat errors:

  • I understand that it’s connected to different types of input pins but I can’t see how this is different to the pinning in the github project overlay I linked to above.

For the Color Convert errors:

  • I’m not sure what to set the number of bytes as. How should I decide this?
  • How do I fix the clock domain error? How can I see what clock domain each is using?

For the Bus Interface AXI error:

  • Is this a knock-on effect from other errors with the M03_AXI or do I manually have to alter the frequency? I can’t see where to edit it.

Would it be easier to take the full PYNQ overlay and just add the HLS IP to that?

Thanks in advance for any help!

1 Like

Hi @walsha61,

Yes. You can start from the base overlay and prune anything you do not need.

Mario

Thanks for the reply @marioruiz. Have you come across any project examples that I could follow? I’m not fully sure how to properly integrate the HLS IP into the base overlay. Would I need to add in a second DMA? Thanks.

1 Like

The color conver IP is already part of the base overlay. It is inside the video hierarchy.

If the data movement is only through stream you can integrate your IP easily on the base overlay video datapath.

You can also check this topic:

I do not know any tutorial that provides step by step integration of a custom IP in the video pipeline.

You can also experiment with the Composable Video Pipeline

Mario

Thanks again for your help @marioruiz. I expanded the hdmi_in and hdmi_out in the PYNQ base overlay and saw the color convert IP like you said. I’m now trying to integrate a different image processing IP into the base overlay video datapath.

I found this example which integrates an edge detect IP:
https://web-pcm.cnfm.fr/wp-content/uploads/2017/04/Workbook-Digilent_ZYBO_Video_Workshop.pdf (Page 39).

The PYNQ base video datapath is a lot more complicated than this but I’m going to see if I can use this to help me integrate the IP!

2 Likes