# Pynq Z2 HLS opencv colorconversion **URL:** \ **Category:** Support\ **Created:** [April 30, 2020, 10:30pm UTC](https://discuss.pynq.io/t/pynq-z2-hls-opencv-colorconversion/1219 "2020-04-30T22:30:24Z")\ **Posts on this page:** 10\ **Page:** 1
**Author:** ![momo\_flyer](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.pynq.io/momo_flyer/32/874_2.png) [@momo\_flyer](https://discuss.pynq.io/u/momo_flyer)\ **Post date:** [April 30, 2020, 10:30pm UTC](https://discuss.pynq.io/t/pynq-z2-hls-opencv-colorconversion/1219/1 "2020-04-30T22:30:24Z")
Hi, I’d like to send an RGB image from the PS to the PL, do some opencv processing (colorconversion to grayscale for now) , then transfer it back to PS and show it. however the code gets stuck at dma.recvchannel.wait(). earlier i made a readback using only the DMA and the transfer is done well, but when i add my HLS core the image doesn’t get transferred back. what i can see is that the IP is idle on start but AP\_ready and AP\_done are low. it exits it idle state when it received the start signal (0x01 or 0x81 on addr 0x00). It doesn’t clear the start signal and it doesn’t go back to its idle and ready state. I can also see something does happens when starting the IP: if i don’t start the IP, it stays stuck at dma.sendchannel.wait() Vivado +HLS 2019.2 pynq 2.5 python code: from pynq import Overlay import numpy as np import pynq.lib.dma import time from pynq import Xlnk from PIL import Image overlay = Overlay(‘/home/xilinx/pynq/overlays/made\_by\_benjamin/opencvtest/opencvtest.bit’) dma = overlay.axi\_dma\_0 #dma\_recv = overlay.axi\_dma\_from\_pl\_to\_ps opencvtest= overlay.cvtcolour\_0 xres\_address=0x10 yres\_address=0x18 image\_path = ‘/home/xilinx/pynq/overlays/made\_by\_benjamin/opencvtest/aurora\_background.jpg’ original\_image = Image.open(image\_path) original\_image.load() input\_array = np.array(original\_image) xres, yres = original\_image.size print(xres) print(yres) xlnk = Xlnk() in\_buffer = xlnk.cma\_array(shape=(yres, xres,3), dtype=np.uint8, cacheable=1) out\_buffer = xlnk.cma\_array(shape=(yres, xres,3), dtype=np.uint8, cacheable=1) in\_buffer[:]=input\_array buf\_image=Image.fromarray(in\_buffer) display(buf\_image) print(opencvtest.read(0x00)) opencvtest.write(xres\_address,xres) opencvtest.write(yres\_address,yres) print(opencvtest.read(xres\_address)) print(opencvtest.read(yres\_address)) def run\_kernel(): dma.sendchannel.transfer(in\_buffer) dma.recvchannel.transfer(out\_buffer) print(‘run kernel’) opencvtest.write(0x00,0x01) dma.sendchannel.wait() print(opencvtest.read(0x00)) dma.recvchannel.wait() run\_kernel() buf\_image2=Image.fromarray(out\_buffer) display(buf\_image2) * * * cvtcolour.cpp: #include “cvtcolour.h” void cvtcolour(int xres, int yres, axi\_stream& img\_in, axi\_stream& img\_out){ #pragma HLS INTERFACE axis port=img\_in #pragma HLS INTERFACE axis port=img\_out #pragma HLS interface s\_axilite port=return #pragma HLS DATAFLOW #pragma HLS INTERFACE s\_axilite port=yres #pragma HLS INTERFACE s\_axilite port=xres RGB\_image imginmat(yres,xres); RGB\_image imggray(yres,xres); RGB\_image imgoutmat(yres,xres); ``` hls::AXIvideo2Mat(img_in, imginmat); hls::CvtColor(imginmat, imggray); hls::CvtColor(imggray, imgoutmat); hls::Mat2AXIvideo(imgoutmat,img_out); ``` ## } cvtcolour.h: #include “hls\_video.h” #include “ap\_fixed.h” #include “ap\_int.h” #include “hls\_stream.h” #include “stdint.h” #define max\_width 1920 #define max\_height 1200 typedef ap\_axiu\<32,1,1,1\> axi\_pixel; typedef hls::stream\ axi\_stream; typedef hls::Mat\ RGB\_image; typedef hls::Mat\ gray\_image; ## void cvtcolour(int xres, int yres, axi\_stream& img\_in, axi\_stream& img\_out); blockdesign: ![image](https://us1.discourse-cdn.com/flex019/uploads/pynq1/original/1X/69a125f2cb6559568f6e614607066ea6ea6f6d29.png) ---
**Author:** ![rock](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.pynq.io/rock/32/32_2.png) [@rock](https://discuss.pynq.io/u/rock)\ **Post date:** [May 2, 2020, 9:05pm UTC](https://discuss.pynq.io/t/pynq-z2-hls-opencv-colorconversion/1219/2 "2020-05-02T21:05:19Z")
> [@momo\_flyer](#): > > typedef hls::Mat\ RGB\_image; > typedef hls::Mat\ gray\_image; Is the dimension correct? ---
**Author:** ![momo\_flyer](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.pynq.io/momo_flyer/32/874_2.png) [@momo\_flyer](https://discuss.pynq.io/u/momo_flyer)\ **Post date:** [May 2, 2020, 10:20pm UTC](https://discuss.pynq.io/t/pynq-z2-hls-opencv-colorconversion/1219/3 "2020-05-02T22:20:32Z")
that seems to be an error in the code indeed, but the gray\_image type isn’t used in de .cpp file. i have also tried using images significantly smaller than the maximum resolutions so it shouldn’t matter or can i only use images that are the maximum defined resolution? ---
**Author:** ![rock](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.pynq.io/rock/32/32_2.png) [@rock](https://discuss.pynq.io/u/rock)\ **Post date:** [May 3, 2020, 4:05am UTC](https://discuss.pynq.io/t/pynq-z2-hls-opencv-colorconversion/1219/4 "2020-05-03T04:05:08Z")
Based on my experience, you will have to provide images with consistent resolution. Otherwise the DMA may wait for more data to be transferred. ---
**Author:** ![momo\_flyer](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.pynq.io/momo_flyer/32/874_2.png) [@momo\_flyer](https://discuss.pynq.io/u/momo_flyer)\ **Post date:** [May 3, 2020, 3:04pm UTC](https://discuss.pynq.io/t/pynq-z2-hls-opencv-colorconversion/1219/5 "2020-05-03T15:04:49Z")
The DMA is not the problem i think: a simple readback through a fifo executes as it should be. in the HLS IP the matrix dimensions are set by xres and yres. and i coded the IP according to this tutorial: [Leveraging OpenCV and High Level Synthesis with Vivado (v2013.1) - YouTube](https://www.youtube.com/watch?v=Y2iHh-HtXn4) Meanwhile i tried with fixed resolution and it still fails ---
**Author:** ![rock](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.pynq.io/rock/32/32_2.png) [@rock](https://discuss.pynq.io/u/rock)\ **Post date:** [May 4, 2020, 7:26pm UTC](https://discuss.pynq.io/t/pynq-z2-hls-opencv-colorconversion/1219/6 "2020-05-04T19:26:39Z")
If the HLS core is expecting more data, you cannot provide a smaller amount of data. You tested DMA without connecting the HLS IP, that is no problem; but after you connect HLS IP, the HLS IP has some requirements on how many data you provide. ---
**Author:** ![momo\_flyer](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.pynq.io/momo_flyer/32/874_2.png) [@momo\_flyer](https://discuss.pynq.io/u/momo_flyer)\ **Post date:** [May 5, 2020, 11:42am UTC](https://discuss.pynq.io/t/pynq-z2-hls-opencv-colorconversion/1219/7 "2020-05-05T11:42:09Z")
Is it possible that the data format is incorrect? for example that the input stream streams data in 3x8b format and that the IP core replies in 1x32b format. that way it would be possible that the DMA awaits the buffer to be filled in 3x8b format. If i don’t write 0x01 to the control signals, the ip core hangs at dma.sendchannel.wait(), if i do start the core, it waits on dma.recvchannel.wait(). this would imply that the core does something at least. ---
**Author:** ![rock](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.pynq.io/rock/32/32_2.png) [@rock](https://discuss.pynq.io/u/rock)\ **Post date:** [May 5, 2020, 4:33pm UTC](https://discuss.pynq.io/t/pynq-z2-hls-opencv-colorconversion/1219/8 "2020-05-05T16:33:43Z")
That’s exactly what I was talking about. If HLS expects 100 1x32b = 3.2kb data, and you only provide 100 3x8b = 2.4kb data, since HLS core is expecting more data, it will not generate TLAST signal on its output side. This way the DMA is hanging assuming the transaction is not complete. ---
**Author:** ![momo\_flyer](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.pynq.io/momo_flyer/32/874_2.png) [@momo\_flyer](https://discuss.pynq.io/u/momo_flyer)\ **Post date:** [May 17, 2020, 5:39pm UTC](https://discuss.pynq.io/t/pynq-z2-hls-opencv-colorconversion/1219/9 "2020-05-17T17:39:57Z")
i’ve been trying to make it work by changing the data types. I fed my IP core random data in all shapes and sizes and tried receiving it in all shapes and sizes and none of it worked. Are there any good resources on the use of HDMI? I may try it with video data: many more people do it that way. ---
**Author:** ![rock](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.pynq.io/rock/32/32_2.png) [@rock](https://discuss.pynq.io/u/rock)\ **Post date:** [May 18, 2020, 4:26pm UTC](https://discuss.pynq.io/t/pynq-z2-hls-opencv-colorconversion/1219/10 "2020-05-18T16:26:03Z")
I am guessing it should not be that complicated - maybe it is just a small thing you ignored. From your HLS code, it looks like (I have changed the gray\_image dimension as you can notice): ```nohighlight typedef hls::Mat RGB_image; typedef hls::Mat gray_image; ``` You have ```nohighlight RGB_image imggray(yres,xres); ``` which I think should be changed to: ```nohighlight gray_image imggray(yres,xres); ``` I don’t know why you converted the color space back to RGB again (so the HLS IP is doing RGB-\>gray-\>RGB?). The reason why you need to change from `RGB_image` to `gray_image` is that you need to check if this is consistent with what `hls::CvtColor` requires.