Vitis vision core fails on PYNQ V2.5.1

Support
1

I used Vitis vision Github library to replicate the resize stream IP (Hello world example) on PYNQ V2.5.1.

It fails by stalling on the receive side.
On the width converters I use both keep and strb and tuser set to 1.

Any idea what has changed.
For the jupyter notebook I use the existing example with RGB format.

1
11250×875 43.6 KB

2
2966×790 26.8 KB 

    #include "xf_resize_config.h"


#define NPC_T XF_NPPC1


#if RGB
#define TYPE XF_8UC3
#define CH_TYPE XF_RGB
#else
#define TYPE XF_8UC1
#define CH_TYPE XF_GRAY
#endif

#define INPUT_PTR_WIDTH   24
#define OUTPUT_PTR_WIDTH  24

void resize_accel(hls::stream< xf::cv::ap_axiu<24, 1, 1, 1> > &srcPtr, hls::stream< xf::cv::ap_axiu<24, 1, 1, 1> > &dstPtr, int rows_in, int cols_in, int rows_out,  int cols_out);




void resize_accel(hls::stream< xf::cv::ap_axiu<INPUT_PTR_WIDTH, 1, 1, 1> > &srcPtr, 
				  hls::stream< xf::cv::ap_axiu<OUTPUT_PTR_WIDTH, 1, 1, 1> > &dstPtr, 
				  int rows_in, int cols_in,  int rows_out,  int cols_out) 
{
// clang-format off

    #pragma HLS INTERFACE axis port=srcPtr
    #pragma HLS INTERFACE axis port=dstPtr

    #pragma HLS INTERFACE s_axilite port=rows_in              
    #pragma HLS INTERFACE s_axilite port=cols_in              
    #pragma HLS INTERFACE s_axilite port=rows_out              
    #pragma HLS INTERFACE s_axilite port=cols_out              
    #pragma HLS INTERFACE s_axilite port=return
    // clang-format on

    xf::cv::Mat<TYPE, HEIGHT, WIDTH, NPC_T> in_mat(rows_in, cols_in);

    xf::cv::Mat<TYPE, NEWHEIGHT, NEWWIDTH, NPC_T> out_mat(rows_out, cols_out);

// clang-format off
    #pragma HLS DATAFLOW
    // clang-format on


    xf::cv::AXIvideo2xfMat<INPUT_PTR_WIDTH,TYPE,HEIGHT,WIDTH,NPC_T>(srcPtr, in_mat);

    xf::cv::resize<INTERPOLATION, TYPE, HEIGHT, WIDTH, NEWHEIGHT, NEWWIDTH, NPC_T, MAXDOWNSCALE>(in_mat, out_mat);
    
    xf::cv::xfMat2AXIvideo<OUTPUT_PTR_WIDTH,TYPE,NEWHEIGHT,NEWWIDTH,NPC_T>(out_mat, dstPtr);    

}
2

Can you post your block design as well? Have you changed anything there? For the axi width converter, I don’t remember configuring it manually. If that IP also exists in the resizer design, it would be better to use the same configuration.

3

Attached is the Vitis Vision stream IP core and block design.VitisResize_MainDesign.tcl (31.1 KB) xilinx_com_hls_resize_accel_1_0.zip (132.3 KB)

1
11224×443 32.1 KB

2
2808×462 40.3 KB

Nothing was changed. The ip was replaced. I noticed that the generated IP’s have extra sideband signals.

4

I did some comparison but it looks like a lot of things have changed?

image
image1594×857 53.8 KB

Can you make sure you start with the original design and only replace that core?

5

This is the original design.

Apart from Vivado version and IP core I don’t see any other difference.

I have 2019.1 and 2020.1 installed so I had to downgrade to the 2019.1 version. Not sure if 2020.1 is compatible with PYNQ2.5.1 so I went with 2019.1

The only other change was adding the Vitis IP. There are only 4 IP blocks (DMA, resize and AXIS width converters) so I don’t see how can there be many changes.

Unless I am missing something looks like the issue is with the Vitis Core not the Vivado design.

6

When you say you changed the core, how did you change it? There might be configurations that are not good for the python code. For example, the pixel byte width. For the side band signals, I would just use the original settings.

7

I used the attached zip file under L1 Vitis Vision folder and issued vivado_hls -f run.tcl

VitisStreanResizePYNQ.zip (8.5 KB)

The sidebands is the part that is not clear.
Using the HLS code above infers the sideband signals automatically, whereas the resize IP core from the old xfopencV did not have these sidebands .
The designers in that release wrote C++ custom code to convert from the axis stream to xfmat in the old library.

void axis2xfMat (xf::Mat<TYPE, HEIGHT, WIDTH, NPC1> &_src, axis_t *src, int src_rows, int src_cols) {
#pragma HLS inline off

	for (int i=0; i<src_rows; i++) {
		for (int j=0; j<src_cols; j++) {
#pragma HLS pipeline
#pragma HLS loop_flatten off
			_src.data[i*src_cols+j] = src[i*src_cols+j].data;
		}
	}
}

void xfMat2axis (xf::Mat<TYPE, HEIGHT, WIDTH, NPC1> &_dst, axis_t *dst, int dst_rows, int dst_cols) {
#pragma HLS inline off

	for (int i=0; i<dst_rows; i++) {
		for (int j=0; j<dst_cols; j++) {
#pragma HLS pipeline
#pragma HLS loop_flatten off
			ap_uint<1> tmp = 0;
			if ((i==dst_rows-1) && (j== dst_cols-1)) {
				tmp = 1;
			}
			dst[i*dst_cols+j].last = tmp;
			dst[i*dst_cols+j].data = _dst.data[i*dst_cols+j];
		}
	}
}

I tried to use Vitis Vision L1 layer to generate IP cores and I see that Xilinx has released API calls for doing this now:
AXIvideo2xfMat and xfMat2AXIvideo.

But , the catch is that you cannot remove the sidebands as far as I understand. Otherwise the API calls fail.

Attached is the Vitis vision modified project.
I also tested a gray only (X8UC1) version and that also fails.
My guess , the issue is with the side-bands and the way the PYNQ DMA is configured but I may be missing something.

8

I think it might be related to the Vitis version (but I am not sure). To my best knowledge, Vitis is released from 2019.2 so I can’t tell the impact of back-porting it to 2019.1.

9

I can create the project using Vivado 2020.1 but I think the sideband issue is the same. How did the PYNQ team create the IP for resize on the current release? I don’t see the sources for the resize IP on the Hello World repo.

10

Source is HLS and is here:
https://github.com/Xilinx/PYNQ-HelloWorld/tree/master/boards/ip

Cathal

11

The IP source code is not functional with the current Vitis Vison library. (2020.1)
I tried to compile and I am getting the following errors:

~/Documents/resizePYNQ$ vivado_hls -f script.tcl 

****** Vivado(TM) HLS - High-Level Synthesis from C, C++ and SystemC v2020.1 (64-bit)
  **** SW Build 2902540 on Wed May 27 19:54:35 MDT 2020
  **** IP Build 2902112 on Wed May 27 22:43:36 MDT 2020
    ** Copyright 1986-2020 Xilinx, Inc. All Rights Reserved.

source /tools/Xilinx/Vivado/2020.1/scripts/vivado_hls/hls.tcl -notrace
INFO: [HLS 200-10] Running '/tools/Xilinx/Vivado/2020.1/bin/unwrapped/lnx64.o/vivado_hls'
INFO: [HLS 200-10] For user 'iti' on host 'iti-VirtualBox' (Linux_x86_64 version 5.4.0-42-generic) on Wed Oct 07 10:51:48 EDT 2020
INFO: [HLS 200-10] On os Ubuntu 18.04.5 LTS
INFO: [HLS 200-10] In directory '/home/iti/Documents/resizePYNQ'
Sourcing Tcl script 'script.tcl'
INFO: [HLS 200-10] Opening project '/home/iti/Documents/resizePYNQ/resize'.
INFO: [HLS 200-10] Adding design file '../Vitis_Libraries/vision/L2/examples/resize/xf_resize_config.h' to the project
INFO: [HLS 200-10] Adding design file '../Vitis_Libraries/vision/L2/tests/resize/resize_DOWN_BILINEAR_NO_RGB/xf_config_params.h' to the project
INFO: [HLS 200-10] Adding design file 'xf_axis_config.h' to the project
INFO: [HLS 200-10] Adding design file 'xf_resize_accel.cpp' to the project
INFO: [HLS 200-10] Adding test bench file 'xf_resize_tb.cpp' to the project
INFO: [HLS 200-10] Opening solution '/home/iti/Documents/resizePYNQ/resize/solution1'.
INFO: [SYN 201-201] Setting up clock 'default' with a period of 10ns.
INFO: [SYN 201-201] Setting up clock 'default' with an uncertainty of 2.7ns.
INFO: [HLS 200-10] Setting target device to 'xc7z020-clg484-2'
INFO: [SCHED 204-61] Option 'relax_ii_for_timing' is enabled, will increase II to preserve clock frequency constraints.
INFO: [HLS 200-10] Analyzing design file 'xf_resize_accel.cpp' ... 
ERROR: [HLS 200-70] Compilation errors found: In file included from xf_resize_accel.cpp:1:
xf_resize_accel.cpp:42:11: error: type 'hls::stream<DATATYPE, 2>' does not provide a subscript operator
 _src.data[i*src_cols+j] = src[i*src_cols+j].data;
 ~~~~~~~~~^~~~~~~~~~~~~
xf_resize_accel.cpp:62:38: error: type 'hls::stream<DATATYPE, 2>' does not provide a subscript operator
   dst[i*dst_cols+j].data = _dst.data[i*dst_cols+j];
                            ~~~~~~~~~^~~~~~~~~~~~~
In file included from xf_resize_accel.cpp:1:
In file included from xf_resize_accel.cpp:30:
In file included from ./xf_axis_config.h:36:
In file included from /tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_int.h:367:
In file included from /tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_fixed.h:55:
/tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_fixed_base.h:838:13: warning: shift count is negative [-Wshift-count-negative]
      ret.V <<= (_AP_I - _AP_W);
            ^   ~~~~~~~~~~~~~~~
/tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_int_base.h:359:18: note: in instantiation of member function 'ap_fixed_base<53, 21, true, 5, 3, 0>::to_ap_int_base' requested here
    Base::V = op.to_ap_int_base().V;
                 ^
/tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_int.h:249:9: note: in instantiation of function template specialization 'ap_int_base<17, false>::ap_int_base<53, 21, true, 5, 3, 0>' requested here
      : Base((ap_fixed_base<_AP_W2, _AP_I2, true, _AP_Q2, _AP_O2, _AP_N2>)op) {}
        ^
../Vitis_Libraries/vision/L1/include/imgproc/xf_resize_nn_bilinear.hpp:233:22: note: in instantiation of function template specialization 'ap_uint<17>::ap_uint<53, 21, 5, 3, 0>' requested here
        nextYScale = indexy_pre + scaley;
                     ^
../Vitis_Libraries/vision/L1/include/imgproc/xf_resize_nn_bilinear.hpp:397:13: note: in instantiation of function template specialization 'computeInterpolation<1, 42, 20, 17, 48, 16, 48, 16, 1>' requested here
            computeInterpolation<INTERPOLATION_TYPE, COMP_INDEX_WIDTH, COMP_INDEX_INT, INDEX_INT, SCALE_WIDTH,
            ^
../Vitis_Libraries/vision/L1/include/imgproc/xf_resize.hpp:63:9: note: in instantiation of function template specialization 'resizeNNBilinear<9, 2160, 3840, 1, 1080, 1920, 1, 2>' requested here
        resizeNNBilinear<TYPE, SRC_ROWS, SRC_COLS, NPC, DST_ROWS, DST_COLS, INTERPOLATION_TYPE, MAX_DOWN_SCALE>(_src,
        ^
xf_resize_accel.cpp:87:2: note: in instantiation of function template specialization 'xf::cv::resize<1, 9, 2160, 3840, 1080, 1920, 1, 2>' requested here
 xf::cv::resize<1, XF_8UC3, 2160, 3840, 1080, 1920, XF_NPPC1, 2>(src_mat, dst_mat);
 ^
In file included from xf_resize_accel.cpp:1:
In file included from xf_resize_accel.cpp:30:
In file included from ./xf_axis_config.h:36:
In file included from /tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_int.h:367:
In file included from /tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_fixed.h:55:
/tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_fixed_base.h:838:13: warning: shift count is negative [-Wshift-count-negative]
      ret.V <<= (_AP_I - _AP_W);
            ^   ~~~~~~~~~~~~~~~
/tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_int_base.h:359:18: note: in instantiation of member function 'ap_fixed_base<42, 20, true, 5, 3, 0>::to_ap_int_base' requested here
    Base::V = op.to_ap_int_base().V;
                 ^
/tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_int.h:249:9: note: in instantiation of function template specialization 'ap_int_base<17, false>::ap_int_base<42, 20, true, 5, 3, 0>' requested here
      : Base((ap_fixed_base<_AP_W2, _AP_I2, true, _AP_Q2, _AP_O2, _AP_N2>)op) {}
        ^
../Vitis_Libraries/vision/L1/include/imgproc/xf_resize_nn_bilinear.hpp:234:18: note: in instantiation of function template specialization 'ap_uint<17>::ap_uint<42, 20, 5, 3, 0>' requested here
        indexy = (ap_uint<INDEX_INT>)indexy_pre;
                 ^
../Vitis_Libraries/vision/L1/include/imgproc/xf_resize_nn_bilinear.hpp:397:13: note: in instantiation of function template specialization 'computeInterpolation<1, 42, 20, 17, 48, 16, 48, 16, 1>' requested here
            computeInterpolation<INTERPOLATION_TYPE, COMP_INDEX_WIDTH, COMP_INDEX_INT, INDEX_INT, SCALE_WIDTH,
            ^
../Vitis_Libraries/vision/L1/include/imgproc/xf_resize.hpp:63:9: note: in instantiation of function template specialization 'resizeNNBilinear<9, 2160, 3840, 1, 1080, 1920, 1, 2>' requested here
        resizeNNBilinear<TYPE, SRC_ROWS, SRC_COLS, NPC, DST_ROWS, DST_COLS, INTERPOLATION_TYPE, MAX_DOWN_SCALE>(_src,
        ^
xf_resize_accel.cpp:87:2: note: in instantiation of function template specialization 'xf::cv::resize<1, 9, 2160, 3840, 1080, 1920, 1, 2>' requested here
 xf::cv::resize<1, XF_8UC3, 2160, 3840, 1080, 1920, XF_NPPC1, 2>(src_mat, dst_mat);
 ^
In file included from xf_resize_accel.cpp:1:
In file included from xf_resize_accel.cpp:30:
In file included from ./xf_axis_config.h:36:
In file included from /tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_int.h:367:
In file included from /tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_fixed.h:55:
/tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_fixed_base.h:838:13: warning: shift count is negative [-Wshift-count-negative]
      ret.V <<= (_AP_I - _AP_W);
            ^   ~~~~~~~~~~~~~~~
/tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_int_base.h:359:18: note: in instantiation of member function 'ap_fixed_base<43, 21, true, 5, 3, 0>::to_ap_int_base' requested here
    Base::V = op.to_ap_int_base().V;
                 ^
/tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_int.h:249:9: note: in instantiation of function template specialization 'ap_int_base<17, false>::ap_int_base<43, 21, true, 5, 3, 0>' requested here
      : Base((ap_fixed_base<_AP_W2, _AP_I2, true, _AP_Q2, _AP_O2, _AP_N2>)op) {}
        ^
../Vitis_Libraries/vision/L1/include/imgproc/xf_resize_nn_bilinear.hpp:237:22: note: in instantiation of function template specialization 'ap_uint<17>::ap_uint<43, 21, 5, 3, 0>' requested here
        nextYScale = indexy_pre + (ap_fixed<COMP_INDEX_WIDTH, COMP_INDEX_INT>)scaley;
                     ^
../Vitis_Libraries/vision/L1/include/imgproc/xf_resize_nn_bilinear.hpp:397:13: note: in instantiation of function template specialization 'computeInterpolation<1, 42, 20, 17, 48, 16, 48, 16, 1>' requested here
            computeInterpolation<INTERPOLATION_TYPE, COMP_INDEX_WIDTH, COMP_INDEX_INT, INDEX_INT, SCALE_WIDTH,
            ^
../Vitis_Libraries/vision/L1/include/imgproc/xf_resize.hpp:63:9: note: in instantiation of function template specialization 'resizeNNBilinear<9, 2160, 3840, 1, 1080, 1920, 1, 2>' requested here
        resizeNNBilinear<TYPE, SRC_ROWS, SRC_COLS, NPC, DST_ROWS, DST_COLS, INTERPOLATION_TYPE, MAX_DOWN_SCALE>(_src,
        ^
xf_resize_accel.cpp:87:2: note: in instantiation of function template specialization 'xf::cv::resize<1, 9, 2160, 3840, 1080, 1920, 1, 2>' requested here
 xf::cv::resize<1, XF_8UC3, 2160, 3840, 1080, 1920, XF_NPPC1, 2>(src_mat, dst_mat);
 ^
In file included from xf_resize_accel.cpp:1:
In file included from xf_resize_accel.cpp:30:
In file included from ./xf_axis_config.h:36:
In file included from /tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_int.h:367:
In file included from /tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_fixed.h:55:
/tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_fixed_base.h:838:13: warning: shift count is negative [-Wshift-count-negative]
      ret.V <<= (_AP_I - _AP_W);
            ^   ~~~~~~~~~~~~~~~
/tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_int_base.h:359:18: note: in instantiation of member function 'ap_fixed_base<32, 22, true, 5, 3, 0>::to_ap_int_base' requested here
    Base::V = op.to_ap_int_base().V;
                 ^
/tools/Xilinx/Vivado/2020.1/common/technology/autopilot/ap_int.h:249:9: note: in instantiation of function template specialization 'ap_int_base<8, false>::ap_int_base<32, 22, true, 5, 3, 0>' requested here
      : Base((ap_fixed_base<_AP_W2, _AP_I2, true, _AP_Q2, _AP_O2, _AP_N2>)op) {}
        ^
../Vitis_Libraries/vision/L1/include/imgproc/xf_resize_nn_bilinear.hpp:57:17: note: in instantiation of function template specialization 'ap_uint<8>::ap_uint<32, 22, 5, 3, 0>' requested here
        pixel = (typename DataType<DEPTH, NPPC>::cname)((ap_fixed<32, 22>)(P1 + P2 + P3 + P4));
                ^
../Vitis_Libraries/vision/L1/include/imgproc/xf_resize_nn_bilinear.hpp:140:13: note: in instantiation of function template specialization 'interpolatePixel<9, 1, 1>' requested here
            interpolatePixel<DEPTH, INTERPOLATION_TYPE, NPPC>(
            ^
../Vitis_Libraries/vision/L1/include/imgproc/xf_resize_nn_bilinear.hpp:498:21: note: in instantiation of function template specialization 'computeOutputPixel<9, 1, 1, 17, 2, 48, 16>' requested here
                    computeOutputPixel<SRC_TYPE, INTERPOLATION_TYPE, NPPC, INDEX_INT, BUFFER_WORDS, WEIGHT_WIDTH,
                    ^
../Vitis_Libraries/vision/L1/include/imgproc/xf_resize.hpp:63:9: note: in instantiation of function template specialization 'resizeNNBilinear<9, 2160, 3840, 1, 1080, 1920, 1, 2>' requested here
        resizeNNBilinear<TYPE, SRC_ROWS, SRC_COLS, NPC, DST_ROWS, DST_COLS, INTERPOLATION_TYPE, MAX_DOWN_SCALE>(_src,
        ^
xf_resize_accel.cpp:87:2: note: in instantiation of function template specialization 'xf::cv::resize<1, 9, 2160, 3840, 1080, 1920, 1, 2>' requested here
 xf::cv::resize<1, XF_8UC3, 2160, 3840, 1080, 1920, XF_NPPC1, 2>(src_mat, dst_mat);
 ^
4 warnings and 2 errors generated.
Failed during preprocessing.
    while executing
"source script.tcl"
    ("uplevel" body line 1)
    invoked from within
"uplevel \#0 [list source $arg] "

INFO: [Common 17-206] Exiting vivado_hls at Wed Oct  7 10:52:00 2020...
1 Like
12

OK, answering my own question ; the Vitis version linked vitis_lib @ 40b98f7 has a couple of bugs on the xf_infra.hpp and xf_axi.hpp files.

I still don’t understand why using IP’s with sidebands inferred by cvMat2AXIvideoxf() does not work?

13

@rock, @cathalmccabe.

Hi,
a) Is there any explanation why the PYNQ team has not used xf::cv::cvMat2AXIvideoxf and *xf::cv::AXIvideo2cvMatxf() but implemented a custom stream datatype to xfmat for the conversion functions?

Is there any updated version that uses the new Vitis vision release for the xfresize example?

I tried to modify the above functions by removing the keep signals and using the same custom stream datatype. .Now the sidebands do not get generated but the core still fails with the exact error as above.

So that would imply that the issue seems to be related to the stream conversion functions and dma library. Is that correct?

b) Does PYNQ support opening xclbin files with Vitis on ZYNQ devices?

14

I tesed Vitis L1 IP (resize) in streaming and AXI mode and even though they pass C Simulation and COSIM then stall on actual hardware.
Yet somebody on Xilinx forum states that the cores are working. Any idea what is wrong with the Pynq side? I am testing on PYNQ 2.6. The notebook is taken verbatim from the examples.

15

I’m still having the same problem as well using vivado 2019.1 + HLS.
I’m streaming a color image to PL. Using HLS opencv I want to convert it to a HLS::mat, convert the color image to grayscale and send it back to PS.
In my latest search, I have found that it’s possible that the TLast signal isn’t triggered. Even after the IP finishes its work, the memory is not available to the PS. I’ve tried triggering it manually, but to no avail.
Also Outstream->TLast=Instream.TLast; is not available to me.
I’m still looking into it, but perhaps it may help you

16

The TLAST gets generated correctly if you use the Vitis Vision libraries. I have tested with both Vitis_HLS and Vivado_HLS.

XFopencv 2019.1 has bugs that have not been fixed. I opened an issue last year that still has not been addressed. it looks abandoned now on Git.

The issue you are reporting looks the same as in this thread.

https://forums.xilinx.com/t5/High-Level-Synthesis-HLS/Vitis-HLS-aggregates-xf-cv-ap-axiu/m-p/1169162#M22375

The issue I am reporting is simple. The Vitis Vision core pass simulation and COSIM but stall on hardware. I have tested both AXI and streaming modes and both fail.

17

@rock @cathalmccabe

I solved this by following what you guys have done. Using custom axis type and custom MAT to stream functions instead of
xf::cv::AXIvideo2xfMat(_src, src_mat);
xf::cv::xfMat2AXIvideo(src_mat, _dst);

The issue is indeed the PYNQ DMA code is not compatible with the sidebands and tuser signal.
It gets stuck on the receive side.

struct axis_t {
ap_uint<8> data;
ap_int<1> last;
};

/*
Unpack a AXI video stream into a xf::cv::Mat<> object
*input: AXI_video_strm
*output: img
*/
template <int TYPE, int ROWS, int COLS, int NPPC>
int AXIstream2xfMat(hls::stream<axis_t> & AXI_video_strm, xf::cv::Mat<TYPE, ROWS, COLS, NPPC>& img) {
axis_t pixelpacket;
int res = 0;

int rows = img.rows;
int cols = img.cols;
int idx = 0;

assert(img.rows <= ROWS);
assert(img.cols <= COLS);

loop_row_axi2mat:   for (int i = 0; i < rows; i++) {
	loop_col_axi2mat: 	for (int j = 0; j < cols; j++) {
			// clang-format off
    		#pragma HLS loop_flatten off
    		#pragma HLS pipeline II=1
			// clang-format on
			AXI_video_strm >> pixelpacket;
	        img.write(idx++, pixelpacket.data);
	}
   }
return res;

}

You can test this using a simple passthrough functions like axiconv on Vitis vision.

1 Like
18

Apologies, I missed some of the replies on this thread. Good you got it resolved - there are different ways you can do this.

I believe this is a known issue with the Vitis vision libraries and will be fixed in the next release. I think xf::cv::ap_axiu will be removed (in xf_axi_sdata.hpp). There is a version of this declared globally in the hls headers (ap_axi_sdata.h). If you want to try the vision libs, you could try comment out the xf::cv:ap_axiu (and ap_axis) in xf_axi_sdata.hpp

Cathal

19

You are right. I have checked the HLS IP recently. It no longer works for the Vitis 2020.1. I changed the resizer IP to use:

void axis2xfMat(axis_t* src, 
			   xf::cv::Mat<TYPE, HEIGHT, WIDTH, NPC_T>& img) {
	int rows = img.rows;
	int cols = img.cols;
	int idx = 0;

	for (int i = 0; i < rows; i++) {
		for (int j = 0; j < cols; j++) {
			// clang-format off
    		#pragma HLS loop_flatten off
    		#pragma HLS pipeline II=1
			// clang-format on
	        img.write(idx++, src->data);
			src++;
		}
	}
}


void xfMat2axis(xf::cv::Mat<TYPE, HEIGHT, WIDTH, NPC_T>& img,
				axis_t* dst) {
	int rows = img.rows;
	int cols = img.cols;
	int idx = 0;

	for (int i = 0; i < rows; i++) {
		for (int j = 0; j < cols; j++) {
			// clang-format off
    		#pragma HLS loop_flatten off
    		#pragma HLS pipeline II=1
			// clang-format on
			ap_uint<1> tmp = 0;
			if ((i==rows-1) && (j== cols-1)) {
				tmp = 1;
			}
			dst->last = tmp;
			dst->data = img.read(idx++);
			dst++;
		}
	}
}

And now it seems happy to regenerate the IP.

For the DMA core, maybe there are some things we are not aware of. IPs / APIs just keep changing so it is hard for us to keep track of all API changes.

1 Like
20

Even with the code you provided, tlast port is not available in the IP with Vitis HLS 2020.1 and stuck at the DMA receiver side.

I have synthesized the IP with the following code.

#include <iostream>
#include <math.h>
#include "hls_stream.h"
#include "ap_int.h"
#include "common/xf_common.hpp"
#include "imgproc/xf_resize.hpp"
#include "xf_resize_config.h"

struct axis_t {
    ap_uint<24> data;
    ap_int<1> last;
};

void axis2xfMat(axis_t* src, xf::cv::Mat<TYPE, HEIGHT, WIDTH, NPC_T>& img) {
	int rows = img.rows;
	int cols = img.cols;
	int idx = 0;

	for (int i = 0; i < rows; i++) {
		for (int j = 0; j < cols; j++) {
			// clang-format off
    		#pragma HLS loop_flatten off
    		#pragma HLS pipeline II=1
			// clang-format on
	        img.write(idx++, src->data);
			src++; }}}

void xfMat2axis(xf::cv::Mat<TYPE, NEWHEIGHT, NEWWIDTH, NPC_T>& img, axis_t* dst) {
	int rows = img.rows;
	int cols = img.cols;
	int idx = 0;

	for (int i = 0; i < rows; i++) {
		for (int j = 0; j < cols; j++) {
			// clang-format off
    		#pragma HLS loop_flatten off
    		#pragma HLS pipeline II=1
			// clang-format on
			ap_uint<1> tmp = 0;
			if ((i==rows-1) && (j== cols-1)) {
				tmp = 1;}
			dst->last = tmp;
			dst->data = img.read(idx++);
			dst++;}}}

extern "C" {
void resize_accel (axis_t *src, axis_t *dst, int src_rows, int src_cols, int dst_rows, int dst_cols) {

#pragma HLS INTERFACE axis port=src //depth=384*288 // Added depth for C/RTL cosimulation
#pragma HLS INTERFACE axis port=dst //depth=192*144 // Added depth for C/RTL cosimulation
#pragma HLS INTERFACE s_axilite port=src_rows
#pragma HLS INTERFACE s_axilite port=src_cols
#pragma HLS INTERFACE s_axilite port=dst_rows
#pragma HLS INTERFACE s_axilite port=dst_cols
#pragma HLS INTERFACE s_axilite port=return

	xf::cv::Mat<TYPE, HEIGHT, WIDTH, NPC_T> src_mat(src_rows, src_cols);
	xf::cv::Mat<TYPE, NEWHEIGHT, NEWWIDTH, NPC_T> dst_mat(dst_rows, dst_cols);
#pragma HLS stream variable=src_mat.data depth=150
#pragma HLS stream variable=dst_mat.data depth=150
#pragma HLS dataflow

	axis2xfMat(src, src_mat);
	xf::cv::resize<INTERPOLATION, TYPE, HEIGHT, WIDTH, NEWHEIGHT, NEWWIDTH, NPC_T, MAXDOWNSCALE>(src_mat, dst_mat);
	xfMat2axis(dst_mat, dst);    }    }

Can anyone share some working code to implement any simple vision library function with stream functionality that could be used in PYNQ.