Composable Overlay not working on PYNQ-ZU

ZI-YIN_Hong · March 10, 2022, 8:00am

Moderator edit: moved to support category

I run 08_webcam_pipeline.ipynb on PYNQ_ZU
(http://localhost:9090/notebooks/composable-pipeline/custom_pipeline/08_webcam_pipeline.ipynb)

and I encounter below error:

why is this happen? and how can I fix it?

ZI-YIN_Hong · March 10, 2022, 8:34am

also, when I am running /composable-pipeline/custom_pipeline/09_mipi_pipeline.ipynb,
below error occurs…

marioruiz · March 10, 2022, 8:37am

Hi @ZI-YIN_Hong,

What PYNQ SD Card image are you using?
I would suggest you use the SD Card image 2.7 and follow the steps to install the composable overlay 1.0.0 GitHub - Xilinx/PYNQ_Composable_Pipeline: PYNQ Composabe Overlays

The second error about the PCam 5C, did you connect the PCam 5C camera into the MIPI connector?

Mario

ZI-YIN_Hong · March 21, 2022, 3:05am

yes, the PYZQ version I’m using is 2.7 and I installed the composable overlay 1.0.0 as you said
(my board is PYNQ-ZU)
and I run PYNQ_Composable_Pipeline/08_webcam_pipeline.ipynb at main · Xilinx/PYNQ_Composable_Pipeline · GitHub

however I got below error:
cv2.error: OpenCV(4.2.0) …/modules/imgcodecs/src/loadsave.cpp:730: error: (-215:Assertion failed) !buf.empty() in function ‘imdecode_’

ZI-YIN_Hong · March 21, 2022, 4:10am

for the second question, yes I did connect the PCAM 5C camera
and I tried below notebook, and can run successfully

github.com

Xilinx/PYNQ_Composable_Pipeline/blob/main/boards/Pynq-ZU/notebooks/custom_pipeline/09_mipi_pipeline.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Custom Pipeline using MIPI as input source\n",
    "---\n",
    "\n",
    "<div class=\"alert alert-box alert-info\">\n",
    "Please use Jupyter labs http://&lt;board_ip_address&gt;/lab for this notebook.\n",
    "</div>\n",
    "\n",
    "This notebook shows your how to create a custom pipeline that uses the [Digilent Pcam 5C Camera](https://reference.digilentinc.com/reference/add-ons/pcam-5c/start) as input video source\n",
    "\n",
    "## Aims\n",
    "* Use Pcam 5C as input video source\n",
    "* Compose branched pipeline on the input video\n",
    "* Enhance the output by using a simple threshold\n",
    "\n",

This file has been truncated. show original

so, what is the problem here, is it because binary file “…/overlay/cv_dfx_4_pr.bit” I used has bugs,
(and it’s the old version? v1.0?)
or because that HDMIVideo function is deprecated.

and how can I probably fix it?

marioruiz · March 21, 2022, 8:43am

I am confused now, what notebook is not working? It seems that you could run the one with the MIPI camera but not the webcam.

so, what is the problem here, is it because binary file “…/overlay/ cv_dfx_4_pr.bit ” I used has bugs,
(and it’s the old version? v1.0?)

Not that I am aware of, however you do have the bitstream in two different places, did you copy this manually?

Mario

ZI-YIN_Hong · March 21, 2022, 8:59am

I just fixed the MIPI notebook, so let’s forget about it.
(I use the URL in setup.py in your repo and download the newest overlay for PYNQ-ZU, and adjust my code, now it’s working properly, thanks for your code and resource. )

However, I am running your notebook file for webcam, and it has below error, how can I fix it?

github.com

Xilinx/PYNQ_Composable_Pipeline/blob/57af006048d2932d94b80a0eb0965a30d873f1a5/pynq_composable/notebooks/custom_pipeline/08_webcam_pipeline.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using a Webcam as input source of the Composable Pipeline\n",
    "----\n",
    "\n",
    "<div class=\"alert alert-box alert-info\">\n",
    "Please use Jupyter labs http://&lt;board_ip_address&gt;/lab for this notebook.\n",
    "</div>\n",
    "\n",
    "This notebook shows your how to use a Webcam as a input source for your composable pipeline\n",
    "\n",
    "## Aims\n",
    "* Use Webcam as an input video source\n",
    "* Compose a video pipeline over the webcam stream\n",
    "\n",
    "## Table of Contents\n",

This file has been truncated. show original

(Is it anything to do with my device?)

marioruiz · March 21, 2022, 9:01am

Is the webcam connected?
Did you try the notebooks in common that use the webcam?

ZI-YIN_Hong · March 21, 2022, 9:06am

yes, it’s connected.
notebooks/common/usb_webcam.ipynb?
I just tried notebooks/common/usb_webcam.ipynb, and it was working properly.

marioruiz · March 21, 2022, 9:12am

Hi,

Can you please run these commands in a JupyterLab terminal and report the output?

lsusb

and

lsusb -t

What USB camera are you using?

Mario

ZI-YIN_Hong · March 21, 2022, 9:21am

webcam is on USB3.0 port3

I am not sure the webcam resolution, I will check later.
(are there any command line that I can check?)

marioruiz · March 21, 2022, 9:47am

Try to run this

v4l2-ctl --list-devices

My feeling is that the camera is not device 0, hence the error. But, we need to run this command to make sure

ZI-YIN_Hong · March 24, 2022, 2:58am

Hi @marioruiz :
So I can run the webcam notebook from your repo when I start-up the PYNQ-ZU board,

github.com

Xilinx/PYNQ_Composable_Pipeline/blob/main/pynq_composable/notebooks/custom_pipeline/08_webcam_pipeline.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using a Webcam as input source of the Composable Pipeline\n",
    "----\n",
    "\n",
    "<div class=\"alert alert-box alert-info\">\n",
    "Please use Jupyter labs http://&lt;board_ip_address&gt;/lab for this notebook.\n",
    "</div>\n",
    "\n",
    "This notebook shows your how to use a Webcam as a input source for your composable pipeline\n",
    "\n",
    "## Aims\n",
    "* Use Webcam as an input video source\n",
    "* Compose a video pipeline over the webcam stream\n",
    "\n",
    "## Table of Contents\n",

This file has been truncated. show original

However, when I want to run it second time, it cause error at below cell

and seem that it can only run once successfully after starting-up the board,
what is the problem here?

marioruiz · March 24, 2022, 8:49am

Is the previous notebook still running?
Did you release the resources in the previous notebook?

Can you provide the exact steps to reproduce this?

cking · February 8, 2023, 5:03pm

Hi ZI-YIN_Hong

I also get the same issue (using an e-con systems see-3cam usb camera attatched to the Kria Kv260 running Pynq3.0.1 with Ubuntu 22.04).
When running the compsable overlay examples (08_webcam_pipeline or common->usb_webcam) the camera works upon first attempt after booting the board, but presents either this error:

or this error:

very strange behaviour, even resetting the kernel or disconnecting and reconnecting jupyter connection doesn’t work, only a hard reset on the board.

marioruiz · February 8, 2023, 5:33pm

Hi Cameron,

You can try to unplug the camera and plug it back.
This issue should be fixed on Composable 1.1.0 (beta), but it seems that you are still on Composable 1.0.0

Mario

cking · February 9, 2023, 12:03pm

Hi Mario,

Thanks for the feedback, I am using the pynq composable overlay that came pre installed on the PYNQ 3.0.1 image (note: the images in my last post were copied and pasted from the original post not my jupyter nb, but messages are the same).
How would I go about updating to composable 1.1.0 (beta), do I clone the repo?

Thanks,
Cameron

marioruiz · February 9, 2023, 1:32pm

Hi Cameron,

I am using the pynq composable overlay that came pre installed on the PYNQ 3.0.1 image (note: the images in my last post were copied and pasted from the original post not my jupyter nb, but messages are the same).

Then you should have the latest version of the composable overlay.

Mario

cking · February 9, 2023, 1:39pm

Hi Mario,

Yeah I’m not sure why it’s happening. Seems to corrupt the display port output as even after running the stop webcam block, the display port output doesnt return to ubuntu (in my case command line ubuntu 22.04) but remains black. It still can be communicated to through the root terminal notebook within jupyter however.

Cameron

cking · February 9, 2023, 4:04pm

Hi Mario,

Update: changed usb webcam to a logitech C920 from the e-con systems see3CAM_CU30 and it seems to be more stable. Does not crash the display port output with every execution like before.

Thanks,
Cameron

Topic		Replies	Views
Problem with PCAM 5C with KV260 Pynq base overlay Support	6	590	March 14, 2023
PCAM 5c with Kria KV260 Support	5	356	May 29, 2024
Issue Connecting Camera to PYNQ-Z2 Board (Version 3.0.1) Support	2	22	October 29, 2024
Capturing frames from CSI-2 camera in PYNQ-ZU Support	8	1592	June 11, 2024
PyGame compatibility with PYNQ Z1 Support	2	459	April 23, 2023

Composable Overlay not working on PYNQ-ZU

Related topics