No output.npy in tfc_end2end_example

Support
1

Hi everyone,

I’m working with PYNQ 2.7 on a ZCU104. I’m trying to run the tfc_end2end_example notebook from the FINN docker container. On cell 37

import numpy as np
from finn.core.onnx_exec import execute_onnx

input_dict = {iname: x.reshape(ishape)}
ret = execute_onnx(model, input_dict)

I encounter the following error:

---------------------------------------------------------------------------
FileNotFoundError                         Traceback (most recent call last)
<ipython-input-37-6dc8c038df07> in <module>
      3 
      4 input_dict = {iname: x.reshape(ishape)}
----> 5 ret = execute_onnx(model, input_dict)

/home/julien/finn/src/finn/core/onnx_exec.py in execute_onnx(model, input_dict, return_full_exec_context, start_node, end_node)
     99     elif model_exec_mode == "remote_pynq":
    100         # use remote exec metadata built into model to execute on a remote PYNQ
--> 101         remote_exec(model, execution_context)
    102     elif model_exec_mode == "rtlsim":
    103         # use stitched IP for rtlsim

/home/julien/finn/src/finn/core/remote_exec.py in remote_exec(model, execution_context)
    116     process_scp_out = subprocess.Popen(bash_command, stdout=subprocess.PIPE)
    117     process_scp_out.communicate()
--> 118     outp = np.load("{}/output.npy".format(deployment_dir))
    119     execution_context[model.graph.output[0].name] = outp

/opt/conda/lib/python3.8/site-packages/numpy/lib/npyio.py in load(file, mmap_mode, allow_pickle, fix_imports, encoding)
    388             own_fid = False
    389         else:
--> 390             fid = stack.enter_context(open(os_fspath(file), "rb"))
    391             own_fid = True
    392 

FileNotFoundError: [Errno 2] No such file or directory: '/tmp/finn_dev_julien/pynq_deployment_jremieh0/output.npy'

I have seen on another topic to try to execute driver.py via a ssh connexion (output.npy not found in End-to-end-flow notebook · Issue #46 · Xilinx/finn · GitHub). When I do so I have the following error:

xilinx@pynq:~/finn_dev_julien/pynq_deployment_isqt422a$ sudo python3 driver.py 
[sudo] password for xilinx: 
Traceback (most recent call last):
  File "driver.py", line 34, in <module>
    from driver_base import FINNExampleOverlay
  File "/home/xilinx/finn_dev_julien/pynq_deployment_isqt422a/driver_base.py", line 32, in <module>
    from pynq import Overlay, allocate
ModuleNotFoundError: No module named 'pynq'

So I tried to install pynq (even tho it sounded weird):

xilinx@pynq:~/finn_dev_julien/pynq_deployment_isqt422a$ pip3 install pynq
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pynq in /usr/local/share/pynq-venv/lib/python3.8/site-packages (2.7.0)
Requirement already satisfied: cffi in /usr/local/share/pynq-venv/lib/python3.8/site-packages (from pynq) (1.14.5)
Requirement already satisfied: numpy in /usr/local/share/pynq-venv/lib/python3.8/site-packages (from pynq) (1.20.3)
Requirement already satisfied: pandas in /usr/local/share/pynq-venv/lib/python3.8/site-packages (from pynq) (1.3.3)
Requirement already satisfied: setuptools>=24.2.0 in /usr/local/share/pynq-venv/lib/python3.8/site-packages (from pynq) (44.0.0)
Requirement already satisfied: pycparser in /usr/lib/python3/dist-packages (from cffi->pynq) (2.19)
Requirement already satisfied: pytz>=2017.3 in /usr/lib/python3/dist-packages (from pandas->pynq) (2019.3)
Requirement already satisfied: python-dateutil>=2.7.3 in /usr/lib/python3/dist-packages (from pandas->pynq) (2.7.3)
WARNING: You are using pip version 21.2.1; however, version 22.2.1 is available.
You should consider upgrading via the '/usr/local/share/pynq-venv/bin/python3 -m pip install --upgrade pip' command.

Does anyone know what could I do to solve this problem?

Thanks in advance,
Julien

2

Hi,

After more research I found these Github issues but their solutions didn’t work for me :

bistring and setuptools are installed on my board:

xilinx@pynq:~$ pip3 install bitstring setuptools
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: bitstring in /usr/local/share/pynq-venv/lib/python3.8/site-packages (3.1.9)
Requirement already satisfied: setuptools in /usr/local/share/pynq-venv/lib/python3.8/site-packages (44.0.0)
WARNING: You are using pip version 21.2.1; however, version 22.2.1 is available.
You should consider upgrading via the '/usr/local/share/pynq-venv/bin/python3 -m pip install --upgrade pip' command.

I don’t know if pip3 has to be updated, I tried to update it, it threw this error:

xilinx@pynq:~$ python3 -m pip install --upgrade pip
Defaulting to user installation because normal site-packages is not writeable
Requirement already satisfied: pip in /usr/local/share/pynq-venv/lib/python3.8/site-packages (21.2.1)
Collecting pip
  Downloading pip-22.2.1-py3-none-any.whl (2.0 MB)
     |████████████████████████████████| 2.0 MB 3.8 MB/s 
ERROR: Will not install to the user site because it will lack sys.path precedence to pip in /usr/local/share/pynq-venv/lib/python3.8/site-packages
WARNING: You are using pip version 21.2.1; however, version 22.2.1 is available.
You should consider upgrading via the '/usr/local/share/pynq-venv/bin/python3 -m pip install --upgrade pip' command.

Also I noticed that the notebook tries to reach /tmp/finn_dev_julien/pynq_deployment_1g5ct2f0/output.npy, but I can’t see this folder on my board, only /home/xilinx/finn_dev_julien/pynq_deployment_1g5ct2f0, I don’t know if this is normal. Anyway, output.npy doesn’t exist there either:

xilinx@pynq:~/finn_dev_julien/pynq_deployment_1g5ct2f0$ ls
driver_base.py  finn       qonnx        resizer.hwh      validate.py
driver.py       input.npy  resizer.bit  runtime_weights
3

Hi @julienmas,

Welcome to our community.

pynq is installed in a venv, which is not available in the xilinx user by default. I would recommend you login as sudo. To source the venv, please check this

Mario

4

Hi Mario,

Thanks for your answer. When I log as root I can’t find driver.py:

xilinx@pynq:~/finn_dev_julien$ sudo -i
[sudo] password for xilinx: 
root@pynq:~# ls
root@pynq:~#

I used the the venv with the xilinx user, it didn’t help me either:

xilinx@pynq:~/finn_dev_julien/pynq_deployment_599kb_zv$ source /etc/profile.d/pynq_venv.sh
(pynq-venv) xilinx@pynq:~/finn_dev_julien/pynq_deployment_599kb_zv$ sudo python3 driver.py 
Traceback (most recent call last):
  File "driver.py", line 34, in <module>
    from driver_base import FINNExampleOverlay
  File "/home/xilinx/finn_dev_julien/pynq_deployment_599kb_zv/driver_base.py", line 32, in <module>
    from pynq import Overlay, allocate
ModuleNotFoundError: No module named 'pynq'
(pynq-venv) xilinx@pynq:~/finn_dev_julien/pynq_deployment_599kb_zv$
5

Have you tried to run your script from a terminal inside JupyterLab?

When I log as root I can’t find driver.py

You need to move to the directory where the file is.

Mario

6

Thanks, I’ve been able to run driver.py from a terminal inside JupyterLab:

(pynq-venv) root@pynq:/home/xilinx/finn_dev_julien/pynq_deployment_599kb_zv# python3 driver.py
(pynq-venv) root@pynq:/home/xilinx/finn_dev_julien/pynq_deployment_599kb_zv#

However, it didn’t solved my initial problem: I’m still unable to run execute_onnx() from the FINN Jupyter Notebook.

7

I would suggest you seek support in the FINN forums.

Mario

8

It now works, using PYNQ 2.6. I’m not sure what I’ve done, but I think it started to worked after I did a git pull from FINN Github repo, on dev branch.