Use pre-built PYNQ/boards base, logictools, etc?

rfdmod · December 12, 2019, 5:15am

Support,

I manually ran make for the base, logictools, etc for each board in the PYNQ/boards directories. When I ran PYNQ/sdbuild/make BOARDDIR=<my_board_dir> BOARDS=<my_board> it once again had to re-build all the base, logictools, etc for each board. Is there a way to have the build process use all the pre-built overlays from the PYNQ/boards directories ? I assume the PYNQ/sdbuild/build/PYNQ/boards/* are copied from the PYNQ/boards/* directories however they do not copy the pre-built base overlays etc.

regards

patocarr · December 12, 2019, 7:41am

Hi @rfdmod
There’s a new feature that handles the build you’re looking for. It’s so recent it hasn’t made it to the documentation yet. Take a look at this post:
https://discuss.pynq.io/t/problems-getting-build-system-up-and-running/665/18

Thanks,
-Pat

gnatale · December 12, 2019, 12:43pm

I’d recommend you use the fature @patocarr suggested and avoid building the bistreams yourself altogether. I also suggest you to use the prebuilt rootfs so you will save even more time.

The two options are documented here: PYNQ/sdbuild at master · Xilinx/PYNQ · GitHub
and here (not yet updated on readthedocs): PYNQ/pynq_sd_card.rst at master · Xilinx/PYNQ · GitHub
In particular:

The relevant excerpt from the sdbuild readme:

To use a board-agnostic image to build a board-specific image you can pass the PREBUILT variable:

make PREBUILT=<image path> BOARDS=<board>

To use a previously built PYNQ source distribution tarball you can pass the PYNQ_SDIST variable. This will also avoid having to rebuild bitstreams (except for external boards) and MicroBlazes’ bsps and binaries.

make PYNQ_SDIST=<sdist tarball path>

rfdmod · December 14, 2019, 3:40am

Hi Gnatele,

I used the two options mentioned, however it is still trying to re-build the bitstreams
for the boards.

When PYNQ/sdbuild/build/PYNQ/sdbuild/packages/pynq/pre.sh reaches
line 41 overlays=find . -maxdepth 2 -iname 'makefile' -printf '%h\n' | cut -f2 -d"/"
it will not find any overlays since the ‘makefiles’ are not present in the untarred pynq-2.5.tar.gz

I downloaded pynq-2.5.tar.gz from Releases · Xilinx/PYNQ · GitHub

you can see this in my log file below:

++ ls /home/user/local/git/PYNQ/sdbuild/build/PYNQ/sdist_untar/pynq-2.5/boards

boards=‘Pynq-Z1
Pynq-Z2
ZCU104’
for bd_name in ‘$boards’
cd /home/user/local/git/PYNQ/sdbuild/build/PYNQ/sdist_untar/pynq-2.5/boards/Pynq-Z1
++ find . -maxdepth 2 -iname makefile -printf ‘%h\n’
++ cut -f2 -d/
overlays=
for bd_name in ‘$boards’
cd /home/user/local/git/PYNQ/sdbuild/build/PYNQ/sdist_untar/pynq-2.5/boards/Pynq-Z2
++ find . -maxdepth 2 -iname makefile -printf ‘%h\n’
++ cut -f2 -d/
overlays=
for bd_name in ‘$boards’
cd /home/user/local/git/PYNQ/sdbuild/build/PYNQ/sdist_untar/pynq-2.5/boards/ZCU104
++ find . -maxdepth 2 -iname makefile -printf ‘%h\n’
++ cut -f2 -d/
overlays=
rm -rf /home/user/local/git/PYNQ/sdbuild/build/PYNQ/sdist_untar
cd /home/user/local/git/PYNQ/sdbuild/build/PYNQ
./build.sh
./build.sh

Script for building default overlays, microblaze bsp’s and binaries.

skipping bitstream base.bit for SnickerdoodleBlack
skipping bitstream logictools.bit for Pynq-Z2
building bitstream base.bit for Pynq-Z2
make[1]: Entering directory ‘/home/user/local/git/PYNQ/sdbuild/build/PYNQ/boards/Pynq-Z2/base’
vivado -mode batch -source build_base_ip.tcl -notrace

gnatale · December 16, 2019, 11:45am

I am sorry, you are totally right! During testing, I happened to use an sdist that had the makefile in it so I introduced this bug. I’ll immediately file a PR to fix it.
In the meantime, I suggest you change the incriminated line with what the PR will look like

github.com

Xilinx/PYNQ/blob/master/sdbuild/packages/pynq/pre.sh#L47


      
          sudo cp -rf $BUILD_ROOT/PYNQ/REVISION $target/home/xilinx/REVISION
          sudo cp -rf $BUILD_ROOT/PYNQ/REVISION $target/boot/REVISION
          
          
if [ -d /usr/local/share/fatfs_contents ]; then
              sudo cp -rf $BUILD_ROOT/PYNQ/REVISION /usr/local/share/fatfs_contents/REVISION
          fi
          
          
if [ -n "$PYNQ_SDIST" ] && [ "$BOARDDIR" -ef "$DEFAULT_BOARDDIR" ]; then
          	# using prebuilt sdist with non-external board
          	# nothing to do except copy sdist to target folder
          	sudo cp ${PYNQ_SDIST} $target/home/xilinx/pynq_git/dist
          else
          	if [ -n "$PYNQ_SDIST" ] && [ ! "$BOARDDIR" -ef "$DEFAULT_BOARDDIR" ]; then
          		# using prebuilt sdist with external board
          		# copy bitstreams, microblaze bsps and binaries from sdist
          		sdist_untar=$BUILD_ROOT/PYNQ/sdist_untar
          		mkdir -p ${sdist_untar}
          		tar -xzvf ${PYNQ_SDIST} -C ${sdist_untar}
          		cd $BUILD_ROOT/PYNQ
          		rsync -rptL --ignore-existing ${sdist_untar}/*/pynq/lib/* pynq/lib/
          		boards=`ls ${sdist_untar}/*/boards`

into
overlays=find . -maxdepth 2 -iname ‘*.bit’ -printf ‘%h\n’ | cut -f2 -d"/"``

EDIT: PR submitted: Fix bug in pynq pre.sh for external boards by giunatale · Pull Request #979 · Xilinx/PYNQ · GitHub
Thanks again for pointing this out!

rfdmod · December 16, 2019, 6:14pm

I’m not sure if this will affect anyone else but I am using a vmware virtual machine and after patching the overlay line as suggested and restarting the build, I received the following error:

rm -rf base *.jou *.log NA
make[1]: Leaving directory ‘/home/user/local/git/PYNQ/sdbuild/build/PYNQ/boards/ZCU104/base’

cd /home/user/local/git/PYNQ/sdbuild/build/PYNQ
python3 setup.py sdist
Traceback (most recent call last):
File “setup.py”, line 34, in
from setuptools import setup, Extension, find_packages, Distribution
ImportError: No module named ‘setuptools’
unmount_special

To fix this I had to do the following:

sudo apt install python3-pip
sudo pip3 install --upgrade “setuptools>=24.2.0”

After that the build completed fine. The entire build after a make clean now only takes about 27 minutes for my setup !! Thanks for all your work.

gnatale · December 17, 2019, 11:09am

setup_host.sh has this install at the bottom.

github.com

Xilinx/PYNQ/blob/master/sdbuild/scripts/setup_host.sh#L123


      
          
          
# make sure resolv.conf is where it's going to be expected in chroot
          if [ ! -f /run/systemd/resolve/stub-resolv.conf ]; then
              sudo mkdir -p /run/systemd/resolve
              sudo cp -L /etc/resolv.conf /run/systemd/resolve/stub-resolv.conf
          fi
          
          
# update setuptools
          sudo -H python3 -m pip install --upgrade "setuptools>=24.2.0"
          # install dependencies required to build pynq sdist
          sudo -H python3 -m pip install numpy cffi
          
          
echo 'PATH=/opt/qemu/bin:/opt/crosstool-ng/bin:$PATH' >> ~/.profile
          
          
echo "Now install Vitis and Petalinux."
          echo "Re-login to  ensure the environment is properly set up."

So this should not be a problem. Perhaps I am assuming you built the VM before I did the PR to fix the imagebuild (that’s when I added that line in setup_host)

gnatale · December 17, 2019, 11:10am

btw, the PR mentioned above got merged yesterday

Again, thanks for reporting this out!

Topic		Replies	Views
PYNQ 2.5.1 custom board building mixed with pynq boards Support	1	1296	December 20, 2019
PYNQ2.7 Custom Board Build Error - PYNQ_SDIST Support	7	1095	January 30, 2023
Where is the Prebuilt Pynq Source Distribution? Support	2	422	October 15, 2022
Can pre-built rootfs images be used for building all PYNQ versions? Support	3	494	March 6, 2020
How can I skip building the standard machines in the Pynq repo? Support	1	520	September 28, 2022

Use pre-built PYNQ/boards base, logictools, etc?

Related topics