PYNQ Allocate() Max. Size?

Support
1

Hello,
I would like to ask, what is the maximum memory that can be allocated by PYNQ’s allocate() function?

from pynq import Overlay, Bitstream, GPIO, MMIO
from pynq import allocate
import numpy as np

src_buffer1 = allocate(shape=(1920*1080,), dtype=np.float32)
dst_buffer1 = allocate(shape=(1920*1080,), dtype=np.float32)

src_buffer2 = allocate(shape=(1920*1080,), dtype=np.float32)
dst_buffer2 = allocate(shape=(1920*1080,), dtype=np.float32)

src_buffer3 = allocate(shape=(1920*1080,), dtype=np.float32)
dst_buffer3 = allocate(shape=(1920*1080,), dtype=np.float32)

src_buffer4 = allocate(shape=(1920*1080,), dtype=np.float32)
dst_buffer4 = allocate(shape=(1920*1080,), dtype=np.float32)

src_buffer5 = allocate(shape=(1920*1080,), dtype=np.float32)
dst_buffer5 = allocate(shape=(1920*1080,), dtype=np.float32)

src_buffer6 = allocate(shape=(1920*1080,), dtype=np.float32)
dst_buffer6 = allocate(shape=(1920*1080,), dtype=np.float32)

src_buffer7 = allocate(shape=(1920*1080,), dtype=np.float32)
dst_buffer7 = allocate(shape=(1920*1080,), dtype=np.float32)

src_buffer8 = allocate(shape=(1920*1080,), dtype=np.float32)
dst_buffer8 = allocate(shape=(1920*1080,), dtype=np.float32)

and there’s Error:

RuntimeError                              Traceback (most recent call last)
<ipython-input-3-a0b8a9a655ee> in <module>
     19 
     20 src_buffer6 = allocate(shape=(1920*1080,), dtype=np.float32)
---> 21 dst_buffer6 = allocate(shape=(1920*1080,), dtype=np.float32)
     22 
     23 src_buffer7 = allocate(shape=(1920*1080,), dtype=np.float32)

/usr/local/share/pynq-venv/lib/python3.8/site-packages/pynq/buffer.py in allocate(shape, dtype, target, **kwargs)
    170     if target is None:
    171         target = Device.active_device
--> 172     return target.allocate(shape, dtype, **kwargs)

/usr/local/share/pynq-venv/lib/python3.8/site-packages/pynq/pl_server/device.py in allocate(self, shape, dtype, **kwargs)
    290 
    291         """
--> 292         return self.default_memory.allocate(shape, dtype, **kwargs)
    293 
    294     def reset(self, parser=None, timestamp=None, bitfile_name=None):

/usr/local/share/pynq-venv/lib/python3.8/site-packages/pynq/pl_server/xrt_device.py in allocate(self, shape, dtype, **kwargs)
    167 
    168         """
--> 169         buf = _xrt_allocate(shape, dtype, self.device, self.idx, **kwargs)
    170         buf.memory = self
    171         return buf

/usr/local/share/pynq-venv/lib/python3.8/site-packages/pynq/pl_server/xrt_device.py in _xrt_allocate(shape, dtype, device, memidx, cacheable, pointer, cache)
    122         bo, buf, device_address = pointer
    123     else:
--> 124         bo = device.allocate_bo(size, memidx, cacheable)
    125         buf = device.map_bo(bo)
    126         device_address = device.get_device_address(bo)

/usr/local/share/pynq-venv/lib/python3.8/site-packages/pynq/pl_server/xrt_device.py in allocate_bo(self, size, idx, cacheable)
    412                             xrt.xclBOKind.XCL_BO_DEVICE_RAM, idx)
    413         if bo >= 0x80000000:
--> 414             raise RuntimeError("Allocate failed: " + str(bo))
    415         return bo
    416 

RuntimeError: Allocate failed: 4294967295
1 Like
2

Hi,
“allocate()” can keep whatever size you want. Your limitation is your processor, so it depends with the board you are using.

Here, it seems that you tried to allocate 232 = 4294967295 bits, while the limit is 0x80000000 = 231 bits. I Do not know why, because you have an error during dst_buffer6, when you allocated 1920x1080x32x12 bits < 231 bits.

Maybe some bits cannot be allocated to let the processor operate, as a minimum requirement.

3

Hi,
my board is ZCU216 with 4GB DDR4 on PS side, so… why did this happen? :sweat_smile:

4

Hi @H.W,

pynq.allocate() needs physically contiguous memory, the size of this contiguous memory is defined at boot time by CMA in Linux. So, if you check dmesg you’ll probably see an error indicating that no more memory is available.

You can check CMA like this

cat /proc/meminfo | grep -i cma

You can check how to increase it here AMD Customer Community

Mario

5

Hi @marioruiz,
thanks a lot for the solution. It works!
The CMA size before was only 128MB

root@pynq:/home/xilinx# cat /proc/meminfo | grep -i cma
CmaTotal:         131072 kB
CmaFree:          130560 kB

by using PuTTY enter the U-BOOT Console and set the new CMA size:

setenv bootargs "earlycon console=ttyPS1,115200 clk_ignore_unused root=/dev/mmcblk0p2 rw rootwait init_fatal_sh=1 cma=1200M"

Now:

root@pynq:/home/xilinx# cat /proc/meminfo | grep -i cma
CmaTotal:        1228800 kB
CmaFree:         1228288 kB

And the allocate() report no errors.

There’s a little problem: by using this way, I can’t “saveenv” in the U-BOOT console, which means I have to reset CMA size every time I reboot the board.

2 Likes
6

Hi @H.W,

If you’re building the image for your board, you can increase this on the device tree.
Otherwise, you can create a custom device tree overlay that increases the CMA size.