Low throughput to USB 3.0 storage on RFSoC 4x2

Support
1

I’m using an RFSoC 4x2 with a USB SSD (Crucial X8) and seeing fairly low throughput to/from the SSD. I’m running PYNQ 3.1.1 with a patched device tree using the settings from @joshgoldsmith 's PR ( device tree fix for usb3 on rfsoc4x2 by jogomojo · Pull Request #60 · Xilinx/RFSoC-PYNQ · GitHub ).

On the RFSoC 4x2, I’ve confirmed with lsusb that it’s connected at 5000M, but I’m only getting about 30 MB/s write and 105 MB/s read. The same drive running the same benchmark on a PC connected at 5000M gets 353 MB/s write, 446 MB/s read. I’ve tried several other USB 3.0 flash drives as well, and they also perform similarly, but I’m using the USB SSD for this example as it can certainly sustain high rates vs. a cheap flash drive.

Has anyone else used USB storage on the RFSoC 4x2 w/ PYNQ? If so, are you seeing similar performance?

Below are outputs of the tests, comparing the RFSoC 4x2 to a PC.

RFSoC 4x2:

lsusb -tv
/:  Bus 02.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 5000M
    ID 1d6b:0003 Linux Foundation 3.0 root hub
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/2p, 5000M
        ID 0424:5742 Microchip Technology, Inc. (formerly SMSC) 
        |__ Port 2: Dev 3, If 0, Class=Mass Storage, Driver=usb-storage, 5000M
            ID 0634:5602 Micron Technology, Inc. 
/:  Bus 01.Port 1: Dev 1, Class=root_hub, Driver=xhci-hcd/1p, 480M
    ID 1d6b:0002 Linux Foundation 2.0 root hub
    |__ Port 1: Dev 2, If 0, Class=Hub, Driver=hub/3p, 480M
        ID 0424:2742 Microchip Technology, Inc. (formerly SMSC) 
        |__ Port 3: Dev 3, If 0, Class=Vendor Specific Class, Driver=, 480M
            ID 0424:2740 Microchip Technology, Inc. (formerly SMSC)

dd if=/dev/zero of=1gfile.bin bs=1G count=1 conv=fdatasync
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 35.6508 s, 30.1 MB/s

dd if=1gfile.bin of=/dev/null bs=1G count=1 iflag=nocache
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 10.1908 s, 105 MB/s

PC:

dd if=/dev/zero of=1gfile.bin bs=1G count=1 conv=fdatasync
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 3.04494 s, 353 MB/s

dd if=1gfile.bin of=/dev/null bs=1G count=1 iflag=nocache
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 2.40969 s, 446 MB/s

Note that for these benchmarks, you need to run the read (2nd) benchmark twice - I believe the nocache clears after it runs, so the first time it’s still using the cache and ridiculously fast.

One other thing to note - initially USB read/write was somewhat unstable, with dmesg outputs showing cmd cmplt err -71 . I changed from uas to usb-storage by adding usb-storage.quirks=0634:5602:u to /boot/uEnv.txt . It’s now stable, but no difference in performance (when it didn’t fail, it was roughly the same speed using uas).

And lastly, I’m testing this with a FAT32 formatted partition. I also tried exFAT and NTFS, but it seems the PYNQ build doesn’t support them (I tried installing the necessary applications through apt, but it seemed to complain about missing the fuse kernel module - didn’t dig too deeply into this). I didn’t try ext4, though I can try that later.

Anyone else seeing this, or have any thoughts?

2

Just to update, I tried an ext4 partition, with similar results:

dd if=/dev/zero of=1gfile.bin bs=1G count=1 conv=fdatasync
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 36.3682 s, 29.5 MB/s

dd if=1gfile.bin of=/dev/null bs=1G count=1 iflag=nocache
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 11.1044 s, 96.7 MB/s

And lastly, I tried reading the raw device (/dev/sda) with no mounted partitions, also with similar results:

dd if=/dev/sda of=/dev/null bs=1G count=1 iflag=nocache
1+0 records in
1+0 records out
1073741824 bytes (1.1 GB, 1.0 GiB) copied, 10.1598 s, 106 MB/s

There was talk in this thread: Low Ethernet Throughput (200 Mbps) on ZynqMP RFSoC4x2 with PYNQ - Is this a known limitation? about the clock configuration causing reduced Ethernet performance… not sure if those clocks (or different ones) could be related to this.

Also not sure if all USB devices have similar performance, or just storage. E.g. would a USB 3.0 Gigabit Ethernet adapter approach GbE speeds, or be stuck at ~30 MB/s write speed?