RFSoC 4x2 QSFP malfunction to obtain a 10GbE connection

For my application, the main idea is to use only one lane of QSFP to obtain a 10GbE connection with the RFSoC 4x2 board.

I instantiated a 10GbE core (PHY+MAC+UDP), by mapping only one lane, and then a connection to a PC or a switch by a QSFP to SFP breakout cable is performed.

I am experiencing a problem with that, because I observe ARP packets transmitted from the board, but no data are received. As far as I am concerned, I think the core code is OK, because it has been validated and used in a ZCU208.

By checking in schematics,

it seems that MGTYTX and MGTYRX are swapped. More specifically, it appears the following:

  • MGTYTXP0_128 (Y35) and MGTYTXN0_128 (Y36) are connected to QSFP_TX1_P and _N respectively.

  • MGTYRXP0_128 (AA38) and MGTYRXN0_128 (AA39) are connected to QSFP_RX4_P and _N respectively. Instead, I would expect a connection to QSFP_RX1_P and _N.

It could be compatible with what I have observed (TX packet and NO RX). Furthermore, other different lanes have been tried with no change in the final result.

Is there something I’m not understanding and am missing? Any suggestion? Thank you very much in advance.

2 Likes

Hi @atmuriel,

Welcome to the PYNQ community.

Can you share how the 10 G IP is configured?

Mario

You could try to do something like this

And use an xdc file to change the pin polarity.

Mario

1 Like

Hello… I’m facing the same issue with RFSoC 4x2… TX is working okay, No RX… @marioruiz can you kindly clarify what should the definitions in the xdc files be to change pin polarity ?

Can anybody confirm single channel 10GbE RX/TX working ?

Also it is reporting some clock issues on dmesg.

$ dmesg | grep axienet
[ 6.875784] xilinx_axienet a0020000.ethernet end1: renamed from eth1
[ 10.046296] xilinx_axienet a0020000.ethernet end1: XXV MAC block lock not complete! Cross-check the MAC ref clock configuration
[ 10.057922] xilinx_axienet a0020000.ethernet end1: configuring for inband/10gbase-r link mode
[ 470.852440] xilinx_axienet a0020000.ethernet end1: Link is Up - 10Gbps/Full - flow control off
[ 471.876454] xilinx_axienet a0020000.ethernet end1: Link is Down
[ 507.716426] xilinx_axienet a0020000.ethernet end1: Link is Up - 10Gbps/Full - flow control off
[ 508.740445] xilinx_axienet a0020000.ethernet end1: Link is Down

Hi @osschipdesign,

Welcome to the PYNQ community.

Can you please share how the xdc file is looking like for you for these pins?

Mario

Hi @marioruiz… Please find the attached design and its constraints… I am able to do TX only and achieve around 3Gbps data throughput over UDP with static ARP routing..

design_10GbE_mark_one.pdf (168.4 KB)

constraints_xdc.txt (2.3 KB)

Are you using a breakout cable?
Would it be possible to use a different GT? This should make the resolution easier.

Hi.
I have also struggled with this, and after not being able to constrain Vivado to use the ports as described above, I have come up with one verified solution and another potential solution.

The first and possibly easiest one is to try to use one of these type of cables:

With these, it should potentially be possible to just swap which fibers are used together. I have ordered one of these, but it is unverified so far if this works as the cable is not received yet.

But I have verified this solution:
In the 10G/25G Ethernet Subsystem IP in Vivado, select PCS/PMA 64-bit, speed 10.3125G, MII datapath, and set number of cores to 4. Select BASE-R to use the non-licensed version. Under GT selection, select Quad X0Y1, and set Lanes to X0Y4-X0Y7. This enables all 4 10G outputs.

Now, swap the Rx outputs of 1 and 4, and 3 and 2, and wire these to your 10G system. It is important to use the corresponding “rx_clk_out_x” with the MII outputs, and cross clock domains to your other code, as these rx clocks are locked to whatever external 10G equipment you are connecting to. (I myself use the “udp_ipv4_for_10g_ethernet” from OpenCores, which is an excellent lightweight implementation).

The downside with this last implementation is that there is a bit of bloat when enabling all 4 GTYs, but it works and for my full project I am still only using 9% LUTs of the FPGA.
I hope this can help others with the same issue.

Idar

2 Likes