Hello Jenny,
Thanks again for your work. It really helps me a lot!
I’m working on your system, but I would like to know if you have any idea how to communicate with a PC equipped with an SFP card?
By this I mean that the PC normally has an IP address and can communicate with another PC by different means (sockets for example).
But in this case, the ZCU111 doesn’t seem to use the TCP/IP protocol, and sends raw data packets instead.
Maybe there is a method to bypass the TCP/IP part of the SFP card on the PC and send data packets directly to the ZCU111?
I don’t know if it’s very clear?
Thanks in advance
Damien
Hi Damien,
Unfortunately, I’m not sure how to help with this. You might need to mess with the settings of your SFP card or the CMAC settings? I would recommend reaching out to the vendor of the SFP card and getting more information about the expected data format. It’s possible your card is expecting the CMAC to be implementing Auto-Negotiation, etc. which is not available on the free version of the CMAC IP. It’s also possible your card is expecting to see a signature related to a specific vendor. It would be good to check and make sure you can demonstrate external loopback using a passive module on both the ZCU111 SFP and the PC SFP. Hopefully the SFP card vendor can provide more support.
For ZCU111 (and RFSoC 4x2) there is no TCP/IP stack. If you want this, you would need to build it into the hardware design yourself. You can use UDP instead.
With a direct connection from your board to your NIC, you need to assign a static IP address to the NIC and the board.
- Check/set the Link Mode to 100G
- Set the FEC to RS (RS-FEC)
- Set the FEC speed to 100G
You may need to check other settings on your card.
You can also increase the MTU - this lets you transfer larger frames. You will need to check the driver for your NIC for details on how to do this.
From Python, you can use the socket library on the board and on your host PC to transfer data.
E.g.
Cathal
Hello Jenny,
Thanks for your feedback. The SFP card is a FS card: Intel E810-XXVAM2-Based Ethernet Network Interface Card, 25G Dual-Port SFP28, PCIe 4.0 x 8. It is usually used for classical networking (with TCP/IP layer).
It’s a good idea to test loopback modules on this card, I’ll try that.
Damien
Hello Cathal,
Thanks for your feedback and for creating a new topic.
I have some questions about your proposals:
1- “You can use UDP instead” → Do you have more info on this ?
2- "you need to assign a static IP address to the board. → Also, I can’t find anything in the UltraScale+ Integrated 100G Ethernet Subsystem IP configuration
3- “Check/set the Link Mode to 100G
Set the FEC to RS (RS-FEC)
Set the FEC speed to 100G
You may need to check other settings on your card.”
→ I’m not sure what to do. In the IP configuration, I have to check the option “Enable RS-FEC Transcode Bypass”? If yes, then the IP doesn’t seem to be usable at all, because most of the connectors will disappear.
I would like to have more information about this.
4- Concerning Jenny’s example, it seems that all 4 SFP28 modules are working at the same time: you can’t do a transfer with 1, 2 or 3 SFPs, only 4. This is not really a problem, but on the PC side, which “protocol” should I use to link the 4 ports together? I’m used to using LACP for example on more classical networks, but in this case I’m not sure.
Damien
Sorry for any confusion. This design (and the base Overlay) only includes the Ethernet layer.
You would need to add this or similar to have UDP layer:
You can try sending raw packets, but you need to construct the packets yourself.
Cathal
Hello Cathal,
Thanks for your feedback 
I will try all of these. After a first reading, it seems clearer to me.
Concerning point 4, I would like a small precision:
4- Concerning Jenny’s example, it seems that all 4 SFP28 modules are working at the same time: you can’t do a transfer with 1, 2 or 3 SFPs, only 4. This is not really a problem, but on the PC side, which “protocol” should I use to link the 4 ports together? I’m used to using LACP for example on more classical networks, but in this case I’m not sure.
Maybe it is necessary to use a QSFP28 pcie card in the PC and a QSFP28 to 4 * SFP28 cable to connect to the ZCU111? There is this reference at FS : #50494
Thanks in advance
Damien
1 Like
Hello,
I would like to come back to this subject for a small question:
In the documentation (Documentation Portal), it is several times alluded to a MAC (so I understand a transfer with the layer 2 “link” of the OSI model?)
Do you have any information about this? I can’t find anything in the IP configuration on Vivado for this.
The goal is to make a “raw ethernet” transfer (so without the TCP/IP layer).
Thanks in advance
Damien
Hi @damien.fruleux,
The stream of data that you get or send to the CMAC is raw Ethernet.
Layer 2 Ethernet frame from the image here Ethernet frame - Wikipedia, except the FCS (CRC). The FCS in the transmission path is computed by the IP.
Mario
Hello
Thank you very much for this answer.
I already manage to make “raw ethernet” transfers from 2 PCs, without IP address, only using MAC addresses.
What I want to know is how to “choose or find” the MAC address in the IP?
Or I did not understand well and I have to create myself the Ethernet frame from A to Z (without the FCS) ?
What I want to know is how to “choose or find” the MAC address in the IP?
There is no MAC address in the IP. You have to create the Ethernet frames yourself (without the FCS)
Hello marioruiz,
I understand much better now, thank you very much.
In order to verify the correct operation, I reused Jenny’s example and I simply activated the “Enable FCS Insertion” option (without activating the “Disable FCS Stripping” option) in the “UltraScale+ 100G Ethernet Subsystem” IP options.
I send a 60 bytes ethernet frame as indicated here (Ethernet frame - Wikipedia) and normally the IP should add the 4 bytes of FCS (32bits CRC).
So I should have in return my 60 bytes frame and 4 bytes of CRC.
However, I can’t find a correct “theoretical” CRC (i.e. by using online simulators : https://crccalc.com/ and https://www.scadacore.com/tools/programming-calculators/online-checksum-calculator/)
From check FCS ethernet frame CRC-32 online tools - Stack Overflow and checksum - How do I calculate the FCS field in an Ethernet Frame? - Stack Overflow, I understand that there are several CRC algorithms, but none of them give the CRC contained in my 4 CRC bytes.
I specify that my frame is correct and verified with https://hpd.gasmi.net/
Same with a 92 bytes frame (so 96 with the CRC)
Any idea why?
Thanks in advance
Damien
1 Like
Hi @damien.fruleux,
I do not have any specific information about this. The best place to ask these type of questions is the Xilinx forums.
Mario
