DMA RFDC loop stuck in message transfer

[ZhimuG]

Hi everyone, I’m working on a DMA-RFDC loop back design on the RFSoc 2x2 board and the PYNQ version is 3.0.1.

The block design is shown as following, where I have a DMA block sending a stream to a RFDC block, and the RFDC sends the ADC outputs back to the same DMA block.
design_1.pdf (217.1 KB)

Here are the settings of the RFDC, and the notebook I’m using
screenshots_and_notebook.zip (329.2 KB)

When I ran the notebook, I got stuck with starting the DMA transfers, and it would seem that the RFDC is not started properly. I suscepted there might some issues with the clock settings of the RFDC, but I couldn’t figure out what’s wrong or missing.

If anyone has any idea on what might be wrong here I would really appreciate your inputs. Thank you very much!

Best,
Zhimu

[matthew]

What type of error do you get? Can you send a screenshot of the error displayed, please?
Are you stuck in an endless dma_0.recvchannel.wait()?

Did you check this post?
Debugging Common DMA Issues [Part 3] - Learn - PYNQ

[ZhimuG]

Thank you very much for your help! Yes, I’m stuck in an endless dma_0.recvchannel.wait(), but did not receive any error.

I have checked the register_map after I interrupted the kernel, but I couldn’t spot anything unusual there. Here is the output when I print out the DMA register map:

RegisterMap {
  MM2S_DMACR = Register(RS=1, Reset=0, Keyhole=0, Cyclic_BD_Enable=0, IOC_IrqEn=1, Dly_IrqEn=0, Err_IrqEn=0, IRQThreshold=1, IRQDelay=0),
  MM2S_DMASR = Register(Halted=0, Idle=0, SGIncld=0, DMAIntErr=0, DMASlvErr=0, DMADecErr=0, SGIntErr=0, SGSlvErr=0, SGDecErr=0, IOC_Irq=0, Dly_Irq=0, Err_Irq=0, IRQThresholdSts=0, IRQDelaySts=0),
  MM2S_CURDESC = Register(Current_Descriptor_Pointer=0),
  MM2S_CURDESC_MSB = Register(Current_Descriptor_Pointer=0),
  MM2S_TAILDESC = Register(Tail_Descriptor_Pointer=0),
  MM2S_TAILDESC_MSB = Register(Tail_Descriptor_Pointer=0),
  MM2S_SA = Register(Source_Address=0),
  MM2S_SA_MSB = Register(Source_Address=0),
  MM2S_LENGTH = Register(Length=0),
  SG_CTL = Register(SG_CACHE=0, SG_USER=0),
  S2MM_DMACR = Register(RS=1, Reset=0, Keyhole=0, Cyclic_BD_Enable=0, IOC_IrqEn=1, Dly_IrqEn=0, Err_IrqEn=0, IRQThreshold=1, IRQDelay=0),
  S2MM_DMASR = Register(Halted=0, Idle=0, SGIncld=0, DMAIntErr=0, DMASlvErr=0, DMADecErr=0, SGIntErr=0, SGSlvErr=0, SGDecErr=0, IOC_Irq=0, Dly_Irq=0, Err_Irq=0, IRQThresholdSts=0, IRQDelaySts=0),
  S2MM_CURDESC = Register(Current_Descriptor_Pointer=0),
  S2MM_CURDESC_MSB = Register(Current_Descriptor_Pointer=0),
  S2MM_TAILDESC = Register(Tail_Descriptor_Pointer=0),
  S2MM_TAILDESC_MSB = Register(Tail_Descriptor_Pointer=0),
  S2MM_DA = Register(Destination_Address=0),
  S2MM_DA_MSB = Register(Destination_Address=0),
  S2MM_LENGTH = Register(Length=0)
}

Also, I forgot to mention that I’m using Vivado 2022.1, but I’m not sure if that could cause any issues.

[matthew]

Is this the DMA’s register map for the transmission or the reception?

It seems you printed it before the transmission, before the dma.transfer, because MM2S_SA or S2MM_DA are empty. Can you please retry it after the transmission, to check if the address correspond to the physical address of input_buffer or output_buffer?

[ZhimuG]

My apologies, I copied the wrong cell. Here is the register map after I interrupted the DMA transfer:

RegisterMap {
  MM2S_DMACR = Register(RS=1, Reset=0, Keyhole=0, Cyclic_BD_Enable=0, IOC_IrqEn=1, Dly_IrqEn=0, Err_IrqEn=0, IRQThreshold=1, IRQDelay=0),
  MM2S_DMASR = Register(Halted=0, Idle=1, SGIncld=0, DMAIntErr=0, DMASlvErr=0, DMADecErr=0, SGIntErr=0, SGSlvErr=0, SGDecErr=0, IOC_Irq=1, Dly_Irq=0, Err_Irq=0, IRQThresholdSts=0, IRQDelaySts=0),
  MM2S_CURDESC = Register(Current_Descriptor_Pointer=0),
  MM2S_CURDESC_MSB = Register(Current_Descriptor_Pointer=0),
  MM2S_TAILDESC = Register(Tail_Descriptor_Pointer=0),
  MM2S_TAILDESC_MSB = Register(Tail_Descriptor_Pointer=0),
  MM2S_SA = Register(Source_Address=26423296),
  MM2S_SA_MSB = Register(Source_Address=0),
  MM2S_LENGTH = Register(Length=100),
  SG_CTL = Register(SG_CACHE=0, SG_USER=0),
  S2MM_DMACR = Register(RS=1, Reset=0, Keyhole=0, Cyclic_BD_Enable=0, IOC_IrqEn=1, Dly_IrqEn=0, Err_IrqEn=0, IRQThreshold=1, IRQDelay=0),
  S2MM_DMASR = Register(Halted=0, Idle=0, SGIncld=0, DMAIntErr=0, DMASlvErr=0, DMADecErr=0, SGIntErr=0, SGSlvErr=0, SGDecErr=0, IOC_Irq=0, Dly_Irq=0, Err_Irq=0, IRQThresholdSts=0, IRQDelaySts=0),
  S2MM_CURDESC = Register(Current_Descriptor_Pointer=0),
  S2MM_CURDESC_MSB = Register(Current_Descriptor_Pointer=0),
  S2MM_TAILDESC = Register(Tail_Descriptor_Pointer=0),
  S2MM_TAILDESC_MSB = Register(Tail_Descriptor_Pointer=0),
  S2MM_DA = Register(Destination_Address=26427392),
  S2MM_DA_MSB = Register(Destination_Address=0),
  S2MM_LENGTH = Register(Length=100)
}

[matthew]

Did you modify the length of the input_buffer to send more or less elements, as indicated in the tutorial I provided? Like 640 elements?

[ZhimuG]

Thanks for the suggestion, but I have tried changing the length of input buffer before. I tried a length of 1, 5, 10, but none of these lengths changed anything, and I kept getting stuck on recvchannel.wait()

I also tried setting all bits of tkeep to 1 by introducing a constant input of “15” to the tkeep connection on the S2MM port, as mentioned in the tutorial. But this didn’t work either.
design_1.pdf (230.5 KB)

I noticed in the register map that I had IOC_Irq=1 for MM2S port, but I had IOC_Irq=0 for the S2MM port. I’m wondering does this indicate anything in specific?

[matthew]

Did you check in this datasheet the meaning of all bits?
(S2MM_DMACR (S2MM DMA Control Register – Offset 30h) • AXI DMA LogiCORE IP Product Guide (PG021) • Reader • AMD Technical Information Portal)

Did you try a shape for input_buffer_0 above the 100 in the notebook you provided, like 2048?

[ZhimuG]

Thanks again for the reference, and IOC_Irq seems to simply indicate there was an interrupt which was likely my manual interrrupt in the notebook.

I have tried setting the input buffer to a size of 640 and 2048 as you suggested, while keeping the output buffer the same size, at 100. Neither input buffer sizes seemed to make a difference, as the DMA is still stuck on recvchannel.wait().

However, I did notice that when I allocate the input and output buffers, the physical addresses seem to overlap when I set the input buffer size to 2048 with dtype=np.uint8. The input buffer physical address started at 0x18f2000, while the output buffer physical address started at 0x18f3000. Do you think this would cause any problems?

[matthew]

Are you asking why the input buffer and output buffer have different physical adresses between themselves?

Also, in the notebook, can you please explain what you are doing in this DAC configuration?

dac_tile = rfdc.dac_tiles[0]
dac_tile.DynamicPLLConfig(1, 409.6, 4096)
dac_block = dac_tile.blocks[0]

[ZhimuG]

Sorry for the confusion. I understand that the physical addresses for the input and output buffer need to be different, but I just thought the difference needed to be large enough to accommodate the entire input buffer. So if the input buffer has a size of (2048,) with dtype=np.uint8 and an address of 0x18f2000, the output buffer’s physical address should at least start at 0x18f6000. So right now it seems they are overlapping.

For the DAC configuration, I’m setting the PLL to be enabled, with a PLL reference frequency of 409.6 MHz and a sampling rate of 4.096 Gsps. Here is a screenshot of my DAC setup inside Vivado.

It seems like Vivado is suggesting that the DAC in its current config requires an AXI-Stream clock of 256.0 MHz (I used 409.6 MHz instead), and I’m wondering if that’s causing any problems?

[ZhimuG]

I just figured out the cause of this issue: I forgot to enable the transfer on the packet generator which never created any packets to sent to the DMA.

I had to follow the code as written in the transfer(self, packetsize) function (lines 262 to 283) for the base overlay in this file here: RFSoC-PYNQ/boards/RFSoC2x2/packages/rfsystem/package/rfsystem/hierarchies.py at master · Xilinx/RFSoC-PYNQ · GitHub

Thank you very much for your help Matthew!