Hi,
The problem is now solved! The reason was, the SMP clock cables from the CLK104 to the RFSoC were not connected. When operating at higher sampling rates (clock rates) in the RF Data Converter it is recommend to use the clock generated by the CLK104 instead of the system clock from the RFSoC. After connecting the clock cables everything works as expected.
Cheers!