I have since solved all the issues I was having before and figured I should elaborate on how I fixed the current issues.
To get up to 8GHz
First I followed the proposed changed by @rameloni to the RFDC block (changing from 8-16 samples per cycle and increasing the sample rate to 8GHz (DAC) and 5GHz (ADC). Ensure you leave the reference clock at 500MHz (more on this later)
After this, expand the hier_dac_player
the axi_bram_ctrl_0 needs to have it’s data width updated to 512. The same needs to be done for the DACRAMstreamer (the blk_mem_gen_0 will update automatically when generating a new synthesis/implementation as it’s set to take params from the axi_bram_ctrl_0). For the DACRAMstreamer, double the number in Mem size bytes. Doing this will require you to also increase the xlconstant_0 block, double this number as well.
Then in Address controller update the axi_bram_ctrl to be double the size (in the picture I have increased to it’s maximum possible size, but to test I started by doubling it)
After this adjust the axis_clock_convertor, axis_dwidth convertor, and axis_register slice to appropriate sizes (64 bytes before the dwidth convertor and 32 after the convertor).
I then found to get it working properly, I had to delete the axis_broadcaster_0. I am not sure if this was an issue with my board of the IP block itself, but increasing the input to 64 byte and the outputs to 32 bytes resulted in it causing a strange phase jumping that completely messed up the output. As I only needed 1 DAC I routed the output of the hier_dac_player directly to the s20_axis pin on the usp_rf_data_convertor_1
This allowed me to successfully generate signals up to 4GHz on the output of DAC_A
To get to 9.85GHz
The reason setting the sampling rate higher then 8 won’t do anything is the axi streaming clock (the one that dictates how fast samples are passed to the data convertor) is running at 500MHz, and since the maximum number of samples being passed in per clock cycle is 16, the fastest you’ll ever be able to go with this overlay is 8GHz. The bottle neck here isn’t the sample clock, it’s the axi stream clock and how fast it can pass data to the rf data convertor.
It is possible however to increase the spead of that clock. It is set by the on board clock chip (LMK04828 or LMX2594, not sure which one it actually pulls from for the stream clock) which the MTS overlay automatically configures when you load it. These are set by text files that can be found in the MTS overlay. I believe these are generated by a Texas Instruments software and it should be possible to make your own and try get them to run faster so you can pass samples in faster. I have not attempted this yet though as I am currently working on being able to get DAC_A and DAC_B to operate independently with their own buffers. If I manage this I will likely reply here again or make a new post detailing that