I set the reference clock to 409.6MHz, and used xrfclk.set_ref_clks(lmk_freq=245.76, lmx_freq=409.6), and now it works.
The point is to make sure the LMX clock is the same as the reference clock in the IP configuration for the RFDC. Otherwise, the rfdc.adc_tiles[0].StartUp() command would fail because you are basically providing wrong clocks to the rfdc (I guess).
However, if you want a clock frequency which is a multiple of 1000 (say, 300MHz, as in your design) rather than a multiple of 1024 (say, 491.52MHz or 409.6MHz), then you might consider using the register file provided here, which set both LMK and LMX clock to 500MHz. If 500MHz is not an option for your reference clock in the IP config, then you might have to design your own register file using the TICSPRO software, which I failed because I don’t know much about the LMK and LMX chips.