Custom Board crashes because of PL EMIO PHY

Hello Pynq-community,
I build a SD image for a custom board (based on the Zynq 7010). I was able to get Jupyter running, even though it required a bit of bodging. The following picture is the architecture of my base overlay:

The PL EMIO to PHY connections are my best guess. Getting Jupyter working required manually starting the eth0 interface in boot.py after loading the base overlay. Other than that it runs fine, with one big exception. When I try to apply another overlay, the board becomes unresponsive. This extends to the UART shell. After that a hard reset is necessary to get the board back to life. Interestingly this also happens while applying the overlays over UART without re-enabling the eth0 in the boot.py beforehand.
Tips for debugging would be more than welcome!
Thank you!