You shouldn’t have to take the SD card out unless you have made changes to the image/need to reflash the image. Power cycling the board (switching the power switch on/off) is what you want to do to reset your board if you made a new connection that maybe hasn’t been picked up.
If the board boots and everything else works then this sounds like a general networking issue. You should be able to connect to a serial terminal, as Cathal described earlier, and do some debugging… Type ifconfig to see if there’s an IP address that got assigned after boot perhaps.
Try searching this forum and others, there are many posts like this one PYNQ-2.7 on ZCU104 has network connection problem discussing networking issues.
You could try using the microUSB connection with usbgadget and connect directly to your computer as outlined in the rfsoc4x2 getting started page Getting started with your RFSoC 4x2 | RFSoC-PYNQ.
Thanks
Shawn