Low accuracy on FPGA

Hi I am trying to follow this project But, the accuracy I get from FPGA is very low(8%) (Please Have a look at the issue raised on the repo).
Seems like weight are not properly loaded onto the FPGA.
While generating the bitstreams, even though bitstream generation was successful, I came across few critical warnings like width mismatch between axi smart connect and Zynq PS: So, I thought maybe these are the reasons for improper weight loading.

After I manually changed the C_M_AXI_S2MM_DATA_WIDTH width to 32 (the first warning). It was resolved but cannot modify the parameters in the 2nd warning(they are set as auto in the IPs).

Can anyone suggest what may be possible reason of such low accuracy?