Does Binary Neural Network working on software take 35 minutes to process?

So I research BNN on PYNQ-Z1 (v2.5) which compares the detection on Hardware and Software, when I run the hardware detection only take 5 to 7 second, the problem is when I run it on software it takes 35 minutes (it says "Inference took 2109116327.50microseconds, 1585801.75 usec per image Claaasification rate: 0.63 images per second) which is impossible because, in theory (also confirm as my lecturer said), it should be around 25 to 35 second which is around 5 times slower than without FPGA, do I have a problem with my code or maybe my PYNQ have a problem in it?