There are a couple of things to have a look at.
First, be careful when assigning array objects - doing
input_buffer = frame doesn’t copy the data from
input_buffer instead it just changes the
input_buffer reference to be that of
frame thus losing handle to the
input_buffer you allocated before the loop. If you want to copy the data from one buffer to another you should use
input_buffer[:] = frame.
That shouldn’t be necessary in this case though. You should be able to operate on the input and output frames directly - something like:
in_frame = hdmi_in.readframe()
out_frame = hdmi_out.newframe()
You might be able to get a bit of a performance boost if you explicitly interleave the processing with the HDMI read and write commands but I don’t think it will make that much difference.
If you are running at 1080p60 you might want to check at a lower resolution/frame rate. The DDR bandwidth of the board is somewhat limited and you can start running into bottlenecks with two many video streams running.
One final note is that (at least with PYNQ >= 2.5) you shouldn’t need to worry about freebuffer. Buffers will get freed automatically when they go out of scope.