I am trying to interface a robotics simulator (MuJoCo) with my PYNQ-Z2 board (PYNQ-Z1 image) and am running into issues. The general idea is that I am trying to pass vectors between the simulator and the FPGA. Thus, I am interested in two approaches:
Install MuJoCo on the board and directly control the FPGA design via MuJoCo.
Use MuJoCo that is already installed on my laptop and somehow have the running simulation interface with the FPGA on-the-fly.
The robotics simulator is installed by running:
pip install gym[mujoco]
When I try approach 1, I am running into installation errors that I did not receive when installing MuJoCo on my laptop. It says “RuntimeError: MUJOCO_PATH environment variable is not set”. Regarding approach 2, I have no idea how to interface a running Python program with an FPGA i.e. how to set up a communication between laptop Python file and the FPGA design.
Would anyone be able to help me with either of these methods? Any insight is greatly appreciated, thank you!
I don’t know about installing Mujoco on the fpga but installing Mujoco on a laptop and interfacing with the fpga via usb to serial converter should be possible.
It is an interesting link in of its own but the basics of it are that they flash the Teensy microprocessor via an Arduino sketch and then communicate via the Python Serial package. Open AI Gym generates the frames that are then send via the serial port. This should work the same for a vector of observations as in the case of environments that generate state vector observations.
The microprocessor then generates the desired action and sends it back to the laptop via serial.
Nowadays it would probably be better to use Gymnasium instead of Gym as it is the most recent version.
