Hi there,
I’m trying to handle interrupts from a Vitis HLS IP in C by writing a UIO userspace driver. So far I’ve been successful in handling the interrupts in a Jupyter notebook. I’ve followed this tutorial: Read_the_docs/interrupts.rst at master · KastnerRG/Read_the_docs · GitHub. However, when I try to emulate the same behaviour in C the UIO driver gets stuck in the read() call. The C code I’m using is adapted from: https://forums.xilinx.com/t5/Embedded-Linux/Axi-intc-hangs-on-boot-testcase-on-zcu104/m-p/1188729/highlight/true#M47736
It turns out I’ve been able to detect the interrupts with the C driver if I run the Jupyter notebook first. After some investigations I’ve discovered this has to do with calling await ip.interrupt.wait() in Python. What’s special about this call? I’ve had a look at the source code and it seems to just have to do with the events in the asyncio library, so apparently it shouldn’t be messing with the kernel space UIO driver, right?
P.D: I’m using the PYNQ-Z2 with the pre-built PYNQ image v2.4. I can see how the interrupts are fired in /proc/interrupts (fabric) with the Jupyter notebook and also with the C UIO userspace driver once the Jupyter notebook has been run at least once.
1 Like
Are you using an AXI interrupt controller and if so is your C code setting it up? The PYNQ library handles configuring the interrupt controller which otherwise won’t be happening.
Peter
It turns out I was targeting the mapped memory of the HLS block instead of that of the AXI Interrupt Controller. So after properly setting it up I’m able to receive interrupts in my C code. However, I see that the counter in /proc/interrupts increases to 1 the first time I run the C driver when I receive an interrupt and it gets stuck there. No matter how many interrupts I receive or how many times I execute the driver, it won’t change.
What I’m doing in my C code with respect to the AXI Interrupt Controller:
- Clean IER.
- Set the LSB in IER to enable the first interrupt.
- Enable hardware interrupts in by writing 0x3 to MER.
- Poll raised interrupts through IPR and, clear them through CIE and acknowledge them through IAR.
In terms of the HLS IP block I’m just enabling the registers as in Read_the_docs/interrupts.rst at master · KastnerRG/Read_the_docs · GitHub
Thank you,
Gabriel
1 Like
You need to clear the HLS interrupt in it’s control register prior to clearing the interrupt on the interrupt controller. That’s the only thing I can think of - the flow of your interrupt code looks fine.
Peter
I’m doing that too. Just to confirm, I don’t need a UIO driver to make the interrupts available to /proc/interrupts, right?
Thank you,
Gabriel
The UIO driver fabric is already there so you don’t need to create a separate one for your HLS block assuming that you’re using the PYNQ-style design of having an interrupt controller attached to pin 0. The rest of the interrupt stack is implemented in user space.
One thing you could do is use PYNQ to dump the register map of the interrupt controller in your stuck state. That should at least isolate if the issue is with the HLS or interrupt controller part of the code.
Peter
1 Like
I’ve spent quite some time debugging the code following your advice of dumping the register map. I’ve done this both with the interrupt contorller and the HLS block. I’ve done this at several points of the application that seem relevant, such as before and after waiting for the interrupt or when some registers are explicitly modified. I haven’t seen any differences between the Python and the C code for the interrupt controller and after inspecting the source code of the PYNQ library I think the problem might be some register I’m not correctly toggling on the HLS block. I’m doing exactly what’s done in the Python code.
The HLS block is instantiated in the Jupyter notebook as a DefaultIP object. By looking at the library code it seems there is some register manipulation behind the scenes. Is there any resource that explains which values are written to which registers for HLS blocks, i.e., for DefaultIP objects? I think it’s one of those writes or clears what I’m missing.
P.D.: if it helps, every time I execute the Python code, the interrupt count in /proc/interrupts gets unstuck and it increases by one the next time I execute the C code. After that, it gets stuck again until I execute the Python code. That’s why I suspect the PYNQ library is setting some registers on the HLS block I’m missing.
Thank you,
Gabriel
If you’re not using xclbin files and .start/.call there is no manipulation of registers under-the-hood of DefaultIP - all of the control logic for the CTRL register is only active in that case.
Peter
I’ve been able to solve the problem. It seems I had to read the ‘fabric’ file descriptor (/dev/uio1) on top of the register manipulation I was doing. Thank you so much @PeterOgden for your help!
A further question: in a scenario in which I have several accelerators connected to the interrupt controller is there any way to edit the interrupt handler of the ‘fabric’ driver or should I use epoll() instead? Note that I’m interested in non-blocking handling of interrupts.
Thank you,
Gabriel
The fabric UIO driver will become readable whenever an interrupt is sent to it from the interrupt controller. Therefore if you configure the controller to propagate all interrupts from your accelerators it will become readable whenever any one of them finishes. If you want non-blocking operation you can use epoll on the descriptor or use fcntl to set the descriptor as non-blocking - that way you’ll get E_WOULDBLOCK if no interrupt has occurred.
You can load different device drivers but this would require modifying the device-tree which would break compatibility with PYNQ.
Peter
