# Kria base overlay, mipi cam5c, how to change the ColorSpaceConversion?

**URL:** <https://discuss.pynq.io/t/kria-base-overlay-mipi-cam5c-how-to-change-the-colorspaceconversion/6545>\
**Category:** Support\
**Created:** [January 19, 2024, 9:09am UTC](https://discuss.pynq.io/t/kria-base-overlay-mipi-cam5c-how-to-change-the-colorspaceconversion/6545 "2024-01-19T09:09:58Z")\
**Posts on this page:** 6\
**Page:** 1

<div class="post-metadata">

**Author:** ![sergicuen](https://avatars.discourse-cdn.com/v4/letter/s/a5b964/32.png) [@sergicuen](https://discuss.pynq.io/u/sergicuen)\
**Post date:** [January 19, 2024, 9:09am UTC](https://discuss.pynq.io/t/kria-base-overlay-mipi-cam5c-how-to-change-the-colorspaceconversion/6545/1 "2024-01-19T09:09:58Z")

</div>

Hi all,  
I am working on the Kria base overlay ( **mipi\_to\_display notebook** ). I am trying to store a gray image via VDMA ( **8bits/pixel, Y channel** ).

Using the following code:

```auto
videomode = VideoMode(1280, 720, 8)
mipi.configure(videomode)  
frame = mipi.readframe()
frame.shape # (720, 1280)

```

I get a gray image but, looking at the pynq code I see that pixel data is just written with the B channel. The reason is that v\_proc\_sys IP is configure to bypass the BGR pixels coming from the demosaic/gamma pipeline.

My question is, how to reconfigure the v\_proc\_sys IP to convert from RGB to YCRCB and then use the pixel\_packer IP (V\_8 mode) to store a gray image through VDMA?

something like mipi.v\_proc\_sys.colorspace(COLOR\_IN\_YCBCR) doesn´t work  
Thank you

---

<div class="post-metadata">

**Author:** ![sergicuen](https://avatars.discourse-cdn.com/v4/letter/s/a5b964/32.png) [@sergicuen](https://discuss.pynq.io/u/sergicuen)\
**Post date:** [January 21, 2024, 11:28am UTC](https://discuss.pynq.io/t/kria-base-overlay-mipi-cam5c-how-to-change-the-colorspaceconversion/6545/2 "2024-01-21T11:28:49Z")

</div>

My last findings,  
I reused some **pipeline.py** PYNQ code to write to the registers of the v\_proc\_sys IP:

```auto
def colorspace(color):
        if len(color) != 12:
            raise ValueError("Wrong number of elements in color specification")
        for i, c in enumerate(color):
            vproc.write(0x10 + 8 * i, int(c * 256))

def configBGR2YCBCR():            
    colorspace(COLOR_IN_YCBCR)

COLOR_IN_YCBCR = [0.114, 0.587, 0.299,
                  0.5, -0.331264, -0.168736,
                  -0.081312, -0.41866, 0.5,
                  0, 0.5, 0.5]

```

Then configuring and reading a frame

```auto
videomode = VideoMode(1280, 720, 8) #pixel packer in V8 mode to store just the Y channel
mipi.stop()
mipi.configure(videomode)
configBGR2YCBCR()
mipi.start()
frame = mipi.readframe()
PIL.Image.fromarray(frame)

```

the result :

 ![image](https://us1.discourse-cdn.com/flex019/uploads/pynq1/original/2X/d/d59da7f392e84b8729ca15ddb44827551eaa3a95.jpeg)

The Y values are very low.

I changed the original scale of the coeff (256) to 4096, but kept the offset scale to 256.

```auto
def colorspace(color):
        if len(color) != 12:
            raise ValueError("Wrong number of elements in color specification")
        for i, c in enumerate(color):
            vproc.write(0x50 + 8 * i, int(c * 4096)) #vproc.write(0x10 + 8 * i, int(c * 256))
        vproc.write(0x98, 0) # write the offsets 
        vproc.write(0x0a0, 128) # 0.5*256
        vproc.write(0x0a8, 128) # 0.5*256

```

The result looks correct, now.

 ![image](https://us1.discourse-cdn.com/flex019/uploads/pynq1/original/2X/2/2219c25a78687d0a1c71f83f6690e1f7bb93c0f5.jpeg)

So, this rises a new question. Is this a bug in the colourspace PYNQ method or is something missing in my code?  
Regards

---

<div class="post-metadata">

**Author:** ![sergicuen](https://avatars.discourse-cdn.com/v4/letter/s/a5b964/32.png) [@sergicuen](https://discuss.pynq.io/u/sergicuen)\
**Post date:** [January 25, 2024, 1:48pm UTC](https://discuss.pynq.io/t/kria-base-overlay-mipi-cam5c-how-to-change-the-colorspaceconversion/6545/3 "2024-01-25T13:48:46Z")

</div>

Hi all,  
looking at the Xilinx drivers (layer 2): [embeddedsw/XilinxProcessorIPLib/drivers/v\_csc/src/xv\_csc\_l2.c at master · Xilinx/embeddedsw · GitHub](https://github.com/Xilinx/embeddedsw/blob/master/XilinxProcessorIPLib/drivers/v_csc/src/xv_csc_l2.c)  
I have found the following:

```auto
#define XV_CSC_COEFF_FRACTIONAL_BITS (12)
...
s32 scale_factor = (1<<XV_CSC_COEFF_FRACTIONAL_BITS);
s32 bpcScale = (1<<(pixPrec-8));  
...
switch(cstdOut)
  {
    case XVIDC_BT_601:
        switch(cRangeOut)
 	{
...
          case XVIDC_CR_16_235:
              RGB2YCC[0][0] = (s32) ( 0.299*(float)scale_factor); //K11
              RGB2YCC[0][1] = (s32) ( 0.587*(float)scale_factor); //K12
              RGB2YCC[0][2] = (s32) ( 0.144*(float)scale_factor); //K13
              RGB2YCC[1][0] = (s32) (-0.172*(float)scale_factor); //K21
              RGB2YCC[1][1] = (s32) (-0.339*(float)scale_factor); //K22
              RGB2YCC[1][2] = (s32) ( 0.511*(float)scale_factor); //K23
              RGB2YCC[2][0] = (s32) ( 0.511*(float)scale_factor); //K31
              RGB2YCC[2][1] = (s32) (-0.428*(float)scale_factor); //K32
              RGB2YCC[2][2] = (s32) (-0.083*(float)scale_factor); //K33
              RGB2YCC[0][3] = 0*bpcScale; //R Offset
              RGB2YCC[1][3] = 128*bpcScale; //G Offset
              RGB2YCC[2][3] = 128*bpcScale; //B Offset
              break;

```

the scale\_factor matches with my previous findings (for pixPrec=8).

My conclusion is that there is a bug in the PYNQ ColorConverter class (pipeline.py) regarding the scale factor of both coeffs and offsets.

Note: there is a bug in the driver too. Coeff 0.144 should be 0.114 (reported in: [AMD Customer Community](https://support.xilinx.com/s/question/0D54U00007K2i8NSAR/vpss-csc-demo-window-bug?language=en_US&t=1706184168139))  
Regards

---

<div class="post-metadata">

**Author:** ![marioruiz](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.pynq.io/marioruiz/32/2177_2.png) [@marioruiz](https://discuss.pynq.io/u/marioruiz)\
**Post date:** [January 26, 2024, 9:22am UTC](https://discuss.pynq.io/t/kria-base-overlay-mipi-cam5c-how-to-change-the-colorspaceconversion/6545/4 "2024-01-26T09:22:45Z")

</div>

Hi @sergicuen,

Thank you for looking at this. I think you have found the solution. The description of the coefficients representation are here [AMD Adaptive Computing Documentation Portal](https://docs.xilinx.com/r/2.3-English/pg231-v-proc-ss/Filter-Coefficients-and-Offsets)

It would be a good idea to create a driver for this IP to handle this. You could use something like this as reference [PYNQ/pynq/lib/video/pipeline.py at master · Xilinx/PYNQ · GitHub](https://github.com/Xilinx/PYNQ/blob/master/pynq/lib/video/pipeline.py#L9)

The different color conversions are defined here [PYNQ/pynq/lib/video/common.py at master · Xilinx/PYNQ · GitHub](https://github.com/Xilinx/PYNQ/blob/master/pynq/lib/video/common.py#L73-L106)

Would you like to do a PR to the PYNQ repo with the necessary changes?

Mario

---

<div class="post-metadata">

**Author:** ![sergicuen](https://avatars.discourse-cdn.com/v4/letter/s/a5b964/32.png) [@sergicuen](https://discuss.pynq.io/u/sergicuen)\
**Post date:** [February 20, 2024, 12:16pm UTC](https://discuss.pynq.io/t/kria-base-overlay-mipi-cam5c-how-to-change-the-colorspaceconversion/6545/5 "2024-02-20T12:16:23Z")

</div>

Hi @marioruiz ,  
I’m afraid I don’t know enough python to tackle the coding of the driver.  
For the moment I leave here the workaround to get a gray image using the base overlay:

```auto
def colorspace(color):
        if len(color) != 12:
            raise ValueError("Wrong number of elements in color specification")
        for i, c in enumerate(color):
            if i<9: # write coeff
                vproc.write(0x50 + 8 * i, int(c * 4096))   
            else: # write offsets
                vproc.write(0x50 + 8 * i, int(c)) 

def configBGR2YCBCR():            
    vproc.write(0x10,0) #BGR
    vproc.write(0x18,0) #BGR
    colorspace(COLOR_IN_YCBCR_16_235)

def configBGR2BGR():            
    vproc.write(0x10,0) #BGR
    vproc.write(0x18,0) #BGR
    colorspace(COLOR_IN_BGR)

# from Xilinx drivers embeddedsw/XilinxProcessorIPLib/drivers/v_csc/src/xv_csc_l2.c
# YCBCR range 16:235
COLOR_IN_YCBCR_16_235= [0.114, 0.587, 0.299,
                  0.5, -0.331264, -0.168736,
                  -0.081312, -0.41866, 0.5,
                  0, 128, 128]

# YCBCR range 0:255
COLOR_IN_YCBCR_0_255 = [0.0979, 0.5041, 0.2568 ,
                  0.4393, -0.2910, -0.1482,
                 -0.0714, -0.3678, 0.4393, 
                  16, 128, 128]
# no color space conversion
COLOR_IN_BGR = [1, 0, 0,
                0, 1, 0,
                0, 0, 1,
                0, 0, 0]

# get the driver for v_proc_sys IP
vproc = base.mipi.v_proc_sys 
# stop DMA and configure pipeline
videomode = VideoMode(1280, 720, 8) 
mipi.stop()
mipi.configure(videomode) # configure DMA write chanel and PixelPacker (mode V_8)
configBGR2YCBCR() # configure V_proc_sys
mipi.start()

# take a frame and display it
frame = mipi.readframe()
PIL.Image.fromarray(frame)

```

---

<div class="post-metadata">

**Author:** ![juanppalacios](https://sea1.discourse-cdn.com/flex019/user_avatar/discuss.pynq.io/juanppalacios/32/5502_2.png) [@juanppalacios](https://discuss.pynq.io/u/juanppalacios)\
**Post date:** [July 18, 2024, 2:38pm UTC](https://discuss.pynq.io/t/kria-base-overlay-mipi-cam5c-how-to-change-the-colorspaceconversion/6545/6 "2024-07-18T14:38:46Z")

</div>

Hi,

Thanks for your solution, I was also able to swap the color spaces from BGR to RGB during initialization and avoid having to slice the channels once every `readframe` call saving me a few crucial milliseconds and improving fps throughput.

Regards