Daily Snippet

• Wrote a code to test the UART on hardware
• Wasted a lot of time trying to get the driver running. Shenki changed the source code to make it compatible with latest kernel but my terminal hangs when I install the drivers.
• Finally tested the UART using Windows. Working!
• Tested the code for different baud rates. Working for 19200.

Tomorrow:

• Integrate the code with HDMI2USB and test it.
• If things work, write documentation and give to HDMI2USB community to test.

Weekend Review

Work done in last week:

• Investigated the effect of encoding quality.
• Investigated the effect of post jpeg stages
• Changed FSM of JPEG core
• Underclocking of Jpeg core
• Studied Chroma Subsampling
• Wrote code of UART (alpha stage)

Work to be done this week:

• Complete UART implementation with documentation on how to use it so that it can be used by other developers.(2-3 days)
•  Then try one of these: 
• Investigate performance of pre-jpeg blocks.
• Write a test bench to simulate the HDMI2USB firmware.(Will take a week or two).
• Implement chroma subsampling 4:2:0.

Snippet

 Today
[Code] UART done with Testbench

Tomorrow:

Test it on hardware

FSM Changed, FPS Did NOT!

I changed the FSM which controls the JPEG encoder yesterday. I initially went in the wrong direction as I understood the OPB protocol incorrectly. When I finally understood the problem, the implementations was of few lines. Earlier quantisation tables were written every time a new frame was loaded. Now it will change only when encoding quality changes. So it saves 1024 clock cycles per frame if encoding quality does not change. It took me several attempts to get it working. But FPS didn't change. It will probably make a difference when the fps is very high, so the clock cycle saved per frame will be significant.

Also, I tried underclocking. I changed the clock divider of jpeg clock(freq=625Mhz/clock divider) from 7 to 11. FPS did not change for 7,8,9. But I got timeout error for 10 and 11 which I guess means FPS was too low. I honestly don't know what to make out of this.

Also there is a bug. I have no idea about the bug. Here is the video.
 bug video
In the video, things worked after two attempts but it can take more. .

Today I was supposed to start changes chroma subsampling of the encoder but I guess adding UART functionality would be better as it will help me and other developers to debug easily. So I will spend the day trying to get the UART working on the board.

Snippet:
Work done- Changed FSM, Played with underclocking, Studied chroma subsamling
Work to be done today-
- Implement UART on board

Yesterday probably I worked the longest (10 hrs maybe). I was trying to check whether the reason for slow fps is blocks after jpeg core. To check this I removed the the signals which the usb controller gave to the jpeg core. Also I wrote a code which counts the done signal of the jpeg core and I output the range of the count using LEDs(like for count less than 10 LED0 would light up). And it turns out that count of the done signal was in same range as the output fps. I tried different combinations like different encoding quality, allowing some signals of usb controller to control the jpeg core while ignoring other and the result was more or less the same. Meanwhile the code threw a timing error which took me 2 hrs to debug. 

 Also I did some reading on quantisation tables for JPEG. Turns out that quantisation should be low for luminance table (quality-85%) and high for chrominance of around 50%. But this is a general rule and cannot be used for all applications.

Today I plan to change the jpeg top module and see whether it improves fps. I have few ideas.

Finally the FPS changes!

In my previous post I mentioned how it appears that JPEG encoder is working at 100% encoding quality. Turns out it is true. I hardcoded to make the core run at 50% quality and 100% quality and there was a clear change in the fps. At 100%, the fps was same as the normal HDMI2USB frame rate. And at 50%, framerate jumps to around 20 fps.
Now 100% quality encoding is not good.Why? In the quantisation step, a each sample of a 8x8 block is divided by a number which depends on encoding quality and frequency component the sample. Human eye is sensitive to low frequency so in quantisation process, low frequencies are preserved and high frequencies are suppressed. But when for 100% quality all components are preserved, so the output file is large but for the human eye it looks as good as say 75% quality.
Plus at 100% quality, the clock cycles spent on quantisation are waste as we are basically dividing by 1.  

 (^100%-11.50 fps)

(^50%-19 fps)

I will look up some material on chrominance and luminance tables and see what is a good trade off between quality and frame rate.

Today I tried to change the encoding quality manually. The reason for this is that if the jpeg core is running at 100% quality then the quantisation step of JPEG encoding is useless. For that I changed the code of JPEG top and used switched in board to select the encoding quality. Unfortunately after flashing the firmware, mplayer and guvcviewer could not play the HDMI input. Things are going slow because it takes around 15 mins in my system to generate the bitstream. I will try to get things working tomorrow too. If things don't work, I will reach out for help.