The Raspberry Pi camera module board (Rev 1.3) has been identified by the hobbyist embedded community as an inexpensive camera that would be ideal for computer vision projects. However, the camera may not be usable outside of the Raspberry Pi community, due to the closed source nature of its interface's timing and data characteristics.
This means that to understand how it works, in order to use it in our hobbyist computer vision projects, we may need to hook it up to one of our FPGA boards and perform some "creative" engineering. This should provide some insight into how it operates, which should be right up our alley with us being FPGA "experts" and all. Hence, in this blog post we investigate what would be required to use the Raspberry Pi camera module with our FPGA development kits and projects.
Figure: The Raspberry Pi Camera Module Board.
If there is one thing embedded computer hobbyist love is any off-the-shelf product that can be usurped for use in their own projects, especially when it is associated with the word inexpensive. This is especially the case when the so called cheap product, which in this case is the Raspberry Pi camera module board, punches way above its weight. This camera offers a lot, is well built and as mentioned previously is relatively cheap. This is not unexpected of the Raspberry Pi Foundation who are becoming synonymous with good quality embedded products at affordable prices.
The camera module board, which consists of OmniVision's 1/4" OV5647 chip, in question has an impressive list of attributes including:
- A 5 Mega Pixel CMOS camera, multiple video capture resolutions from 640 x 480 pixels at 60 Frames per Second (FPS) to 1080p video capture at 30 FPS.
- A still image capture mode with a maximum resolution of 2592 x 1944 pixels.
- Small size at 25 x 24 x 9mm3.
When these characteristics are combined with an embedded number crunching platform, like the processing power offered by FPGAs, the camera module board should be ideal for a variety of computer vision projects including stereo vision and motion tracking. However, the only caveat to using this camera in non-Raspberry Pi baseboard projects is the camera's Camera Specification Interface (CSI). This interface, which is not openly available, is controlled by the MIPI Alliance who request a joining fee that may not be compatible with hobbyist projects (N.B It has been reported that the organisation offer a reduced joining fee for small organisations/businesses/hobbyists, however we have not investigated this ourselves).
Figure : Raspberry Pi Camera Module Board MIPI Alliance Camera Serial Interface.
Essentially, the camera could be considered to have two separate interfaces on its 15-pin ribbon cable, a CSI-2 interface that transports the image data from the camera to the receiving device and an I²C interface that allows the user to configure the camera and tell it what to do. The control interface (referred as CCI) is a bi-directional control interface compatible with the I²C standard. I²C is a fairly well understood interface, is proprietary to Philips Semiconductors (NXP Semiconductors) and since 2006 implementers of the I²C protocol have not been pursued over licensing fees. Now, although I²C is easy to implement in an FPGA, the CSI-2 interface is a different kettle of fish all together.
The camera module's CSI-2 interface, which consists of two sub-LVDS differential pair data lines and a sub-LVDS differential clock, as mentioned previously has a closed source interface. This interface was developed to provide the mobile industry a standard, robust, scalable, low-power, high-speed, cost-effective interface that supports a wide range of imaging solutions for mobile devices (source: MIPI Alliance). The CSI-2 specification defines standard data transmission and control interfaces between the Raspberry Pi camera module board's transmitter and a FPGA development kit receiver. The CSI-2 data transmission interface that consists of the unidirectional differential data and clock signals, described previously, has a physical layer compatible with the MIPI Alliance Standard for D-PHY.
Figure: The camera module board is also available with its Infrared (IR) filter removed. Known as the Pi Noir camera board, it has been used in photo astronomy.
Now, although this proprietary interface sounds daunting, when one is as "comfortable" with FPGA's as we are, anything is possible. Debugging this interface should be possible because of the multitude of I/Os standards supported by FPGAs. So can an FPGA be used to decode the Raspberry Pi's camera module for use in non-pi related projects? The board I propose for this purpose can be seen, in the Figure, below. So the answer is YES. This proposed debug board in conjunction with one of the few accumulated Altera FPGA development boards, which have a HSMC interface, would be ideal for the job.
These boards include the Cyclone V Soc kit, the Cyclone V GX Starter kit, Cyclone III FPGA Starter kit (Review link), etc. The DE0-Nano FPGA Development Board (Review link), which will require a different interface may also be used if the design of the debug board is altered slightly. The Parallela Kit (Review link) may also be useful for this exercise. I'm quite looking forward to investigating using, in this camera project, Altera's new Arria 10 family of devices with their hard floating-point single precision adders and multipliers.
Figure: FPGA Debug Board. Proposed Raspberry Pi Camera Module Debug for interfacing the camera to FPGA boards.
In the next part of this computer vision blog post we will look at the requirements of a sub-LVDS receiver interface and how it can be typically implemented in an FPGA. We will also accumulate ideas for developing the Raspberry Pi Camera Module debug environment for FPGAs. The practical aspects of developing the PCB shown, in the Figure, above will also be progressed.