Ben's Blog

No description yet.

  • Home
    Home This is where you can find all the blog posts throughout the site.
  • Categories
    Categories Displays a list of categories from this blog.
  • Tags
    Tags Displays a list of tags that have been used in the blog.
  • Archives
    Archives Contains a list of blog posts that were created previously.
  • Login
    Login Login form

Computer Vision (2) - Investigating using the Raspberry Pi camera module with FPGA vision systems, Part 2

Posted by on in Computer Vision In Embedded Systems
  • Font size: Larger Smaller
  • 0 Comments

 The Raspberry Pi camera module or more specifically the OV5647 image sensor consists of a CSI-2 image sensor interface. Quite fortunately, as the OV5647 can be found in many image sensing devices apart from the Raspberry Pi camera, like mobile phones for example, plenty of information can be garnished about this interface from the Internet. The CSI-2 is a standard interface or Camera Sensor Interface (CSI) defined by the Mobile Industry Processor Interface Alliance, usually referred to by their four letter acronym, MIPI.

This blog post is a brain dump of the continuation of the idea of interfacing the OV5647 image interface to FPGA boards and other embedded systems like the Parallella board containing a Xilinx Zynq FPGA and the Epiphany 16-core processor. We will also like to interface this sensor to Altera FPGAs like the SocKit, consisting of the Cyclone IV FPGA and Arrows family of BeMicro evaluation kits, which contain the Cyclone V FPGA. Support should also be provided for the Beagle Bone (black) and Intel's Galileo and possibly Edison development boards.

As eloquently declared, previously, the primary aim of the set of posts in this category is  to use this cheap and readily available camera board for custom embedded computer vision systems projects, when interfaced to OpenCV for example. In this article we will be concentrating on the characteristics of the CSI2 interface. In a not to future article we will consider the characteristics of the OV5647, which are listed in the Table below. In particular we will investigate the optics of the device to determine how their suitablility in stereo vision systems, which will be useful in our 17 DOF robotics project.

Item Value
Dimensions 25mm x 20mm x 9mm
Weight 3g
Sensor size 3.67 x 2.74 mm
Optical Format 1/4"
Pixel size 1.4 x 1.4 um
Pixel count 2592 x 1944
Lens f=3.6 mm f/2.9
Angle of View 54 x 41 degrees
Field of View 2.0 x 1.33 m at 2 m

Table. The OV5647 Camera Sensor Specifications.

a1sx2_Original1_002-002-001-000053a.png

In a future article we will also explore the CS-Mount variant of the OV5647 sensor module, some of which we already have in our possession, to explore the field of view of multiple Raspberry Pi camera arrays and high performance imagining.  Now, we shall concentrate on the CSI-2 MiPI Interface.

 The Raspberry Pi camera module or the OV5647 sensor module (we will be using the two names interchangeably), apart from having an I2C control interface also consists of the CSI-2 interface. This interface consists of two unidirectional, differential data lines and a unidirectional, differential clock line. In order for the module to achieve the extremely high data rates required to transmit 1080p video at say 30 fps, the differential interface can operate at up to 800 Mbps, using the sub-LVDS differential signalling standard. 

002-002-001-000054.png

We are not going to discuss the intricacies of sub-LVDS signalling here, this is left as an exercise to the reader who may wish to become an expert on such matters. However, as seen in the Figure above, which will be explained presently, an LVDS receiver is compatible with a sub-LVDS transmitter. This is excellent news, as it means that we can readily interface any OV5647 module to a FPGA development board or kit that supports the LVDS signalling standard and most do, either directly or through emulation.

 The news gets even better when we also appreciate that an OV5647 module's differential CSI interface is unidirectional. Hence, to interface the module to a FPGA we only need to be concerned with the camera module's transport layer protocol and implementing a MIPI D-PHY physical layer. Well, it was too good to be true wasn't it! Especially, when we consider that to understand the module's, data transport and physical layers we are likely to require a sub-LVDS transmitter IP module. Crumbs!

Hence, as we stand here are the issues that we are currently solving to use the OV5647 camera module in our vision projects. We need to interface the module to a LVDS compatible device, which we have established can be readily done, using our FPGA boards and development kits that support the standard. To implement the physical link layer between the module and our FPGA kit we need to implement a MIPI D-PHY, which is the physical interface specification of the CSI-2 standard. 

Most FPGA vendors provide a reference to their emulated D-PHY interfaces (see references below), which are  typically external TX and RX sub-LVDS interfaces, as well as internal deserialisers to create byte sized packets of the incoming data or serialisers to output byte sized packets of data a bit at a time. Some of the characteristics of the MIPI D-PHY are listed in the Table, below. All of these characteristics, listed in the table, are readily achievable with the FPGAs on the market today.

Characteristic Value
Data Rate Per Lane:  
Maximal High Speed Rate 1 Gb/s
Minimal High Speed Rate 80 Mb/s
Low Power rate ≤ 10 Mb/s
Electrical Signalling:  
High Speed (HS) Sub-LVDS
Low Power (LP) LVCMOS-1.2V
HS Clocking Method DDR source synchronous
HS Line Coding None or 8B9B

Where things begin to become a bit tricky is understanding the data protocol of the OV5647 module. To understand the protocol we will need to use the FPGA development kit as a pass-through module. This means that apart from interfacing the camera's sub-LVDS transmitter to the FPGA's LVDS receiver, the FPGA must also provide the ability to implement a sub-LVDS transmitter to interface to what the camera module would have interfaced to, like a Raspberry Pi for example. 

Hence, bearing in mind the interfaces and protocols listed above, without creating a custom FPGA development board, it has not been easy finding an existing FPGA development board that implements sub-LVDS or sub-LVDS and LVDS IO interface standards along with a LVTTL interface for the I2C controller.  To debug the camera module, without developing a custom FPGA board, a perfect companion for this phase of this project could be Lattice Semiconductor's MachX03L breakout board, pictured below. 

002-002-001-000054a.png

 So far, by studying the user manual and schematics of this development board we have been able to tick all the right boxes. It seems what should be required is a board that acts as an intermediary between this board and the FPGA development board that we wish to interface to.  An example of the schema we should use can be seen in the block diagram below.

002-002-001-000054b.png 

I'll delve in more in the next post how this board and the two custom boards pictured either side of it,will be used, as a flexible OV5647 camera debug module in the next post. I'm a bit pushed for time this week, but more soon.

References

  1. XAPP894, D-PHY Solutions, Xilinx, v1.0, August 25, 2014.
  2. TN1253, Using Differential I/O (LVDS, Sub-LVDS) in iCE40 LP/HX Devices, Lattice Semiconductor, October 2013.
  3. TN1210, Sub-LVDS Signalling Using Lattice Devices, Lattice Semiconductor, March 2014. 
  4. RD1182, MIPI D-PHY Interface IP,  Lattice Semiconductor, January 2014. 
  5. MachX03L Breakout Evaluation Kit User Guide, Lattice Semiconductor, August 2014. 
Last modified on
0