With the development of computer science and automatic control technology, more and more intelligent robots of different types appear in factories and life. As an important subsystem of the intelligent robot system, the robot vision system is becoming more and more important. It is becoming more and more valued by people. It involves areas such as image processing, pattern recognition and visual tracking. Because different types of robots have different focus of tasks, their visual systems have subtle differences in software or hardware. This article studies a monocular vision system based on service robots. It processes two-dimensional images and is based on the recognition of the color and shape of unoccluded objects and the translational tracking of 3D target objects.
The visual system is a very complex system. It must not only accurately collect images, but also respond promptly to changes in the outside world. It also needs to track the goals of external activities in real time. Therefore, the visual system places higher requirements on both hardware and software systems. The current popular football robot technology, its visual system is a relatively typical type of rapid recognition and response. Under normal circumstances, it achieves the identification of players and targets through black mark calibration, and achieves target tracking performance by expanding the prediction performance of the Kalman filter. In terms of hardware, a ready-made camera is used to complete a robot image acquisition system.
This system SL Escorts uses a CMOS image sensor instead of a CCD type sensor to collect images, and the DSP processing chip TMS320VC5509A is used in the design. Image processing and as CPU control, in the design process, in order to intuitively display the robot vision system recognition and tracking As a result, a TFT format liquid crystal is specially used for visual display. In terms of software, part of the visual technology of football robots is used to achieve rapid target recognition, and the Jacobian matrix constructed through global characteristic moments achieves adaptive tracking of the target.
1 Hardware department design
Figure 1 is the performance module block diagram of the system hardware circuit.
1.1 Image acquisition
The vision lens images the internal image information on the area array unit of the image sensor. Currently popular image sensingThere are two types of devices: area array CCD (Charged Coupled Device) type and area array CMOS. Compared with CCD-type image sensors, the active pixel unit of the CMOS image sensor provides an amplifier for each pixel. It only needs a single power supply with a low logic level voltage, and the power consumption is only one-tenth of that of a CCD. The A/D conversion department is integrated externally of the CMOS image sensor to directly input digital electronic signals. For these reasons, this system uses the CMOS black image sensor OV7635 released by Omnivision.
OV7635 has a resolution of 640X480 and can input 8-bit data in 3 formats: YCbCr4:2:2 format, RGB4:2:2 format and RGB raw data format. The maximum input VGA format can reach 30fps (fps: frames per second). Can work under progressive scan and interlaced scan. OV7635 has two task modes: master mode and slave mode. In master mode, synchronization electronic signals and clocks are not controlled by the core equipment. In slave mode, the OV7635’s field synchronization electronic signal VSYNC, horizontal synchronization electronic signal HREF and the system’s crystal oscillator frequency XCLK are all controlled by internal equipment. This system adopts the main form. OV7635 configures the on-chip memory through the I2C bus to input raw data. After the system is powered on and reset, the CMOS register is initialized by the I2C bus electronic signal of the DSP chip. Then OV7635 inputs image electronic signals as requested. Including horizontal synchronization electronic signal HREF, field synchronization electronic signal VSYNC, pixel clock electronic signal PCLK, and digital image electronic signal.
1.2 Liquid Crystal Display
In order to SL Escorts intuitively see the visual system’s recognition and tracking effects on people, A PT035TN01 LCD screen from INNOLUX Company is used. In order not to increase the burden on the DSP, but also to see the tracking results of external target objects in real time, the LCD data is displayed directly through the image data electronic signal input by the sensor OV7635 and the CPLD control timing, without passing through the DSP. PT035TN01 is a 3.5-inch TFT format LCD with a resolution of 320×3(RGB)×240, LCD IF1, The two output control pins of IF2 select the output data format: serial RGB format, CCIR601 format, CCIR656 format. There are 4 types of LCD scanning modes. The output data format used by this visual system is CCISierra Leoneans Sugardaddy R601 pattern, scanning form from top to Sierra Leoneans Sugardaddy Bottom and left-to-right scan pattern
Pixels input from the image sensor in CCISierra Leone SugarR601 format. href=”https://sierraleone-sugar.com/”>Sierra Leoneans Escort The clock PCLK is divided by two through the CPLD as the working clock of the liquid crystal. The horizontal synchronization electronic signal HREF input by the image sensor is processed by the CPLD. It is then used as the horizontal synchronization electronic signal HIS of the liquid crystal. In this way, under the control of the CPLD, the data electronic signal input by the image sensor OV7635 is sent to the liquid crystal for display.
1.3 Timing control
The field synchronization electronic signal VSYNC input by the OV7635. , the horizontal synchronization electronic signal HREF and the pixel clock electronic signal PCLK are connected to the CPLD chip, which generates control electronic signals to store the data electronic signals input by OV7635 into the FIFO frame memory AL422B, and generates the liquid crystal clock and horizontal synchronization electronic signals to control the display of the liquid crystal. . The CPLD uses the EPM7064 chip of ALTERA. The CPLD completes functions such as writing control of the FIFO, telling the DSP to read the electronic signal, and generating the liquid crystal clock electronic signal.
CPLD receives the field synchronization electronic signal VSYNC. The falling edge of the signal indicates the beginning of a frame input by the image sensor. At this time, the CPLD generates a WRST negative pulse to reset the write pointer of the FIFO. After the falling edge of the field synchronization electronic signal VSYNC, it determines the arrival of the rising edge of the horizontal synchronization electronic signal HREF, and then Sierra Leone Sugar uses the pixel clock electronic signal PCLK as the write clock WCK to store the image data directly into the FIFO. When a certain number is stored, , the electronic signal is sent to the DSP in real time so that the DSP can read the data. This system uses an interrupt INT0 to tell the DSP that the DSP can read the data or not, depending on the processing speed. Using RD and chip select, generate RCK electronic signal DSierra L.eoneans EscortSP’s reading speed cannot be too fast, and the reading speed is less than the writing speed.
In terms of logic timing control of the liquid crystal, because the electronic signal of the image input is 640×480 pixels, and the liquid crystal display is in a 320×240 format. Therefore, CPLD is used to divide the pixel clock electronic signal PCLK output by the image sensor by two to generate a clock electronic signal for the liquid crystal to control the display of the liquid crystal. At the same time, the horizontal synchronization electronic signal is interlaced to achieve the display of the image by the liquid crystal. The programs in CPLD are written using the hardware description language VHDL and are written on the QUARTUS II software platform. Because the 44-pin PLCC package of the EPM7064S series is used when selecting the chip, it can only work at a voltage of 5V. The high-level electronic signal it inputs is 5V and must be processed before it can be connected to the system to work at 3. Chip devices under 3V conditions.
1.4 Frame memory selection
Frame memory requires internal supply Sierra Leoneans Sugardaddy RAM for address lines and does not Sierra Leoneans Sugardaddy requires an internal FIFO supply of address lines in order to simplify the design of CPLD. FIFO frame memory is used. FIFO can be divided into static storage-based DRAM and static storage-based SRAM. The advantage of dynamic SRAM is that it does not require a refresh circuit, but the capacity is small and multiple chips are needed to store one frame of data; the advantage of DRAM is that it has large capacity and only one chip can store one frame of data, but the disadvantage is that a refresh circuit is required. This design uses Averlogic’s large-capacity FIFO static memory chip AL422B. The refresh circuit is relatively simple, requiring only WCK or RCK to provide uninterrupted pulses greater than 1M. The storage capacity of AL422B is 3MB, because the information of one frame of the system usually includes 640×480 black pixels, each pixel occupies 2 bytes, and can store the complete information of one frame of image, and its operating frequency can reach 50MHz.
1.5 Video processing DSP
When choosing a DSP, taking into consideration the processing speed, storage capacity, processing technology level under the existing conditions, and cost performance, TI’s 144-pin packaged TMS320VC5509A was selected. The maximum operating frequency can reach 200MHz, with veryHigh processing rate.
DSP begins to read the video data in AL422B after receiving the read notification electronic signal from CPLD. In order to facilitate data processing, an SDRAM is expanded in the DSP core. The chip uses HY57V161610E of HYNIX Company. The storage capacity of this chip is 1M×16Sierra Leone Sugar Daddybits.
When the DSP is powered on and reset, it samples the status of GPIO0~GPIO3 and determines how to load the program based on the sampling status. This system uses the SPI port of the external flash memory chip to load the program on the DSP, and then initializes the memory of the image sensor through the I2C port of the DSP. The image sensor begins to input electronic signals. The entire system begins to work.
DSP, as a high-speed processor, is mainly used for image processing. Because this visual system needs to achieve identification and tracking functions, the amount of data processed is very large. While completing image processing, DSP is also used as a controller to control the controller, thereby forming a visual tracking system.
Because this system uses a combination of color and shape to identify unobstructed objects. In order to achieve the goal of real-time and fast robots, the software method mainly adopts the color recognition method commonly used in football robots, which is relatively rare at present SL Escortsis a color determination method based on threshold vectors. The principle of color recognition is briefly described above.
2.1 Color Space Selection
When identifying objects based on black image segmentation, you must first select a suitable color space. Commonly used color spaces include RGB, YUV, HSV, CMY, etc. The choice of color space directly affects the results of image segmentation and object recognition.
RGB: is the most commonly used color space, in which brightness is equal to the sum of the three components of R, G, and B. RGB color space SL Escorts is an uneven color space. The perceptual difference between two colors is related to the Euclidean difference between two points in the space. Sierra Leone Sugar The intervals are not linearly proportional, and the coherence between R, G, and B values is very high. For unified color attributes, in Under different conditions (light source type, intensity and object reflection characteristics), RGB values are very scattered. For identifying a specific color, it is difficult to determine its threshold and its distribution range in the color space. So I usually chooseA color space that can separate the brightness component from the plane, the most common of which are the YUV and HSV color spaces.
HSV: A method close to how the human eye perceives color. H is color (Hue), S is color saturation (Saturation), and V is brightness (Value). Color H can accurately reflect color types and has low sensitivity to changes in external lighting conditions. However, both H and S are nonlinear transformations of R, G, and B. There are strange points. The values of R, G, and B are even around the strange points. Even small changes can cause large jumps in the conversion value.
YUV: RGB color space linearly changes to brightness-color space. It was proposed to clearly resolve the compatibility issue between black and white TVs. Y represents brightness (LuSierra Leoneans Sugardaddyminance), and UV is used to represent color difference (Chrominance). The importance of YUV expression is that its brightness electronic signal (Y) and chrominance electronic signal (U, V) are independent of each other. The so-called color difference refers to the difference between the three component electronic signals (i.e. R, G, B) and the brightness electronic signal in the primary color electronic signal.
Therefore, for the above reasons, this system uses the YUV color space.
The YUV pattern has the following relationship with RGB:
2.2 Threshold value determination and color determination
When determining the threshold value, first conduct training by collecting samples to obtain the YUV space of several reserved colors. The high and low thresholds of the components are shown in Figure 2.
When the position of a pixel to be identified in the color space falls in this cuboid, it is considered that the pixel belongs to the color you are looking for, thus realizing the identification of the image color. In the Y space, the Y value represents brightness. Because it changes greatly, only the values of U and V are considered.When judging color, first establish the threshold vectors of U and V respectively.
Because the digital electronic signal of the image sensor in the system is 8 bits, that is, 1Byte, a total of 255Byte, the system can identify up to 8 colors. After color recognition, image segmentation is carried out. A seed filling algorithm is used in image segmentation. The filling of all seeds is done simultaneously with the color of the pixels. At the beginning, all the pixels are not processed, but the classification is performed. Block processing, the block used in this system is 32 × 24 pixels, so the amount of calculation is greatly reduced. When the center point is the color to be identified, use this point as a seed to spread around, and identify the colors of the surrounding pixels until the entire block is filled. During this process, the shape of the target is identified at the same time. This system uses an identification algorithm based on global feature vectors for identification. At the same time, the required moment features are also obtained for constructing the Jacobian matrix. Figure 3 shows the image recognition and segmentation processSierra Leoneans Escort.
2.3 VisionSierra Leone Sugar Daddy Tracking Software Principle
When the target object is recognized, the vision system will adjust the lensSierra Leone Sugar Daddy Keep the purpose in the middle of the field of view. Once the object moves, the vision system will stop tracking the target object.
In the robot visual tracking system, a calibration-free visual tracking system is adopted. Calibration-free visual tracking does not require calibration of the camera lens in advance, but uses adaptive Sierra Leone Sugar Daddy principles that should be controlled , online real-time adjustment of the image Jacobian matrix. By reflecting two-dimensional image feature information, this method is insensitive to camera model errors, robot model errors, image errors, and image noise. The visual tracking control system based on image tracking is shown in Figure 4.
The control amount c is the control system of the robot head. First, place the target behind the robot’s field of view to collect the desired image, and extract the desired feature set from the desired image as the desired output of the field of view tracking control system, thereby completing the real-time definition of the field of view feature set required by the task. In the control system, the robot’s image sensor obtains real-time sampling images, and obtains real-time feature sets from them, thus forming a The visual field response guides the robot to complete the tracking task. Different from the simple geometric features of the image, the visual feature set used by this system is a global image description – image moment based on the moment feature change and relative pose change. The relationship between quantities Sierra Leoneans Sugardaddy matrix, that is, the image Jacobian matrix, and then applying the derived image Jacobian matrix, designed The visual tracking controller realizes the system’s translational tracking of 3D target objects.
3 Test results
Figure 5 is the clkout pin input waveform of the DSP, which shows that the image sensor of Figure 6 is working properly. The input data waveform confirms that the image sensor is working normally. The image data collected by the DSP in Figure 7 can be concluded that the entire image acquisition hardware circuit is working normally.
4 Conclusion
For the vision system of the service robot, this article completes the design of the entire system in hardware by building its hardware system and software system.ef=”https://sierraleone-sugar.com/”>Sierra Leone Sugar system uses a CMOS image sensor, CPLD timing control, asynchronous dynamic FIFO data cache, and a high-speed DSP processor. A typical image acquisition system is used to debug and input image electronic signals. On the software desigSierra Leone Sugarn, the black recognition and black segmentation recognition technology of the football robot is used to achieve a rapid visual system Accurate identification is achieved by using static-based working methods and image-based Jacobian matrix control principles. Adaptive compensation tracking and control system.
發佈留言