"1. Introduction
For Chinese people who value family affection and friendship, videophone is a long-awaited means of communication for ordinary people. Videophone is a product integrating communication technology, computer technology and audio and video processing technology. It is the development direction of information technology. At present, there are two types of videophones: one is based on PC; The other is an integrated videophone separated from PC; From the signal transmission line, there are mainly broadband videophone based on ISDN and ADSL and narrowband videophone based on PSTN. PSTN has been developed in China for decades. It is the communication network with the widest coverage, the most economic and the widest mass base. Therefore, it is still of great practical significance to realize a simple and easy to popularize videophone on PSTN network.
The standard of PSTN based videophone in China is formulated according to ITU-T H.324. At present, all PSTN videophones on the market basically use MCU + special ASIC chip to realize audio and video coding and protocol control required by H.324 protocol, or the scheme of processor + DSP architecture, and the system composition is complex. This paper presents a new videophone based on RISC architecture embedded platform, and realizes the wireless videophone with simple system structure. The scheme can develop an independent videophone terminal and can be used in communication, monitoring and other fields.
2.h.324 protocol introduction
Figure-1 is the block diagram of H.324 protocol, and the box in the figure is the recommended scope of H.324 protocol. The proposed contents mainly include h.263/h.261 video codec protocol, G.723 audio codec protocol, V.14 data protocol, H.223 multiplexing / demultiplexing protocol, H.245 control protocol and modem (control protocol). See reference [1]
Figure-1 H.324 protocol block diagram
The transmission of low bit rate audio and video information on PSTN network must be compressed and encoded. In H.324 protocol, audio and video coding and decoding is one of the core contents. H.263 is defined by ITU and supports video coding and decoding for video conference and video telephony applications. H. 263 is developed based on H.261 coding, which is especially set for low bit rate video coding and can transmit video streams with a minimum bandwidth of 20K to 24kbit / sec. G. 723.1 is a dual rate voice codec standard launched by ITU for voice signals with telephone bandwidth. It can transmit audio streams at a minimum rate of 5.3kbps.
3. Hardware platform composition of s3c2440a system based on ARM processor
Figure 2 system hardware architecture
The overall hardware architecture is shown in figure-2.
(1) The core of the system is ARM processor S3C2440 based on RISC architecture, which is a 16 / 32-bit RISC small high-performance microprocessor launched by Samsung Electronics of Korea. The ARM920T core with a speed up to 533MHz can provide various functions for ordinary system peripherals, so there is no need to add additional devices. While reducing the complexity of the system, it provides end users with camera interface, TFT and STN LCD functions, SD / MMC / SDIO card slots and other additional devices. The processor has powerful performance and can meet the operation requirements of h.263 codec and G.723.1 codec. See references [9].
(2) The video acquisition part of the system adopts 300000 pixel CMOS camera ov9650esl. The device can collect 1280 * 1024 pixel array at most, which is enough to meet the needs of the system. S3c2440a proprietary camera controller is used to collect video data, and IIC bus is used to configure exposure time, white balance and other parameters.
(3) The display part of the system: LG's 3.5-inch 65K color TFT LCD screen with a resolution of 320 * 240 is adopted to meet the system requirements. In this design, the LCD data interface of the processor is used to configure the display format of the LCD screen, SDRAM is used as the display storage area, and the s3c2440a integrated LCD controller is used to output and display the LCD screen.
(4) Audio input and output part of the system: codec chip uda1341 is adopted. ADC and DAC are integrated in the audio chip, and the audio acquisition and playback function of the system is composed by connecting mic and speaker. The chip transmits data with CPU through IIS bus, and can set its acquisition rate and gain through l3-bus.
(5) Interface with PSTN network: in order to realize data transmission on PSTN network, CX06833 of Conexant is adopted in accordance with H.324 protocol. The chip is connected with CPU through UART interface and provides PSTN network interface. See reference [10]
(6) System storage part: SDRAM with model is42s32200 is used as the memory, and the capacity of the chip is 8mbyte, which is used to run the software system. Sst39vf320x is used for program storage. The chip is nor flash for program storage, and the capacity is 4mbyte
(7) Accessories: including 4 * 4 keyboard, working status display LED, etc.
4. Implementation of system software
The flow of the system is shown in figure-3.
Figure-3 system software flow
After the hardware system is powered on, the software initializes the hardware and enters the standby state. When off hook, the CPU performs analog call through the analog telephone control built in the modem. Both sides of the call set up the visual communication system according to the H.245 protocol and started to set up the visual communication system.
The system collects video through CMOS camera, opens up a memory in SDRAM for video content cache, and the H.263 coding part takes out the image from the cache according to a certain frame rate for coding. The CPU controls the codec chip to collect voice through mic. Similarly, a buffer for storing audio is opened in SDRAM, and G.723.1 is encoded into a code stream.
H. 223 multiplex the video code stream, audio code stream and control data code stream, and modulate them to the PSTN network through modem for analog line transmission.
The data code stream obtained from modem is decompound into audio and video code streams. H. After the 263 code stream is decoded, the video memory is refreshed at a fixed frame rate. Arm transmits the data in the video memory to the 4-inch 320 * 240 LCD through the built-in TFT controller. G. 723.1 code flows through the G.723.1 decoder, decodes it, outputs it to the codec chip, and pushes the speaker to play through the power amplifier.
4.1 initialization of system hardware
S3c2440 contains 4K SRAM named stepping stone, which can be used as bootloader for system initialization. When the system is powered on and started, read the initialization program segment in flash into SRAM to initialize the clock inside CPU, SDRAM controller, watchdog, interrupt, etc. And initialize the memory system, move the program in flash to SDRAM, and the main program of the system runs in SRAM. After entering the main program, initialize GPIO, LCD screen, camera, keyboard and other peripherals. After initialization, the system enters standby and waits for off hook.
4.2 h.263 codec module
How to realize h.263 codec is one of the keys to realize visual call in RISC embedded system. H. 263 protocol is a mature video coding and compression protocol, as shown in figure-4. In this paper, H.263 protocol is not described in detail, but only how to implement it in the system based on ARM processor. For details, see reference [2]
4.2.1 H.263 code
Figure-4 H.263 coding block diagram
As shown in figure-4, the H.263 encoder is mainly composed of four parts: transform coding T, quantization Q, prediction P, entropy coding VLC; For the practical coding system, code stream control CC is also an essential part. There are two single pole two position switches in the figure, which are used to select intra and Inter. Judge the type of the current frame. After judgment, it is divided into branches of intra and inter for processing respectively.
In this system, the process of H.263 encoder is:
In the function processing layer, the processing function video is called after the H.263 coding task is issued_ Encode, read out a frame of image from memory and judge the type of current frame. As shown in figure-5.
Figure-5 H.263 coding software flow
1) Intra frame. At this time, turn the switch to the upper switch. The input of the current block (currently processed video block) after video frame blocking is transformed and encoded, quantized, entropy encoded and output as a code stream; The quantization result is stored as a reference frame after inverse quantization and inverse transformation. Finally, the reconstructed image is called deblocking filter.
The function codeoneintra() encodes the current frame according to I frame. The input of the function is:
H263ENCODE_ Memory * MEMS encoded information structure pointer.
H.263ENCODE_ MEMORY{
Reconstructed frame structure pointer, reference frame structure pointer, reconstructed macroblock structure pointer, reference macroblock structure pointer, decoded image parameter structure pointer, decoded coefficient, macroblock parameter structure pointer, decoding process structure pointer}
2) Inter frame condition. At this time, turn the switch to the lower switch. The current block after the video frame is divided into blocks, firstly, the motion estimation is carried out in the search area of the reference frame to obtain the prediction block (the block closest to the current block in the reference frame), and the motion vector is obtained. The motion vector entropy coding is output as the code stream. The difference image is obtained by subtracting the prediction block from the current block. The difference image is transformed and encoded, quantized, entropy encoded and output as a code stream.
4.2.1 H.263 decoding
Figure 6 below is the system block diagram of H.263 decoder:
Figure-6 block diagram of H.263 decoder
As shown in figure-6, the H.263 decoder corresponds to the encoder and also has four main parts: inverse entropy coding, inverse quantization (scale), inverse DCT (IDCT) and motion estimation. Figure 2 briefly illustrates the decoding process of inter frame, that is, the results after IDCT are added to the motion estimation results, but they do not need to be added to the motion estimation prediction results in intra frame. It can be seen that the decoder is simpler than the encoder.
H. 263 decoding process:
1) In the case of intra frame, the intra digital stream in the code stream is de entropy coded and de quantized, and IDCT completes the decoding of video frame to obtain video data output; 2) In the case of inter frame, the difference is subjected to inverse entropy coding, inverse quantization and IDCT. The obtained results are added to the motion estimation prediction block to complete motion compensation, and then the video data output is obtained; 4.3 G.723.1 encoding and decoding.
Another key of the system is to realize the codec of G.723.1 on the embedded architecture. The standard can sample the input analog speech signal with 8kHz and compress the speech signal quantized by 16bit linear PCM into a bit stream of 6.3kbps or 5.3kbps. G. 723.1 can be selected according to different requirements. The high bit rate is 6.3kb/s. The multi pulse maximum likelihood quantization (MP-MLQ) coding algorithm is adopted, which has high reconstructed speech quality; The low bit rate is 5.3kb/s. Based on ACELP algorithm, the computational complexity is low. In this system, ACELP algorithm is used for low bit rate speech compression. Due to space limitations, this paper will not elaborate here. Please refer to references [6] and [8]
4.4 implementation of H.245 protocol
H. The full name of 245 protocol is multimedia communication control protocol. It gives a set of syntax and semantics used to describe terminal information, bandwidth negotiation and other food messages. It mainly provides end-to-end signaling function, so that both communication parties can correctly interconnect. H. 245 signaling controls the operation of H.324 system, including capability exchange, opening and closing of logical channels, mode priority request, multiplexing table item transmission, flow control messages, general commands and instructions. See reference [4]
When the system is implemented, the SRP simple retransmission protocol is used to provide reliable transmission of H.245 code stream. A transmission mark is adopted. When a command frame is sent, the mark is activated. At this time, the subsequent command frames are not sent. When the corresponding frame is received, the mark is cleared and the next command frame is sent. If no response frame is received during the timeout period, the current frame is retransmitted; If the response frame is not received after multiple retransmissions, the communication is terminated and the application communication error is reported. The whole process is controlled by arm.
4.5 implementation of H.223 protocol
H. 223 protocol is a low bit rate multimedia communication multiplexing protocol. It is divided into multiplexing layer (MUX) and adaptation layer (AL). Al is responsible for converting different data streams obtained from user layer and IO layer. Al is divided into al1, al2 and Al3 layers to transmit data and control code stream, audio code stream and video code stream respectively. MUX multiplexes the code stream of Al according to the multiplexing table. The protocol is completed by arm. See reference [5] for details
5 system implementation results
After establishing a stable analog call, the communication parties press the visual key to establish a connection. You can choose the quality of the video of the call. Under QCIF (176x144), the video can stably reach the rate of 10 frames, the characters are clear, the lip shape is synchronized, and the synchronization between the moving image and the accompanying sound is good.
6 Conclusion
This paper introduces an embedded terminal platform based on pure RISC architecture. On this platform, it complies with H.324 protocol and well realizes the function of videophone. Under QCIF resolution, it can stably reach the rate of 10 frames, clear voice and meet the requirements of video call. The structure of the system platform is simple, and it is easy to expand and intelligentize on this platform., Read the full text“
Our other product:
