"The implementation method of videophone system in H.324 standard generally has the following same methods: software mode, general DSP mode, special processor and special chip mode.
Taking PC as the platform, using the existing sound card, video collection card and modem as the input and output equipment, and adopting the multithreading technology software based on windows to realize multiple tasks such as H.263 video decoder, decoding / sending, receiving and display. In the practical application of the system, the processing capacity of PC, the connection with the receiving end and the corresponding image format need to be considered. Assembly language can be used in the coding module to improve the execution speed, and MMX technology can be introduced to further improve the execution speed. It is characterized by low cost, less configuration and easy to transplant and upgrade. However, it has high requirements for computer processing capacity and encoding and decoding algorithm speed, which is generally difficult to meet the real-time requirements.
Special processor and special chip are another hardware processing method. Its advantages are high integration, low price, but poor scalability.
General DSP is a hardware implementation method. The reason for using general DSP is that it can realize a variety of functions through programming. When the software function for hardware is upgraded and new standard algorithms appear, it only needs to locally change the microcode of DPS to adapt to different applications. When implementing H.263 with DSP, it is first necessary to estimate the amount of calculation required for encoding, decoding and other processing processes, and these calculations are related to the specific video coding process. High speed DSP should be selected.
From the above three implementation methods, DSP has the advantages of flexible development, wide adaptability and strong scalability. At present, the DSP chips with high-speed and super processing ability that can process H.263 algorithm in real time include ad21065l, TMS320C55x and TMS320C6x. Therefore, using DSP to realize video image processing systems such as videophone has become the first choice for developers.
1 H.324 basic framework of videophone
H. 324 is the ITU framework standard for low bit rate multimedia communication terminals suitable for public telephone switching network (PSTN). It uses a V.34 modem and can transmit voice, data, video images or a combination of the three in real time. H. 324 includes H.263 recommendation for very low bit rate video coding, G.723 recommendation for audio coding, control protocol H.245, multiplexing and multiplexing protocol H.223, etc.
The video H.324 videophone block diagram of PSTN is shown in Figure 1.
In the block diagram of H.324 codec system, video coding H.263 provides a transmission rate lower than 64Kbps, which is the technical core of H.324. The video encoder mainly completes the compression of the source image sequence. First, DCT transform is adopted in block mode, then the transformed DCT coefficients are quantized, and then enter the video multiplex encoder. In the process of coding, the selection of coding mode and the size of code rate are required to be controlled; Video multiplex coding is mainly to compile each frame of image data into four levels of data structure, so as to insert necessary auxiliary data information in each level. At the same time, the AC DCT coefficient (AC) is VLC coded, the DC coefficient is FLC coded, and the compressed coded data is multiplexed with the control information. Because the output bit rate of H.263 is generally not constant, in practical application, due to the limitation of transmission network bandwidth, buffers must be set at the transmitting end and receiving end to transform the code stream with coding rate into a fixed rate code stream to prevent data loss and destruction. The information of the buffer is transmitted to the coding controller, which controls the quantization step in the encoder, and sends the step auxiliary information to all levels of video multiplex coding for use by the decoder. The decoding part can be regarded as the inverse process of coding, but it is not completely symmetrical. The computation and complexity of the encoding process is much larger than that of the decoding process.
2 performance requirements of videophone system
Before the design of videophone, the image format, video and audio codec operation and storage of videophone are analyzed.
For the videophone on PSTN, due to the narrow channel bandwidth (the maximum transmission rate does not exceed 64kbits / s), the image should not be too large. QCIF (176) is adopted for image format × 144) format with a frame rate of 15fps (frames per second). The following video and audio codec calculation and storage are estimated under the above conditions.
H. 263 the computation of video coding mainly focuses on motion estimation and prediction coding, discrete cosine transform (DCT) and variable length coding (VLC). The computation of this part is estimated to be 90mips (megainstructions / second). Similarly, the amount of decoding computation is mainly concentrated in motion compensation (MC), inverse discrete cosine transform (IDCT) and variable length decoding (VLD), which is estimated to be 40mips. G. 723.1 the computation of audio coding is much smaller than that of video, about 30mips.
In addition to other operations, such as audio and video mixing, data transmission, control of external devices, etc., it is estimated to be about 60mips. Therefore, the calculation amount of DSP is about 220mips.
The memory includes program memory and data memory. Program memory and some constants (DCT coefficient, VLC coefficient, etc.) are stored in ROM and booted into RAM after running. This part is estimated to be 400kbytes.
The data memory mainly stores the sampled video data and the data stream to be decoded, that is, the buffer of video and audio signals. The sampling data of each frame in QCIF format is about 50kbytes. At least 4 ~ 5 frames should be stored in RAM, and this part needs 250kbytes. In order to synchronize video and audio, the audio data buffer is about 50kbytes.
3 implementation of videophone system based on DSP
According to the resource analysis, in order to realize the video phone with PSTN as the transmission line, first select the appropriate video and audio codec DSP chip, then construct the video phone hardware flow diagram, allocate storage resources, and then write the protocol software related to transplantation to make it suitable for the efficient operation of DSP chip.
3.1 selection of video and audio decoding DSP chip
The video codec chip adopts adsp-21065l of ad company. Adsp-21065l is a powerful 32bit DSP in the super Harvard structure (share) ADSP series. Its high-performance core and integrated peripheral devices make it cost-effective and widely used. Its main characteristics are:
·198 MFLOPS (32bit floating point operation);
·180 mops (32bit fixed-point operation);
·16K 32bit dual port chip memory;
·64M 32bit word length external extended address space;
·2 serial ports supporting 32 time division multiplexing (FDM) channels;
·10 DMA channels;
·2 clocks;
·Seamless interface with 32bit word length SDRAM, etc.
The fast computing capability (198mflops) and large external storage space (64M) of adsp-21065l enable it to complete the parts with large amount of computing and storage capacity in H.324, such as video codec H.263, video audio hybrid H.223 and control H.245.
Adsp-21065l adopts multi bus to enable high-speed data transmission among DPS core, I / O port, internal dual port memory and external synchronous DRAM. This is very important for video data coding and transmission, because video data has the characteristics of large amount and fast transmission. The multi bus structure of adsp-21065l can solve these problems well. Moreover, the calculation of 32 bit word length can improve the signal-to-noise ratio of video and audio.
The audio codec chip adopts adsp-2189m, and its main characteristics include:
·The on-chip memory is up to 1.5mbit, which can be configured into 32K word program memory and 48K byte data memory;
·The operation capacity is 70mips and the instruction cycle is 13ns;
·Three bus structures allow two fetching operations and operations in each instruction cycle;
·4m external expansion memory;
·2 serial ports, which can automatically buffer data;
·6 external interrupts;
·Rich instructions, low power consumption, etc.
In view of the characteristics of adsp-2189m and the calculation amount and storage capacity requirements of audio codec G.723.1 discussed earlier, adsp-2189m can be used for audio codec G.723.1. In addition, the multifunctional serial port of adsp-2189m can complete the task of V.34 modem.
3.2 hardware structure of videophone system
The hardware structure of DSP videophone is shown in Figure 2.
In Figure 2, adsp-21065l is the core of videophone. Its main function is to complete the functions of video coding H.263, multiplexing protocol H.223 of multimedia communication and transmission control protocol H.245. H. The core technology of 263 is based on block DCT and motion estimation and prediction technology. The former eliminates image 8 by orthogonal transform × 8. Spatial redundancy in sub block data to obtain certain data compression; For the latter, first, each input image is divided into levels until it is divided into 8 × 8 blocks are used as the basic processing unit of the motion estimation algorithm, and then motion estimation and prediction are carried out to eliminate the time redundancy between image sequences and achieve the purpose of moving image compression.
H. The multiplexing protocol of 223 low bit rate multimedia communication is mainly composed of Al and MUX. Al is responsible for converting different data streams (al-sdu) obtained from user layer and I / O layer into al-pdu. MUX is responsible for sending Al layer data to the physical layer. The logical unit exchanged between Al and MUX layer is called mux-sdu, and each mux-sdu is actually al-pdu. Al is divided into three types according to different receiving media: al1, al2 and Al3. Al1 is responsible for transmitting general data and control information, al2 is responsible for audio, and Al3 is responsible for video. Since multiple al-pdus (mux-sdus) may reach the multiplexing layer at the same time, the multiplexing layer combines different al-pdus into a data stream according to a predefined descriptor table. The descriptor table has 16 items, and each item defines a different al-pdu combination mode.
H. 245 transmission control protocol specifies the end-to-end signaling to control the correct operation of H.324 terminal. It defines the control process of in band negotiation at the beginning of inter terminal communication, the syntax and semantics of messages, and defines various services. Some are suitable for all terminals and some are only suitable for special terminals. It defines various processes to allow the exchange of audio-visual and data capabilities; Request the transmission of specific audio-visual and data modes; Manage the channel for transmitting audio-visual and data information; Determining a master terminal and a slave terminal for managing a bidirectional channel; Carry various control and indication information; Controlling the bit rate of an independent channel or the entire multiplexing layer; Test round trip delay, etc.
Adsp-2189m plays the role of audio coding protocol G.723 and modem V.34. G. 723 audio coding protocol has two rates of 5.3kbps and 6.3kbps. Its theme is "5.3-6.3kbps multimedia transmission symmetric rate audio encoder". G. 723 encoder is optimized to increase the limited complexity and obtain high-quality dialogue. It encodes speech or other audio signal frames with linear prediction comprehensive analysis coding method. The excitation signal of 6.3kbps is MP-MLQ (multi pulse maximum similarity quantization), and ACELP (arithmetic codebook excitation linear prediction) is used for 5.3kbps; The frame length is 30ms, plus a probe of 7.5ms, and the algorithm delay is 37.5ms in total. The additional delay depends on the processing delay during implementation, the transmission delay of communication link and the buffer delay of multiplexing protocol. 5. 34 protocol modulates and demodulates the data stream to make it suitable for telephone line transmission.
3.3 allocation of storage resources and compilation and migration of software
According to the application characteristics, rational allocation of resources, especially storage resources, has always been an important part of DSP design. In this system, the frequently used DCT coefficients and VLC coefficients are placed in the on-chip RAM of adsp-21065l, while other data are placed in the external RAM. According to the specific functional requirements and reference materials, the corresponding H.324 program in C language is developed, compiled, linked, generated, executed, evaluated and optimized. Evaluate the efficiency of program execution, specifically analyze the time-consuming modules such as DCT and motion estimation and prediction, and write this part of assembly program with efficient assembly language according to the instruction characteristics and hardware structure of adsp-21065, so as to further improve the execution efficiency of program and make full use of ADSP system hardware resources, and finally realize H.324 program in ADSP in real time as required.
In order to ensure the unity of interfaces between chips, the core chips such as video and audio codec chip and video audio frequency digital to analog conversion chip based on video phone on PSTN are all products of ad company. The software of videophone system based on DSP is scalable and easy to realize function expansion and interworking with other network video terminals complying with itu standards and ISO standards. Nowadays, ADSL communication on telephone line is developing rapidly. Due to the flexibility of videophone system based on DSP, as long as ADSL terminal equipment is added to it, code rate control is changed in software, and video and audio quality is improved, broadband video and audio services can be carried out., Read the full text“
Our other product:
