MPEG4 is a compression technology suitable for surveillance
MPEG4 was announced in November 1998. The international standard MPEG4, which was originally expected to be put into use in January 1999, is not only for video and audio coding at a certain bit rate, but also pays more attention to the interactivity and flexibility of multimedia systems. The experts of the MPEG expert group are working hard for the formulation of MPEG-4. The MPEG-4 standard is mainly used in Video Phone, Video Email and Electronic News, etc. Its transmission rate requirements are relatively low, between 4800-64000bits/sec, and the resolution is between 4800-64000bits/sec. It is 176X144. MPEG-4 uses a very narrow bandwidth, compresses and transmits data through frame reconstruction technology, in order to obtain the least data and obtain the best image quality.
Compared with MPEG-1 and MPEG-2, the characteristic of MPEG-4 is that it is more suitable for interactive AV services and remote monitoring. MPEG-4 is the first dynamic image standard that changes you from passive to active (no longer just watching, allowing you to join in, that is, interactive); another feature of it is its comprehensiveness; from the source , MPEG-4 attempts to blend natural objects with man-made objects (in the sense of visual effects). The design goal of MPEG-4 also has wider adaptability and scalability. MPEG4 tries to achieve two goals:
1. Multimedia communication under low bit rate;
2. It is the synthesis of multimedia communication in multiple industries.
According to this goal, MPEG4 introduces AV objects (Audio/Visaul Objects), making more interactive operations possible. MPEG-4's video quality resolution is relatively high, and the data rate is relatively low. The main reason is that MPEG-4 adopts ACE (Advanced Decoding Efficiency) technology, which is a set of coding algorithm rules used in MPEG-4 for the first time. Target orientation related to ACE can enable very low data rates. Compared with MPEG-2, it can save 90% of storage space. MPEG-4 can also be widely upgraded in audio and video streams. When the video changes between 5kb/s and 10Mb/s, the audio signal can be processed between 2kb/s and 24kb/s. It is especially important to emphasize that the MPEG-4 standard is an object-oriented compression method. It is not simply dividing the image into some blocks like MPEG-1 and MPEG-2, but according to the content of the image, the objects (objects, characters, Background) It is separated to perform intra-frame and inter-frame encoding and compression, and allows flexible allocation of code rates among different objects. More bytes are allocated to important objects, and fewer bytes are allocated to secondary objects. Thus, the compression ratio is greatly improved, so that it can obtain better results at a lower code rate. The object-oriented compression method of MPEG-4 also makes the image detection function and accuracy more reflected. The image detection function enables the hard disk video recorder system to have a better video motion alarm function.
In short, MPEG-4 is a brand-new video coding standard with low bit rate and high compression ratio. The transmission rate is 4.8~64kbit/s, and it occupies a relatively small storage space. For example, for a color screen with a resolution of 352×288 , When the space occupied by each frame is 1.3KB, if you select 25 frames/second, it will require 120KB per hour, 10 hours per day, 30 days per month, and 36GB per channel per month. If it is 8 channels, 288GB is required, which is obviously acceptable.
There are many kinds of technologies in this area, but the most basic and the most widely used at the same time are MPEG1, MPEG2, MPEG4 and other technologies. MPEG1 is a technology with a high compression ratio but poorer image quality; while MPEG2 technology mainly focuses on image quality, and the compression ratio is small, so it requires a large storage space; MPEG4 technology is a more popular technology nowadays, using this technology can be It saves space, has high image quality, and does not require high network transmission bandwidth. In contrast, MPEG4 technology is relatively popular in China and has also been recognized by industry experts.
According to the introduction, since the MPEG4 standard uses telephone lines as the transmission medium, decoders can be configured on-site according to the different requirements of the application. The difference between it and the compression coding method based on dedicated hardware is that the coding system is open and new and effective algorithm modules can be added at any time. MPEG4 adjusts the compression method according to the spatial and temporal characteristics of the image, so as to obtain a larger compression ratio, lower compression code stream and better image quality than MPEG1. Its application goals are for narrow-band transmission, high-quality compression, interactive operations, and expressions that integrate natural objects with man-made objects, while also particularly emphasizing broad adaptability and scalability. Therefore, MPEG4 is based on the characteristics of scene description and bandwidth-oriented design, which makes it very suitable for the field of video surveillance, which is mainly reflected in the following aspects:
1. Storage space is saved-the space required to adopt MPEG4 is 1/10 of that of MPEG1 or M-JPEG. In addition, because MPEG4 can automatically adjust the compression method according to scene changes, it can ensure that the image quality will not be degraded for still images, general sports scenes, and intense activity scenes. It is a more effective video encoding method.
2. High image quality--The highest image resolution of MPEG4 is 720x576, which is close to the picture effect of DVD. MPEG4 based on the AV compression mode determines that it can guarantee good definition for moving objects, and the time/time/image quality is adjustable.
3. The requirement for network transmission bandwidth is not high--because the compression ratio of MPEG4 is more than 10 times that of MPEG1 and M-JPEG of the same quality, the bandwidth occupied during network transmission is only about 1/10 of that of MPEG1 and M-JPEG of the same quality. . Under the same image quality requirements, MPEG4 only needs a narrower bandwidth.
====================
Technical Highlights of the New Video Coding Standard H.264
Summary:
For practical applications, the H.264 recommendation jointly formulated by the two major international standardization organizations, ISO/IEC and ITU-T, is a new development in video coding technology. It has its unique features in multi-mode motion estimation, integer transformation, unified VLC symbol coding, and layered coding syntax. Therefore, the H.264 algorithm has high coding efficiency, and its application prospects should be self-evident.
Keywords: video coding image communication JVT
Since the 1980s, the introduction of two major series of international video coding standards, MPEG-x formulated by ISO/IEC and H.26x formulated by ITU-T, ushered in a new era of video communication and storage applications. From H.261 video coding recommendations to H.262/3, MPEG-1/2/4, etc., there is a common goal that is constantly pursued, that is, to obtain as much as possible under the lowest possible bit rate (or storage capacity). Good image quality. Moreover, as the market's demand for image transmission increases, the problem of how to adapt to the transmission characteristics of different channels has become increasingly apparent. This is the problem to be solved by the new video standard H.264 jointly developed by IEO/IEC and ITU-T.
H.261 is the earliest video coding suggestion, the purpose is to standardize the video coding technology in ISDN network conference TV and video phone applications. The algorithm it uses combines a hybrid coding method of inter-frame prediction that can reduce temporal redundancy and DCT transform that can reduce spatial redundancy. It matches the ISDN channel, and its output code rate is p×64kbit/s. When the value of p is small, only images with low definition can be transmitted, which is suitable for face-to-face TV calls; when the value of p is large (such as p>6), conference TV images with better definition can be transmitted. H.263 recommends a low bit rate image compression standard, which is technically an improvement and expansion of H.261, and supports applications with a bit rate less than 64kbit/s. But in fact H.263 and later H.263+ and H.263++ have been developed to support full bit rate applications. It can be seen from the fact that it supports many image formats, such as Sub-QCIF, QCIF , CIF, 4CIF and even 16CIF and other formats.
The code rate of the MPEG-1 standard is about 1.2Mbit/s, and it can provide 30 frames of CIF (352×288) quality images. It is formulated for the video storage and playback of CD-ROM discs. The basic algorithm of the MPEG-l standard video coding part is similar to H.261/H.263, and measures such as motion-compensated inter-frame prediction, two-dimensional DCT, and VLC run-length coding are also adopted. In addition, concepts such as intra frame (I), predictive frame (P), bidirectional predictive frame (B) and DC frame (D) are introduced to further improve the coding efficiency. On the basis of MPEG-1, the MPEG-2 standard has made some improvements in improving image resolution and compatibility with digital TV. For example, the accuracy of its motion vector is half pixel; in coding operations (such as motion estimation and DCT) Distinguish between "frame" and "field"; introduce coding scalability technologies, such as spatial scalability, temporal scalability, and signal-to-noise ratio scalability. The MPEG-4 standard introduced in recent years has introduced coding based on audio-visual objects (AVO: Audio-Visual Object), which greatly improves the interactive capabilities and coding efficiency of video communications. MPEG-4 also adopted some new technologies, such as shape coding, adaptive DCT, arbitrary shape video object coding and so on. But the basic video encoder of MPEG-4 still belongs to a kind of hybrid encoder similar to H.263.
In short, the H.261 recommendation is a classic video coding, H.263 is its development, and will gradually replace it in practice, mainly used in communications, but the numerous options of H.263 often make users at a loss. The MPEG series of standards have evolved from applications for storage media to applications that adapt to transmission media. The basic framework of its core video coding is consistent with H.261. Among them, the eye-catching "object-based coding" part of MPEG-4 is due to still There are technical obstacles, and it is difficult to universally apply. Therefore, the new video coding proposal H.264 developed on this basis overcomes the weaknesses of the two, introduces a new coding method under the framework of hybrid coding, improves coding efficiency, and faces practical applications. At the same time, it was jointly formulated by the two major international standardization organizations, and its application prospects should be self-evident.
1. JVT's H.264
H.264 is a new digital video coding standard developed by the joint video team (JVT: joint video team) of VCEG (Video Coding Experts Group) of ITU-T and MPEG (Moving Picture Coding Experts Group) of ISO/IEC. It is part 10 of ITU-T's H.264 and ISO/IEC's MPEG-4. The solicitation of drafts started in January 1998. The first draft was completed in September 1999. The test model TML-8 was developed in May 2001. The FCD board of H.264 was passed at the 5th meeting of JVT in June 2002. . The standard is currently under development and is expected to be officially adopted in the first half of next year.
H.264, like the previous standard, is also a hybrid coding mode of DPCM plus transform coding. However, it adopts a concise design of "return to basics", without many options, and obtains much better compression performance than H.263++; it strengthens the adaptability to various channels and adopts a "network-friendly" structure and syntax. Conducive to the processing of errors and packet loss; a wide range of application targets to meet the needs of different speeds, different resolutions, and different transmission (storage) occasions; its basic system is open and no copyright is required for use.
Technically, there are many highlights in the H.264 standard, such as unified VLC symbol coding, high-precision, multi-mode displacement estimation, integer transformation based on 4×4 blocks, and layered coding syntax. These measures make H.264 algorithm have very high coding efficiency, under the same reconstructed image quality, it can save about 50% of the code rate than H.263. H.264's code stream structure has strong network adaptability, increases error recovery capabilities, and can well adapt to the application of IP and wireless networks.
2. Technical highlights of H264
Layered design
The H.264 algorithm can be conceptually divided into two layers: the video coding layer (VCL: Video Coding Layer) is responsible for efficient video content representation, and the network abstraction layer (NAL: Network Abstraction Layer) is responsible for the appropriate way required by the network. Pack and transmit data. The hierarchical structure of the H.264 encoder is shown in Figure 1. A packet-based interface is defined between VCL and NAL, and packaging and corresponding signaling are part of NAL. In this way, the tasks of high coding efficiency and network friendliness are completed by VCL and NAL respectively.
VCL layer includes block-based motion compensation hybrid coding and some new features. Like the previous video coding standards, H.264 does not include functions such as pre-processing and post-processing in the draft, which can increase the flexibility of the standard.
NAL is responsible for using the segmentation format of the lower layer network to encapsulate data, including framing, logical channel signaling, timing information utilization, or sequence end signal, etc. For example, NAL supports video transmission formats on circuit-switched channels, and supports video transmission formats on the Internet using RTP/UDP/IP. NAL includes its own header information, segment structure information, and actual load information, that is, the upper-layer VCL data. (If data segmentation technology is used, the data may consist of several parts).
High-precision, multi-mode motion estimation
H.264 supports motion vectors with 1/4 or 1/8 pixel precision. At 1/4 pixel accuracy, a 6-tap filter can be used to reduce high-frequency noise. For motion vectors with 1/8 pixel accuracy, a more complex 8-tap filter can be used. When performing motion estimation, the encoder can also select "enhanced" interpolation filters to improve the effect of prediction
In the motion prediction of H.264, a macro block (MB) can be divided into different sub-blocks according to Figure 2 to form 7 different modes of block sizes. This multi-mode flexible and detailed division is more suitable for the shape of the actual moving objects in the image, greatly improving
The accuracy of the motion estimation is improved. In this way, each macro block can contain 1, 2, 4, 8, or 16 motion vectors.
In H.264, the encoder is allowed to use more than one previous frame for motion estimation, which is the so-called multi-frame reference technology. For example, if 2 or 3 frames are just coded reference frames, the encoder will select a better prediction frame for each target macroblock, and indicate for each macroblock which frame is used for prediction.
4×4 block integer transformation
H.264 is similar to the previous standard, using block-based transform coding for the residual, but the transform is an integer operation instead of a real number operation, and the process is basically similar to that of DCT. The advantage of this method is that the same precision transformation and inverse transformation are allowed in the encoder and the decoder, which facilitates the use of simple fixed-point arithmetic. In other words, there is no "inverse transformation error" here. The unit of transformation is 4×4 blocks, instead of 8×8 blocks commonly used in the past. As the size of the transform block is reduced, the division of the moving object is more accurate. In this way, not only the transformation calculation amount is relatively small, but also the convergence error at the edge of the moving object is greatly reduced. In order to make the small-size block transformation method not produce the grayscale difference between the blocks in the larger smooth area in the image, the DC coefficient of 16 4×4 blocks of the intra-frame macroblock brightness data (each small block One, a total of 16) performs a second 4×4 block transformation, and performs a 2×2 block transformation on the DC coefficients of 4 4×4 blocks of chrominance data (one for each small block, 4 in total).
In order to improve the rate control ability of H.264, the change of quantization step size is controlled at about 12.5% instead of a constant increase. The normalization of the transform coefficient amplitude is processed in the inverse quantization process to reduce the computational complexity. In order to emphasize the fidelity of color, a small quantization step size is adopted for the chrominance coefficient.
Unified VLC
There are two methods for entropy coding in H.264. One is to use unified VLC (UVLC: Universal VLC) for all symbols to be coded, and the other is to use content-adaptive binary arithmetic coding (CABAC: Context-Adaptive). Binary Arithmetic Coding). CABAC is an optional option, its coding performance is slightly better than UVLC, but the computational complexity is also higher. UVLC uses a code word set of unlimited length, and the design structure is very regular, and different objects can be coded with the same code table. This method is easy to generate a codeword, and the decoder can easily identify the prefix of the codeword, and UVLC can quickly obtain resynchronization when a bit error occurs
Here, x0, x1, x2, ... are INFO bits, and are 0 or 1. Figure 4 lists the first 9 codewords. For example, the 4th number word contains INFO01. The design of this code word is optimized for fast resynchronization to prevent bit errors.
intra pdiction
In the previous H.26x series and MPEG-x series standards, inter-frame prediction methods are used. In H.264, intra-frame prediction is available when encoding Intra images. For each 4×4 block (except for the special treatment of the edge block), each pixel can be predicted with a different weighted sum of the 17 closest previously encoded pixels (some weights can be 0), that is, this pixel 17 pixels in the upper left corner of the block. Obviously, this kind of intra-frame prediction is not in time, but a predictive coding algorithm performed in the spatial domain, which can remove the spatial redundancy between adjacent blocks and achieve more effective compression.
In the 4×4 square, a, b, ..., p are 16 pixels to be predicted, and A, B, ..., P are encoded pixels. For example, the value of point m can be predicted by the formula (J+2K+L+2)/4, or by the formula (A+B+C+D+I+J+K+L)/8, and so on. According to the selected prediction reference points, there are 9 different modes for luminance, but there is only 1 mode for intra-frame prediction of chrominance.
For IP and wireless environments
The H.264 draft contains tools for error elimination to facilitate the transmission of compressed video in an environment with frequent errors and packet loss, such as the robustness of transmission in mobile channels or IP channels.
In order to resist transmission errors, the time synchronization in the H.264 video stream can be accomplished by using intra-frame image refresh, and the spatial synchronization is supported by slice structured coding. At the same time, in order to facilitate resynchronization after a bit error, a certain resynchronization point is also provided in the video data of an image. In addition, intra-frame macroblock refresh and multiple reference macroblocks allow the encoder to consider not only the coding efficiency, but also the characteristics of the transmission channel when determining the macroblock mode.
In addition to using the change of the quantization step size to adapt to the channel code rate, in H.264, the method of data segmentation is often used to cope with the change of the channel code rate. Generally speaking, the concept of data segmentation is to generate video data with different priorities in the encoder to support the quality of service QoS in the network. For example, a syntax-based data partitioning method is adopted to divide the data of each frame into several parts according to its importance, which allows the less important information to be discarded when the buffer overflows. A similar temporal data partitioning method can also be used, which is accomplished by using multiple reference frames in P and B frames.
In the application of wireless communication, we can support large bit rate changes of the wireless channel by changing the quantization precision or space/time resolution of each frame. However, in the case of multicast, it is impossible to require the encoder to respond to varying bit rates. Therefore, unlike the FGS (Fine Granular Scalability) method used in MPEG-4 (with lower efficiency), H.264 uses stream switching SP frames instead of hierarchical coding.
========================
3. TML-8 performance
TML-8 is the test mode of H.264, use it to compare and test the video coding efficiency of H.264. The PSNR provided by the test results has clearly shown that compared to the performance of MPEG-4 (ASP: Advanced Simple Profile) and H.263++ (HLP: High Latency Profile), the results of H.264 have obvious advantages. As shown in Figure 5.
The PSNR of H.264 is obviously better than that of MPEG-4 (ASP) and H.263++ (HLP). In the comparison test of 6 speeds, the PSNR of H.264 is 2dB higher than MPEG-4 (ASP) on average. It is 3dB higher than H.263 (HLP) on average. The 6 test rates and their related conditions are: 32 kbit/s rate, 10f/s frame rate and QCIF format; 64 kbit/s rate, 15f/s frame rate and QCIF format; 128kbit/s rate, 15f/s Frame rate and CIF format; 256kbit/s rate, 15f/s frame rate and QCIF format; 512 kbit/s rate, 30f/s frame rate and CIF format; 1024 kbit/s rate, 30f/s frame rate and CIF format.
4. difficulty of realization
For every engineer considering practical applications, while paying attention to the superior performance of H.264, it is bound to measure the difficulty of its implementation. Generally speaking, the improvement of H.264 performance is obtained at the cost of increased complexity. However, with the development of technology, this increase in complexity is within the acceptable range of our current or near future technology. In fact, considering the limitation of complexity, H.264 has not adopted some particularly computationally expensive improved algorithms. For example, H.264 does not use global motion compensation technology, which is used in MPEG-4 ASP. Increased considerable coding complexity.
Both H.264 and MPEG-4 include B-frames and more precise and complex motion interpolation filters than MPEG-2, H.263 or MPEG-4 SP (Simple profile). In order to better complete motion estimation, H.264 has significantly increased the types of variable block sizes and the number of variable reference frames.
H.264 RAM requirements are mainly used for reference frame images, and most coded videos use 3 to 5 frames of reference images. It does not require more ROM than the usual video encoder, because H.264 UVLC uses a well-structured look-up table for all types of data
5. concluding remarks
H.264 has broad application prospects, such as real-time video communication, Internet video transmission, video streaming services, multi-point communication on heterogeneous networks, compressed video storage, video databases, etc.
The technical characteristics of H.264 recommendations can be summarized into three aspects. One is to focus on practicality, adopt mature technology, pursue higher coding efficiency, and concise expression; the other is to focus on adapting to mobile and IP networks and adopt hierarchical Technology, which separates the encoding and the channel formally, in essence, takes into account the characteristics of the channel more in the source encoder algorithm; the third is that under the basic framework of the hybrid encoder, its main key components are all made. Major improvements, such as multi-mode motion estimation, intra-frame prediction, multi-frame prediction, unified VLC, 4×4 two-dimensional integer transformation, etc.
So far, H.264 has not been finalized, but because of its higher compression ratio and better channel adaptability, it will be more and more widely used in the field of digital video communication or storage, and its development potential is unlimited.
Finally, it must be noted that the superior performance of H.264 is not without cost, but the cost is a large increase in computational complexity. According to estimates, the computational complexity of encoding is approximately three times that of H.263, and the complexity of decoding Approximately 2 times of H.263.
===========================
Correctly understand the H.264 and MPEG-4 technology products, and eliminate the manufacturer’s false propaganda
It is recognized that the H.264 video codec standard has a certain degree of advancement, but it is not the preferred video encoder standard, especially as a surveillance product, because it also has some technical defects.
is included in the MPEG-4 Part 10 standard as the H.264 video codec standard, which means it is only attached to the tenth part of MPEG-4. In other words, H.264 does not exceed the scope of the MPEG-4 standard. Therefore, it is incorrect that the H.264 standard and video transmission quality on the Internet are higher than MPEG-4. The transition from MPEG-4 to H.264 is even more incomprehensible. First, let us correctly understand the development of MPEG-4:
1. MPEG-4 (SP) and MPEG-4 (ASP) are the early product technologies of MPEG-4
MPEG-4 (SP) and MPEG-4 (ASP) were proposed in 1998. Its technology has developed to the present, and there are indeed some problems. Therefore, the current state-owned technical personnel who have the ability to develop MPEG-4 have not adopted this backward technology in MPEG-4 video surveillance or video conferencing products. The comparison between H.264 products (technical products after 2005) and the early MPEG-4 (SP) technology promoted on the Internet is really inappropriate. Can the performance comparison of IT products in 2005 and 2001 be convincing? . What needs to be explained here is that this is a technical hype behavior of manufacturers.
Please take a look at the technology comparison:
Some manufacturers misguided comparisons: Under the same reconstructed image quality, H.264 reduces the bit rate by 50% compared to H.263+ and MPEG-4 (SP).
These data essentially compare H.264 new technology product data with MPEG-4 early technology product data, which is meaningless and misleading for comparing current MPEG-4 technology products. Why didn't H.264 products compare data with new MPEG-4 technology products in 2006? The development of H.264 video coding technology is indeed very fast, but its video decoding video effect is only equivalent to the video effect of Microsoft's Windows Media Player 9.0 (WM9). At present, for example, the MPEG-4 technology used by Huayi's hard disk video server and video conferencing equipment has reached the (WMV) technical specifications in video decoding technology, and the audio and video synchronization is less than 0.15s (within 150 milliseconds). H.264 and Microsoft WM9 can't match
2. The developing MPEG-4 video decoder technology:
At present, MPEG-4 video decoder technology is developing rapidly, not as the manufacturers hype on the Internet. The advantage of the current H.264 image standard is only in its compression and storage, which is 15-20% smaller than the current MPEG-4 storage file of Huayi products, but its video format is not a standard format. The reason is that H.264 does not adopt an internationally-used storage format, and its video files cannot be opened with internationally-used third-party software. Therefore, in some domestic governments and agencies, when selecting equipment, it is clearly stated that the video files must be opened with internationally accepted third-party software. This is really important for monitoring products. Especially when theft occurs, the police need to obtain evidence, analyze, etc.
The upgraded version of MPEG-4 video decoder is (WMV), and the audio is different according to the coding technology and experience of each manufacturer. The current mature MPEG-4 new technology products from 2005 to 2006 are far higher than H.264 technology products in terms of performance.
In terms of transmission: Compared with the new MPEG-4 technology product H.264, there are the following defects:
1. Audio and video synchronization: H.264 audio and video synchronization has some problems, mainly in terms of delay. The transmission performance of H.264 is equivalent to Microsoft's Windows Media Player 9.0 (WM9). At present, the MPEG-4 technology adopted by Huayi network video server achieves a delay of less than 0.15 seconds (150 milliseconds) in the field of video surveillance and video conferencing, which is beyond the reach of H.264 products;
2. Network transmission efficiency: adopt H.2
Our other product:
