I. Basic concept
IF: Intra Picture, use intra compression to remove space redundant information.
P Frame: PredicTive-frame, compress the encoded image of the data amount by compressing the amount of time redundant information in front of the image in the image sequence. Refer to the previous I frame or P frame.
B frame: Bi-Directional Interpolated Prediction Frame, considering the encoded frame in front of the source image sequence, and emphasizes the redundant information between the encoded frames behind the source image sequence, compressing transmission data. The encoded image of the amount is also referred to as a bidirectional coding frame. Refer to one of the I frames or P frames and a P frame behind it.
GOP: A set of pictures between two I frames, consists of one I frame and multiple B / P frames, which are the basic units of codec access.
IDR frames: I and IDR frames are used in intra prediction. They are all the same thing. In the coding and decoding for convenience, the first I frame and other I frame are different, so the first I frame is called IDR, so it is convenient to control the encoding and decoding process; IDR frame The role is to refresh immediately, make the error dissemble, starting from the IDR frame, re-calculate a new sequence to start encoding. The I frame does not have a random access, and the IDR image must be a I image, but the i image is not necessarily an IDR image.
Two compression
The core algorithm used by the H264 is the intra compression and inter-frame compression. The intra compression is an algorithm for generating I frames, and the inter-frame compression is an algorithm for generating B frame and P frames. INTRAME compression is also called spatialcompression. When a frame of image is compressed, only the data between the frames is considered, which is actually similar to still image compression. The intra synthetic algorithm is generally used. Since the intra compression is encoded a complete image, it can be independently decoded and displayed. The intra compression is generally less than a high compression, which is similar to the encoding JPEG. The principle of interframe compression is that the data of the adjacent frames has a large correlation, or the two frames of information in front and rear files are small. That is, a continuous video thereof has redundant information between adjacent frames, depending on this characteristic, the redundancy amount between compressed adjacent frames can further increase the amount of compression and reduce the compression ratio. Inter-frame compression is also referred to as time compression, which compresses data between different frames between the time axes. Inter-frame compression is generally non-destructive. The frame difference algorithm is a typical time compression, which compares the difference between the frame and the neighboring frame, which can greatly reduce the amount of data.
3.H.264 code stream
H.264 Original code stream (also known as "nude") is composed of one one NALU. An original H.264 NALU unit is often composed of [Startcode] [NALU HEADER] [NALU PAYLOAD], StartCode is divided into two: 0x000001 (3byte) or 0x00000001 (4byte). If the corresponding SLICE corresponding to the NALU is 0x00000001 in the start code of the frame, otherwise use 0x000001; their structure is shown below.
The H264 functional layer is divided into two levels, VCL (video encoding layers), and NAL (network extract layer).
The VLC layer is the definition of the grammar level of the core algorithm engine, block, macroblock, and the film, which is responsible for effectively representing the content of video data, and finally outputs the data SODB encoded;
The NAL layer defines the syntax level above the tabular (such as the sequence parameter set parameter set and image parameter set, which will be described later), which is responsible for formatting data and provides header information as required by the network. Ensure that the data is suitable for transmission on various channels and storage media. The NAL layer packed SODB into RBSP and then forms a NAL header into a NALU unit.
SODB (STRING OF DATA BITS): The data bit string is the original data after encoding;
RBSP (Raw Byte Sequence PayLoad): The original byte sequence load adds a finite bit, a bit "1" and several bit "0" after the original coded data is added to the byte alignment.
Be
The following is the H264 code stream structure. SPS, PPS, IDR, and SLICE are a type of data for NAL units. As we have seen, each slice also includes both heads and data, and the fraction head contains a slice type, the macroblock type, the number of fragment frames, and the settings and parameters of the corresponding frames. And the fragmentation data is a macroblock, which is where the storage pixel data we have to find is. Be
Be
The following is the NAL unit sequence. Each NAL unit contains an RBSP and a NAL header.
The following is a NAL unit. Be
Be
The following is the NAL header.
Composed of three parts, Forbidden_bit (1bit), NAL_REFERENCE_BIT (2BITS) (priority), NAL_Unit_type (5bits). The Type of NAL illustrates the current NAL package data structure and data meaning, which may be a slice, a parameter set, or a padding data.
The following is a few types of RBSP. As shown above NAL_Unit_type (5bits) (Type)
NAL data types are mainly divided into VCL types and non-VCL types.
NANAL DATA Type: 1) SPS (Sequence Parameter Set): SPS is recorded for decoding parameters such as identifier, frame number, and reference frame number, decoded image size, and frame field mode. 2) PPS (image parameter set): PPS logo records for decoding parameters such as entropy encoding type, number of valid reference images, and initialization. 3) SEI (Supplemental Enhancement Information): This partial parameter can be transmitted as bit stream data of H264, each SEI information being encapsulated into a NAL unit. SEI may be useful for the decoder, but for the basic decoding process, it is not necessary.
VCL NAL Data Type 1) Head Information Block, including macroblock type, quantization parameter, motion vector. This information is the most important because you leave them, the symbols in the data block cannot be used. This data block is referred to as a Class A data block. 2) Inframe encoding information data block, called Class B data block. It includes an intra-frame encoded macroblock type with an intra coding factor. Corresponding SLICE, the availability of class B data block depends on the Class A data block. The inter-frame encoded information data block is not that the intra encoded information can prevent further deviations, so it is more important than the inter inter-frame encoding information. 3) Interframe encoding information data block, called Class C data block. It includes inter-frame encoded macroblock type, inter-frame encoding coefficient. It is usually the largest part of the SLICE. Interframe encoding information data block is an unimportant part. The information it contains does not provide synchronization between the codec. The availability of Class C data blocks also depends on the Class C class data block, but is independent of the class B data block. Each segmentation is stored separately in one NAL unit alone, so it can be transmitted separately. Be
The following is the H.264 hierarchical structure.
1 frame (a image) = 1 ~ n piece (slice) / / can also be said to 1 to more pieces as a piece
1 piece = 1 ~ n macroblock (MARCROBLOCK)
1 macroblock = 16x16 YUV data (original video acquisition data)
Slice: The film is a new concept proposed by H.264, and the concept is integrated by coding the picture. A picture has one or more pieces, while the film is loaded by NALU and transmitted by the network. However, NALU is not necessarily slice, which is a full unnecessary condition because NALU may also load other information used as a description video. The purpose of the set tablet is to limit the diffusion and transmission of the error, and the encoders should be independent of each other. A prediction of a piece cannot be referred to in the macroblock in other slices, so that the prediction error in a piece will not propagate into other slices.
The slice has the following five types:
The sheet meaning I contains only the I macroblock P sheet contains the P and I macroblocks B tablets containing B and I macroblock SP films contains P and / or I macroblocks, and the switching Si on different streams is a special Type encoding macroblock
Macro block: macroblock is the main carrier of video information. An encoded image is usually divided into multiple macroblocks. The brightness and chrominance information of each pixel is included. The main work of video decoding is to provide an efficient way to get the pixel array in the macroblock in the code stream.
The following is a macroblock structure diagram.
From the above figure, it can be seen that the macroblock contains the macroblock type, the prediction type, the brightness block pattern, the QU atization Parameter, the pixel brightness and chroma dataset. Be
The macroblock classification meaning I macroblock utilizes the pixel of the decoded from the current piece to perform intra prediction of the P macroblock using the previously encoded image as a reference for intra prediction, and an intra-encoded macroblock can be further macro block. Split: 16 × 16.16 × 8.8 × 16.8 × 8 brightness pixel block. If the 8 × 8 sub-macroblock is selected, it can be divided into various sub-macroblocks, and the size is 8 × 8, 8 × 4, 4 x 8, 4 × 4b macroblock utilizes two-way reference images (current And future encoded image frames) Perform intra prediction
Our other product:
