"First, H.265 (HEVC) encoding process
Like H.264, H.265 encodes steps such as intra prediction, inter prediction, quantization, linear transformation. The process is approximately as follows;
1, block
A frame screen is first cut into a plurality of mutually not overlapping block regions, referred to as the encoding unit (h.264 is called a macroblock), which is transmitted to the encoder.
2, intra prediction
The first screen of the image sequence (and each of the frames that can be dragged) use only intra predictive encoding (prediction using only other areas in the same frame, not relying on other frames).
3, inter-frame prediction
Most of the blocks in other frames use inter-frame predictive encoding, the process includes selecting a prediction mode, a reference image motion data, and a motion vector (MV) generating each block. The encoder calculates motion compensation by bypass transmission prediction mode information and motion vector (mV), and then reconstructs inter prediction data.
4, compressed output
The predictive results between the intra or inter-frames and the residual data between the actual screens are output together by spatial-linear transformation, sampling, quantization, entropy encoding, and prediction information.
HEVC integrates many key technologies to improve performance, such as a quadrature, more refined intra prediction technique based on four-fork tree division coding unit, predicting direction, with inter prediction technology, high-precision motion compensation technique It is used to improve the sequest block filtering and pixel adaptive compensation technique of reconstructed image quality.
Second, the noun explanation
1. Coding Tree Unit (CTU) and Coding Tree Block (CTB):
The core coding layer in the H.264 standard is a macroblock, including a 16 x 16 brightness block sample, and for the general YUV420, it is accompanied by two 8 × 8 chroma blocks. Similar structures are called CTUs in HEVC, but it can be set by the encoder, and can be surpassed 16 × 16. The CTU consists of a brightness CTB, several chroma CTBs, and some associated syntax elements. The selective size of the brightness CTB has 16 × 16, 32 × 32, 64 × 64, and the larger block will have a better compression ratio. HEVC also supports segmentation of CTB into smaller blocks using tree structures and quadroes.
2, encoding unit (Cu) and coding block (CB):
The four fork in the CTU determines the size and position of the brightness and chroma Cb, and the root node of the quadrush is associated with the CTU. The brightness CB can support the size of the brightness CTB. The process of cutting the CTU into brightness and chroma Cb is integral. A brightness CB, two chroma Cb, and associated syntax elements constitute a CU. A CTB can only contain only one CU, or several CUs, each CU has a partition associated prediction unit (PU) and a transform tree unit (TU).
3, prediction unit (PU) and prediction block (PB):
The PU is a predicted basic unit that specifies all prediction modes of the coding unit, and its maximum unit is the same as the current CU size. In the HEVC, the SKIP mode, intra mode, and inter-frame mode are different from the PU segmentation. A frame image uses intra prediction or inter prediction is based on the Cu level decision. The root of the PU partition structure is also in the Cu layer. The brightness and chrominance Cb are further split into brightness and chrominance Pb based on the result of predictive decisions. HEVC supports PB from 4 × 4 to 64 × 64 size. As shown in the figure above, for SKIP mode, the size of the PU is 2N2N. The size of the intra prediction mode PU may be 2 N2N and NN, where, when the size of the CU is 88, the intralated PU can take n * n.
There are 8 cases of inter prediction PU segmentation mode, mainly divided into two categories: symmetric segmentation and asymmetric segmentation. Among them, 2 n2n, 2nn, N2N and NN are 4 symmetrical modes, 2nnu, 2nnd, NL2N, and NR2N are 4 non-symmetrical modes, u, d, l, and r represent the upper and lower left and so, respectively, and the asymmetric division is only used for size. In CUs of 3232 and 1616, NN in the symmetric divided form is only used in CUs having a size of 88. For example, 2nnu and 2Nnd are divided by 1: 3 and 3: 1, respectively, and NL2N and NR2N are divided by left and right 1: 3 and 3: 1, respectively.
4, transform unit (TU) and transform block (TB):
TU is a basic unit of transform and quantization, and the transform tree is a quadrush tree consisting of a conversion unit. Starting from the CU size, the transform unit is divided into four sub-blocks according to the syntax element Split_Transform_FLAG calibration. Depending on the depth of iterative division, the size can be one of 3232, 1616, 88, and 44. The maximum / minimum value of the conversion unit can be set in the sequence parameter set. The predictive residual use block transform coding. The root of the Tu tree structure is at the Cu level. The residual of the brightness Cb may be exactly the same as the brightness Tb, or further divide a smaller brightness Tb. The chrominance Tb is also the same. The 4 × 4, 8 × 8, 16 × 16, 32 × 32 Tb each defines the basic integer method of approximate DCT transformation. For the residual transformation of 4 × 4 brightness intra prediction, there is also a DCT conversion table derived integer transformation option. As shown in the figure above, the maximum division depth of TU is 3, and the size can be greater than the PU but cannot exceed CU. When the PU is a square, the TU uses a square transform, and when the PU is a rectangle, the Tu uses a rectangular transformation, which may be one of 328, 832, 164, and 416.
5, sports vector:
Advanced motion vector prediction (AMVP) based on neighboring PB and reference frame data is used in standards. MV encoding can also use a merge mode that allows MVs that inherit neighboring PB. In addition, HEVC also includes a reinforced version of H.264 / AVC direct motion prediction.
6, exercise compensation:
MV uses quarter sampling prediction. The interpolation of hierarchical sampling uses a 7-or-order or 8-order filter (interpolated in H.264 / AVC, using one-half of the 6-order filter and quarter-sampling linear interpolation). Like H.264 / AVC, HEVC also uses multi-reference frames. Each PB can transmit one or two motion vectors based on one-way or bidirectional prediction. Like H.264 / AVC, the prediction signal can be added and offset to declare predictive weight.
7, intra prediction:
When the inter-frame prediction is not used, it can only be based on the adjacent block to perform intra prediction on the space. The intra prediction of HEVC supports 33 direction modes (8 in H.264 / AVC). HEVC also designed enhanced two-dimensional transform and optional DC prediction mode. The optimal intra prediction mode is required to be calculated from a pre-decoded adjacent PB. As shown in the figure above, HEVC adopts 35 intra prediction modes, including DC mode, 33 angular modes, and PLANAR modes. HEVC is more meticulous to the selection process of intra prediction mode, and the PU of different sizes corresponds to different prediction modes. This makes the intra prediction more accurate and reduces spatial redundancy. The specific process of intra prediction is as follows:
1> First traverse all prediction modes, calculate the SAD (definition?) Predicted in each mode and sort by small to large. 2> SAD minimum set of prediction modes as subset of this prediction mode. 3> After determining the predictive subset, it is judged whether or not the direction of the PU left and on the pixel block is in a subset, and the mode is added to the subset. 4> Finally Distortion Optimization (RDO) of all prediction modes in subset.
8, quantitative control:
Like H.264 / AVC, HEVC also uses URQ (consistent quantization reduction). Since HEVC introduces more transform blocks, the quantitative parameter matrix also increases.
9, entropy encoding:
HEVC uses Cabac to make entropy encodings, removes adaptive variable length coding (CAVLC) based on context. Compared with the CABAC in H.264 / AVC, it has been greatly improved in terms of the newly introduced parallel processing architecture, speed, compression ratio, and memory usage. As shown in the figure above, the HEVC's CABAC entropy coding process is basically similar to H.264. Mainly include: binary, text model selection, probability estimation and binary arithmetic code, but HEVC has improved in terms of probability estimation accuracy and adaptive speed.
10, built-in loop filter:
Like H.264 / AVC, the filter of the block effect is also built into the block effect of the block prediction cycle of HEVC. Compared to H.264 / AVC, HEVC design more emphasizes the process of simplifying decisions and filters, and optimized for parallel processing.
11, Sampling Adaptive Offset (SAO):
A nonlinear amplitude mapping is introduced after the inter-block prediction cycle. The main purpose is to analyze some parameters through the histogram of the encoder end to enhance the amplitude signal restore of the decoder end. The SAO is adaptive compensation for the reconstructed pixel after each LCU after filtering each LCU, thereby reducing the distortion of the reconstructed image. SAO is divided into two categories: sideband compensation (BO) and boundary compensation (EO). As shown in the figure above, Bo first divides the brightness level into 32 strips, then count the pixels within one LCU fall into the number of each strip. For 32 strips, each four is a group, the left boundary is marked as the starting position. Calculating 4 consecutive values should be compensated, and the pixels in the LCU are compensated, and finally the RDO is performed, and the least RDO minimized 4 continuous strip is compensated.
As shown in the figure above, EO is mainly compensated for the contour of the image, and the compensation direction is mainly divided into four categories, each of which can be divided into four situations in Figure 17. Statistics Each case is compensated according to the type of pixels based on the pixels based on pixels.
12, parameter set structure:
The decoded information that can be shared by multiple blocks is included in the parameter set. This structure is responsible for ensuring that the necessary decoding information is transmitted to the decoding end. It expands from the image sequence parameter set in H.264 / AVC, and is called VPS in HEVC.
13, NAL syntax unit:
Each syntax structure is placed in a logical packet called NAL (network abstraction layer, typical frame image as a NAL). Two bytes of this bag showed what it was loaded (used to determine if retransmitted).
14, piece:
The portion of the entropy encoding can be referred to as "slice", which can be referred to as independent of other data in the same frame. The film can be a frame or a portion of a frame. One of the main roles of the film is to synchronize synchronization when packet loss. The amount of data that can be included in the cladding transmission is strictly limited, and the number of CTUs included in this restriction film can effectively minimize the additional overhead generated by the package transmission.
15. Enhance Information (SEI) and Video Availability Information (VUI):
SEI and VUI are used to store video metadata, such as timestamp, color space, 3D filling methods, and so on.
16, parallel block (TILE)
HEVC defines an optional way to divide one frame image into a parallel block. The main purpose of the parallel block is to enhance the ability to deal with parallel and does not introduce new error spreads. Parallel blocks are some areas that use some common information within one frame of images, but can decode independently. The general approach is to cut the image into a parallel block comprising substantially the same number of CTUs. The introduction of the parallel block makes the simple coarse particle size of parallelism possible, and the thread will no longer need to consider complex synchronization and lock.
17, wrong peak parallel processing (WPP)
Mainly used for entropy coding. When WPP is turned on, the chip is divided into several CTU lines. The first line is processed, and the second line starts processing after the first line is processed, and the third line starts processing after the second line is processed after 2 CTUs. There are 2 CTU delays in each row. WPP provides a way in which parallel is parallel in an appropriate level (such as a film). WPP can provide better compression efficiency than parallel blocks, and will not introduce block effects.
18, relying on smaller slices (Dependent Slice Segments)
This design allows data to associate with the error parallel parallel or parallel block, in the fragmentation package, which is a decoding process faster to reducing the delay. When used with the wrong peak parallel processing, it also requires a mechanism for the peak. This design is particularly suitable for parallel processing under low delay.
19, inter-frame prediction
HEVC inter prediction proposes new tools such as exercise mergers, advanced motion vector forecasts to improve coding efficiency. Inter prediction mode is divided into SKIP mode, motion merge technology (MERGE mode), advanced motion vector prediction technology (AMVP). Where SKIP mode is a special mode in MERGE, which distinguishes no transmission residual information and MV information when transmitting. The following describes the inter prediction modes.
MERGE mode
The MERGE mode uses the motion information of the adjacent PU block to estimate the motion information of the current PU block, and the encoder selects the optimal motion information from the reference list composed of the time-space domain adjacent PU block, and transmits it to the decoding end.
AMVP technology
The AMVP technology and the Merge mode prediction process are similar, but also to predict the motion information of the current PU block using the motion information of the airspace and the time domain adjacent block. AMVP technology estimates the process of estimating the current encoding block motion information, including the Merge mode, including the candidate list, the selection process of the optimal motion vector, just the number of candidate, as shown in the figure, the AMVP candidate list build process The five positions of the airspace are identical to the five positions in the MERGE mode, but the final selection is two optimal positions, one from the upper block, the other from the left block. The selection of time domain motion vectors is the motion vector of the time domain adjacent prediction units in two different prediction directions as the measurement value, and select an optimal one as the time domain motion vector. When the domain and the null galler candidate subset were completed, the repeated motion vector was first removed. Second, the total number of motion vectors was 2, and if it was more than 2, the first two removing the cable value greater than 1, if less than 2 added zero Sport vector.
20, go to square filter
The HEVC's goal filter is similar to H.264, but since the size of the TU in the HEVC can be larger than the PU, it is not possible to select the block boundary as H.264, but from the TU and PU to select a smaller boundary to filter. Moreover, in order to reduce the responsibility, it is not filtered with the 4 * 4 border. As shown in the figure above, the boundary filtering is divided into three situations: filtering, weak filtering, and strong filtering. The filter type is determined by the boundary strength BS, threshold β, and TC.
Reference Blog: HEVC Standard Overview - Very detailedHEVC video encoding technology
Our other product:
