HM reference frame management analysis
The HEVC uses a technique of reference frame set (RPS) to manage the decoded frame, used as a reference for subsequent images. Unlike the previous video coding standard, the reference frame management policy is different, the RPS technology in the HEVC, by transmitting the status of each frame in the DPB directly in each frame, and H.264 / AVC The two reference frame management techniques of sliding windows and MMCO (Memory Management Control Opre) are achieved by transmitting relative changes in each piece of DPB. Once data is lost, there will be a sustained impact.
H.265 Inter Bloom Predictive Frame is divided into: low delay P frame, low delay B frame, random access B frame. 1) The P frame is one-way predictive frame-forward prediction, but the encoding unit in the P frame may be an inter prediction block (forward prediction) or may be an intra prediction block. The P frame typically has four reference frames, the encoder searches for a match block of the current block in 4 reference frames according to the rate distortion; 2) B frame adopts two-way prediction, the encoding unit in the B frame can use inter-frame prediction ( Forward prediction, rearward prediction, two-way predictions can be), or may be intra prediction. The B frame typically has two reference frame lists (LIST0, LIST1), which generally have two reference frames for each reference list. LIST0 is the forward reference list, which is stored in front of the current frame (i.e., the POC is smaller than the current frame of the POC). List1 is a rear reference list, stores the frame behind the current frame (ie, POC is greater than the current frame of the current frame), sometimes There is also a front frame, but for LOWDELAY B, LIST0 and LIST1 are stored in front of the current frame, which is to adapt to low delay requirements.
1. Basic knowledge of reference frame management
Multi-frame reference techniques are used in HEVC HEVC to improve the accuracy of inter prediction. Decoding the completed image, is slowed in the DPB, which is used as a reference image, so the standard needs to be managed in the DPB. Each image in the DPB has three states "unused for reference", "buy for short-term reference", "Used for Long-Term Reference", and the transformation between these three states is by decoding image tagging process. Control. Two ways of sliding windings and MMCO are used in H.264 / AVC. Sliding window management mode, is the number of frames that can be stored in DPB, as the current decoded image, move the new decoded image into the first out, remove the decoded image of the window, so save in DPB It is a recent decoded multiple images. The MMCO is the status tag of the image in the DPB by transmitting the control command in the code stream. It can mark a "Used for Reference" as "unuse for reason" or will The current frame or "Used for short-term referene" frame is marked as "Used for Long-Term Reference", etc.
2, HEVC reference frame set technology (RPS)
In order to enhance the anti-strain resistance of reference frame management, the HEVC uses a reference frame set technology, which transmits the current frame and subsequent frames by transmitting the status change of each frame in the DPB in the slot stream started in each frame. The reference frame is described in the DPB and describes the identity identifier as a frame in PoC. Therefore, it does not need to rely on the DPB status of the previous frame, thereby enhancing anti-strain performance. In short, the first piece of each frame contains an RPS, which is identification information from a set of "ΔΔ PoC" (current frame POC and its POC) and "Reference to the current frame / subsequent frame" Composition, the DPB is marked according to the description of the RPS in the sheet. For example, the B frame of POC = 4 can only refer to the I frame of POC = 0, and its RPS is {(ΔΔPoc = 3, subsidByCurPic)}; POC = 3 B frame, can refer to POC = 4 and POC = 2 two A B frame, RPS is {(ΔΔPoc = -1, subsidBycurPic), (ΔΔPoc = 1, subsidBycurPic)}; if the I frame of POC = 0, it is no longer appearing in the RPS, then it is marked as "unused for reference for Reference" "And then remove from DPB.
Reference frame RPS update:
a) 3 reference frames in the DPB before decoding the frame of POC = 1, respectively, the frames of PoC = 0, 4, 2; b) When receiving the RPS information transmitted in the tip of POC = 1 (describes the decoding POC When the frame of = 1, the state of the DPB, that is, the frame of POC = 0 and 2 is used as a short-terminal reference of the current frame, and the frame of the POC = 4 is used as a reference for subsequent frames. The frame in the DPB is labeled. The frame of POC = 0 and 2 is marked as "Used for short-term reference" and the frame of POC = 4 is retained to be used as a subsequent reference; then decoded the frame of POC = 1, decoding the completion, POC = 1 The frame is placed in the DPB. At this time, there is a POC = 0, 4, 2, 1 four frames; c) When the RPS information transmitted in the tip of POC = 3 (describes the frame of the decoded POC = 3, the DPB, POC = 2 The frame of 4 is used as a short-terminal reference of the current frame, and the frame of POC = 0 is used as a reference for subsequent frames. When the frame of POC = 1 is no longer used as a reference, the frame in the DPB is labeled, and POC The frame labeled "Used for Short-Term Reference" is "Used for Short-Term Reference", and the frame of POC = 1 is removed, and then the frame of POC = 3 is decoded, the decoding is completed, and it is placed in the DPB. At this time, there are POC = 0, 4, 2, 3 four frames in DPB;
3, RPS prediction
As can be seen from the above description, when the reference frame is used to be identified by the POC value, the difference between the plurality of ΔΔPoC values in the RPS of different frames is the same, such as the RPS description of the frame of the POC 1. The ΔΔPoC values are {1, -3, 1}, and the RPS description of the frame of the POC 3, the ΔΔPoC values are {3, -1, 1}, respectively. Therefore, in the RPS description of different frames, ΔΔPoC values can still be predicted, ie ΔΔrps = ΔΔPOC (RPSCURR) -ΔΔPoC (RPSREF) to further reduce the transmission cost.
4, multi-reference frame configuration in HM
The reference software HM uses the GOP structure of Hierachical-B in the RA configuration. The number of reference frames is 4 (or 3), but in the encoding configuration, the number of active reference frames is 2, that is, only the most recent reference frames. Motion estimation and motion compensation. Similarly, under the Lowdelay configuration, the number of reference frames is 4, and the number of reference frames of the active reference frame is 4.
5, HM multi-reference frame optimization selection
In the previous analysis, both HEVCs and their reference software HM support multi-reference frame techniques, and 4 or 2 active reference frames can be configured in HM, which makes the complexity of each PU multiple complexity. Traditional block matching technology is still available in the HEVC reference software, and select the best prediction direction and reference frame index. The process can be simple to summarize as a process:
1) From Listx = 0 (ie forward prediction), set the reference frame index IREFINDEX to 0; 2) Under the current IREFINDEX, generate advanced motion vector prediction (AMVP) candidate list, and select the AMVP index; 3) Amvp selected in (2) is the starting point, in the current Irefindex frame, exercise estimates for SA (T) D + λpred λpred * bits, to get the best motion vector and minimum price; 4) Set IREFINDEX = IREFINDEX +1, repeat steps 2) to 3), get the best motion vector and minimum cost under Irefindex; until all the reference frames in the reference frame queue Listx complements the best motion vector and minimum Calculation, and select the least price of IREFINDEX as the current forecast direction, the best reference index; 5) Put Listx = 1 (ie, backward prediction), repeat 2) to 4), and select the least cost of IREfIndex The best reference index under the current prediction direction; 6) Fixed one of the best reference indexes and motion vectors in LIST0 (or LIST1) as one of the two-way predictions, and all reference indexes in LIST1 (or LIST0) To 4) step, and find the best two-way predictive reference index and motion vector; 7) Compare forward prediction, backward prediction, and two-way predictive minimum cost, select the best prediction direction;
Our other product:
