First, introduction
In the H.264 / AVC video coding standard, the entire system framework is divided into two levels: video coding level (VCL) and network abstraction level (NAL). Among them, the former is responsible for effectively representing the content of video data, while the latter is responsible for formatting data and provides header information to ensure that data is suitable for transmission of various channels and storage media. Therefore, we usually per frame data is a NAL unit (except SPS and PPS). In the actual H264 data frame, the front of the frame often has 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 0000-00-00
As shown below:
The H264 is transmitted in the network NALU, NALU structure is: NAL head + RBSP, the data flow in the actual transmission is shown in the figure:
The NALU head is used to identify what type of data behind the RBSP, and whether he will be used by other frame references and whether there is an error in network transmission.
Among them, RBPs are divided into several types:
2, NAL HEADER
The NALU type is our judged tool, from the official documentation as shown below:
The H264 frame consists of NALU head and NALU body. The NALU head consists of one byte, and its syntax is as follows:
+---------------- + | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | + - + + + - + + - + + + - + | f | NRI | TYPE | + ------------------ +
F: 1 bit forbidden_zero_bit. This bit must be 0 in the H.264 specification.
NRI: 2 Bits NAL_REF_IDC. Take 00 ~ 11, it seems to indicate the importance of this NALU, such as 00's NALU decoder can discard it without affecting the playback of the image, the greater the value, and it means that the more important the current NAL is. It is necessary to be prioritized. If the current NAL is a film belonging to the reference frame, or when the sequence parameter set, or the image parameter set, this syntax element must be greater than 0.
TYPE: 5 BITS NAL_UNIT_TYPE. This type of NALU unit, 1 to 12 are used by H.264, 24 ~ 31 is used by the application other than H.264, as is it:
The next byte after 00 00 00 01 is the NALU type, and after converting it into binary data, the interpretation order is from left to right.
For example, there are 67,68 and 65 after 00000001 above
The binary code of 0x67 is: 0110 01114-8 is 00111, converted to decimal 7, reference secondary: 7 correspondence sequence parameter set SPS
The binary code of 0x68 is: 0110 10004-8 is 01000, converted to decimal 8, reference second map: 8 correspondence image parameter set PPS
The binary code of 0x65 is: 0110 01014-8 is 00101, converted to decimal 5, refer to the second map: 5 corresponds to the IDR image (I frame)
Be
Therefore, it is determined whether or not I frame is: (NALU type & 0001 1111) = 5 NAALU type & 0x1f = 5
For example, 0x65 & 1F = 5
Judgment P Frame 0x61 & 0x1f = 1
Several examples:
SPS and PPS
SPS and PPS contain information parameters required to initialize the H.264 decoder.
The SPS contains parameters for a continuous encoding video sequence, such as identifier SEQ_PARAMETER_SET_ID, frame number, and constraints of POC, number of reference frames, decoded image size, and frame field encoding mode.
The PPS corresponds to a sequence in a sequence, a few images, parameters such as identifier PIC_Parameter_Set_ID, optional seq_parameter_set_id, entropy encoding mode selection identifier, number, initial quantization parameter, and devices filter coefficient adjustment identification.
The beginning and end of NAL
The encoder puts each NAL independently, and puts a group in a packet, because the packets have a header, and the decoder can easily detect NAL's boundary and remove NAL to decode.
There is a start code 0x00 00 01 (or 0x00 00 00 01) before each NAL, the decoder detects each start code, as a starting identifier of NAL, when the next start code is detected, the current NAL ends .
At the same time, H.264 provides that when 0x000000 is detected, the end of the current NAL can be characterized. So what should I do when data is 0x000001 or 0x000000 in NAL? H.264 introduces a prevention of competition mechanisms. If the encoder detects that NAL data exists 0x000001 or 0x000000, the encoder will insert a new byte 0x03 before the last byte, which:
0x000000->0x00000300
0x000001->0x00000301
0x000002->0x00000302
0x000003->0x00000303
When the decoder detects 0x000003, the 03 is discarded and the original data (shell operation) is restored. When the decoder is decoded, first read the NAL data, statistically NAL's length, and then start decoding.
Our other product:
