"Article Directory
1. The code stream H264 / H265 in the NAL unit Header Introduction parsing NAL unit HeaderH264 NAL Header embodiment the H264 type: H.264 type commonly used type
H265 NAL Header resolve H265 type way: H.264 type commonly used type
Packed single NAL principle 2.H.265 RTP packet: Single NAL unit packet polymerization package: Aggregation packet (AP) slicing unit: Fragmentation unit (FU) PACI carrying RTP packet (not used, did not understand) the practical application
3. VLC media player rtsp request, Wireshark capture frame configuration analysis, single NAL unit packet transmission sub-slices, Fragmentation unit (FU)
Reference Documents
1. The code stream in H264 / H265 NAL unit Header Introduction
Code stream are made of one NALU (Network Abstract Layer) consisting of a continuous network abstraction layer, wherein the NALU = [StartCode] + [NALU Header] + [NALU Payload], where
StartCode: indicates the start of a NALU, under normal circumstances is a 4-byte "00000001" or 3 bytes "000 001", generally 4 bytes majority.
NALU Header: header information represents a set of video coding, and specifically the following analysis.
Payload: payload indicates the original sequence of bytes.
NAL unit Header
H264 and H265 compare the original code stream NAL unit header below. Nal Unit which is a header byte H264, H265 into two bytes, each bit specifically as follows:
H264 NAL Header
--------------------
| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
--------------------
| F | N_R | Type |
1234
F: forbidden_zero_bitN_R: nal_reference_idctype: nal_unit_type1 bit2 bits5 bits typically 0, an error of 1 indicates that this unit, the decoder discards the data loss by the processing. VCL reference frame may be characterized property, non-zero reference frame, non-reference frames 0, Non-VCL may be discarded or not decoded during characterization, such as non-discardable SPS PPS is 1, SEI NAL current disposable type 0
H264 type analytical way:
// define the byte following the header [4] is to get rid of "00000001"
type = header [4] & 0x1F
12
H.264 type commonly used type
typetype valuesps7pps8sei (enhanced information frame, may not) 5 6 I frame commonly used P frame 1
H265 NAL Header
---------------------------------------
| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
---------------------------------------
| F | Type | LayerId | Tid |
12345
F: forbidden_zero_bitType: nal_unit_typeLayerId: nuh_layer_idTID: nuh_temporal_id_plus11 bit6 bits6 bits3 bitsH.265 requirement is 0, it is an indication of a grammar is illegal, the .vps 32, sps is 33, pps 34, sei prefix is 39. IDR 19 and 20. encoded information is now level 0, with possible future expansion. time ratings information
H265 type analytical way:
We will resolve nal_unit_type value under normal circumstances, to distinguish in the end is what type of frame.
// define the byte following the header [4] is to get rid of "00000001"
type = (header [4] & 0x7E) >> 1;
//or
type = (header [4] >> 1) & 0x3f;
1234
H.264 type commonly used type
typetype valuevps32sps33pps34SEI39IDR19 common P-frame or 20, the rear frame image 1
2.H.265 RTP packaged principle
If the NALU Slice corresponding to the start of a frame, it is represented by 4 bytes, i.e., 0x00 00 00 01; otherwise, represented by 3 bytes, 0x00 00 01. The following four bytes are to be described as a frame. RTP package is the first four bytes of 0x00 00 00 01 removed, added on PayloadHdr, and then sent out.
According to the standard document is divided into four packages ways:
Single NAL packet: Single NAL unit packet
0123
2. 5. 4. 3. 1 0. 6. 7. 8. 9. 1 0 2 3,456,789,012,345,678,901
+ - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
| PayloadHdr | DONL (conditional) |
+ - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
| |
| NAL unit payload data |
| |
| + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
|: ... OPTIONAL RTP padding |
+ - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
1234567891011
In the SDP sprop-max-don-diff time = 0, DONL can be omitted; when 0
> 8);
APData [3] = (byte) (vps.length & 0xFF);
// Write NALU 1 size into the array (NALU 2 is the sps).
APData [vps.length + 4] = (byte) (sps.length >> 8);
APData [vps.length + 5] = (byte) (sps.length & 0xFF);
// Write NALU 2 size into the array (NALU 3 is the pps).
APData [vps.length + sps.length + 6] = (byte) (pps.length >> 8);
APData [vps.length + sps.length + 7] = (byte) (pps.length & 0xFF);
// Write NALU 1 into the array, then write NALU 2 into the array, then write NALU 3 into the array.
System.arraycopy (vps, 0, stapA, 4, vps.length);
System.arraycopy (sps, 0, stapA, 6 + vps.length, sps.length);
System.arraycopy (pps, 0, stapA, 8 + vps.length + sps.length, pps.length);
return APData;
}
123456789101112131415161718192021222324252627282930313233
Typically, the decoder generally decodes support this, but I do not appear out of the polymerization solution when the package is connected Kang the NVR, these configurations can only transmit frames to the case of a single package.
Slicing unit: Fragmentation unit (FU)
1.The structure of an FU
0123
2. 5. 4. 3. 1 0. 6. 7. 8. 9. 1 0 2 3,456,789,012,345,678,901
+ - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
| PayloadHdr (Type = 49) | FU header | DONL (cond) |
+ - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - |
| DONL (cond) | |
| - + - + - + - + - + - + - + - + |
| FU payload |
| |
| + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
|: ... OPTIONAL RTP padding |
+ - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
2.H.265 PayloadHdr
---------------------------------------
| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
---------------------------------------
| F | Type | LayerId | Tid |
3.The structure of FU header
+ --------------- +
| 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
+ - + - + - + - + - + - + - + - +
| S | E | FuType |
+ --------------- +
* S = variable
* E = variable
* FuType = NAL unit type
* First packet: S = 1, E = 0;
* Tundish: S = 0, E = 0
* Last packet: S = 0, E = 1
12345678910111213141516171819202122232425262728293031323334
Refers to a single NAL packet size <1500-RTP_HEADER_LENGTH - bits (1500 In general, some may be less 1300), the use of a single NAL packet transmission. When this value is larger than the transmission mode when the FU usually, the RTP sections, is divided into first, tundish, the end of the packet data.
In type to the PayloadHdr 49, by setting the start S and the end position E
// Set PayloadHdr (16bit type = 49)
header [0] = 49 << 1;
header [1] = 1;
// Set FU header
// + --------------- +
// | 0 | 1 | 2 | 3 | 4 | 5 | 6 | 7 |
// + - + - + - + - + - + - + - + - +
// | S | E | FuType |
// + --------------- +
header [2] = (byte) type; // FU header type
header [2] + = 0x80; // Start bit
// ........
//
// rtpHeaderLenth = 12; // tcp or udp head
// .......
while (sum = naluLength) {
buffer [rtphl + 2] + = 0x40;
socket.markNextPacket ();
}
senddata ()
// tundish header
buffer [rtphl + 2] + = 0x40;
}
12345678910111213141516171819202122232425262728293031323334353637
PACI carrying RTP packet (not commonly used, did not understand)
0123
2. 5. 4. 3. 1 0. 6. 7. 8. 9. 1 0 2 3,456,789,012,345,678,901
+ - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
| PayloadHdr (Type = 50) | A | cType | PHSsize | F0..2 | Y |
+ - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
| Payload Header Extension Structure (PHES) |
| = + = + = + = + = + = + = + = + = + = + = + = + = + = + = + = + = + = + = + = + = + = + = + = + = + = + = + = + = + = + = + = |
| |
| PACI payload: NAL unit |
|... |
| |
| + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
|: ... OPTIONAL RTP padding |
+ - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - + - +
1234567891011121314
Practical application
In actual fact, used in three formats:
1. a nal unit rtp packed into a package.
2.nal unit is relatively large, a plurality of slices packaged in the rtp.
3. The VPS PPS SPS SEI rtp frame into one package.
In some cases a third way out possible solutions, are usually the first two players must support.
3. VLC media player rtsp request, Wireshark packet capture
Analysis configuration frame, single NAL unit packet
A first data packet received sequentially comprising: VPS, SPS, PPS, SEI the frame data are direct transmitted.
VPS is the beginning of the frame 40, (0x40 >> 1) & 0x3f = 32
SPS frame begins with 42, (42 >> 1) & 0x3f = 33
PPS is the beginning of the frame 44, (44 >> 1) & 0x3f = 34
SEI frame begins with 4E (4E >> 1) & 0x3f = 39
Transmission sub-sections, Fragmentation unit (FU)
The first sub-package
Sub tundish
Sub-packet tail
Here again look FU header defined in this unit:
* The FU header
*
* 01234567
* + - ++ - + + + - + + + + - + +
* | S | e | futype |
* +------------- +
*
* S = variable
* E = variable
* FutyPE = NAL Unit Type
* The first package: s = 1, E = 0;
* Middleware: s = 0, E = 0
* Last packages: s = 0, E = 1
12345678910111213
From the grabbed sub-package picture, you can see the first two bytes of 62 01, where the third bytes are:
93, == "1001 0011 S: 1 E: 0 fytype10011
13, == "0001 0011 S: 0 E: 0 fytype10011
53, == "0101 0011 s: 0 E: 1 fytype10011
12345
FyType is a 10011 conversion decimal 19, so the actual NAL TYPE type 19 is NAL_IDR_W_RADL frame in H265 encoding.
As far as we said, the H265 slice of PayLoadHDR is 2 bits, the value of the fu is 1 bit, so the values of 62 and 01 are obtained, of which payloadHDR is header [0] + header [1], and the value of the value is Header [ 2] General H265 frames TYPE: 19, 20, 1, etc. I frame or P frame
HEADER [0] = 49 << 1; // 62
HEADER [1] = 1; // 01
12
Header [2] = (byte) TYPE: 19, 20, 1 and other I frame or P frame
1
/ / The following analysis of the modifications 6 and 7 points distinguishes the first package, intermediate package, and tail packets
Header [2] + = 0x80; // Start bit 10 + Type
Header [2] = (Byte) (Header [2] & 0x7f); // Middle package 00 + Type
Header [2] + = 0x40; // Tail 1 + TYPE
1234
Reference documentation
: The H.265 RTP package unpacking standard. "
Our other product:
