FMUSER Wirless Transmit Video And Audio More Easier !
es.fmuser.org
it.fmuser.org
fr.fmuser.org
de.fmuser.org
af.fmuser.org ->Afrikaans
sq.fmuser.org ->Albanian
ar.fmuser.org ->Arabic
hy.fmuser.org ->Armenian
az.fmuser.org ->Azerbaijani
eu.fmuser.org ->Basque
be.fmuser.org ->Belarusian
bg.fmuser.org ->Bulgarian
ca.fmuser.org ->Catalan
zh-CN.fmuser.org ->Chinese (Simplified)
zh-TW.fmuser.org ->Chinese (Traditional)
hr.fmuser.org ->Croatian
cs.fmuser.org ->Czech
da.fmuser.org ->Danish
nl.fmuser.org ->Dutch
et.fmuser.org ->Estonian
tl.fmuser.org ->Filipino
fi.fmuser.org ->Finnish
fr.fmuser.org ->French
gl.fmuser.org ->Galician
ka.fmuser.org ->Georgian
de.fmuser.org ->German
el.fmuser.org ->Greek
ht.fmuser.org ->Haitian Creole
iw.fmuser.org ->Hebrew
hi.fmuser.org ->Hindi
hu.fmuser.org ->Hungarian
is.fmuser.org ->Icelandic
id.fmuser.org ->Indonesian
ga.fmuser.org ->Irish
it.fmuser.org ->Italian
ja.fmuser.org ->Japanese
ko.fmuser.org ->Korean
lv.fmuser.org ->Latvian
lt.fmuser.org ->Lithuanian
mk.fmuser.org ->Macedonian
ms.fmuser.org ->Malay
mt.fmuser.org ->Maltese
no.fmuser.org ->Norwegian
fa.fmuser.org ->Persian
pl.fmuser.org ->Polish
pt.fmuser.org ->Portuguese
ro.fmuser.org ->Romanian
ru.fmuser.org ->Russian
sr.fmuser.org ->Serbian
sk.fmuser.org ->Slovak
sl.fmuser.org ->Slovenian
es.fmuser.org ->Spanish
sw.fmuser.org ->Swahili
sv.fmuser.org ->Swedish
th.fmuser.org ->Thai
tr.fmuser.org ->Turkish
uk.fmuser.org ->Ukrainian
ur.fmuser.org ->Urdu
vi.fmuser.org ->Vietnamese
cy.fmuser.org ->Welsh
yi.fmuser.org ->Yiddish
HLS and Http Live Streaming are protocols for real-time streaming defined by Apple. HLS is implemented based on the HTTP protocol. The transmission content includes two parts, one is the M3U8 description file, and the other is the TS media file.
1. M3U8 file
The media file is described in text, which is composed of a series of tags.
#EXTM3U
#EXT-X-TARGETDURATION:5
#EXTINF:5,
./0.ts
#EXTINF:5,
./1.ts
#EXTM3U: The first line of each M3U8 file must be this tag.
#EXT-X-TARGETDURATION: Specify the maximum media segment time length (seconds). The time length specified in #EXTINF must be less than or equal to this maximum value. This value can only appear once.
#EXTINF: Describe the length of a single media file. Media files follow, such as ./0.ts
2. ts file
The ts file is a transport stream file, the main video coding format is h264/mpeg4, and the audio is acc/MP3.
The ts file is divided into three layers: ts layer Transport Stream, pes layer Packet Elemental Stream, es layer Elementary Stream. The es layer is the audio and video data, and the pes layer is the data frame description information such as a timestamp added to the audio and video data. The ts layer is to add the necessary information for the identification and transmission of the data stream in the pes layer
Note: The details are as follows
(1) ts layer The ts packet size is fixed at 188 bytes, and the ts layer is divided into three parts: ts header, adaptation field, and payload. The ts header is fixed at 4 bytes; the adaptation field may or may not exist, and its main function is to fill in data less than 188 bytes; the payload is pes data.
ts header
sync_byte 8b synchronization byte, fixed at 0x47
transport_error_indicator 1b Transport error indicator, indicating that there is a useless byte after the adapt field of the ts header, usually 0, this byte is counted in the length of the adapt field
payload_unit_start_indicator 1b Load unit start indicator, which is marked as 1 at the beginning of a complete data packet
transport_priority 1b Transport priority, 0 is low priority, 1 is high priority, usually 0
pid 13b pid value
transport_scrambling_control 2b Transport scrambling control, 00 means unencrypted
adaptation_field_control 2b Whether to include the adaptation field, '00' is reserved; '01' means no adaptation field, only the payload; '10' means only the adaptation field, no payload; '11' means the adaptation is also included Domain and payload.
continuity_counter 4b Increasing counter, from 0-f, the starting value does not necessarily take 0, but it must be continuous
The content of the ts layer is identified by the PID value, and the main content includes: PAT table, PMT table, audio stream, and video stream. To parse the ts stream, you must first find the PAT table, as long as you find the PAT, you can find the PMT, and then you can find the audio and video streams. The PID value of the PAT table is fixed at 0. The PAT table and the PMT table need to be inserted into the ts stream regularly, because the user may join the ts stream at any time, this interval is relatively small, usually every few video frames to add PAT and PMT. PAT and PMT tables are necessary, and other tables such as SDT (service description table) can also be added, but the hls stream can be played as long as there are PAT and PMT.
PAT table: Its main function is to indicate the PID value of the PMT table.
PMT table: Its main function is to specify the PID value of the audio and video stream.
Audio stream/video stream: Carry audio and video content.
adaption
adaptation_field_length 1B adaptation field length, the number of bytes behind
flag 1B takes 0x50 to indicate PCR or 0x40 to not include PCR
PCR 5B Program Clock Reference, program clock reference, used to recover the system timing clock STC (System Time Clock) consistent with the encoder.
stuffing_bytes xB stuffing bytes, the value is 0xff
The length of the adaptation area shall include one byte identified by the transmission error indicator. pcr is the program clock reference, pcr, dts, pts are all sampled values of the same system clock, pcr is incremental, so it can be set as dts value, audio data does not need pcr. If there is no field, ipad can play, but vlc can not play. When packing the ts stream, the PAT and PMT tables do not have an adaptation field. If the length is not enough, just fill in 0xff. Both video stream and audio stream need to add an adaptation field, which is usually added to the first ts packet and the last ts packet of a frame, and the middle ts packet is not added.
PAT format
table_id 8b PAT table is fixed to 0x00
section_syntax_indicator 1b is fixed at 1
zero 1b is fixed at 0
reserved 2b is fixed to 11
section_length 12b The length of the following data
transport_stream_id 16b Transport stream ID, fixed as 0x0001
reserved 2b is fixed to 11
version_number 5b Version number, fixed at 00000, if PAT changes, the version number will increase by 1
current_next_indicator 1b is fixed to 1, indicating that this PAT table can be used, if it is 0, wait for the next PAT table
section_number 8b is fixed to 0x00
last_section_number 8b is fixed to 0x00
Start loop
program_number 16b When the program number is 0x0000, it means this is NIT, when the program number is 0x0001, it means this is PMT
reserved 3b is fixed to 111
PID 13b PID value corresponding to the content of the program number
End loop
CRC32 32b CRC32 check code of the previous data
PMT format
table_id 8b PMT table has any value, 0x02
section_syntax_indicator 1b is fixed at 1
zero 1b is fixed at 0
reserved 2b is fixed to 11
section_length 12b The length of the following data
program_number 16b channel number, which means the channel associated with the current PMT, the value is 0x0001
reserved 2b is fixed to 11
version_number 5b Version number, fixed at 00000, if PAT changes, the version number will increase by 1
current_next_indicator 1b is fixed at 1
section_number 8b is fixed to 0x00
last_section_number 8b is fixed to 0x00
reserved 3b is fixed to 111
PCR_PID 13b The PID of the TS packet where PCR (program reference clock) is located, designated as the video PID
reserved 4b is fixed to 1111
program_info_length 12b Program description information, designated as 0x000 means no
Start loop
stream_type 8b Stream type, mark is Video or Audio or other data, h.264 encoding corresponds to 0x1b, aac encoding corresponds to 0x0f, mp3 encoding corresponds to 0x03
reserved 3b is fixed to 111
elementary_PID 13b PID corresponding to stream_type
reserved 4b is fixed to 1111
ES_info_length 12b Description information, designated as 0x000 means no
End loop
CRC32 32b CRC32 check code of the previous data
(2) pes layer
The pes layer adds time stamp and other information to each video/audio frame. The pes package contains a lot of content, and we only leave the most commonly used ones.
pes start code 3B start code, fixed at 0x000001
stream id 1B audio value (0xc0-0xdf), usually 0xc0
Video value (0xe0-0xef), usually 0xe0
pes packet length 2B The length of the pes data behind, 0 means unlimited length,
Only the video data length will exceed 0xffff
Flag 1B usually takes the value 0x80, which means the data is not encrypted, has no priority, and is backed up data
Flag 1B value 0x80 means only pts, value 0xc0 means pts and dts
pes data length 1B The length of the following data, the value is 5 or 10
pts 5B 33bit value
dts 5B 33bit value
pts is the display timestamp and dts is the decoding timestamp. Both timestamps are required for video data. The pts and dts of audio data are the same, so only pts is needed. There are two timestamps, pts and dts, which are caused by B frames. The pts of I and P frames are equal to dts. If a video has no B frames, pts will always be the same as dts. Read the video frames sequentially from the file, and the sequence of the extracted frames is the same as the dts sequence. The dts algorithm is relatively simple, the initial value + increment is enough, the pts calculation is more complicated, and the offset needs to be added to the dts.
There are only pts in the audio pes (same as dts), the I and P frames of the video must have both timestamps, and the video B frame only needs pts (same as dts). To pack pts and dts, you need to know the video frame type, but we cannot judge the frame type through the container format. You must parse the h.264 content to get the frame type.
for example:
I P B B B P
Reading order: 1 2 3 4 5 6
dts order: 1 2 3 4 5 6
pts order: 1 5 3 2 4 6
On-demand video dts algorithm:
dts = initial value + 90000 / video_frame_rate, the initial value can be arbitrarily specified, but it is best not to take 0, video_frame_rate is the frame rate, such as 23, 30.
pts and dts are based on timescale, 1s = 90000 time scale, one frame should be 90000/video_frame_rate timescale.
Dividing the timescale of one frame by the sampling frequency can be converted to the playback duration of one frame
On-demand audio dts algorithm:
dts = initial value + (90000 * audio_samples_per_frame) / audio_sample_rate, the value of audio_samples_per_frame is related to the codec, aac value is 1024, mp3 value is 1158, audio_sample_rate is the sampling rate, such as 24000, 41000. AAC decodes a frame of 1024 samples per channel, which means that the duration of a frame is 1024/sample_rate seconds. So the timestamp of each frame is 0, 1024/sample_rate, ..., 1024*n/sample_rate seconds.
The dts and pts of the live video should directly use the time in the live data stream, and should not be calculated by formulas.
(3) es layer
The es layer refers to audio and video data, we only introduce h.264 video and aac audio.
h.264 video:
To pack the h.264 data, we must add a nalu (Network Abstraction Layer unit) to the video data. The nalu includes the nalu header and the nalu type. The nalu header is fixed to 0x00000001 (start of frame) or 0x000001 (in frame). The data of h.264 is composed of slices, and the contents of slices include: video, sps, pps, etc. The nalu type determines the content of the subsequent h.264 data.
F 1b forbidden_zero_bit, h.264 stipulates that it must be 0
NRI 2b nal_ref_idc, value 0~3, indicates the importance of this nalu, I frame, sps, pps usually take 3, P frame usually takes 2, and B frame usually takes 0
Type 5b refer to the table below
nal_unit_type description
0 not used
1 Non-IDR image, IDR refers to key frame
2 slice partition A
3 slice partition B
|
Enter email to get a surprise
es.fmuser.org
it.fmuser.org
fr.fmuser.org
de.fmuser.org
af.fmuser.org ->Afrikaans
sq.fmuser.org ->Albanian
ar.fmuser.org ->Arabic
hy.fmuser.org ->Armenian
az.fmuser.org ->Azerbaijani
eu.fmuser.org ->Basque
be.fmuser.org ->Belarusian
bg.fmuser.org ->Bulgarian
ca.fmuser.org ->Catalan
zh-CN.fmuser.org ->Chinese (Simplified)
zh-TW.fmuser.org ->Chinese (Traditional)
hr.fmuser.org ->Croatian
cs.fmuser.org ->Czech
da.fmuser.org ->Danish
nl.fmuser.org ->Dutch
et.fmuser.org ->Estonian
tl.fmuser.org ->Filipino
fi.fmuser.org ->Finnish
fr.fmuser.org ->French
gl.fmuser.org ->Galician
ka.fmuser.org ->Georgian
de.fmuser.org ->German
el.fmuser.org ->Greek
ht.fmuser.org ->Haitian Creole
iw.fmuser.org ->Hebrew
hi.fmuser.org ->Hindi
hu.fmuser.org ->Hungarian
is.fmuser.org ->Icelandic
id.fmuser.org ->Indonesian
ga.fmuser.org ->Irish
it.fmuser.org ->Italian
ja.fmuser.org ->Japanese
ko.fmuser.org ->Korean
lv.fmuser.org ->Latvian
lt.fmuser.org ->Lithuanian
mk.fmuser.org ->Macedonian
ms.fmuser.org ->Malay
mt.fmuser.org ->Maltese
no.fmuser.org ->Norwegian
fa.fmuser.org ->Persian
pl.fmuser.org ->Polish
pt.fmuser.org ->Portuguese
ro.fmuser.org ->Romanian
ru.fmuser.org ->Russian
sr.fmuser.org ->Serbian
sk.fmuser.org ->Slovak
sl.fmuser.org ->Slovenian
es.fmuser.org ->Spanish
sw.fmuser.org ->Swahili
sv.fmuser.org ->Swedish
th.fmuser.org ->Thai
tr.fmuser.org ->Turkish
uk.fmuser.org ->Ukrainian
ur.fmuser.org ->Urdu
vi.fmuser.org ->Vietnamese
cy.fmuser.org ->Welsh
yi.fmuser.org ->Yiddish
FMUSER Wirless Transmit Video And Audio More Easier !
Contact
Address:
No.305 Room HuiLan Building No.273 Huanpu Road Guangzhou China 510620
Categories
Newsletter