Keywords: Asynchronous Audio and Video MPEG-2 PCR DTS PTS Encoder Decoder
With the rapid development of digital television in my country and the advancement of the digital transformation of urban radio and television networks, more and more people have begun to use set-top boxes to watch digital television programs. But in the process of watching TV programs through a set-top box, viewers sometimes find that some audio and video are out of sync. This also caught our attention.
Phenomenon and test
Guiyang City basically completed the digital transformation of its radio and television network at the end of 2007, and the programs of Guizhou TV Station have also entered the digital network transmission. After entering the digital network, we found that several programs of our station had the phenomenon of non-synchronization of audio and video in some areas, especially when the news was broadcast on the satellite video channel and the people channel. In order to find out where the problem is, we decided to conduct a lip sync test on the entire transmission path of our program. The equipment used for the test is Tektronix WFM7120. When doing audio/video delay measurement, it is also necessary to generate a series of short color bar video signals through the TG700 DVG7, and the audio sequence is embedded in this group of video signals with an interval of 5s, send such a signal to the system under test, and finally send the signal to WFM7120 to measure the timing difference between audio and video.
Broadcast control center internal test
As shown in Figure 1, in order to measure whether there is audio/video delay difference in the TV station system, we use the inspection time to record the test signal generated by TG700 into the broadcast hard disk, play it through the hard disk, and input the test signal to the delayer. After the frame synchronization module, it is broadcast on a channel, and then we measure these three signals before the transmission department transmits the signal to the encoder of the network company. The measurement results show that the audio/video delay difference of these three signals does not exceed 12ms, that is, one field is not enough, indicating that the signal does not have the problem of audio and video synchronization in the broadcast control center.
Testing of different set-top boxes
For the second measurement point, we chose the front-end computer room of the network company. As shown in Figure 2, here, we have selected the main brands of set-top boxes currently used in China for testing. After encoding the TG700 test signal through the original encoder we are using, insert it into the channel we are currently broadcasting. Then use a set-top box in the front-end computer room to demodulate the TV signal. The decoded audio/video signal is then sent to WFM7120 for measurement after A/D and embedding of the analog signal through a Panasonic D950 video recorder. The measurement results show that the audio/video delay difference of these types of set-top boxes is different, some are ahead of 150ms, and some are lagging by 300ms. This shows that different set-top boxes have different capabilities to maintain the synchronization relationship between audio/video signals after demodulating and decoding the same digital TV signal.
Testing of different encoders
As shown in Figure 3, we still use the TG700 signal generator to test different encoders, and enable the encoder, modulator and set-top box to build a simulated broadcast/viewing environment. Here, we use several encoders of different brands. After encoding the test signal of TG700, it is modulated by the same modulator, and then the signal is decoded by the same set-top box. It is also processed by D950 and sent to WFM7120 for measurement. The final measurement result is that some of their audio/video delay differences are 30ms, and some reach 300ms, indicating that different encoders have a greater impact on the audio/video synchronization of the final viewing signal of the set-top box.
Cause Analysis
The timing principle of MPEG-2 system
At present, in my country's digital TV transmission system, MPEG-2 standard is an important audio and video compression standard. It compresses, encodes, and multiplexes program signals at the source end, and demultiplexes and decodes signals at the receiving end. Has been widely used. The digital transmission system we are using is based on the MPEG-2 standard. Let's take a look at the system structure of MPEG-2, as shown in Figure 4.
It can be seen from Figure 4 that the audio and video signals form a basic stream after the redundant information is removed by the compression encoder. This elementary code stream cannot be stored or transmitted directly. It must be sent to a specific packer. The elementary code stream is divided into paragraphs according to a certain format, and specific identification characters are added to form the so-called packed elementary code stream (PES). PES packets are audio and video data packets with variable lengths. Then the audio and video PES packets and auxiliary data are sent to the transmission subsystem, which are divided into small data packets with a fixed length of 188b and multiplexed by time division multiplexing. A single TS stream is formed, and the TS stream reaches the receiving end after transmission through the channel.
As we all know, synchronization is a necessary condition for correct TV display. For digital TV, since the buffer is used to store the signal during the compression and encoding process, the time axis of the signal in the multiplexer is changed, plus the amount of data redundancy is different, the compression ratio is also different, so the time axis Great changes, especially in the frame group layer processing, the order of B frames and P frames has also changed. All these make the synchronization of digital TV signals completely lose the concept of the original sequence. An effective way to achieve synchronization is to add a time label to the signal code stream every time a specified interval has passed. With this tag, the receiving end can be re-ordered according to this time tag during the decoding process before display, reconstruct the order of the image before compression and encoding, and the time relationship between sound and image, thereby achieving image synchronization and The sound is synchronized with the image.
It can also be seen from Figure 4 that there is a single common system clock STC (27MHz) in the MPEG-2 encoder. This clock is used to generate a time stamp indicating the correct decoding and display timing of audio/video. At the same time, it can be used to indicate sampling The instantaneous value of the instantaneous system clock time. The clock is phase-locked by the line synchronization of the input video. When the input is an SDI signal, the system clock of the encoder is generated by the clock divided by 10. It is the emergence of a common system clock in the encoder, as well as the regeneration of the clock in the decoder and the correct use of time stamps, that provide the basis for the correct synchronization of operations in the decoder. In order to realize the clock synchronization of the codec, the STC system clock is counted in the encoder, and the sampling value of the counter is transmitted to the receiver in the adaptation header of the selected TS packet every certain transmission time, as a decoding The program clock reference signal of the processor, which is PCR. The PCR valid bit is 42b, among which the high 33b is PCR_Base, which is the count value in the unit of the 27MHz clock and the clock divided by 300, and the low 9b is PCR_Extension, which is the count value in the 27MHz clock as the unit. In addition to PCR, the decoding time label DTS and the display time label PTS are also very important. They are similar to PCR_Base. They are also created with the encoder's 27MHz system clock, divided by 300 as the unit count value. Among them, DTS is used to instruct the decoder when to decode the received image and audio frame, and PTS is used to notify when to display the decoded image frame.
When using two-way encoding, the decoding of a certain image must be carried out within a period of time before it is displayed, so that it can be used as the source data for decoding the B-frame image. For example, the display order of images is IBBP, but the transmission order of images is IPBB. The MPEG reference model believes that decoding occurs instantaneously, that is, decoding and display are performed at the same time. For audio frames and image B frames, the decoding time and display time are the same, and PTS is the same as DTS, so only PTS needs to be transmitted. For video I frames and P frames, due to frame reordering, the decoding time and display time are different, and PTS and DTS must be transmitted at the same time. When the decoder receives the IPBB image sequence, it must decode the I-frame and P-frame images before decoding the first B-frame image. The decoder can only decode one frame of image at a time, so it first decodes the I frame image and stores it. When the P frame image is decoded, it outputs and displays the decoded I frame image, and then decodes and displays the B frame image. Tables 1, 2, 3, and 4 show the sequence of the input and output images of the encoder, the PTS and DTS values of each frame, and the decoding and display sequence of each frame of the image by the decoder.
In Table 1, 13 frames of images constitute a group of images, the first frame I frame uses intra-frame coding, the second and third B frames are obtained by bidirectional prediction from the first and fourth frames, and the fourth frame P frame is passed by the first frame. Derived from forward prediction. After encoding the first frame, the encoder first buffers the second and third frames, encodes the fourth frame, and then encodes the second and third frames, and so on, and the final encoded output sequence is shown in the table 2 shown.
It can be seen from Table 3 and Table 4 that when the decoder receives a certain access unit containing an I frame image, the file data packet should contain DTS and PTS, the time between the values of these two tags The interval is one image period. After the I frame image is the P frame, there should also be a DTS and a PTS in the file data packet, and the time interval between the values of the two tags is three image periods. Then there are two B-frames, the file data packets of which only contain PTS. That is to say, the I frame image will be played and displayed after a delay of one frame after decoding. When the I frame is displayed, the fourth frame P frame is decoded, but it is not played and displayed. It is cached first, and after the 1I frame is played and displayed , Decode and display 2B frames immediately, then 3B frames, then display the buffered 4P frames, and decode and buffer the 7P frames at the same time, and so on. It can be seen that the sequence of decoded and displayed images is consistent with the sequence of image input in Table 1.
Timing principle of decoder (set-top box)
PTS and DTS are just 33b values. If there is no reference to the time axis represented by PCR, this value is meaningless. In order to maintain correct decoding, the system clocks of the encoder and decoder (set-top box) must be kept locked, that is, their frequencies are kept the same, and the initial values of their respective counters are the same.
There is a voltage controlled oscillator (VCO) with a frequency of about 27MHz in the decoder (set-top box). The output signal is sent to the counter as the system clock to generate the current STC sample value, which is a value of 42b like PCR. Among them, the high 33b is the count value in the unit of 27MHz clock after 300 pink frequency, and the low 9b is the count value in the unit of 27MHz clock. When a new program arrives at the decoder (set-top box), the decoder (set-top box) obtains the PCR value from the code stream, compares its PCR_Extention value with the lower 9b bits of the current STC, and obtains the error signal, and then goes through the phase-locked loop circuit. Adjust the voltage-controlled oscillator so that the system clock frequency of the decoder (set-top box) is consistent with the system clock frequency of the encoder. Obtain the PTS and DTS values of each frame sequentially from the code stream, and compare them with the high 33b bits of the current STC value. If the DTS value is greater than the STC value, the code stream is buffered and the STC value change is monitored at the same time. When the STC value increases to equal the DTS value, the frame code stream is decoded. When the STC value is equal to the PTS value, Play the frame. If due to the buffer delay jitter of the transmission network, when the code stream reaches the decoder (set-top box), its PTS value is already less than the STC value, then the decoder (set-top box) skips this frame and discards the frame data. Since PTS and DTS are generated based on the PCR value, the first PCR value obtained must be used as the initial value to set the STC counter of the decoder (set-top box) to make their values the same, otherwise, the time base will be different. , Thus decoding error. The processing of audio and video is similar, but there is no problem of timing rearrangement. Figure 5 shows the working principle diagram of the decoder (set-top box) PCR.
Reasons for out-of-sync audio and video
In practical applications, some encoders cause jitter in their output clock due to the unstable time base of the input video signal, and the frame synchronization interval is not 40ms. For these encoders, after setting the initial DTS value according to PCR and buffering delay, the DTS value of each frame is obtained by adding a fixed value to the previous DTS (this value can be calculated as follows: 27MHz is divided by 300 It is 90kHz, and PAL TV is 25 frames per second. Therefore, the value is 90000/25=3600), and the PTS value is calculated according to the frame type and GOP type. However, the PCR value did not increase by 3600 during this period, which caused DTS and PTS to become larger or smaller relative to PCR. Some decoders (set-top boxes) do not use a voltage-controlled oscillator, and their system clock is a fixed 27MHz, but uses the received PCR value to initialize the value of the local system clock counter. The encoder and the decoder (set-top box) cannot maintain a strict lock, which may cause the decoder (set-top box) to drop frames. However, some decoders (set-top boxes) no longer strictly decode and display according to DTS and PTS after frame loss, but decode according to the situation of the buffer, because the delay of video and audio encoding is different, it may cause audio The painting is out of sync.
In addition, in the transmission process from the encoder to the decoder (set-top box), due to the existence of variable delay buffer links such as multiplexers and modulators, the transmission delay of PCR packets may not be constant, varying from large to small. If PCR is not corrected, the above problems may also occur.
to sum up
From the above analysis, it can be seen that both the encoder and the decoder (set-top box) may cause the occurrence of asynchronization of audio and video. After testing the encoders of various brands, our station chose an encoder with better test indicators and replaced the original encoder, which greatly improved the phenomenon that the TV audio and picture are out of sync. In the next step of introducing set-top boxes, network companies will also strengthen the testing of relevant indicators to improve the quality of audience ratings. Of course, in the process of advancing the digitalization of my country's radio and television, we still need the joint efforts of our television workers and equipment manufacturers in order to finally achieve a complete success.v
Our other product:
