JP5863254B2 - CONVERSION DEVICE, INFORMATION DISTRIBUTION DEVICE, COMMUNICATION SYSTEM, CONTROL METHOD, AND PROGRAM - Google Patents

Description

  The present invention relates to a technique for converting various types of data into MP4 file format data.

  The MP4 file format data consists of various box information. As shown in FIG. 6, "ftyp box" for storing file identification information, "mdat box" for storing stream data, and metadata for stream data are stored. It has a “moov box” and so on.

  The stream data includes video data, audio data, caption data, and the like, and is stored in the “mdat box” in units of “chunk”. Note that video data, audio data, caption data, and the like are stored in an “mdat box”. One “chunk” is data of a unit obtained by bundling a plurality of “samples”. “Sample” is data of the minimum unit of reproduction.

  The box information has a hierarchical structure, and the box information has other box information. For example, the “moov box” has the “stbl box” shown in FIG. 7, and the “stbl box” is further hierarchized. As shown in FIG. 7, the “stsd box”, “stts box” It has box information of “box”, “stsz box”, “stsc box”, and “stco box”.

  The “stbl box” is access information of stream data stored in the mdat box. The “stsd box” is codec information, which is information that is referred to when reproducing stream data. The “stts box” is time information in units of samples. The “stsz box” is size information in units of samples. The “stsc box” is information on the number of samples constituting each chunk. The “stco box” is the head position information (offset value from the head of the ftyp box) of each chunk.

  To access video and audio data stored in the “mdat box” for standard MP4 file format data, refer to the contents of the “stsc box”, “stco box”, and “stsz box” shown in FIG. The position and size of the video data and the audio data are grasped from these three box information. Also, in order to synchronize the video data and the audio data during data reproduction, the “stts box” is referred to and the reproduction time is grasped.

  Note that standard MP4 file format data does not have chapter information (chapter number, playback reference time (equivalent to PTS), stream data position information, etc.), so when performing special playback such as skip or seek, The playback start position corresponding to the playback start time specified at the time of playback request is specified based on the metadata stored in the “moov box”, and the stream data corresponding to the specified playback start position is specified in the “mdat box”. Will be identified and played.

  Specifically, referring to the “stbl box”, the time in samples described in the “stts box” of the “stbl box”, the number of samples in chunks described in the “stsc box”, and Based on, it is specified which chunk the playback start time is included in (step S′1). Since the time in one chunk is obtained from the time in samples and the number of samples in chunks (time in one chunk = time in samples x number of samples in chunks), the playback start time in step S'1 It is possible to specify which chunk is included (reproduction start time≈total time of each chunk unit (time of sample unit × α)) (where α is the number of samples).

  Next, based on the position information of each chunk described in “stco” (offset value from the ftyp box head (= file head)), the chunk position information (ftyp box) obtained in step S'1 (Offset value from the top) is specified (step S'2). Since it was possible to identify the number of chunks (α) in the process of step S'1, the start position of the specified chunk is specified based on the position information of each chunk (offset value from the top of the ftyp box) can do.

  Next, based on the sample unit size information (byte) described in the “stsz box” and the chunk position information (offset value from the top of the ftyp box) obtained in the process of step S′2 above. Then, the reproduction start position corresponding to the reproduction start time is calculated (step S′3). Since the start position of the chunk can be specified by the process of step S'2, and what number sample (α) can be specified by the process of step S'1, it was specified by the process of step S'1 The reproduction start position corresponding to the reproduction start time can be calculated by multiplying the sample number (α) by the size information in units of samples. The above process is performed for each of video data and audio data.

  Thereby, based on the metadata stored in the “stbl box”, it is possible to identify the reproduction start position corresponding to the reproduction start time specified at the time of the reproduction request and reproduce the stream data.

  However, when special reproduction such as skip or seek is performed, it is necessary to perform the above-described processing one by one, and thus a load is imposed before the reproduction is started.

  For this reason, when performing special reproduction such as skip or seek, it is necessary to develop a mechanism capable of grasping the reproduction start position without performing the above-described processing one by one.

  In order to grasp the reproduction start position of the stream data in the MP4 file format stream data without performing the above-mentioned processes one by one, the chapter information (chapter number, reproduction reference time (corresponding to PTS), stream data position information) Etc.). However, as described above, since standard MP4 file format stream data does not have chapter information, it is preferable to embed chapter information in MP4 file format stream data. That is, it is preferable to embed related information related to the stream data in the stream data in the MP4 file format.

  As a technical document filed prior to the present invention, a technique for recording reproduction information of an MP4 file in a thumbnail file by embedding and recording the reproduction information of a moving image when the thumbnail file is acquired. (For example, see Patent Document 1: Japanese Patent Laid-Open No. 2007-166553).

JP 2007-166553 A

  Japanese Patent Application Laid-Open No. 2004-151858 discloses a technique for recording reproduction information of an MP4 file in a thumbnail file so that the reproduction information of a moving image can be acquired when the thumbnail file is acquired.

  However, the thumbnail file of Patent Document 1 is stored in the “moov box” of MP4 and is played back using the “moov box” of MP4. Normally, the “moov box” is often placed behind the “mdat box” as shown in FIG. For this reason, in the invention of Patent Document 1, a thumbnail file cannot be acquired until a “moov box” arranged behind the “mdat box” is read.

  As a result, in the invention of the above-mentioned Patent Document 1, after reading a moving image, reproduction information related to the moving image is acquired, and it takes time to acquire reproduction information related to the moving image.

The present invention has been made in view of the above circumstances, and an object thereof is to obtain the relevant information related to the stream data efficiently.

  In order to achieve this object, the present invention has the following features.

The conversion device according to the present invention includes:
Format conversion means for sampling stream data having chapters in the MPEG2-TS format into samples in the MP4 file format, generating stream data in units of samples, and outputting a reproduction reference time of each sample;
Chunk data generating means for generating stream data in units of chunks by collecting stream data in units of samples;
Read chapter information that reads the chapter reproduction reference time in the MPEG2-TS format stream data with reference to a chapter table that stores the chapter in the MPEG2-TS format stream data in association with the chapter reproduction reference time. Means,
The playback reference time of each sample output by the format conversion means is compared with the playback reference time of the chapter read by the chapter information reading means, and is placed in front of the moov box for storing stream data metadata. And at least a chapter identifier for identifying the chapter at the beginning of the stream data in units of chunks including the sample associated with the reproduction reference time that matches the reproduction reference time of the chapter. Marker generating means for embedding chapter marker information including time, and
It is characterized by having.

  According to the present invention, related information related to stream data can be efficiently acquired.

It is a figure which shows the system configuration example of the communication system of this embodiment. It is a figure which shows the internal structural example of the information delivery server 100 of this embodiment. 3 is a diagram illustrating an internal configuration example of an MP4 format conversion unit 12 included in the information distribution server 100. FIG. It is a figure which shows the state which embedded the chapter marker information in the head part of the chunk. 6 is a diagram illustrating an example of processing operation of an MP4 format conversion unit 12. FIG. It is a figure which shows the data structural example of MP4 file format. It is a figure which shows the correspondence of the data hierarchy of a stbl box, the data in a mdat box, and a stbl box.

<Outline of Communication System of this Embodiment>
First, the outline of the communication system of this embodiment will be described with reference to FIGS. 1 and 2.

  As shown in FIG. 1, the communication system of the present embodiment includes an information distribution server 100 that distributes stream data, terminal devices 200-1 to 200-n (n is an arbitrary integer) that receives stream data, It is the communication system which comprises.

  As shown in FIG. 2, the information distribution server 100 according to the present embodiment includes a data generation unit 1 that generates stream data in the MP4 file format in which relation information related to stream data is embedded in an mdat box, and an MP4 file format stream. And a distribution control unit 2 that distributes data.

  Also, the terminal devices 200-1 to 200-n according to the present embodiment receive the MP4 file format stream data, read the relationship information embedded in the mdat box of the MP4 file format stream data, and acquire the relationship information. .

  The communication system according to the present embodiment has the above-described configuration, so that the terminal devices 200-1 to 200-n can efficiently acquire related information related to stream data. Hereinafter, the communication system of the present embodiment will be described in detail with reference to the accompanying drawings.

<System configuration example of communication system>
First, a system configuration example of a communication system according to the present embodiment will be described with reference to FIG.

  The communication system according to this embodiment includes the information distribution server 100 and terminal devices 200-1 to 200-n (n is an arbitrary integer).

  The information distribution server 100 is a device that distributes stream data to each of the terminal devices 200-1 to 200-n.

  The terminal devices 200-1 to 200-n are devices that receive stream data distributed from the information distribution server 100 and reproduce the received stream data.

<Internal configuration example of information distribution server 100>
Next, an internal configuration example of the information distribution server 100 according to the present embodiment will be described with reference to FIG.

  The information distribution server 100 of this embodiment includes a data generation unit 1 and a distribution control unit 2. The data generation unit 1 converts MPEG2-TS format stream data stored in the HDD 300 into MP4 file format stream data, and generates MP4 file format stream data to be distributed to the terminal devices 200-1 to 200-n. To do. The distribution control unit 2 performs distribution control for distributing the MP4 file format stream data generated by the data generation unit 1 to the terminal devices 200-1 to 200-n. The delivery method when delivering the stream data to the terminal devices 200-1 to 200-n is not particularly limited, and any delivery method can be applied. For example, DTCP-IP distribution can be applied.

  The HDD 300 stores MPEG2-TS format stream data (recorded programs A, B, and C) 301 and a chapter table 302. The chapter table 302 manages chapter information (chapter number, playback reference time (corresponding to PTS), stream data position information, etc.) corresponding to MPEG2-TS format stream data (recorded programs A, B, and C). Yes.

  The data generation unit 1 includes a descrambling unit 11, an MP4 format conversion unit 12, a data embedding unit 13, and an encryption unit 14.

  The descrambling unit 11 decrypts the MPEG2-TS format stream data stored in the HDD 300.

  The MP4 format conversion unit 12 converts the MPEG2-TS stream data that has been descrambled by the descrambling unit 11 into MP4 file format stream data.

  The data embedding unit 13 generates various data to be embedded in the stream data in the MP4 file format. For example, when performing DTCP-IP distribution, copy control information and the like are generated.

  The encryption unit 14 embeds various data generated by the data embedding unit 13 in the stream data of the MP4 file format converted by the MP4 format conversion unit 12. Also, the MP4 file format stream data embedded with various data is encrypted, and the MP4 file format stream data to be distributed to the terminal devices 200-1 to 200-n is generated. The encryption method is not particularly limited, and any encryption can be applied depending on the distribution method.

  The data generation unit 1 of the present embodiment has the above configuration, converts the stream data in the MPEG2-TS format stored in the HDD 300 into stream data in the MP4 file format, and the terminal devices 200-1 to 200-n MP4 file format stream data can be generated.

<Detailed configuration example of MP4 format converter 12>
Next, a detailed configuration example of the MP4 format conversion unit 12 of the present embodiment will be described with reference to FIG.

  The MP4 format conversion unit 12 of the present embodiment includes a stream analysis unit 121, a metadata generation unit 122, a format conversion unit 123, a chunk data generation unit 124, an access information generation unit 125, and a chapter information reading unit 126. A marker generating unit 127 and a data combining unit 128.

  The stream analysis unit 121 analyzes the MPEG2-TS format stream data received by the MP4 format conversion unit 12.

  The metadata generation unit 122 generates metadata based on the analysis result of the stream analysis unit 121. Metadata is data for the moov box.

  The format conversion unit 123 converts the MPEG2-TS format stream data into the MP4 sample format stream data. Thereby, video data and audio data in units of samples can be generated. The format converter 123 manages the PTS included in the MPEG2-TS stream data, and manages time information so that the video data and audio data in units of samples can be synchronized.

  The chunk data generation unit 124 generates chunk data composed of a plurality of samples from the sample-unit stream data sampled by the format conversion unit 123. Thereby, video data and audio data in units of chunks can be generated. As a method of dividing into chunk units, in the case of MPEG2-TS format stream data, a sample corresponding to one GOP unit is divided into samples of one chunk unit. Note that the chunk size (number of samples) in units of chunks may be set as a default in advance, or may be dynamically changed according to a request from the terminal device 200-n that is a delivery destination for delivering stream data. Is possible.

  The access information generation unit 125 generates access information necessary for accessing the chunk-unit stream data generated by the chunk data generation unit 124. The access information is data for the stbl box.

  The chapter information reading unit 126 refers to the chapter table 302 and reads chapter information (chapter number, playback reference time (corresponding to PTS), position information of stream data, etc.) corresponding to MPEG2-TS format stream data. For example, when the MPEG2-TS stream data analyzed by the stream analysis unit 121 corresponds to the recorded program A, the chapter information corresponding to the recorded program A is read.

  Based on the chapter information read by the chapter information reading unit 126, the marker generation unit 127 includes chapter marker information (chapter number, reproduction reference time (equivalent to PTS), next chapter position information (next chapter marker position information), etc.) And the generated chapter marker information is embedded in the head portion of the chunk data corresponding to the chapter information. The marker generation unit 127 compares the PTS of each sample managed by the format conversion unit 123 with the PTS of the chapter information, and the chapter marker is included at the beginning of the chunk data including the PTS of the sample that matches the PTS of the chapter information. Embed information. Thereby, as shown in FIG. 4, chapter marker information can be embedded in the head part of a chunk.

  The data combining unit 128 combines the metadata generated by the metadata generating unit 122, the access information generated by the access information generating unit 125, and the chunk data generated by the chunk data generating unit 124, and the MP4 file format Generate stream data. As a result, the data combining unit 128 can combine box information such as ftyp, mdat, moov, and stbl.

  The MP4 format conversion unit 12 of the present embodiment has the above-described configuration, and can convert MPEG2-TS format stream data to MP4 file format stream data. Also, the MP4 format conversion unit 12 converts the MPEG2-TS format stream data into the MP4 file format stream data, so that the chapter information corresponding to the MPEG2-TS format stream data (chapter number, playback reference time (PTS Equivalent), stream data position information, etc.) to generate chapter marker information (chapter number, playback reference time (equivalent to PTS), next chapter position information (next chapter marker position information), etc.) Chapter marker information can be embedded at the head of the chunk corresponding to.

<Example of processing operation of MP4 format converter 12>
Next, an example of processing operation of the MP4 format conversion unit 12 will be described in detail with reference to FIG.

  The stream analysis unit 121 analyzes MPEG2-TS format stream data (step S1).

  The metadata generation unit 122 generates metadata based on the analysis result of the stream analysis unit 121 (step S2).

  The format conversion unit 123 samples the MPEG2-TS format stream data in units of samples, and generates video data and audio data in units of samples (step S3).

  The chunk data generation unit 124 divides the sample-unit stream data sampled by the format conversion unit 123 into chunk units, and generates chunk data including a plurality of samples (step S4). Thereby, video data and audio data in units of chunks can be generated.

  Next, the access information generation unit 125 generates access information necessary for accessing the chunk-unit stream data generated by the chunk data generation unit 124 (step S5).

  The chapter information reading unit 126 refers to the chapter table 302 and reads chapter information (chapter number, playback reference time (corresponding to PTS), position information of stream data, etc.) corresponding to MPEG2-TS format stream data ( Step S6).

  Based on the chapter information read by the chapter information reading unit 126, the marker generation unit 127 determines whether or not the chunk unit stream data generated by the chunk data generation unit 124 corresponds to the chapter position (step S7). When it corresponds to the chapter position (step S7 / Yes), based on the chapter information read by the chapter information reading unit 126, the chapter marker information (chapter number, playback reference time (corresponding to PTS), next chapter position information (next Etc.), and the generated chapter marker information is embedded in the head portion of the chunk data (step S8), and the data is multiplexed (step S9). In addition, when the chunk unit stream data generated by the chunk data generation unit 124 does not correspond to the chapter position (step S7 / No), the marker generation unit 127 does not generate and embed chapter marker information in step S8, Data multiplexing is performed (step S9).

  The data combining unit 128 determines whether or not the data is at the end (step S10). If the data is not at the end (step S10 / No), the process proceeds to step S4 and step S5, and the chunk data generation unit 124 Unit stream data is generated (step S4), and the chapter information reading unit 126 reads chapter information (step S6).

  Further, when the data is the end (step S10 / Yes), the data combining unit 128, the metadata generated by the metadata generating unit 122, the access information generated by the access information generating unit 125, and the chunk data generating unit The chunk data generated in 124 is combined (step S11), and stream data in the MP4 file format is generated. Thereby, the MP4 format conversion unit 12 converts the MPEG2-TS format stream data into the MP4 file format stream data, so that the chapter information (chapter number, reproduction reference time ( Chapter marker information (chapter number, playback reference time (equivalent to PTS), next chapter position information (next chapter marker position information), etc.) is generated based on Chapter marker information can be embedded at the head of the chunk corresponding to the information.

<Example of processing operation of terminal device 200-n>
Next, a processing operation example of the terminal device 200-n according to the present embodiment will be described.

  When the terminal device 200-n receives MP4 file format stream data from the information distribution server 100, the terminal device 200-n analyzes the stream data, and chapter marker information (chapter number, playback reference time (corresponding to PTS) included in the mdat box Next chapter position information (next chapter marker position information) is read and acquired. Thereby, the terminal device 200-n can acquire the reproduction start position of the stream data based on the chapter marker information.

  In the case of an existing terminal device that does not have a function for reading chapter marker information included in the mdat box, the chapter marker information included in the madat box is not recognized and ignored as in the conventional case. For this reason, even if MP4 file format stream data containing chapter marker information in the mdat box is distributed to an existing terminal device, the existing terminal device can perform the reproduction processing of the stream data without any problem as before. it can. However, when special playback such as skip or seek is performed in an existing terminal device, the playback start position is specified based on the metadata included in the moov box as in the conventional case.

<Operation / Effect of Communication System of this Embodiment>
As described above, the information distribution server 100 according to the present embodiment generates the MP4 file format stream data in which the chapter marker information related to the stream data is embedded in the mdat box, and the generated MP4 file format stream data is the terminal device. 200-1 to 200-n, the terminal devices 200-1 to 200-n receive the MP4 file format stream data, and read the chapter marker information embedded in the mdat box of the MP4 file format stream data The chapter marker information is acquired. Thereby, the terminal devices 200-1 to 200-n can efficiently acquire chapter marker information related to the stream data. As a result, the terminal devices 200-1 to 200-n can acquire the reproduction start position of the stream data based on the chapter marker information.

  The above-described embodiment is a preferred embodiment of the present invention, and the scope of the present invention is not limited to the above-described embodiment alone, and various modifications are made without departing from the gist of the present invention. Implementation is possible.

  For example, the control operation in each device configuring the communication system of the present embodiment described above can be executed using hardware, software, or a combined configuration of both.

  In the case of executing processing using software, it is possible to install and execute a program in which a processing sequence is recorded in a memory in a computer incorporated in dedicated hardware. Alternatively, the program can be installed and executed on a general-purpose computer capable of executing various processes.

  For example, the program can be recorded in advance on a hard disk or ROM (Read Only Memory) as a recording medium. Alternatively, the program can be stored (recorded) temporarily or permanently in a removable recording medium. Such a removable recording medium can be provided as so-called package software. Examples of the removable recording medium include a floppy (registered trademark) disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto optical) disk, a DVD (Digital Versatile Disc), a magnetic disk, and a semiconductor memory.

  The program is installed in the computer from the removable recording medium as described above. In addition, it is wirelessly transferred from the download site to the computer. In addition, it is transferred to the computer via a network by wire.

  In addition, the information distribution server 100 and the terminal device 200-n in the present embodiment are not only executed in time series according to the processing operation described in the above embodiment, but also the processing capability of the device that executes the processing or necessary It is also possible to construct to execute in parallel or individually according to the above.

100 Information distribution server (information distribution device)
1 Data generator 11 Decryptor 12 MP4 format converter (conversion device)
121 Stream analysis unit 122 Metadata generation unit 123 Format conversion unit 124 Chunk data generation unit 125 Access information generation unit 126 Chapter information reading unit 127 Marker generation unit 128 Data combination unit 13 Data embedding unit 14 Encryption unit 2 Distribution control unit 200 -1 to n terminal device 300 HDD
301 Stream data 302 Chapter table

Claims (7)

Format conversion means for sampling stream data having chapters in the MPEG2-TS format into samples in the MP4 file format, generating stream data in units of samples, and outputting a reproduction reference time of each sample;
Chunk data generating means for generating stream data in units of chunks by collecting stream data in units of samples;
Read chapter information that reads the chapter reproduction reference time in the MPEG2-TS format stream data with reference to a chapter table that stores the chapter in the MPEG2-TS format stream data in association with the chapter reproduction reference time. Means,
The playback reference time of each sample output by the format conversion means is compared with the playback reference time of the chapter read by the chapter information reading means, and is placed in front of the moov box for storing stream data metadata. And at least a chapter identifier for identifying the chapter at the beginning of the stream data in units of chunks including the sample associated with the reproduction reference time that matches the reproduction reference time of the chapter. Marker generating means for embedding chapter marker information including time, and
The conversion apparatus characterized by having. The marker generating means includes
Based on the chapter read by the chapter information reading means, it is determined whether or not the chunk unit stream data generated by the chunk data generating means corresponds to a chapter position. When the marker information is generated, the generated chapter marker information is embedded in the head portion of the stream data in chunk units, and the stream data in chunk units generated by the chunk data generation means does not correspond to the chapter position, the chapter marker The conversion apparatus according to claim 1, wherein information is not generated and embedded. Storage means for storing MPEG2-TS stream data and the chapter table;
The format conversion means includes
3. The conversion apparatus according to claim 1, wherein the stream data in the MPEG2-TS format stored in the storage means is sampled in units of samples in the MP4 file format. An information distribution device having the conversion device according to any one of claims 1 to 3,
An information distribution apparatus that generates MP4 file format stream data based on chunk-unit stream data generated by the chunk data generation means, and distributes the generated MP4 file format stream data. A communication system comprising an information distribution device and a terminal device,
The information distribution apparatus includes:
Format conversion means for sampling stream data having chapters in the MPEG2-TS format into samples in the MP4 file format, generating stream data in units of samples, and outputting a reproduction reference time of each sample;
Chunk data generating means for generating stream data in units of chunks by collecting stream data in units of samples;
Read chapter information that reads the chapter reproduction reference time in the MPEG2-TS format stream data with reference to a chapter table that stores the chapter in the MPEG2-TS format stream data in association with the chapter reproduction reference time. Means,
The playback reference time of each sample output by the format conversion means is compared with the playback reference time of the chapter read by the chapter information reading means, and is placed in front of the moov box for storing stream data metadata. And at least a chapter identifier for identifying the chapter at the beginning of the stream data in units of chunks including the sample associated with the reproduction reference time that matches the reproduction reference time of the chapter. Marker generating means for embedding chapter marker information including time, and
Distribution control means for generating stream data in MP4 file format based on the chunk-unit stream data generated by the chunk data generation means, and distributing the generated MP4 file format stream data;
The terminal device
A communication system that receives stream data in the MP4 file format, reads the chapter embedded in the stream data in the MP4 file format, and acquires the chapter. A control method for a conversion device,
A format conversion step of sampling stream data having a chapter in the MPEG2-TS format into samples in the MP4 file format, generating stream data in units of samples, and outputting a reproduction reference time of each sample;
Chunk data generation step of generating stream data in chunk units by collecting the stream data in sample units;
Read chapter information that reads the chapter reproduction reference time in the MPEG2-TS format stream data with reference to a chapter table that stores the chapter in the MPEG2-TS format stream data in association with the chapter reproduction reference time. Process,
The playback reference time of each sample output by the format conversion step is compared with the playback reference time of the chapter read by the chapter information reading step, and is placed in front of the moov box for storing stream data metadata. And at least a chapter identifier for identifying the chapter at the beginning of the stream data in units of chunks including the sample associated with the reproduction reference time that matches the reproduction reference time of the chapter. And a marker generation step of embedding chapter marker information including the time. Sample conversion of MPEG2-TS format stream data into MP4 file format samples, generates stream data in units of samples, and outputs a playback reference time for each sample;
Chunk data generation processing for generating stream data in units of chunks by collecting the stream data in units of samples;
Read chapter information that reads the chapter reproduction reference time in the MPEG2-TS format stream data with reference to a chapter table that stores the chapter in the MPEG2-TS format stream data in association with the chapter reproduction reference time. Processing,
The playback reference time of each sample output by the format conversion process is compared with the playback reference time of the chapter read by the chapter information reading process, and placed before the moov box for storing the metadata of the stream data And at least a chapter identifier for identifying the chapter at the beginning of the stream data in units of chunks including the sample associated with the reproduction reference time that matches the reproduction reference time of the chapter. A program for causing a computer to execute marker generation processing for embedding chapter marker information including time.

Images