WO2008056284A1

WO2008056284A1 - System for encoding, transmitting and storing an audio and/or video signal, and a corresponding method

Info

Publication number: WO2008056284A1
Application number: PCT/IB2007/054182
Authority: WO
Inventors: Olaf Seupel
Original assignee: Nxp B.V.
Priority date: 2006-11-06
Filing date: 2007-10-15
Publication date: 2008-05-15

Abstract

To reduce the computing and storage capacity required for the management of stored data, in a system (100) for encoding, transmitting and storing an audio and/or video signal in which the signal is encoded in an encoding means (11) and is then transmitted to a storage means (15), it is proposed that the encoding means (11) have associated with it a content-sensing means (12) that is able to sense and transmit a content feature (27), and that the storage means (15) have associated with it a content-analyzing means (16) for processing the content features (27). A corresponding method is also specified.

Description

DESCRIPTION

System for encoding, transmitting and storing an audio and/or video signal, and a corresponding method

FIELD OF THE INVENTION

The invention relates to a system for encoding, transmitting and storing an audio and/or video signal, the signal being encoded in an encoding means and then being transmitted to a storage means, and to a corresponding method.

PRIOR ART

Particularly in the field of home entertainment electronics but also in, amongst other things, office communications, the quantities of data that have to be processed are becoming larger and larger. By means of, for example, hard-disk recorders, or optical storage devices such as DVD recorders, entire feature films and the like can easily be stored to enable the film to be watched, or a particular piece of music to be listened to, etc. at a time when the user wishes. Because of the increasing amounts of data involved, the storage of data and the management of the stored data are of fundamental importance to enable, for example, certain holiday photos or certain feature films that have been recorded to be found again easily even after quite long periods of time. Data management of this kind may for example be effected by means of a hierarchically organized file structure, although in this case each stored file or quantity of data is merely assigned an identifier, which means that the file has to be called up and, for example, looked at if one wishes to make sure what it actually contains. To overcome this problem, it is known for the data, such as a video signal for example, which represents a feature film and is downloaded from the internet, to be encoded in an encoding means. This encoding normally comprises a compression of the data stream, to save on storage space and computing and transmission capacity. The encoded data stream is then transmitted to a storage means, which for example the user has in his home. This may for example be a hard-disk memory or a DVD burner, to enable the data to be stored temporarily or permanently on the relevant media. To allow this data to be managed, a content analysis may be performed on the compressed and stored data stream, but this calls for the data stream to be at least partly, or completely, decoded, i.e. usually to be decompressed, to enable features relating to the content of the original data stream, by means of which it is possible for the stored data to be clearly identified and assigned, to be detected. This however calls for considerable computing and storage capacity at, for example, the end user's end, which results for example in home entertainment electronics products becoming more expensive. As a rule, this also involves offline processing, i.e. the content analysis takes place after recording, taking place in, for example, a standby mode. The information on the content is thus not available immediately on recording.

US 2005/01693667 Al describes a means of monitoring a video signal to enable, for example, a change in the video signal to be detected and an alarm to be given if necessary. The signals are compressed and stored in the process. In this way, a movement in a room for example can be detected and it can thus be concluded that an unauthorized person is present. Simplified management of stored data and the processing of audio signals are not disclosed in this case.

WO/9802827 Al discloses a system for transmitting and storing video signals in which the video signals are encoded and compressed. In the course of this, further signals, for the beginning and end of a video sequence for example, may be assigned to these compressed signals. However, these additional signals are stored in a data memory together with the compressed video signal, which means that, to enable the additional features to be analyzed, the entire stored data stream has, in this case too, to be called up again and decompressed. Passing on of the data stream, together with assigned content signals for example, is not disclosed in this case. Finally, there is known from JP 10-285046 A a system for signal processing in which additional features are likewise added to an audio and/or video signal to enable the content of the data stream to be analyzed. However, these additional signals are compressed and stored too, which means that a relatively large amount of work has to be done to decode them. To reduce the computing and storage capacity required, the analysis of signals for their content is performed only on signals that have already been compressed.

Taking this prior art as a point of departure, the object underlying the invention is to provide a system for encoding, transmitting and storing signals, of the kind specified in the opening paragraph, with which it is possible for various items of stored data to be managed in a simplified manner with no major cost or complication. The intention is also to specify a corresponding method.

These objects are achieved by virtue of the features specified in claims 1 and 6. The key idea behind the invention is for the encoding means to have associated with it, in addition, a content-sensing means by which content features, which may, in essence, be of any desired kind but are preferably as described below, can be sensed and can be transmitted as additional data. This additional data stream is likewise directed to the storage means, the storage means also having an additional content-analyzing means by which the data, or rather content features, transmitted can be processed and stored. A processing operation on the content features transmitted is performed in the content- analyzing means. As soon as the transmission and storage of the data stream proper has been completed, the content-analyzing means compiles a list of all the content features available and stores it separately. It goes without saying that what may be used as a data source is any desired audio and/or video signal such as, amongst others, an analog TV or satellite-receiver signal, a stored film file on a central server or the signal from an image recording device. What may be used as an encoding or compressing means is any desired encoding means such for example as one based on the MPEG-2, MPEG-4 or WMV format. The encoded data stream may be transmitted either via an optical or electrical cable, or by wireless means, or over the internet. The storage means stores the compressed signals on for example a DVD, a hard disk or some other storage medium.

The advantage of the invention lies in the fact that, because a list of content features for analyzing the content features of an encoded and compressed data stream is stored separately, it is possible for the data material that is stored to be managed and found again easily. The entire stored file does not have to be decoded and decompressed but only the list of content features, which is considerably smaller from the point of view of the volume of data represented by it, which means in particular that less storage and computing capacity is required for managing different stored files, and particularly feature films and the like. The content analysis in question can take place in real time, which means that the information becomes available immediately.

Advantageous embodiments of the invention are described in the respective sets of dependent claims. For the transmission of the additional data which represents the content features of the audio and/or video signal, it is proposed in claims 2 and 7 that this data is transmitted either in parallel with the signals encoded or compressed in a known manner, i.e. on a dedicated transmission channel, or as a metadata stream embedded in these encoded signals. In the case of embedded transmission, it will be clear to the person skilled in the art that, by means of a multiplexer, the data that needs to be transmitted in addition is, as it were, inserted in the original data stream, with a corresponding demultiplexer being provided at the storage means to separate the respective data streams. No such embedding and subsequent separation of the two data streams is needed in the case of the first type of transmission, but it does require an additional transmission channel, which means that one or other of the two ways of transmission can be selected to suit the technical circumstances in the given case.

It is further proposed, in claims 3 and 8, that the content features be placed in a hierarchy. This takes place particularly in three levels ranking as low, medium and high. What may be used as content features in this case are for example any desired items of statistical information on the content of the audio or video signal. What may be used as features for the low level are features such as "brightness", "motion" or "signal strength". These features are assigned either to the audio signal or the video signal and apply to one sequence of the transmitted data. What may be used as content features for the medium level are, amongst others, "cut between scenes in a film" or "silence" in an audio file, which content features are usually more complex than the low-level content features. They too are usually divided into audio and video features. Content features on the highest level may be, amongst others, "film chapter" or "inserted commercial" and are the most complex content features and are preferably not divided into audio and video features. Given this placing in a hierarchy, it is easily possible, when the stored content features are analyzed at a later stage, to go quickly to the stored file by making a search for content features on a desired level. As characterized in claims 4 and 9, three different data packets are used to transmit these content features on different levels in the hierarchy. These are a data packet for content features on the low level for an audio sequence, a data packet for content features on the low level for a video sequence, and a data packet for content features on the medium and high levels for a video sequence. Each of these virtual packets for the data, or rather content features, transmitted begins with a signal for the start of the packet, for the size of the packet and for the type of packet and after this it contains the data for the various content features. If required, a signal may also follow for the end of the data packet. To give an unambiguous assignment of the additional content features to the compressed data stream proper, it is specified in claims 5 and 10 that the content features are synchronized with the transmitted data stream. This can be done in a simple way with time stamps that are for example assigned to a data packet of the kind described above. These time stamps are identical with the reproduction time stamp and the input time stamp for the encoded data stream and are produced by the content-sensing means during the encoding process. As soon as the recording process has been completed, a list of the content features which is synchronized with the associated audio and/or video file is produced in the content- analyzing means with the help of these time stamps and is stored likewise, to allow the said content features to be accessed easily at a later stage.

It goes without saying that the present invention can be used both in the field of private home entertainment electronics and also in the professional field, for compressing video files and for television transmission. Use in digital camcorders is equally possible as also is the processing of DVBT signals.

DETAILED DESCRIPTION OF PREFERRED EMBODIMENTS

These and other aspects of the invention are apparent from and will be elucidated with reference to the embodiments described hereinafter.

In the drawings:

Figs. 1 and 2 show different types of data transmission. Fig. 3 shows the transmission of a data stream and of the associated content features in parallel.

Fig. 4 shows the hierarchical structure of a data packet for content features.

Shown in each of Figs. 1 and 2 is a system 100 for encoding, transmitting and storing an audio and/or video signal. The signals from a data source 10, such for example as a digital or analog television signal, are fed to an encoding means 11 in which they are encoded into, for example, the WMV format and compressed. The encoding means 11 has associated with it in the present case a content-sensing means 12 by which content features 27, preferably hierarchically organized, of the data stream to be encoded can be sensed and passed on. In the embodiment shown in Fig. 1, the encoded date stream is transmitted and passed on to a storage means 15 via a channel 13, and the content features 27 that are generated in addition are transmitted and passed on to the said storage means 15 via a second channel 14. The storage means 15 has for example a hard disk 17 or a DVD burner for the permanent or overwriteable storage of the data transmitted. The storage means 15 has, in addition, a content-analyzing means 16 by which the content features transmitted from the content-sensing means 12 can be sensed, analyzed and processed. Finally, a list of the content features 27 compiled by the content-analyzing means 16 is stored on, for example, the hard disk 17. In the embodiment shown in Fig. 2, the two transmission channels 13, 14 have connected downstream of them a multiplexer 18 that incorporates the signal relating to the content features 27 in the encoded original data stream and feeds it via the channel 19 to a demultiplexer 20, which separates the two different data or signal streams again and conveys them to the storage means 15 via two channels 13, 14, i.e. to the hard disk 17 and the content- analyzing means 16 respectively.

Shown in Fig. 3 is the assignment of data packets 21 to audio and video sequences 22, 23 as it takes place over time, as indicated by the time axis "t". In this case, a data packet 21a for low-level audio content features 27 is assigned to an audio sequence 22_n, a data packet 21b for low -level video content features 27 is assigned to a video sequence 23_n and a data packet 21c for medium and high-level content features 27 is also assigned to the video sequence 23_n. This is repeated for succeeding data packets 21 associated with succeeding audio and video sequences 22, 23, as indicated by the subscripts _n, _n+1.

The hierarchical structure of a data packet 21 of this kind for content features 27 is shown in Fig. 4. Each data packet 21 begins with a start signal 24 followed by a statement of size 25 and a signal for the type of packet 26. Transmitted in succession to this are the various content features 27i to 26_n. A content feature 27 of this kind comprises in turn details of the nature of the feature 28, i.e. whether it is of a high or low level for example, of the size 29 of the feature, and a time stamp 30, followed by the actual data 31 that describes the content feature. This time stamp 30 in turn is made up of an input time stamp 32, a reproduction time stamp 33 and a statement of position 34 within the data stream. Using this statement of position 34, the content feature 27 can be unambiguously assigned, via the time stamps 32, 33, to the stored data stream, to enable the desired sequence to be found easily and without any great amount of computing work. LIST OF REFERENCE NUMERALS

10 Data source

11 Encoding means

12 Content-sensing means

13 Channel

14 Channel

15 Storage means

16 Content-analyzing means

17 Hard-disk memory

18 Multiplexer

19 Channel

20 Demultiplexer

21 Data packet

22 Audio sequence

23 Video sequence

24 Start of packet

25 Size of packet

26 Type of packet

27 Content feature

28 Type of feature

29 Size of feature

30 Time stamp

31 Content-feature data

32 Input time stamp

33 Reproduction time stamp

34 Position in data stream

Claims

1. A system (100) for encoding, transmitting and storing an audio and/or video signal, the signal being encoded in an encoding means (11) and then being transmitted to a storage means (15), characterized in that the encoding means (11) has associated with it a content-sensing means (12) that is able to sense and transmit a content feature (27), and the storage means (15) has associated with it a content-analyzing means (16) for processing the content features (27).

2. A system (100) as claimed in claim 1, characterized in that the content features (27) can be transmitted in parallel with the encoded signals or as an embedded metadata stream.

3. A system (100) as claimed in claim 1 or 2, characterized in that the content features (27) are placed in a hierarchy, in particular on three levels.

4. A system (100) as claimed in any one of claims 1 to 3, characterized in that the content features (27) can be transmitted in data packets (21).

5. A system (100) as claimed in any one of claims 1 to 4, characterized in that the content features (27) are synchronized with the data stream.

6. A method of encoding, transmitting and storing an audio and/or video signal, the signal being encoded in an encoding means (11) and then being transmitted to a storage means (15), characterized in that a content feature (27) is sensed in and transmitted by a content-sensing means (12) associated with the encoding means (11), and is processed in a content-analyzing means (16) associated with the storage means (15).

7. A method as claimed in claim 6, characterized in that the content features (27) can be transmitted in parallel with the encoded signals or as an embedded metadata stream.

8. A method as claimed in claim 6 or 7, characterized in that the content features (27) are placed in a hierarchy, in particular on three levels.

9. A method as claimed in any one of claims 6 to 8, characterized in that the content features (27) are transmitted in data packets (21).

10. A method as claimed in any one of claims 6 to 9, characterized in that the content features (27) are synchronized with the data stream.