CN104462354A - Multimedia system with multiple retrieval modes and processing method - Google Patents

Abstract

The invention relates to a multimedia system which comprises a shooting terminal, a server and a query terminal, and is characterized in that the query terminal comprises a third shooting unit, a third input unit, a query unit and a third transmission unit, wherein the query terminal is used by a user for retrieving multimedia through a plurality of methods. The invention further relates to a multimedia method using the multimedia system.

Description

A kind of multimedia system and disposal route with multiple retrieval mode

Technical field

The present invention relates to MultiMedia Field, particularly can carry out the MultiMedia Field retrieved.

Background technology

Along with the widespread use of network and multimedia technology, online training has become the inevitable development trend of staffs training.But online training is faced with " originate single, depart from put into practice, delayed scene " problem, especially lacking with scene cases is the production skills class training of content, and this will certainly affect and be operating as main staffs training effect with production skills.Therefore, actively introduce scene cases teaching, explore the application of intelligent terminal and mobile network, will the new direction of online training and Resources Construction be become.

Summary of the invention

The present invention relates to a kind of multimedia system, comprise camera terminal, server and inquiry terminal; It is characterized in that: inquiry terminal comprises the 3rd image unit, the 3rd input block, query unit, the 3rd transmission unit; Wherein, described inquiry terminal is used for retrieving described multimedia for user by multiple method.

The invention still further relates to a kind of multi-media processing method using described multimedia system, it is characterized in that comprising the following steps:

Step S100, user uses camera terminal to take multimedia, and described multimedia comprises Audio and Video; When a certain position and/or zoom shot, recorded start time and the end time of shooting by timing unit; Subsequently, event, start time and end time are saved in the record sheet of the first storage unit by timing unit in the lump, and wherein said event comprises position, zoom;

Audio conversion in multimedia is word by step S200, after conversion, camera terminal is shown to user, confirms for user; If user thinks that the word content of conversion is wrong or need adjustment, then the first input block of camera terminal is used to modify;

Step S220, camera terminal by the first transmission unit by the record sheet in the multimedia of shooting, the first storage unit and word second transmission unit of uploading onto the server that confirms through user, the multimedia of shooting is saved in database by server, and recording multimedia ID, described multimedia ID is the multimedia unique identification uploaded;

Step S240, server is according to the event in record sheet, start time and end time, use resolution unit to extract key frame from multimedia video, and determine the recording duration of key frame, described key frame is the representative of the frame of video that camera terminal is taken under same event;

Step S260, obtains the duration of key frame that in video, each event is corresponding and event, wherein obtains the duration according to start time of each event and end time;

Step S300, server extracts text index from the word uploaded; For each multimedia, server forms multimedia index table, and is stored in the second storage unit; Described multimedia index table comprises: multimedia ID, text index, key frame set and incident duration set;

Step S400, user uses inquiry terminal to search described multimedia;

Step S420, if what input in inquiry terminal is pure words, then only retrieves, and result for retrieval is presented to user in text index, provides the multimedia number not having explanatory note to user simultaneously;

Step S440, if what input in inquiry terminal is only M photos, then inquiry terminal forms photograph collection X:

X＝{X1,X2,…X i,…XM}

Then compare the every photos in photograph collection X and whether each key frame in key frame set mates, then according to the matching degree of photo and key frame, retrieve corresponding video, and the result for retrieval according to sequencing of similarity is presented to user;

The computing formula of similarity Sim is as follows:

$\mathrm{Sim}=\frac{\underset{j=1}{\overset{M}{\mathrm{\Σ}}}\underset{i=1}{\overset{e}{\mathrm{\Σ}}}\mathrm{Tij}}{M*\sqrt{\underset{i=1}{\overset{e}{\mathrm{\Σ}}}{\mathrm{Ti}}^2}}$

Wherein, M is the number of pictures in photograph collection X; E is the event number in video, i.e. key frame quantity; Ti is the duration of i-th event; The value of Tij has two, when photo and key frame do not mate, and Tij=0; When photo and key frame coupling, Tij is the duration of i-th event of coupling;

Step S460, if inquiry terminal input not only comprise word but also comprise photo, then first retrieve in text index, if result for retrieval is less than specific threshold, then direct result presented to user; Otherwise, if result for retrieval is more than or equal to specific threshold, then character search to result in use the method in step S440 again to retrieve, and result for retrieval is presented to user.

Multimedia system of the present invention and multi-media processing method are specially adapted to take production scene as the scene cases training based on mobile intelligent terminal of background, obtain based on production practices, training contents originates diversified beneficial effect.Further, multimedia system of the present invention and multi-media processing method can make user adopt multiple method to search described multimedia, for user's inquiry provides accuracy and convenience.

Accompanying drawing explanation

Fig. 1 is the block diagram of multimedia system;

Fig. 2 is the schematic diagram of camera terminal change in location.

Embodiment

For making the object, technical solutions and advantages of the present invention clearly, by the present invention is described in further detail by reference to the accompanying drawings.This description is that unrestriced mode describes the embodiment consistent with principle of the present invention by example, the description of these embodiments is enough detailed, to make those skilled in the art to put into practice the present invention, other embodiments can be used without departing from the scope and spirit of the present invention and can change and/or replace the structure of each key element.Therefore, the following detailed description should do not understood from restrictive, sense.

As shown in Figure 1, multimedia system of the present invention comprises camera terminal 100, server 200 and inquiry terminal 300.

Camera terminal 100 comprises the first image unit 110, recoding unit 120, record cell 130, first input block 140, first transmission unit 150, first storage unit 160, display unit 170 etc.Preferably, camera terminal 100 also comprises timing unit 180 (such as timer), focal length detecting unit 190.Server 200 comprises the second transmission unit 210, resolution unit 220, database 230, second storage unit 240 etc.Inquiry terminal 300 comprises the 3rd image unit 310, the 3rd input block 320, query unit 330, the 3rd transmission unit 340 etc.

The treatment scheme of multimedia system is as follows:

In step S100, user uses camera terminal to take multimedia, and described multimedia comprises Audio and Video; When a certain position shooting, recorded the start time and end time of taking in a certain position by timing unit; When follow shot position or conversion focal length, the start time of taking after recording follow shot position or conversion focal length by timing unit and end time; Subsequently, event (comprising position/zoom), start time and end time are saved in the record sheet of the first storage unit by timing unit in the lump.

Describe camera terminal in detail below with reference to Fig. 2 and move to by position A the process that position B moves to position C again, and the timing course of timing unit.

First, at position A, when the first image unit that user opens camera terminal starts to take, automatic triggering timing unit is started timing, now recorded the start time (such as 0 second) of position A by timing unit.Optionally, also by the specific physical button of user operation camera terminal or the specific keys of screen display can be touched with timing triggering timing unit.After a period of time, when having taken at position A, terminate the timing at position A by the above-mentioned specific keys of user operation, and record the end time (such as 180 seconds) by timing unit.Subsequently, the position A of record, start time and end time are saved in the record sheet of the first storage unit by timing unit.

Afterwards, camera terminal is moved to position B and proceed shooting, in the timing of position B by the above-mentioned specific keys of user operation, and recorded the start time (such as 181 seconds) of position B by timing unit.Again after a period of time, when having taken at position B, by the timing of user operation above-mentioned specific keys end position B, and record the end time (such as 270 seconds) by timing unit.Subsequently, the position B of record, start time and end time are saved in the record sheet of the first storage unit by timing unit.

Afterwards, then camera terminal is moved to position C proceed shooting, in the timing of position C by the above-mentioned specific keys of user operation, and recorded the start time (such as 271 seconds) of position C by timing unit.After taking a period of time again, when having taken at position C, first image unit of being closed camera terminal by user stops shooting, now by the timing of automatic triggering timing unit end position C, and records the end time (such as 300 seconds) by timing unit.Optionally, also timing can be terminated by the above-mentioned specific keys of user operation.Subsequently, the position C of record, start time and end time are also saved in the record sheet of the first storage unit by timing unit.

In addition, when user suspends the first image unit shooting of camera terminal, automatic triggering timing unit is terminated timing; And, when the first image unit of camera terminal recovers shooting, also automatic triggering timing unit is continued to start timing.

The automatic triggering of above-mentioned timing unit, without the need to the button of user's operation bidirectional camera terminal, thus improves efficiency.

Record sheet in first storage unit comprises event, start time and end time, and form preferred exemplary is as follows:

Table 1: record sheet

Event	Start time Tb	End time Te
			Position A	0 second	180 seconds
Position B	181 seconds	270 seconds
			Position C	271 seconds	300 seconds

In another preferred embodiment, in shooting process when user operation camera terminal change focal length, start time and the end time of change focal length generation can be recorded equally.In the preferred embodiment, suppose when position B uses nearly Jiao to take, by user operation camera terminal specific physical button or touch the specific keys of screen display with the timing of triggering timing unit end position B, and record the end time (such as 200 seconds) by timing unit.Then, user change focal length change into use Jiao far away take, in the timing of position B/ zoom by the above-mentioned specific keys of user operation, and the start time (such as 201 seconds) of position B/ zoom is recorded by timing unit, shooting a period of time, after Jiao far away has taken, terminate timing by the above-mentioned specific keys of user operation, and recorded the end time (such as 270 seconds) of position B/ zoom by timing unit.Recording unit records in the preferred embodiment is as follows:

Table 2: record sheet

Event	Start time Tb	End time Te
			Position A	0 second	180 seconds
Position B	181 seconds	200 seconds
			Position B/ zoom	201 seconds	270 seconds
Position C	271 seconds	300 seconds

In another preferred embodiment, camera terminal also comprises the focal length detecting unit be connected with timing unit.When focal length detecting unit detects focal length variations, automatic triggering timing unit is terminated timing; And after focal length variations completes, automatic triggering technique unit continues to start timing.

In step S200, be word by the audio conversion in multimedia, the audio frequency obtained when audio frequency is camera terminal shooting multimedia.Such as adopting audio conversion disclosed in Chinese patent application 201210102950.6 is the method for word.After conversion, camera terminal is shown to user, confirms for user.If user thinks that the word content after changing is wrong or need adjustment, the first input block of camera terminal can be used to modify.User also can by the illustrative words outside the first input block amount of imports.

In step S220, camera terminal by the first transmission unit by the record sheet in the multimedia of shooting, the first storage unit and word second transmission unit of uploading onto the server that confirms through user, the multimedia of shooting is saved in database by server, and records multimedia ID.Multimedia ID is the multimedia unique identification uploaded, such as, adopt the combination etc. of multimedia names and the uplink time uploaded.

In step S240, server is according to the event in record sheet, start time and end time, and use resolution unit to extract key frame from multimedia video, described key frame is defined as the representative of the frame of video that camera terminal is taken under same event.Such as: for each event, according to the standard for manual sampling preset, from video, choose the key frame K that each event is representative.

In step S260, according to start time and the end time of event each in video, obtain the duration T of each event.

In step S300, server extracts text index from the word uploaded.For each multimedia, server forms multimedia index table, and is stored in the second storage unit.Multimedia index table comprises: multimedia ID, text index, key frame set K and incident duration set T.Wherein text index can be null value NULL, and it represents in the multimedia of camera terminal shooting does not have audio frequency, and user does not also carry out explanatory note to multimedia.

In step S400, user uses inquiry terminal to search multimedia.User by the 3rd input block of inquiry terminal, can input multiple inquiry mode.The first inquiry mode is input photo, and photo can be a width, also can be several, and photo can derive from the photo of the 3rd image unit shooting; The second inquiry mode is on the basis of the first, input characters and photo; The third inquiry mode is pure words input.

In step S420, if what input in inquiry terminal is pure words, then only retrieves in text index, and result for retrieval is presented to user, provide to user the multimedia number not having explanatory note simultaneously.

In step S440, if what input in inquiry terminal is only M photos, then inquiry terminal forms photograph collection X.

X＝{X1,X2,…X i,…XM}

Then compare the every photos in photograph collection X and whether each key frame in key frame set K mates, the concrete mode compared can adopt such as image matching method disclosed in Chinese patent application 200710176039.9 or other image matching methods of the prior art.Then according to the matching degree of photo and key frame, retrieve corresponding video, and the result for retrieval according to sequencing of similarity is presented to user.The computing formula of similarity Sim is as follows:

$\mathrm{Sim}=\frac{\underset{j=1}{\overset{M}{\mathrm{\Σ}}}\underset{i=1}{\overset{e}{\mathrm{\Σ}}}\mathrm{Tij}}{M*\sqrt{\underset{i=1}{\overset{e}{\mathrm{\Σ}}}{\mathrm{Ti}}^2}}$

Wherein, M is the number of pictures in photograph collection X; E is the event number in video, i.e. key frame quantity; Ti is the duration of i-th event.The value of Tij has two, when photo and key frame do not mate, and Tij=0; When photo and key frame coupling, Tij is the duration of i-th event of coupling.

In step S460, if inquiry terminal input not only comprise word but also comprise photo, then first retrieve in text index, if result for retrieval is less than specific threshold, then direct result presented to user; Otherwise, if result for retrieval is more than or equal to specific threshold, then character search to result in use the method in step S440 again to retrieve, and the result of retrieval is presented to user, to improve recall precision.

Multimedia system of the present invention and multi-media processing method are specially adapted to take production scene as the scene cases training based on mobile intelligent terminal of background, obtain based on production practices, training contents originates diversified beneficial effect.Further, multimedia system of the present invention and multi-media processing method can make user adopt multiple method to search described multimedia, for user's inquiry provides accuracy and convenience.

Method disclosed by the invention comprises the one or more steps for realizing the object of the invention, and method step can exchange each other and not leave scope of the present invention.In other words, unless the normal running of embodiment needs the step of particular order, the order of concrete steps can be revised, and the scope of spirit of the present invention can not be left.Although the present invention essentially describes specific embodiment and application, it will be understood by those skilled in the art that the present invention is not limited thereto.According to method and system disclosed by the invention, obvious various amendment for those skilled in the art, change and change all do not deviate from the spirit and scope of the present invention.

Claims (2)

1. a multimedia system, comprises camera terminal, server and inquiry terminal; It is characterized in that: inquiry terminal comprises the 3rd image unit, the 3rd input block, query unit, the 3rd transmission unit; Wherein, described inquiry terminal is used for retrieving described multimedia for user by multiple method.

2. use a multi-media processing method for multimedia system according to claim 1, it is characterized in that comprising the following steps:

Step S100, user uses camera terminal to take multimedia, and described multimedia comprises Audio and Video; When a certain position and/or zoom shot, recorded start time and the end time of shooting by timing unit; Subsequently, event, start time and end time are saved in the record sheet of the first storage unit by timing unit in the lump, and wherein said event comprises position, zoom;

Audio conversion in multimedia is word by step S200, after conversion, camera terminal is shown to user, confirms for user; If user thinks that the word content of conversion is wrong or need adjustment, then the first input block of camera terminal is used to modify;

Step S220, camera terminal by the first transmission unit by the record sheet in the multimedia of shooting, the first storage unit and word second transmission unit of uploading onto the server that confirms through user, the multimedia of shooting is saved in database by server, and recording multimedia ID, described multimedia ID is the multimedia unique identification uploaded;

Step S240, server is according to the event in record sheet, start time and end time, use resolution unit to extract key frame from multimedia video, and determine the recording duration of key frame, described key frame is the representative of the frame of video that camera terminal is taken under same event;

Step S260, obtains the duration of key frame that in video, each event is corresponding and event, wherein obtains the duration according to start time of each event and end time;

Step S300, server extracts text index from the word uploaded; For each multimedia, server forms multimedia index table, and is stored in the second storage unit; Described multimedia index table comprises: multimedia ID, text index, key frame set and incident duration set;

Step S400, user uses inquiry terminal to search described multimedia;

Step S420, if what input in inquiry terminal is pure words, then only retrieves, and result for retrieval is presented to user in text index, provides the multimedia number not having explanatory note to user simultaneously;

Step S440, if what input in inquiry terminal is only M photos, then inquiry terminal forms photograph collection X:

X＝{X1,X2,…Xi,…XM}

Then compare the every photos in photograph collection X and whether each key frame in key frame set mates, then according to the matching degree of photo and key frame, retrieve corresponding video, and the result for retrieval according to sequencing of similarity is presented to user;

The computing formula of similarity Sim is as follows:

$\mathrm{Sim}=\frac{\underset{j=1}{\overset{M}{\mathrm{\Σ}}}\underset{i=1}{\overset{e}{\mathrm{\Σ}}}\mathrm{Tij}}{M*\sqrt{\underset{i=1}{\overset{e}{\mathrm{\Σ}}}{\mathrm{Ti}}^2}}$

Wherein, M is the number of pictures in photograph collection X; E is the event number in video, i.e. key frame quantity; Ti is the duration of i-th event; The value of Tij has two, when photo and key frame do not mate, and Tij=0; When photo and key frame coupling, Tij is the duration of i-th event of coupling;

Step S460, if inquiry terminal input not only comprise word but also comprise photo, then first retrieve in text index, if result for retrieval is less than specific threshold, then direct result presented to user; Otherwise, if result for retrieval is more than or equal to specific threshold, then character search to result in use the method in step S440 again to retrieve, and result for retrieval is presented to user.