Embodiment
The invention provides a kind of image editor that has pin function that in mobile multimedia processor, uses and carry out picture editting's method and system.Information processing is included in mobile multimedia chip inediting view data, and described mobile multimedia chip includes integrated image editor, and described image editor uses connected at least one plug-in unit.Described plug-in unit can be installed in the on-chip memory in the mobile multimedia chip that includes integrated image editor.Based on the plug-in unit of being installed, can select at least a picture editting's effect, for example cutting or adjustment size.
Reading command and/or write instruction and can send at least one plug-in unit that is connected with described image processor to.At least one pointer is stored in the interior storer of described mobile multimedia chip.Described pointer and the reading command that is transmitted and/or transmitted to write instruction relevant.Based on the plug-in unit that is connected with described image editor, can use at least a picture editting's effect to described view data.View data behind the editor can be stored in on-chip memory and/or the chip external memory.The described plug-in unit that is connected with image editor can obtain status information to be used for the editor of view data.Described plug-in unit can be carried out or resets based on the described status information of obtaining.
Figure 1A is the structural representation of the mobile device of the mobile multimedia chip that has integrated image editor of the use according to one embodiment of the invention.As shown in Figure 1A, mobile device 102a comprises display 107a, host-processor 108a, mainframe memory 110a, antenna 104a and mobile multimedia chip 114a.Mobile multimedia chip 114a is used for handling audio frequency and/or the video data of mobile device 102a.Mobile multimedia chip 114a comprises image editor 116a, on-chip memory 122a and plugin manager 109a.
According to one embodiment of the present of invention, image editor 116a can be mobile device 102a PC is provided applied editting function.For example, image editor 116a realizes for example Photoshop of PC application program
TMEditting function.Image editor 116a can comprise suitable circuit, logic and/or code, provide multiple image editing function edit a plurality of image 118a ..., 120a.Mobile multimedia chip 114a among the mobile device 102a and/or host-processor 108a can use this image editing function in the process of processing video data.Image editor 116a can carry out multiple image editing function at least a portion of a pictures at every turn, uses and/or handle bandwidth with this storer of optimizing in the mobile device 102a.Some characteristic features of image editor 116a comprise that the very color complete clear picture frame of generation, the CamPlus effect that can be applicable to offline condition and picture merge (part-shot combining).Other features of some of image editor 116a comprise by shielding padding (masked fill operation) adds literal on image, use plug-in unit to carry out senior or self-defined picture operation, for example, adjusts brightness/contrast/color balance etc.
In the video editing process, image file format and image file position are transparent or hide mobile multimedia chip 114a and/or host-processor 108a, this makes that manageable file layout is more, and can handle the compressed image that is loaded among the on-chip memory 122a.In this, for realizing handling, the physical location of image file format and video data is isolated.In addition, the memory buffer of the video data that storage will be handled can use for example user-defined ID of identifier to identify, rather than absolute storage address.Mobile multimedia chip 114a can pass through antenna 114a from the external video data source, or receives the video data that will handle from external memory storage 106a by connecting 105a.
In another embodiment of the present invention, image editor 116a can under offline condition, handle any size image 118a ..., 120a, and the raw pixel data of permission mobile multimedia chip 114a and/or host-processor 108a visit arbitrary image, and with its part as graphic user interface (GUI).According to another embodiment of the present invention, image editor 116a can allow third party (3
Rd) write a plurality of plug-in unit 112a of being used to realize user-defined function or effect ..., 112n, with to image (for example any one image 118a ..., 120a) operate.For example, image editor 116a can allow third party (3
Rd) write plug-in unit 112a, when being used on video display 107a display process rear video data, in mobile device 102a, realize eliminating the blood-shot eye illness function.
Mobile multimedia chip 114a also can use other pin function in the edited image data process, as cutting and shake.In one embodiment of the invention, mobile multimedia chip 114a includes plugin manager 109a.Plugin manager 109a can comprise suitable circuit, logic and/or code, is used to manage one or more plug-in units.For example, plugin manager 109a can be provided with, carries out, resets and forbid picture editting's effect of one or more plug-in units.Although plugin manager 109a is implemented in the mobile multimedia chip 114a, the present invention is not limited to this.In this, plugin manager 109a can be implemented in outside the mobile multimedia chip 114a and be implemented among the mobile device 102a.
The picture format that image editor 116a is supported comprises JPEG (joint photographic experts group) (JPEG) form, BIIF (GIF) and transplantable network representation file layout (PNG).JPG is a kind of compress technique, is used for compressed color and gray level continuous-tone image.Jpeg image can be supported 1,600 ten thousand pixels, and can be applicable to photo and complex figure.GIF is a kind of compress technique of supporting 256 looks, can be used for line style, black white image and the high small text of several pixel.The GIF form also can be supported transparent form, and background colour is set to transparent, and the color of the webpage below making can display.The PNG form uses the ratio of compression higher than GIF form, also can use opacity, or the control of transparency.
According to one embodiment of the present of invention, image editor 116a can support PNG and GIF form.If image editor 116a uses the PNG form, when the editing video data, can use the form of every pixel 24 (bpp) so, and use Alpha's (alpha) channel and need not interlock.If image editor 116a uses the GIF form, video data editing just can be realized need not to adopt under the situation of staggered, local color map and/or cartoon making so.However, the present invention is not limited to this.PNG decoding, PNG coding and GIF coding can be set to default option.This can be by to be PNG CODEC and/or GIF CODEC at compiling image editor 116a be provided with sign when realizing PNG coding/decoding and GIF coding or parameter realizes.
Figure 1B is the structured flowchart of mobile multi-medium system according to having of one embodiment of the invention of integrated image editor.As shown in Figure 1B, mobile multi-medium system 100 comprises core processor 102, audio-frequency module 108, memory stick module 110, SanDisk (SD) card module 112, test module 114, TV module 116, PC module 118 and a plurality of camera module, i.e. camera 1120 and camera 2120.System 100 also can further comprise base band flash memory module 124, baseband module 126, Keysheet module 128, RF module 130, antenna 132, main LCD module 134, inferior LCD module 136, optional flash memory module 138 and optional SDRAM module 140.Core processor 102 can comprise RAM module 104, image editor 103 and analog module 106.
Core processor 102 can comprise suitable logic, circuit and/or code, is used to handheld multimedia devices to carry out video and/or multi-medium data processing.For example, core processor 102 can design and be optimized for and carry out videograph/playback, mobile TV and 3D moving game.Core processor 102 is complete programmable structure, so just can use software to carry out multiple multimedia function, and need not to increase the cost expense relevant with hardware accelerator and bring the cycle because of market uncertainty.Core processor 102 can comprise suitable logic, circuit and/or code, is used for video and Audio Processing, multitone is synthetic, lcd controller and SRAM and is integrated in same chip.This integral design method has been simplified design, saves cost, space and power simultaneously.Core processor 102 both can be used as the independent multimedia processor and had used, and the coprocessor that also can be used as main frame uses.
Core processor 102 can be 150MHz double arithmetic logic units (ALU) VideoCore that for example has 128K bit instruction and data cache
The ll processor.Core processor 102 can be the digital rights manager (DRM) that for example has unique chip id code.Core processor 102 have for multiple product and algorithm for example SD card, Advanced Encryption Standard (AES), 3 data encryption standardss (3DES) and Rivest-Shamir-Adleman (RSA) but the function that provides the recording medium content protecting to encrypt.Core processor 102 can comprise the encryption post code that has the key of freely programming.H.263 or H.264 core processor 102 can be used for carrying out video compress, for example moves video CODEC such as MPEG-4.Core processor 102 can be used for Video processing, for example imageing sensor front-end processing, blur prevention or noise reduction process, or be used for video analysis, for example cutting operation, Target Recognition or intelligence sensor.Core processor 102 can comprise for example 52 general purpose I/O (GPIO) pin.This GPIO pin can be used for user-defined I/O or is connected with inner peripheral equipment.
Core processor 102 comprises image editor 103, RAM module 104 and analog module 106.Image editor 103 comprises suitable circuit, logic and/or code, realizes the video data editing function of mobile multi-medium system 100.For example, image editor 103 can provide with Figure 1A in the same image editing function of image editor 116a, comprise the use pin function.In this, image editor 103 can comprise plugin manager 150.Plugin manager 150 comprises suitable circuit, logic and/or code,, can manage one or more plug-in units, as plug-in unit 152 and 154.For example, plugin manager 150 can be provided with, carries out, resets and forbid picture editting's effect of plug-in unit 152 and 154.Picture editting's effect that plug-in unit 152 and 154 provides for example comprises that " blood-shot eye illness " elimination function, cutting, shake and/or text cover.RAM module 106 comprises suitable circuit, logic and/or code, is used for data on the memory feature, for example data on the sheet of 10Mb.Analog module 106 can comprise for example Switching Power Supply (SMPS) module and phaselocked loop (PLL) module.
Analog module 106 can comprise SMPS controller on the sheet, realizes the generation of its core voltage.This core voltage can be carried out software programming according to the rate requirement on the core processor 102, with this power management is further controlled.Core voltage working range under the normal condition is between 0.8V-1.2V, and under park mode, this value is reduced to about 0.6V.Analog module 106 also can comprise a plurality of phaselocked loops, is used to for example clock of external unit generation 195kHz-200MHz.Core processor 102 can comprise a plurality of working power patterns, for example, and operation, standby, dormancy and power-down mode.According to one embodiment of the present of invention, core processor 102 can comprise bypass mode, allows the peripherals of main frame reference-to storage mapping under power-down mode.Under bypass mode, core processor 102 can directly be controlled display in course of normal operation, makes main frame can keep shown content under standby mode.
Audio-frequency module 108 can comprise suitable logic, circuit and/or code, is used for by built-in IC audio frequency (I
2S), pulse code modulation (pcm) or audio coding decoding (AC ' 97) interface is communicated by letter with core processor 102.AC ' 97/I
2The S controller can provide AC ' 97/I in holotype or under pattern
2The output of S audio frequency.But pcm interface input and output speech or high-quality stereo audio.The PCM controller comprises and independently transmits and receives first-in first-out (FIFO) buffer memory, and can use DMA further to reduce the processor expense.Audio-frequency module 108 also can comprise audio frequency input, audio output port and loudspeaker/microphone port.Memory stick module 110 can comprise suitable logic, circuit and/or code, is used for communicating by letter with core processor 102 by the memory stick supporting interface.SD card module 112 can comprise suitable logic, circuit and/or code, is used for communicating by letter with core processor 102 by SD I/O (I/O) interface.Multimedia card (MMC) also can be used for communicating by letter with core processor 102 by SD I/O (I/O) interface.
Test module 114 can comprise suitable logic, circuit and/or code, is used for communicating by letter with core processor 102 by JTAG (JTAG) interface.The address space of test module 114 addressable core processors 102, and by emulation interface execution boundary scan.TV module 116 can comprise suitable logic, circuit and/or code, is used for communicating by letter with core processor 102 by capable (the PAL)/NTSC of phase alternation (NTSC) output port.
PC module 118 can comprise suitable logic, circuit and/or code, is used for communicating by letter with core processor 102 from port by USB (universal serial bus) (USB) 1.1.A plurality of utilizing camera interfaces such as camera 1120 and camera 2122 can comprise suitable logic, circuit and/or code, are used for communicating by letter with core processor 102 by the original CCIR601 utilizing camera interface of multi-format.Core processor 102 can further comprise 8/16 CMOS/CCD camera input port able to programme, can support nearly two 8M pixel image sensor and original Bayer rgb format 8/10/12/16 bit data.Core processor 102 can comprise the utilizing camera interface that has windowing and sub sampling function, is used to connect mobile TV front end.
Core processor 102 can comprise a plurality of serial line interfaces, for example from USB1.1, main built-in integrated circuit (I
2C), main Serial Peripheral Interface (SPI) and be used for bluetooth or universal asynchronous receiver/transmitter of IrDA (UART).Main I
2C interface may command imageing sensor, and can be connected to intelligent power or other peripherals.The SPI main interface can be used for the control chart image-position sensor.Can provide two chips to select by interruption or DMA works under poll pattern.Core processor 102 can comprise lcd controller and interface, supports a plurality of displays of XGA resolution, and handles 8/9/16/18/21 digital video data.
Base band flash memory module 124 can comprise suitable logic, circuit and/or code, receives data by 8/16 bit parallel host interface from core processor 102.Host interface can provide two channels with absolute address and data register, and the storage space of core processor 102 can directly be read and/or write to host-processor thus.Baseband module 126 can comprise suitable logic, circuit and/or code, is used for by communicating by letter with core processor 102 as 8/16 bit parallel host interface.Keysheet module 128 and RF module 130 can comprise suitable logic, circuit and/or code, are connected with baseband module 126.Antenna 132 can be connected to RF module 130.Core processor 102 can comprise baseband interface, concerning the main frame that has 3 bit address and 8/16 bit data bus, with the form work of memory mapped peripherals.This baseband interface can comprise the two software channels that possess power supply effective bypass pattern.
Main LCD module 134 can comprise suitable logic, circuit and/or code, receives data by the display controller and second external memory interface from core processor 102.Display controller can comprise suitable logic, circuit and/or code, is used to drive inner TV output function or is connected to a plurality of LCD.Display controller can be supported multiple screen cache form, uses directly access cache device of direct memory visit (DMA), and the minimizing processor expense.Display controller can generate NTSC and the PAL raster format drives TV output.
Inferior LCD module 136 can comprise suitable logic, circuit and/or code, is used for communicating by letter with core processor 102 by second external memory interface.Core processor 102 can comprise the RGB external data bus.Core processor 102 can use Pixel-level interpolation and the refresh rate that can be provided with that output image is carried out with convergent-divergent.
Optional flash memory module 138 can comprise suitable logic, circuit and/or code, is used for communicating by letter with core processor 102 by external memory interface.Optional SDRAM module 140 can comprise suitable logic, circuit and/or code, is used for receiving data by external memory interface from core processor 102.Core processor 102 can comprise the high-speed memory interface, is connected to SDRAM, SRAM, flash memory and external peripheral.The control and the clocking information of SDRAM and other asynchronous devices can be configured by core processor 102.Core processor 102 can further comprise the supplementary storage interface, is connected to memory mapped LCD and external peripheral.This supplementary storage interface can be used for connecting low-speed device, but can not influence the access speed of external memory storage.This supplementary storage interface can provide 16 data lines and 6 chip selection/address lines, and the employed programmable bus timing of setting, visit and retention time.Core processor 102 can be supported the NAND/NR flash memory, comprises that NAND starts and high speed direct memory visit (DMA).
Core processor 102 can be used for the multiple application in the mobile video field, and has minimum power consumption.The programmable features of core processor 102 provides dirigibility and adaptability, makes it can move existing and new CODEC, can make mobile device 102a possess the applied editting function of PC.For example, the image editor 116a among Figure 1A can realize PC application program such as Photoshop
TMThe same editting function.Core processor 102 can move high-quality 3D recreation, and performance can compare favourably with family's control desk.The parallel characteristics of vector processing unit makes the game developer can develop usually the painted and volume light source high-end features such as (volumetric lighting) of pixel by the desktop game of hardware supported, and consumes power seldom.For example, core processor 102 can use image editor 103 to realize the image editing function of handheld mobile communication device such as mobile phone.
Fig. 2 A is the structured flowchart according to the image editor of one embodiment of the invention.As shown in Fig. 2 A, image editor 200a can realize in a mobile multimedia chip, comprise image processing command module 202a, picture editting's module 204a, resource management module 206a and plug-in unit 208a.
Image processing command module 202a can comprise suitable logic, circuit and/or code, is used to obtain input picture processing command 201a, is translated into image editor operation 203a then.Image processing command module 202a also can use the image editor utility routine, is used for realizing other image editing functions of image editor 200a, covers as cutting, shake and/or text.
Resource management module 206a can comprise suitable logic, circuit and/or code, is used for obtaining resource 207a from the view data position, and before being edited by picture editting's module 204a it is carried out pre-service.For example, resource management module 206a can obtain view data from document location, on-chip memory and/or chip external memory, comprises video data.In this, this resource can comprise the video data that is stored in a certain position, and this position can be physical file such as jpeg file, or a position in the storer such as a position in the mainframe memory, or is null resource.In one embodiment of the invention, resource management module 206 can use one or more CODEC that the view data of obtaining is encoded and/or decoded.For example, resource management module 206 can use JPEG CODEC that the jpeg image data are encoded and/or decoded.In addition, resource management module 206 also can use bar tape handling (stripe processing) in the preprocessing process that the view data of obtaining is carried out.A band of the view data of obtaining in this, or a part will mail to picture editting's module 204a edits.
Picture editting's module 204a can comprise suitable logic, circuit and/or code, is used for the view data 205a that resource management module 206a obtains being edited with the one or more image editor commands relevant with effective plug-in unit 208a according to the image editor operation 203a that generates.In one embodiment of the invention, picture editting's module 204a can use one or more image effects, for example image cutting.In another embodiment of the present invention, picture editting's module 204a can include plugin manager module 210a, makes to use at least one plug-in unit that is connected to image editor 200a, as plug-in unit 208a.View data 209a behind processing back or the editor will mail to outside picture editting's module 204a, and be stored in on-chip memory and/or the chip external memory.
Plugin manager module 210a can comprise suitable logic, circuit and/or code, is used to manage one or more user-defined plug-in units.For example, plugin manager module 210a can manage plug-in unit 208a.In this, plugin manager module 210a can handle with the process of editing in the view data 205 that resource management module 206a is obtained in the relevant multiple function of one or more plug-in unit effects used.According to one embodiment of the present of invention, some senior plug-in unit operation can be obtained the part of view data, and for example a frame in this case, in the process of picture editting's module 204a edited image, can carry out the bar tape handling to view data.In operating process, plugin manager module 210a can send one or more image processing commands based on the picture editting effect relevant with plug-in unit 208a.
Fig. 2 B is the detailed block diagram according to the image editor of one embodiment of the invention.Shown in Fig. 2 B, image editor 202b can be embodied as the unified resource module.Image editor 202b can comprise jpeg file format module 204b, screen size buffer memory 206b, mainframe memory module 208b, plugin manager module 244b, new resources (NEW) module 210b, format converting module 214b, data outputting module 216b and data input module 218b.New resources (NEW) module 210b can comprise suitable logic, circuit and/or code, can specify a resource under the situation that does not have relevant storer.
Jpeg file format module 204b can comprise suitable logic, circuit and/or code, can be used as the jpeg format file source.But the present invention is not limited only to this, and jpeg file format module 204B also can be used as GIF formatted data source, PNG formatted data source and/or extended formatting data source.CODEC module 212b can comprise suitable logic, circuit and/or code, realizes the compression and decompression to jpeg file formatted data module 204b.CODEC module 222b can carry out the compression and decompression operation to screen size cache module 206b.CODEC module 232b can carry out the compression and decompression operation to mainframe memory module 208b.CODEC module 242b can carry out the compression and decompression operation to new resources (NEW) module 210b.Can be configured CODEC 212b, 222b and 242b, make when the data of being stored in the storer are raw data, not need deal with data.However, the present invention is not limited only to this.
Format converting module 214b can comprise suitable logic, circuit and/or code, can be second form from first format conversion with the input data from data input module 218b.In addition, format converting module 214b can be converted to the form that can send to data outputting module 216b with the data that CODEC 212b, 222b, 232b and 242b generate.For example, format converting module 214b can change between multiple form, includes but not limited to RGB888, RGB565 and YUV442.
Plugin manager module 244b can comprise suitable logic, circuit and/or code, can be used for user-defined one or more plug-in units are managed.In this, plugin manager module 244b can handle the relevant multiple function of one or more plug-in unit effects of carrying out using in the edited image data process in image editor 202b.
In another embodiment of the present invention, image editor 202b can be used as unified resource and uses, and realizes being positioned at the treatment of picture on the different physical mediums.In this, this resource comprises the view data that is stored in a certain position, can be to be stored in for example view data on the jpeg file format module 204b of physical file, be stored in the view data on storer such as the mainframe memory 208b, or the view data among null resource such as the null resource module 210b.Null resource module 210b is used for creating " newly " image, and can be used as the input use.In this, picture format will separate with the physical location of data, realizing jpeg file or to be stored in the processing of the extended formatting file in the storer, or adds new CODEC and handles new images/video format.Each resource includes the band interface, is used for the external world mutual.If any, this band interface returns next band from input picture, in the output image that this next band is stored to correspondence, and changes between original image form and operation picture format.Image manipulation among the image editor 202b can carry out based on band.
According to one embodiment of the present of invention, the senior plug-in unit operation of some among the image editor 202b may require to obtain a frame of view data, in this case, will walk around this band interface.In this, can be directly from resource reading of data and/or write data to it.Same resource can not be used as input and output simultaneously, because handle a band and write back file at every turn when handling image, this will make this document damage.When input and output all in storer, and data are when being unprocessed form, same resource can be used as input and output simultaneously.
In one embodiment of the invention, image editor 202b can obtain view data such as jpeg file data by jpeg file module 204b.The jpeg file data of obtaining are decoded by CODEC 212b subsequently.Decoded jpeg image data will mail to format converting module 214b.Format converting module 214b is converted to decoded jpeg image data the view data of different images form.In addition, image editor 202b can use one or more picture editting's effects and/or use plug-in unit in the process that decoded jpeg image data are edited.In one embodiment of the invention, can in the process of edited image data, use one or more plug-in units of plugin manager module 244b management to provide " blood-shot eye illness " to eliminate effect.View data behind the editor mails to the outside of image editor 202 subsequently, and for example, 216b is stored in the storer by data outputting module.
Fig. 2 C is the structured flowchart according to the image editor of one embodiment of the invention.In Fig. 2 C, image editor 200b comprises main frame 202b, storer 203c, disk 204b, cache manager 206b, resource module 208b, processing module 210b, CODEC module 212c, card module 226b and image editor corn module (IECFB) 228c.Resource module 208b, processing module 210b among the image editor 200c, CODEC module 212b, card module 226b and IECFB 228c can be implemented in the mobile multimedia chip by software and/or firmware, in the mobile multimedia chip 114a among Figure 1A.
Main frame 202c comprises host-processor, is integrated in mobile multimedia equipment such as the mobile phone.Storer 203c comprises suitable storer, but store video information, code and/or service data.Disk 204c can store for example video information.Cache manager 206c can comprise suitable logic, circuit and/or code, is used for the storage in the processing memory module 203c.Resource module 208c can comprise suitable logic, circuit and/or code, handles the allocation and deallocation of resource for main frame 202c, storer 203c, disk 204c and/or cache manager 206c.
Processing module 210c can comprise suitable logic, circuit and/or code, is used to handle input picture.CODEC module 212c can comprise suitable logic, circuit and/or code, can manage one or more CODEC modules, for example RAW CODEC module, PNG CODEC module, JPEG CODEC module and/or GIF CODEC module.
Image processor corn module (IECFB) 228c can comprise suitable logic, circuit and/or code, is used to the multiple image effect of one or more image applications of handling the image manipulation request and/or processing module 210c being handled.For example, IECFB 228c can handle cutting, adjustment size, rotation, transparent filling, mask filling (mask fill) and/or CamPlus
TMEffect.Card module 226c can comprise suitable logic, circuit and/or code, is used for one or more plug-in units of leading subscriber definition.
In one embodiment of the invention, IECFB 228c can comprise plugin manager module 227c.Plugin manager module 227b can comprise suitable logic, circuit and/or code, can manage one or more user-defined plug-in units, as plug-in unit 226c.In this, plugin manager module 227c can handle and the relevant multiple function of using in IECFB 228c edited image data procedures of one or more plug-in unit effects.
In operating process, video data can mail to processing module 210c from main frame 202c, storer 203c and/or disk 204c by cache manager 206c.Processor module 210c can use software and/or firmware to realize, uses CODEC module 212c and resource module 208c to come the video data of receiving is handled and decoded.Processor module 210c processed video data will mail to IECFB 228c further to be handled.For example, IECFB228c is the video data after the editing and processing further, and/or uses one or more visual effects by card module 226c.
Fig. 2 D is the structured flowchart according to the image editor of one embodiment of the invention.Main frame 202, storer 203, disk 204, cache manager 206, resource module 208, band module 210, CODEC manager module 212, card module 224 and plugin manager module 226 have been shown in Fig. 2 D.Also show a plurality of CODEC among the figure, comprise RAW CODEC module 214, PNG CODEC module 216, JPEG CODEC module 218, GIF CODEC module 220 and be used for the image C ODEC of video module 222.Fig. 2 D also comprises image manipulation request module 228, image effect module 230, CAMPLUS module 236, image editor functional module 232, image editor command module 234 and image editor message processing module 238.
Main frame 202 can comprise host-processor, is integrated in mobile multimedia equipment such as the mobile phone.Storer 203 can comprise suitable storer, is used for store video message code and/or service data.Disk 204 can be stored for example video information.Buffer manager for use 206 can comprise suitable logic, circuit and/or code, is used for the storage in the processing memory module 203.
Resource module 208 can comprise suitable logic, circuit and/or code, is used to main frame 202, storer 203, disk 204 and/or cache manager 206 to handle the allocation and deallocation of resource.The resource that resource module 208 is handled can comprise uses for example memory buffer that identifies of user definition ID rather than absolute storage address of identifier.In addition, image file information is transparent or hiding as image file format and/or image file position for main frame 202, and this makes that manageable file layout is more, and can handle the compressed image that is loaded in the storer 203.Image editor 200 can use the unified resource notion, so just can use one or more processing commands to the image that is stored on the different physical medias.In this, this resource can comprise the view data that is stored in a certain position, and it can be certain position in physical file, the storer, certain position, null resource and/or the transmission buffer of mainframe memory.
But the position of file resource identification document comprises pathname, is used in reference to place, prescribed paper position.Except the file of sign storage inside, file resource also can identify the file of exterior storage.Memory resource can be used for storing original image.This has reduced the data access time significantly.For the data that are stored on the host computer system, can use host resource to visit these data by host address.Null resource can be used for creating new images, also can be used as input.File layout can separate with the physical location of data, and this is beneficial to the processing of video data, as is stored in the JPED file in the storer, or adds new CODEC and handle new images/video format.Each resource can comprise the band interface, and as band module 210, resource module 208 can use them to handle the video data of input.
Band module or band interface 210 can comprise suitable logic, circuit and/or code, are used to handle an input picture.Each resource can send image manipulation request module 228 to by band module 210.If there is next base band, then band module 210 can be returned next band from input picture, this next band is carried out the rake formula scan and it is stored in the corresponding output image.This band interface also is used between original image form and the operation picture format and changes.Most of image manipulations are based on that band carries out, and comprise from input picture and read next band, handle this band, and this band is write output image.Because the restriction of storer, image will be that handle on the basis with every.In this, can read, handle a band and be written into storer such as storer 203 in.
CODEC manager module 212 can comprise suitable logic, circuit and/or code, is used to manage one or more CODEC modules.For example, CODEC manager module 212 can be managed RAW CODEC module 214, PNG CODEC module 216, JPEG CODEC module 218, GIF CODEC module 220 and is used for the image C ODEC of video module 222.In this, CODEC manager module 212 can be managed loading and the unloading of different images CODEC.
RAW CODEC module 214 can comprise suitable code, and what these codes can be direct or static is compiled among the image editor REF#.PNG CODEC module 216 can comprise suitable logic, circuit and/or code, can handle the PNG compression and decompression of different images.PNG CODEC module 216 can be loaded among the image editor REF# dynamically.JPEG CODEC module 218 can comprise suitable logic, circuit and/or code, can handle the JPEG compression and decompression of different images.One or more this CODEC for example JPEG CODEC module 216 can be loaded among the image editor REF# dynamically.GIF CODEC module 220 can comprise suitable logic, circuit and/or code, is used to handle the GIF compression and decompression of different images.The image C ODEC that is used for video module 222 can comprise suitable logic and/or code, is used to handle the compression of images and the decompression of different images.
Card module 224 can comprise suitable logic, circuit and/or code, and user-defined multiple plug-in unit effect can be provided.For example, can create elimination " blood-shot eye illness " plug-in unit, image editor 200 handled images are operated as third party application.Plugin manager module 226 can comprise suitable logic, circuit and/or code, the plug-in unit of leading subscriber definition.In this, plugin manager module 226 can be card module 224 and handles multiple function, as initialization and function of reset.
According to one embodiment of the present of invention, some senior plug-in unit operation may need the entire frame of view data, can walk around band interface 210 in this case.In this, view data can read from the resource that resource module 208 provides or write direct wherein.Sometimes may have such situation, promptly same resource can not be used as input and output simultaneously.In this, same resource will be caused the damage of data as input and output simultaneously, because handle a band and write back file at every turn when handling image.But input and output all be arranged in storer and data be former be that the situation of form is an exception.
Image manipulation request module 228 can comprise suitable logic, circuit and/or code, can handle the image manipulation request.Image effect module 230 can comprise suitable logic, circuit and/or code, can handle various image effects, fills and/or CamPlus as cutting, adjustment size, rotation, transparent filling, mask
TMDeng." transparent filling " operation is used in pastes Target Photo on the original image, wherein the color of Target Photo is not the Transparent color of definition.CAMPLUS module 236 can comprise suitable logic, circuit and/or code, can handle CamPlus
TMImage effect.Image editing function module 232 can comprise suitable logic, circuit and/or code, can realize the plurality of picture editting function of operation on the main frame 202.Image editor command module 234 can comprise suitable logic, circuit and/or code, can generate the operation requests relevant with image editor command.
Image editor message processing module 238 can comprise suitable logic, circuit and/or code, can handle the management of image editor message.For example, picture editting's message processing module 238 can be realized indicating its process of commands that stops, ending, finish, starts processing, and send notice when finishing dealing with.For example, starting the order of handling causes making it to begin to handle the next band of input picture.
In one embodiment of the invention, cache manager 206, resource module 208, band module 210, CODEC manager module 212, plug-in unit effects module 224, plugin manager module 226, CODEC module 212 ..., 222, image manipulation request module 228, image effect module 230, CAMPLUS module 236, image editor functional module 232, image editor command module 234 and image editor message processing module 238 can be realized in the mobile multimedia chip by software and/or firmware, the mobile multimedia chip 114a among Figure 1A for example.
In operating process, the cache manager 206 in the image editor 200 can use user-defined identifier to come view data in the diode-capacitor storage 203.Based on a kind of picture format, the CODEC manager can be loaded into appropriate C ODEC in the storer 203.For the image of JPEG or PNG form, will distinguish and dynamically load JPEG CODE218 or PNG CODEC216.Based on the order of receiving, image editor command module 234 can be set up correct request, is applied in subsequently on the band of the indicated view data of band module 210.According to one embodiment of the present of invention, image editor 200 can be handled the view data as one or more rest images.But the present invention is not limited only to this, and image editor 200 can be handled the view data such as video data of Tathagata from video flowing equally.
If image editor 200 can use external memory storage, externally create the memory resource that is used for stored video data in the storer.If externally there are not enough storage spaces in the storer, image editor 200 will generate an error message.In one embodiment of the invention, memory buffer is pressed reverse sequence and is discharged, and such distribution method is reduced to memory fragmentation minimum.In addition, allow the maximum quantity of the memory buffer of use can be fixed as particular value, and can change to implement more flexible.Memory resource also can be used as temporary file and stores raw data, to quicken image manipulation.
If external memory storage is unavailable, image editor 200 can the low-speed memory mode operation, during this period, will not support the original image and the PNG of 24/pixel (bpp).To replace with 16bpp at the image manipulation of internal operation with 24bpp.But the color parameter in these orders still is appointed as the value of 24bpp.Some image manipulation for example shows non-jpeg image or rotation jpeg image, will be slow when using external memory storage, because image editor will use temporary file.
In one embodiment of the invention, except main The Application of Thread is the message handling device, image editor 200 can comprise worker thread.In this, a worker thread will be specifically designed to carries out all uncompleted image manipulations, and other worker threads will be specifically designed to display, and as the display on the mobile device, the rear video data are handled in output.
In another embodiment of the present invention, image editor command module 234 can be set up and the relevant operation requests of a plurality of image editor orders.Each order of image editor 200 can be used as input resource string and mails to the image editor command module, and corresponding order output will be mail to image manipulation request module 228 as output resource string.Some image editor order that image editor command module 234 is handled can be used for returning the state of image editor 200, for image manipulation current and/or subsequently is set up input picture, return the size of images of editing by integrated image editor 200, and the image manipulation that stops to move.Other image editor orders that image editor command module 234 is handled can be used for setting up the display window that uses the specific program that image editor 200 distributes to, refresh or the conversion displaying contents, and duplicate choosing partly of shown image and further handle.
Other image editor orders that image editor command module 234 is handled can be used for using the image feature image rotating, this function is optional, after rotation, use is by a zone of the definite color blank map picture of for example 24bpp value, second image pasted open filling on the color or transparent first image of editing, opening Fill Color on the part of image, to opening image applications CamPlus effect.Other image editor orders that image editor command module 234 is handled can be used for image is adjusted to the new size of appointment, and image is cropped to fixed new region.Use relevant image editor command to handle with storer, be used to remove or delete memory resource or memory buffer, list all buffer memorys in the storer 203, delete all memory resources by image editor command module 234.Use relevant image editor command to handle with plug-in unit by image editor command module 234, be used to add the input picture that plug-in unit will use, for plug-in unit is provided with output image, list one or more resource strings to be used for test, replacement resource string, the operation plug-in unit, and/or from plug-in unit, obtain the visual effect tabulation.
Fig. 3 is the process flow diagram according to the use band method processing video data of one embodiment of the invention.In an embodiment of this operation, when in mobile multimedia processor, carrying out the picture editting, can use the band disposal route.As Fig. 2 B and shown in Figure 3, the most of image manipulations in the image editor 200 all carry out based on band.Band interface 210 also is used between the video data original image form received from main frame 202, storer 203 and/or disk 204 and the operation picture format and changes.In this, the band module 210 in the image editor 200 can be used for reading next band from input picture, handles this band, and this band is write in the output image.Because the restriction of storer, image will be that handle on the basis with every.In this, can read, handle a band and be written into storer such as storer 203 in.In step 302, the band module 210 in the image editor 200 can read next band from the input picture that is received from disk 204 or storer 203.In step 304, the band that obtains can be handled in image editor 200.In step 306, processed video data band can write in the output image.Output image after the processing can be stored in the storer 203, and picture editting's request will be mail to image manipulation request module 228.Image-editing operations can be applied on the image after the processing by image effect module 230.In addition, can be by plugin manager 226 with on the image of one or more plug-in unit effects applications after handle.
Fig. 4 is the synoptic diagram that carries out image manipulation according to image editor shown in Fig. 2 B of one embodiment of the invention.In Fig. 4, show image manipulation module 402, unified resource module 404a, 404b, 404c and 404d, band cache module 406a and 406b, image manipulation request module 410, plugin manager module 412 and plug-in unit 414.Unified resource module 404b, 404c and 404d are input unified resource modules, and unified resource module 404a is the output resource module.Among input unified resource module 404b, 404c and the 404d each can be handled one or more input resources, and output unified resource module can be handled an output resource.Unified resource module 404b, 404c and 404d can comprise the view data that is stored in a certain position, as the view data of a position in the view data in the physical file, the storer, the view data of a position in the mainframe memory and/or the view data of null resource.
The input resource can comprise and mails to the data that image editor 200 is edited.The output resource can comprise the processed video data, and these data are handled by image editor 200.Null resource can be used for creating new images, also can be used as input.Picture format will separate with the physical location of data, is beneficial to video data as being stored in the processing of the jpeg file in the storer, or adds new CODEC and handle new images/video format.Each resource all can comprise a band interface, and image manipulation module 402 will use this interface to handle the video data of input.
In an operation, use band cache module 406b to handle from the one or more unified output resource of unified resource module 404b, and mail to image manipulation module 402.Image manipulation module 402 receives and is used for input parameter that the output that band buffer memory 406 generates is handled.Band buffer memory 406a is mail in the output that image manipulation module 402 generates, and offers unified resource module 404a.Unified resource module 404a can comprise the output resource.In this, image editor 200 editors' video data can mail to and be stored in storer 203, disk 204 or image editor 200 external memory storages such as the chip external memory.
Image manipulation operation requests module 410 can comprise suitable logic and/or code, is used to handle the image manipulation request.Card module 414 can comprise suitable logic, circuit and/or code, is used to provide user-defined multiple plug-in unit effect.For example, can create elimination " blood-shot eye illness " plug-in unit and use, on application and the image editor processed images as the third party.Plugin manager module 412 can comprise suitable logic, circuit and/or code, the plug-in unit of leading subscriber definition.In this, plugin manager module 412 can be handled the multiple function of card module 414, as initialization and function of reset.
In operating process, can receive the image manipulation request by image manipulation request module 410.Image manipulation request module 410 is obtained unified resource or view data subsequently to handle.The subsequent picture operation requests will indicate plugin manager 412 to carry out the pin function of wanting, and these functions will be used when the view data that editor obtains.Plugin manager 412 is by the application of the pin function of card module 414 administrative institutes request subsequently.
Fig. 5 is the synoptic diagram that communicates according between the image editor of one embodiment of the invention and plug-in unit.As Figure 1A and shown in Figure 5, image editor 502 can receive I/O (I/O) the resource string that the host-processor in the subscriber equipment 501 sends, and this subscriber equipment 501 can be handheld communication devices such as mobile phone or mobile device 102a.In this, will produce I/O resource string when the user imports data in handheld communication devices 501 sends.In step 506, the user can be mail to image editor 502 with I/O resource string such as view data.After step 512 is received I/O resource string, image editor 502 will be provided with this I/O resource string in step 514, and preserve this resource string.In step 508, image editor 502 can receive instruction, and it provides service to plug-in unit 504 requests.In step 516, image editor 502 loading of plug-in 504.In step 518, picture editting's effect that image editor 502 is provided with corresponding to plug-in unit 504.In step 534, plug-in unit 504 can be preserved the effect of being asked.
In step 522, input and/or picture data type that plug-in unit 504 needs when image editor 502 returns some picture edittings.In response, in step 520, image editor 502 is provided with the I/O resource based on input of returning and/or data type.In step 524, image editor 502 will send one or more read-write operations to plug-in unit 504 for these resources.In step 536, plug-in unit 504 is a pointer with these operation store.In step 510, image editor 502 turns back to message handling device, receives other instruction by the subscriber equipment relevant with edited image data 501.
In step 526, image editor 502 is initiated request and is come to start the execution of plug-in unit 504.In step 538, plug-in unit 504 will be carried out one or more read write commands relevant with view data, and preserves the row that some generate for output image.In step 530, plug-in unit 504 will be to image editor 502 return state information.After having finished based on the state confirmation that returns, in step 528, image editor 502 request plug-in units 504 are carried out once more.In step 532, if the work of plug-in unit 504 is not finished as yet, image editor 502 will ask plug-in unit 504 to be carried out once more.In another embodiment of the present invention, if if plug-in unit 504 is finished or makes a mistake, image editor 502 is with the executive process of replacement plug-in unit.In response, in step 540, plug-in unit 504 will move or be reset.
Fig. 6 is the process flow diagram according to the processing multi-medium data of one embodiment of the invention.As Figure 1A and shown in Figure 6, in step 602, at least one plug-in unit 112a is loaded among the on-chip memory 122a of mobile multimedia chip 114a, and this mobile multimedia chip 114a includes integrated image editor 116a.In step 604, select at least a picture editting's effect based at least one the plug-in unit 112a that loads.In this, plugin manager 109a can realize that setting, execution, replacement and/or the forbidding of plug-in unit 112a are to carry out the picture editting.In step 606, based on the plug-in unit 112a that is connected to image editor 116a, image editor 116a with picture editting's effects applications of choosing on view data.In step 608, in the on-chip memory and/or chip external memory of image data storage in mobile device 102a behind the editor.
Therefore, the present invention can be realized by the combination of hardware, software or hardware and software.The present invention can realize at least one computer system in a concentrated manner, perhaps realizes with the distributed way of distribution of different parts in the computer system of several mutual connections.The computer system of any kind of or other equipment that can realize method of the present invention all are suitable for.A typical combination of hardware, software and firmware is the general-purpose computing system with computer program, when this computer program is uploaded and carried out, controls this computer system so that realize method of the present invention.
Can be used as independent chip in the enforcement of the present invention, application specific integrated circuit (ASIC) is embodied as the circuit board level product, or be integrated on the independent chip as different assemblies other parts with system.The degree of the system integration is mainly by speed and cost factor decision.Because modern handle complicated specific can be used commercial available processors, realize the outside that this processor can be realized at the ASIC of native system.As selection, if processor can be used as ASIC core or logic module, the part that so commercial available processors can be used as the ASIC equipment that has multiple function of form of firmware realizes.
The present invention also can embed in the computer program that includes the various features that can realize described method, can realize the described method of the application when this program is loaded in the computer system.Described herein computer program is meant, for example, one group of instruction with any language, code or symbolic representation, can directly make system carry out specific function, perhaps through making system carry out specific function after the following a kind of or various processing: a) convert another kind of language, code or symbol to information processing capability with information processing capability; B) duplicate with different materials.But the implementation method of those of ordinary skill in the art's other computer programs as can be known also can be used for the present invention.
Below invention has been described in conjunction with certain embodiment, and those of ordinary skill in the art can carry out various changes or be equal to replacing and not departing from the scope of the present invention as can be known to the present invention.In addition, the various modifications of carrying out according to instruction of the present invention to adapt to particular environment or material do not depart from the scope of the present invention yet.Therefore, the present invention is not limited to disclosed specific embodiment, the present invention includes all embodiment that fall in the claim scope.
The application quote in full and require the applying date be February 12 in 2005 day, be called the U.S. Provisional Patent Application No.60/652 of " mobile multimedia processor in picture editting method and system ", the right of priority of 481 (act on behalf of case number be No.16436US01).
The application also quotes in full following patented claim:
U.S. Patent application No._____ (act on behalf of case number for No.16430US02), the applying date is 2006 years _ month _ days;
U.S. Patent application No._____ (act on behalf of case number for No.16431US02), the applying date is 2006 years _ month _ days;
U.S. Patent application No._____ (act on behalf of case number for No.16432US02), the applying date is 2006 years _ month _ days;
U.S. Patent application No._____ (act on behalf of case number for No.16433US02), the applying date is on Dec 14th, 2005;
U.S. Patent application No._____ (act on behalf of case number for No.16434US02), the applying date is 2006 years _ month _ days;
U.S. Patent application No._____ (act on behalf of case number for No.16435US02), the applying date is on Dec 14th, 2005;
U.S. Patent application No._____ (act on behalf of case number for No.16436US02), the applying date is 2006 years _ month _ days;
U.S. Patent application No.______ (act on behalf of case number for No.16437US02), the applying date is 2006 years _ month _ days;
U.S. Patent application No.______ (act on behalf of case number for No.17063US02), the applying date is 2006 years _ month _ days.