JP2003528538A - Transmarking watermark embedding function as rendering command and watermark embedding processing based on multimedia signal features - Google Patents

Abstract

(57) Summary The watermarked media signal (20) is transmarked to adapt the watermark to the robustness and perceptibility constraints of the new environment (34). A first watermark is detected in the media signal (22), and message information from the first watermark is embedded in a second watermark (44) before the media signal undergoes a conversion process. This second watermark (44) is adapted to survive the conversion process. Additionally, a watermark embedding command is included in a set of rendering commands used during the media object creation process to specify how the media object should be rendered. The watermark embedding commands include identifiers used to link to customer or related content information and customer websites, the strength of embedding the watermark, areas not to embed, batch processing options, image printing preferences, and different media types. Or the format of the media object or the watermark embedding method used in the different parts and the desired rendering quality.

Description

Detailed Description of the Invention

RELATED APPLICATION DATA This patent application is incorporated herein by reference and is filed on March 18, 2000 by Ken Levy, in “Embedded Data an”.
US Provisional Patent Application No. 60/190, entitled "Data Scrambling Implementations" (improving embedded data and data scrambling).
It seeks the benefits of No. 481. This patent application is also incorporated herein by reference and is incorporated by reference in US provisional patent application No. 60/60, entitled "Watermark Systems and Methods," filed December 21, 2000, by Ken Levy et al. It seeks the benefits of 257,822.

This patent application is also incorporated herein by reference and is referenced by Ken Lev
filed on May 2, 2000 by y and Geoff Rhoads
connected Audio and Other Media Objec
It is a continuation-in-part application of US patent application Ser. No. 09 / 563,664 entitled "ts (Connected Audio and Other Media Objects)".

This patent application is incorporated herein by reference and is incorporated by reference by Seder and Car.
filed on August 1, 2000 by r. Perry, Graham, and Rhoads, "Management of Document and Oth."
er Objects Using Optical devices (management of documents and other objects using optical devices) ".

This patent application is also incorporated herein by reference, McKinle.
filed on November 2, 2000 by Y and Hein, "Batch Id
entity Registration and Embedding
In Media Signals (Registering and Embedding Batch Identifiers in Media Signals) ", US patent application Ser. No. 09 / 706,505.

This patent application is hereby incorporated by reference and US Provisional Patent Application No. 60 / 101,851 filed by Ken Levy on September 25, 1998 and December 2, 1998, respectively. And US Provisional Patent Application No. 60 / 110,68
No. 3, for U.S. patent application Ser. No. 09 / 404,292 filed by Ken Levy on Sep. 23, 1999 and assigned to AIPL.

[0006]

TECHNICAL FIELD OF THE INVENTION

The present invention relates to multimedia signal processing, and more particularly to steganography, digital watermarking and data concealment.

[0007]

BACKGROUND OF THE INVENTION

Digital watermarking is the process of modifying a physical or electronic medium to embed a machine-readable code in this medium. The medium is modifiable so that the embedded code is imperceptible to the user or imperceptible to the user, but still detectable via an automated detection process. Most commonly, digital watermarks are added to media signals such as images, audio signals, video signals. However, it may also be applied to other types of media objects, including documents (eg via line, word or character shifts), software, multidimensional graphic models, surface textures of objects, etc.

Digital watermarking systems typically detect embedded watermarks from two components, an encoder that embeds the watermark in the host media signal and a signal that is suspected of containing the watermark (a suspect signal). It has a decoder to read by. The encoder embeds the watermark by modifying the host media signal. The reading component analyzes the suspect signal to detect if the watermark is present. In applications where the watermark encodes information, the reader extracts this information from the detected watermark.

Several specialized watermarking techniques have been developed, and for robust watermarks the goal is to design an imperceptible watermark that survives the transformation. However, this has not always been achieved. The reader is assumed to be familiar with the literature in this area. Particular techniques for embedding and detecting imperceptible watermarks in media signals are described in assignee's co-pending application Ser. No. 09 / 503,881 and US Pat. No. 5,862. , 26
No. 0 is detailed. Watermarking techniques specifically adapted to graphic arts and halftone images are described in "Methods a."
nd Systems for Watermark Processing
No. 09 / 074,034 and “Halfto” of of Line Art Images (Method and System for Watermarking Line Drawing Images) ”.
ne Watermarking and Related Applicat
09 / entitled "ions (halftone watermarking and related applications)"
689,226 and "Halftone Primitive Waterma.
60/263, entitled "Ranking and Related Applications".
No. 987.

In watermarking applications and related literature, digital watermarks are classified as robust, fragile and semi-fragile. Robust watermarks are designed to survive the typical and even fraudulent processing of watermarked signals, which distorts the watermarked signal, making it more difficult to reliably detect and read. Refers to the watermark. A fragile watermark can be printed, duplicated, or
A watermark that degrades in response to some form of processing, such as scanning, compression, and so on. Weak watermarks are commonly used in authentication applications to detect signal tampering. Semi-fragile watermarks combine the ideas of fragile and robust watermarks. These types of watermarks are designed to survive some type of processing, such as compression, and to detect tampering, such as trimming or exchanging signals. Fragile watermarks and semi-fragile watermarks can be used to trigger certain actions or suppress the use of the watermarked content when degradation of the fragile watermark is detected.

There are numerous challenges and trade-offs in digital watermarking media signals such as audio, still images, and video. One challenge is to embed the watermark so that it is sufficiently robust against the particular set of attacks envisioned for that application, but sufficiently imperceptible to that application. In some applications, it is not possible to fully predict the type of processing a media object will encounter even before it is delivered. For example, music tracks are produced and distributed in a number of different formats (different compression ratios, different compression codes, different broadcast formats, etc.). Each of these formats causes various watermark degradations or distortions. In addition, music tracks are rendered using hi-fi (high fidelity) audio devices and low quality devices, causing different perceptual quality constraints. In particular, low-quality rendering allows the watermark to be embedded more robustly because the perceptual constraint on the watermark is less stringent. The same thing
The same is true for video signals such as movies, TV shows, advertisements, etc.

In the case of a still image, the image may undergo transformations such as compression, color transformations, halftoning, etc. before it is finally printed or rendered. Consider, for example, graphic arts used in advertisements, packaging materials, brochures, etc. Such art images include a collection of raster images that are combined to form the final image. For a particular design project, a graphic artist generally creates a piece of graphic art for a customer, including a collection of component images in various formats. Some of these images are line drawings, vector graphics,
Color halftone or color multi-level pixel image per pixel (RGB,
Color format (such as CMYK or YUV).
The entire image product is described in a work form that encapsulates a rendering function that controls the assembly of component images and the printing process.

The customer may want to watermark the final image product for various applications such as inserting a customer identifier for tracking purposes, linking the image to the customer's website, etc. . There are two major interrelated issues that are potentially interrelated. One problem is caused by the content flow and the timing of adding the watermark flow. Another problem occurs by adding a watermark to vector graphics. The step where the watermark message payload and the embedding parameters are defined may not necessarily be the proper step for embedding the watermark in the host signal. One place for embedding a watermark message payload in graphic arts is at the Raster Interface Processing (RIP) stage. At this stage, the component images are assembled and converted into a particular halftone image format compatible with the printer. This halftone image format includes one or more color planes of pixels that specify whether ink is present at corresponding pixel locations. RI
The P stage usually occurs at prepress offices or printers and requires someone with the strictest eye on color. Further by definition, this step results in a perfect raster image. Watermarks can be defined in terms of vector graphics (or line drawings), but will eventually be embedded in the raster image when printed on typical modern devices. Customers typically do not interact with prepress offices or printers except perhaps when proofing images. Furthermore, these locations are under tight time and cost constraints and do not want to handle inefficient and costly customer interactions.
Finally, many graphic art pieces contain little or no raster portion. Therefore it is not possible to add a watermark before this art is rasterized at the RIP stage. Despite the difficulty of watermarking prior to rasterization for printing, it is often necessary to preview the watermarked final image product on a display screen or desktop printer, which is for previewing purposes. We raise the question of how to embed a watermark.

If the graphic artist has to add a watermark in front of the prepress office or the printer, the graphic artist must rasterize the image. This causes two problems. First, graphic artists now have to deliver files with a huge number of bits (or sizes). Second, graphic artists are not the best people to handle the color management needed to create high quality images.

The difficulty is that the customer is already working with a graphic artist and wants to finalize the content of the watermark, but the watermark is ultimately a rasterized image at a prepress office or printer. Is to be embedded in.
Similar problems exist with other media types, such as audio and video, where the watermark payload is specified at a different than optimal stage for embedding watermarks in content.

If the image file is a vector graphic, a participant, such as the owner, can create a vector graphic, whether rendered for printing as described above or electronically distributed as on the web. You may want to watermark the shape. Participants want the watermark to be embedded in the rendered image whenever the vector file is rendered, such as possible on a computer screen in a web browser or printer. This allows identification of illegal copies, such as copies made with the print screen feature.

[0017]

[Means for Solving the Problems]

A method of controlling watermark embedding on a media object by using the watermark embedding command is described below. In the process of creating a media object, the method includes a watermark embedding command between a set of two or more rendering commands that specifies how to render the media object. For example PCL, PDF or Postscript for images, MIDI and structured audio for audio signals, MPE for audio video signals.
Some signal formats, such as G-4 and MPEG-7, contain descriptors that control how a particular media signal should be rendered. The watermark embedding command is a combination of the following items: customer or related content information, an identifier used to link to the customer's website or store, watermark embedding strength, no embedding area, batch processing options, printing of images. It includes a combination of items such as preferences, different media types, formats, or watermark embedding methods to use in different parts of the media object, and desired rendering quality.

The watermark embedding command allows the customer or creator to specify the watermark message payload, embedding parameters and preferences, and also allows the rendering device to embed the watermark to suit a particular rendering process. . In the case of graphic arts, the customer may rasterize the image for display, embed the watermark on command, and preview the watermarked content on a graphic artist's monitor or a low-cost printer that renders the watermarked image. You can In addition, the prepress office or printer can add and modify watermarks without interacting with the customer, thereby saving time and money.

The watermark embedding command typically includes a message payload to be embedded and a rule or link to how to embed these bits. The watermark function is thus implemented according to the embedding method desired when the graphic art is rendered, such as on the display screen, proof, or on the final printing plate.

This method extends to other types of media objects, including audio or music tracks, video sequences, etc.

The advantages of the watermark embedding command are: vector graphics, MI
Includes the ability to embed watermarks in rendering description content such as DI, structured MPEG audio video. Further, the watermark can be embedded at a different time and place than when and where the watermark and content are rendered. This reduces costs by allowing proper interaction between content owners and authors with different responsibilities and proficiency.

The present invention further provides a method and associated system, apparatus and software for transmarking a media signal. Transmarking involves converting auxiliary data embedded in a media signal from one digital watermark format to another. This is the process of transforming a media signal such as compression, broadcasting, editing, rendering, etc. to change the characteristics of an embedded watermark so that the watermark improves its robustness or perceptual characteristics for its new environment. Used in. In some cases, transmarking is performed when out-of-band data files, media file headers or footers or other metadata with this media file are transmarked into or derived from the watermark. Can be expanded. Thus these watermarks appear robust to all transformations.

One aspect of the present invention is a method of transmarking a media signal pre-embedded with a first digital watermark using a first digital watermark embedding method. This transmarking method detects a first digital watermark in a media signal. The method then embeds the message information from the first digital watermark in the second digital watermark in the media signal before the media signal undergoes a conversion process. This second digital watermark is adapted to survive this conversion process.

Another aspect of the invention is another method of transmarking a media signal. The method detects a first digital watermark in a media signal, converts the media signal to a different format, and then adds a message information from the first digital watermark to a second digital watermark in the converted media signal. Embed. This second digital watermark is adapted to the robustness or perceptual parameters associated with this new format.

Further features will become apparent with reference to the following detailed description and the accompanying drawings.

[0026]

DETAILED DESCRIPTION OF THE INVENTION

Feature-Based Watermarking Noise reduction techniques such as Weiner filtering or spectral subtraction allow the reader to capture the embedded watermark as noise. This noise is
Represents the sum of all watermark layers. This noise can be rescaled and embedded within other images to imitate the look of the original image.

However, embedding another noise layer consisting of a local array around up to N (probably 5) features such as the peak of the derivative of the image can stop this attack. This idea is similar to US Pat. . When using peaks, these peaks should have a certain average slope around the peaks. Alternatively, the reader could use the peak of the derivative of the image because the edge is correlated to this peak and the edge is a good place to hide the data.

To this end, when all noise layers are transferred from one image to another as one grouped noise, as is done in this copy attack, the most likely new feature is Will not match old features. As expected, the more features that are used, the less likely these features are to match between old and new images. The decoder thus knows that the image is fake. In addition, features such as peaks or derivative peaks are robust to most transformations. Finally, since these features occur in the image, no global database is needed to determine where the image-specific watermarks occur.

There is a problem with backward compatibility, which means how the decoder knows if an image has been tampered with or if it is an old image created before the peak noise layer was added. There may be. There are three proposals described below. The first suggestion is
The different groups of global PN sequences could be used in this new version rather than the old version. The second proposal is to add a noise layer that defines this version. Third is the use of different spacings or positions for the grids used to determine scaling and rotation of the embedded data.

Furthermore, feature-based watermarking is advantageous when trying to find watermarked regions of large images. As is well known, searching the entire image for small areas can be time consuming.

As shown in FIGS. 1a and 1b, this process embeds spatially limited data, such as a local PN sequence, which uses picture features such as derivative peaks to provide information about the position and scaling of picture corners. That is. Further, a P × Q pixel area for embedding (for example, P and Q are preferably the same and are multiples of 2).
An entire block structure of watermarks such as could be grounded around this feature. In this way, the embedded data carrying the feature-based watermark and the message do not overlap. The use of derivative peaks is ideal as the eye does not perceive noise near the edges and is robust to scaling and scanning. This is efficiently found in the decoding process as there are only a few appearances of features in the rest of the image. Finally, this is advantageous if the feature does not exist at the edge of the embedded region. If the feature is near the edge, some embedded data, i.e. the PN sequence, will be lost.

This embedded local feature PN sequence will inherently inform the decoder that this feature is part of the picture due to its presence. This local feature PN
The column should also include a grid layer so that the scaling factor can be determined once this is found. Instead of a grid layer, the watermark decoder could use the autocorrelation and cross-correlation scaling methods that compensate for the scaling and rotation discussed herein. This local feature PN sequence should also contain a few layers that give where the lower left (or other corner) of the picture is located. For example, the two layers could tell the decoder in which quadrant the feature is located. A global P that carries a message by scaling information and quadrant information
Finding N columns will be easier and faster.

Scaling This method is illustrated by the following two embodiments. In the first embodiment, autocorrelation between the image and the self-similar noise layer is used to determine the scaling and rotation of the image.

FIG. 2 shows a self-similar noise array layer that can be embedded in the image to determine temporal scaling and rotation, or sequentially in the audio only for 2D images. The PN variable is, for example, a 10 × 10 array of noise and each PN column is the same. The 0 variable is, for example, a 10 × 10 array of zeros. There is a tradeoff between the size of the larger PNO array, which is less visible, and the calculation of autocorrelation. For example, when using a 10 × 10 array, the autocorrelation is 0.5 to 2 ×
It is only necessary to include 20 multiply, add instructions per pixel to capture this change.

The second embodiment involves estimating the image transform by cross-correlating the original PN noise layer and the image that has been modified by adding the PN noise layer beforehand. Assuming that the image is transformed linearly, such as by rotation or scaling, that the PN noise layer is white noise, and that the PN noise layer is orthogonal to the image, this cross correlation The result of is the impulse function of the transformation. This impulse function can be used to improve the reproduction of the watermark. Finally, the idea from spectral estimation can be applied to improve the accuracy of the estimation, since these assumptions are usually only partially true.

In transition audio applications, embedding bit 0 and bit 1 of auxiliary information
The transitions to and from the padding are done by instantly changing the phase of the PN sequence, i.e. by switching from multiplying by -1 and 1, and vice versa. After representing auxiliary bit 0, for example by subtracting 100 ms of shaped noise from the signal, auxiliary bit 1 is represented by adding the shaped noise to the next signal sample for more than 100 ms and so on. It This is also true in video applications. However, the eyes and ears are extremely sensitive to changes.

Therefore, as shown in FIG. 3, the transition between bit 0 and bit 1 of the auxiliary information should have a transition period in which the phase of the noise train changes gently. This will reduce the embedded bit rate, but this should reduce the perceptibility of the watermark. The length of the transition period could be from 1 millisecond to several 100 milliseconds.

Auto-Correlated Watermarks A problem with reading watermarks by digital cameras, typically CCD or CMOS based cameras, is that these cameras integrate space to obtain color values. This integration is used because each camera light receiving element, such as a CCD, images the space and an RGB or CMYK color grid is used. This integration does not degrade the quality of the photo because the real-world photo has data points that are correlated with its neighbors. However, for a watermark based on white noise whose value changes for each pixel, in white noise all pixels are independent, so
Not only does the camera remove noise, it also produces false data. The current solution is
The use of noise whose value changes in units of blocks of pixels.

An alternative solution uses a watermark based on autocorrelation that is designed to take a copy of the image, reduce its level, and place it slightly offset from the original image. Either the offset value or the copy level can be used to transfer 0's and 1's. For example, the upper left shift represents 1, while the lower right shift represents 0. The watermark is retrieved by calculating the autocorrelation and finding the offset value of the peak given by the embedded low-level and shifted copy of the image.

This type of watermark breaks through the integration, as neighbors will be related and will break through the camera's integral, as it would have real-world data. This watermark will also be invisible as it inherently puts the data where it can be hidden. In other words, the offset copy of the image is already prepared to be hidden in the image.

The prior art shows this type of mark used in audio, and each bit is included in the reference here by Aris on Aug. 17, 1999.
Sequential embedding, as per US Pat. No. 5,940,135 assigned to Technologies. However, this process can only work on images in a video. Thus, for a single image, if the entire image is used, then just one bit per image could easily be embedded and retrieved.

As shown in FIG. 4 a, using a process that uses several blocks per image,
The embedded data rate can be increased. Block size is a balance between the number of embedded bits and the amount of noise embedded to search for 1 bit. Moreover, the smaller the block size, the more information is lost in the edge pattern. Finally, the shift used in embedding the low level copy of the image should be minimized so that there is no loss of quality such as edge blurring. It seems desirable to have a single camera pixel element, i.e., a shift greater than one CCD grid.

Finally, when dividing an image into multiple blocks, orientation of these blocks with respect to the retrieved image is required. Traditionally, a noise grid covering each block is used. However, in order to locate the center or similar part of the image, skipping the embedding process in some blocks can be used. In Figure 4b, the X block contains a watermark and the block without X has no watermark. As you can see, the unwatermarked block points to the center of the image,
Moreover, since these are asymmetric, the rotation is determined.

A problem with encrypting or scrambling dynamic media scrambled content files is that they will likely be stored on a hard disk or optical disc or the like for an extended period of time, perhaps 10 years or more. This would give pirates (piracyers) ample time to break protection. If the pirate does not break the code during the transaction, as compared to other encrypted transactions such as bank withdrawals, then the next transaction uses a new key, which is too late. The current solution is
Rejecting a broken key. However, this may be noticed when your content is not playing or requires re-encryption when the legitimate user is doing nothing, or your device requires a firmware upgrade. Means there is. This would confuse and confuse the customer.

Dynamic media scrambling involves re-encrypting content using new techniques or new keys each time the content is rendered, or at some other interval, perhaps a regular or irregular interval. Or re-scramble (assuming the device is re-writable). This technique is invisible to the customer. Moreover, when the key is noticed to have been broken, removal of that key from the system will occur overtime without any inconvenience to legitimate customers.

When the content is rendered on the user's machine, the encryption routine uses the current key to decrypt the content. A new key is then created and the encryption routine encrypts the content for storage on the user's machine. The encryption routine modifies some or all of the previous keys to generate a new key. In particular, some of the keys can be based on the processor ID or unique to the machine or software running on the machine, such as an operating system waste bin or recycle bottle. The rest of the key changes for each rendering according to a random or pseudo-random function. When a new key is created, it is stored in a secure, encrypted, tamper-proof file on your machine. This key will be used the next time the content is played.

The key does not change each time the content is rendered. Alternatively, the key may change every Nth time the content is rendered. Here, N is a predetermined integer. Alternatively, the key is a key management system or registry that instructs the encryption routine on the user's device to receive a new key from a local or remote key management system, or to update the key the next time the content is played. It can also change based on some external event trigger, such as receiving a key update flag from the database.

This key renewal process allows the encryption key to be renewed after hours and ultimately removes the old or broken key from the system.

Transmarking In many applications, a digital watermark signal embedded in a media signal such as audio, video or still image can be modified when this signal is transformed. Digital watermark transmarking can be used to modify the embedded digital watermarking scheme at the signal transformation points that should fit the new signal.

For example, when playing DVD audio on a radio, analog or digital radio, the watermarks can be retrieved and re-embedded at different levels or at different broadcast locations. In addition, the watermark could be modified at the relay station due to the increased noise level in the signal. Although this method of audio application can retrieve watermarks, the original DVD can minimize the perceptual changes due to watermarks. More specifically, the audio application may be searching for the watermark in a noisy room, and the artist D
You won't complain that VD watermarks are ruining your recordings.

This is the same as US Provisional Application No. 60 / 101,851, filed September 25, 1998 and December 2, 1998 by Ken Levy, all of which are hereby incorporated by reference. No. 09 / 404,292 filed by Ken Levy on Sep. 23, 1999 and assigned to AIPL, based on 60 / 110,683. These patent applications show that audio is from raw PCM format to MP3, A
It discusses changing the type of watermark as it is converted to AC, Real, Liquid and similar compressed formats.

This method also applies to video signals. For example, a watermarked DVD video is R
When transferred to low-bandwidth Internet video, such as that provided by ealNetworks, this DVD watermark is read to overcome the strong compression needed to stream video over low-band and amplitude amplified. Be done or re-embedded. This watermark can be used for copy protection, but could also be used to make available links or information about the video.

In some applications it may be useful to convert the auxiliary information embedded in the media signal from one format to another. This conversion process is another application of transmarking. Transmarking may also include converting out-of-band identifiers such as header / footer tags into watermarks and vice versa. This may also include converting a message in one watermark format to another format. This process includes decoding operations on the input media object and encoding the decoded information into the media object. This may also include the process of first removing the marks in the media object to avoid interference with the newly inserted marks.

There are various reasons for performing transmarking. One is to convert from one watermark used for packaged media to another watermark used for compressed and electronically distributed media, or for watermarking radio or wireless telephone broadcast transmission applications. , To make this embedded information more robust to the types of processing media objects are likely to encounter.

This type of transmarking can be done at various stages in the distribution path of media objects. The identifier in the watermark or in the file header / footer may be encoded in either an electronic distribution format or a physical packaging medium such as an optical disk or magnetic memory device when packaging the content for distribution. it can. At some point the media signal can be converted from one format to another. This stage of format conversion is an opportunity to perform transmarking adapted for the new format with regard to robustness or perceptual issues. This new format may be a broadcast format such as digital radio broadcast or AM or FM radio broadcast. In this case, the identifier can be transmarked into a strong watermark format or other metadata format for broadcast applications. This new format may be a compressed file format (eg truncation from optical disc to MP3 format). In this case, the identifier can be transmarked to a file header / footer format or watermark format that is robust and compatible with the compressed file format.

The transmarking process leaves the existing embedded identifiers intact and inserts additional identifiers as layers within the media object. This may include encoding a new watermark that does not interfere with the existing watermark (eg, inserting the new watermark in an unmarked part of the media object, or in a transform area that does not interfere). This may also include adding additional or new identifier tags to headers or footers within the file format.

FIG. 5 is a flowchart showing the transmarking process. The input to the transmarking process is a digital watermarked signal 20 such as an audio signal (eg music track), video signal or still image. A digital watermark is one or more symbols that carry information such as a content identifier, transaction identifier, database index, use or replication management parameters (flags that instruct a device or process to not replicate, copy once, not transfer, etc.). For example, it carries a message payload in binary or M-ary. For digital watermarking of multimedia content, statutory dispute tracking, broadcast monitor, replication management and rendering with watermarked signal either in response to user input or automatically when the watermark signal is activated. There are various applications including using a watermark as a trigger or link for interactive content to be done.
Some of these applications are part of co-pending patent applications 09 / 571,422, 09 / 563,664, 09 / 574,726 and 09/597, which are incorporated herein by reference. , 209.

In these applications, there are many reasons to transmark watermark signals embedded in host signals. Some examples are to improve the robustness of the watermark when the watermark undergoes format conversion (compression, transmission, digital-analog conversion, increase / decrease of sample number, printing, display, etc.), or perception of the watermark before playback. And / or balancing the perceptual level vs. robustness level trade-off of the watermark signal for the new format as the host signal undergoes conversion from one format to another.

The transmarking process shown in FIG. 5 begins by detecting the first watermark in the watermarked signal (22). The watermark detector uses the watermark key to identify the presence of the watermark. The specific operation of this detector depends on the watermarking process used. In many approaches, the watermark key is spatial, temporal and / or
Or specify the frequency domain position. It can also specify how to decode messages modulated with pseudo-random numbers (eg frequency or phase hopping, spread spectrum modulation). To simplify the watermark search, the watermark detector searches for a reference signal attribute of the embedded signal such as a specific time, space, a known embedded symbol sequence in the transform domain, or a known signal pattern. These attributes allow the detector to determine if the watermark is present in the suspect signal and also to determine the position of the watermark in the time, space or transform domain.

The watermark detector can then optionally decrypt the embedded message (26) such as replication management parameters, content identifier, owner identifier, transaction identifier, etc. This step is optional because the initial detection operation can convey enough information to trigger the rest of the transmarking operation. For example, merely detecting the presence of a watermark signal at a particular time, space or location of the transform domain can convey one or more bits of message information.

Some samples will help explain this detection, message decoding process. One type of watermark embedding process encodes a symbol by inserting an amplitude-shifted and shifted version of the host signal. This shift is
It is a combination of temporal, frequency and / or spatial shifts of the host signal, depending on the nature of the signal (eg temporal frequency for audio, spatial frequency for images).
This shifted version depends on the presence or absence of the shifted version (s) at a particular shift position relative to the host, and / or on the statistical characteristics of the host signal due to the embedding of the shifted version. The message symbol value is conveyed by the amount of change that is introduced. Other types of embedding processes embed watermarks by modulating perceptual domain samples and / or transform domain frequency coefficients. In both cases, the message applies a pseudo-randomization process prior to making changes to the host to hide the resulting message sequence in the host signal (eg, multiply or exclusive OR operations on the PN sequence). Can be randomized by expanding the message by doing). The message can be embedded by an addition process of the modified signal and / or by quantization of sample values or frequency coefficient values or statistical characteristic values.

In these embedding techniques, the detector searches for the attributes of the watermark signal by detecting a shifted version or modulated samples / coefficients using correlation or statistical analysis. By identifying known symbols or evidence of watermark signal attributes, the detector determines if a watermark signal is present. In some cases, the watermark detector determines that there is an additional message payload message based on the detection of certain watermark signal attributes. The detector then proceeds to decode the additional signal attributes and map them to message symbols. Furthermore,
The message payload can also be extracted using error correction decoding such as BCH, Turbo, Reed Solomon, and convolutional decoding.

Next, the transmarking process removes the first watermark signal (28). Again, this process is optional, as the transmarking process can proceed by embedding the second watermark without any particular attempt to remove or mitigate the effects of the first watermark. Once the watermark detector detects the watermark,
When determining the spatial and / or frequency domain location, the detector can remove the watermark or reduce its effects. This effectively renders a watermark when the embedding function is reversible, such as a reversible addition operation, by performing the inverse operation of the embedding function with a watermark key that specifies the attributes and position of the first watermark. Can be removed. It can also remove watermarks without knowing the inverse function such as using a whitening filter with watermarking function based on PN sequence.

Interestingly, this would allow adding more imperceptible watermarks to content that is moving from low quality media to high quality media. Although the content is still in the quality of the original medium, this watermark will minimize the quality degradation or cause no further quality degradation. When converting from high quality media to low quality media, removing the first watermarks still improves quality and robustness by reducing the interference between each watermark.

In some applications, the watermarked signal may be converted to other formats such as a transmarking process or compression of the signal before proceeding. These applications are the applications in which new format signals are available for watermarking. In this case, the transmarking process proceeds by embedding the second watermark in the host signal after the format conversion has been performed. This allows the watermark embedding process to adapt the watermark to the perceptual quality and robustness parameters of the new format signal. In other applications, where the signal is broadcast, it is difficult or impractical to block the signal to embed a new watermark after the format conversion has taken place. For example, format conversion can occur as a result of broadcast transmission. In this case, the transmarking process proceeds to embed the second watermark, adapting the watermark to the robustness and perceptual quality parameters suitable for the new format of the signal before the format conversion is performed.

The transmarking process then encodes the second watermark (44) using the same or somewhat different embedding process as the first watermark (30). This second watermark can be added by a feedback loop before conversion, after conversion, or during conversion. For example, the first watermark can be embedded by adding a shifted version of the host signal, while the second watermark is perceptually in a perceptual domain or some transform domain (such as Fourier, DCT, wavelet, etc.). It can be embedded by adding an adapted pseudo-random carrier signal and vice versa. The second watermark is different from the first watermark in time of the host signal,
The spatial or frequency part can be modified, or the two watermarks can overlap at one or more of these parts of the signal. Whether the watermark embedding function is basically the same as or different from the function used to embed the first watermark, this embedding process (30) uses a new format or environment perceptual and robustness constraints. Especially adapted to and. This watermark embedding process uses a robustness parameter (32) (eg, watermark signal gain, degree of redundancy, frequency domain position) and watermark strength and redundancy specifically adapted to the watermark for survival in the new format. Specifies degrees and frequency domain position. This second watermark adds new information about the transformation that this information can use for legal dispute tracking.
This information can be in any combination of the following: the translation device identifier (MPE
G coders or manufacturers) and delivery system identification information such as broadcast network or cable system identifiers could be included. This new information augments the original information embedded in the first watermark, does not change its meaning, but instead adds additional payload information.

In order to ensure that the second watermark satisfies the robustness constraint, this embedding process optionally applies a feedback path that adds the watermarked signal to the degradation process and then decodes the known message. The number of errors that have occurred in doing so, and the part of the signal in which these errors occur (temporal, spatial or frequency part)
Selectively increase the gain of the watermark signal at. These degrading operations may include compression operations, or operations that model likely degradation to occur in new formats such as digital-to-analog conversion, print / scan, broadcast transmission, time scaling, and the like. This process is repeated until the measured error rate falls below the acceptable threshold.

Furthermore, the embedding process uses a perceptual quality parameter 33 which specifies constraints on the perceptual quality of the signal for the new format. These parameters are
One can specify limits on the watermark strength or define automatically measurable perceptibility thresholds such as the peak signal-to-noise ratio typically used in the analysis of digital watermarking methods. Again, as described above, this embedding process optionally includes a feedback path for measuring the perceptual quality of the watermarked signal, and also the portion of the signal (time Signal, in the spatial, spatial or frequency part).

FIG. 5 graphically illustrates the interaction between the watermark embedding process 30 on the one hand and the rendering / editing or transmission environment (34, 36) on the other hand. This figure shows how the embedding means adapts the new watermark to the environment in which the transmarked signal is used. For example, if the signal is a still image used in a photo editing environment, the robustness of the watermark can be adapted to the image processing operation of the editing tool. If the watermark needs to cut through the print,
The transmarking process survives the process and embeds in the signal a new watermark designed to be reproducible from the printed image through the scanned image. In this case, the watermark embedding means can include additional calibration signal information as described in US Pat. No. 5,862,260 to ensure that the watermark can be detected despite geometric distortions.

By the way, since the second watermark is adaptable to the intended environment, the operation of the editing tool can be modified so as to improve the survivability of the watermark. In this case, image editing operations such as blurring, color conversion, etc. are adapted to preserve the watermark signal as much as possible. In particular, low pass filtering or blurring operations, which generally reduce high frequency components, are performed to pass those selected high frequency components in order to preserve the high frequency component watermark signal. The operation to add Gaussian noise is
It can be modified by shaping or reducing the noise at these frequencies in order to reduce interference with the watermark signal at certain frequencies. If the watermark is inserted by modifying a particular color channel such as intensity, the color conversion operation can be designed to preserve the intensity of the watermarked image.

Furthermore, the signal editing tool can be integrated with a transmarking process to decode the watermark before the operation and then re-encode the watermark after the operation to ensure the preservation of the watermark. For example, the watermark can be reapplied after image editing tools have been used to perform affine transformations on the image, or after the image has been cropped.

For the transmission of media signals over a communication channel, the watermark can be transmarked at the point where the signal (audio, video or image signal) over the communication channel is transformed. These are the case when the signal is uncompressed and compressed in other formats, and when the signal is transformed at the router or repeater (eg watermark when the signal is amplified at the router or repeater node of the channel). High strength transmarking), the signal is transformed into packets in the switching network and the watermark signal is decoded and re-encoded within individual packets, or the signals are re-combined and then re-encoded. Includes cases and. This re-encoding is done by forwarding the watermark addition command to the header of the packet that specifies the watermark payload and watermark embedding protocol used for the recombined signal.

For audio and video compression codecs, the transmarking process can be integrated into the compression codec. This allows the codec to modify the compression operation or the bitrate to ensure the survival of the watermark. In the first case, the compression codec can be designed to preserve certain frequency components that would otherwise be substantially reduced to preserve the watermark. In the latter case, the codec chooses a bit rate at which the watermark will survive and the signal will be compressed to an acceptable level.

If the watermarked signal is about to be rendered on a high fidelity device, such as a DVD player, whose use is tightly controlled, the second watermark can be embedded with less impact on the perceptibility. Conversely, if the watermarked signal is to be rendered on a lower fidelity device, such as a personal computer, the second watermark can be embedded to be more robust while remaining within the perceptual quality parameters of the rendering device. . Furthermore, the watermark is a DVD
This can be changed when the audio master is converted to a CD or a cassette tape.

If the watermarked signal is about to be transmitted in a broadcast environment or the like, the embedding process will survive the broadcast and maintain robustness to maintain perceptual fidelity within the less stringent constraints of the broadcast environment. Encode the second watermark that has. The transmarking process can be used to encode triggers used for interactive video or audio. These triggers may originally be encoded in one format, but may be transmarked to another format before broadcasting or at some node in the broadcasting process. The trigger may be transmarked to video, for example, when the trigger is compressed to MPEG2 format for broadcast, or when the content is received at the headend of the cable or a node of the content delivery channel. This trigger may be a network address of interactive content such as an IP address or URL, or may be an index to a network address of interactive content such as HTML or XML.

As another example, a trigger for interactive content in radio broadcasting is prepared for broadcasting such as conventional radio broadcasting, digital satellite broadcasting, or broadcasting that flows on the Internet when the content is transferred from a packaged medium such as an optical disk. You can transmark when you are told.

This second watermark, like the first watermark, uses the watermark key 38 to specify the spatial and temporal and / or frequency attributes of the second watermark. Further, the message decoded from the first watermark such as the identifier 40 and the copy management parameter 42 is embedded.

In the typical case, the result of the transmarking process is the new watermarked signal 4
It is 6. As noted, the watermark information or functionality is
It may be transmarked to out-of-band data such as file headers or footers such as D3 tags. Conversely, out-of-band data may be transmarked into in-band data embedded in the host signal using digital watermarking.

Watermark Embedding Function in Rendering Description File Tools for generating documents and other media objects continue to become more sophisticated and complex. Adobe offers a variety of such tools, including their InDesign software. Watermarking can be advantageously implemented in such systems.

In such an environment, documents are created using various tools, most of which can insert watermarks. One program can use (ie, "composite") the output of one or more other programs as input.

To better handle watermarking in this environment, watermarking functionality (eg, PostScript-like commands) is provided in these tools. This function is invoked with parameters that specify the desired characteristics of the watermark information, eg payload, robustness level, mask used. The watermark is actually added as digital data at the time of rendering such as screen display, proof printing, or ripping (cutting) of the final version. In such an environment, the embedding means knows the nature of the rendering device, such as a printer, and thus adjusts its embedding appropriately. The idea is that the watermark is not lost during the compositing operation and can be embedded in a vector (or line) graphic. Furthermore, the color manager at the ripping stage can be the best entity for adding watermarks.

This idea likewise extends to video-specifically MPEG4 object video and audio-specifically MIDI or MPEG4 structured audio languages and virtual advertising.

The use of PostScript-like features for embedding watermarks is described in US Application No. 09 /
No. 629,401 for further details.

An alternative is that desktop tools do not have the ability to watermark, but instead an online watermarking server is available to support common image formats. Various tools can present an image to the server with information about the desired parameters of the watermark.
The server then returns the image to the application. The integration burden is thus virtually eliminated, and registration and marking take place simultaneously.

When watermarking graphic arts materials such as packaging, it is desirable to have the graphic designer embed the desired watermark for the person or company making the packaging, rather than the printer. It is advantageous to have the watermark embedded in the graphic artist, as the consumer is already talking to the graphic artist and the consumer will never have to talk to the printer. Usually, printers are only required to proof print plates or prototypes. Moreover, printers don't want to remember anything extra, and printing alone is hard enough.

However, many graphic arts materials remain line drawings (also known as vector graphics) until rasterized during the printing process, and the latest watermarking technology is based on rasters. There is.

The solution is to embed the watermark function in the line drawing file in a manner similar to how fonts are described by Bezier curves. The watermark function not only includes the bits to be embedded, but also defines how to embed these bits in different elements. The watermark function could be thought of as a command in the popular Postscript (EPS) format.

For example, if text and watermark creation are included in the line drawing, the watermark bits are
Embedding could be done by slightly adjusting the position of each character vertically or horizontally, or both. Alternatively, the watermark could be embedded at the edge of the character by adding or removing a hump that is too small to be seen but is digitally readable. It is important that any data embedding method can be used according to the bits and rules of the watermark function. Similarly, when creating a construction object, the watermarking function could be achieved by embedding bits in the humps along the edges of this object. Alternatively, if you have a gradient fill inside the object, the watermark function can be achieved by adding a more traditional PN sequence or modulating the halftone dots within this gradient fill. Could be done.

In general, the watermark function has the bits to be embedded and the rules of how to embed these bits or a link to this method. Thus, the watermark function is implemented according to the desired embedding method when rendering the line drawing on a display screen, printer or printing plate.

As noted, the watermarking feature is applicable to various types of media objects and rendering description languages. 6 and 7 show a framework for realizing and using the watermark embedding function as a rendering command. FIG. 6 shows the watermark embedding command (100) and the insertion of this command in the rendering description file (102). The watermark embedding command is specified in text format or other binary format that is compatible with the rendering description file in which it is inserted.

When creating a media signal, the user specifies a watermark embedding command and related parameters. Later at render time, the rendering device calls the watermark embedding module to embed the watermark in the media object according to the watermark embedding command. The watermark command parameters include a combination of parameters that describe the watermark message payload, watermark protocol, watermark embedding method, payload specification, embedding location, robustness parameter, and perceptual quality parameter. Any combination of these and other parameters can be used, depending on the application.

The watermark message contains a certain number of binary or M-ary symbols. These symbols can represent various types of information related to the media signal in which they are embedded, including replication management parameters that control the rendering or transfer of the object and the media object and its ownership, to name a few. A transaction identifier of a person or an object (user ID, machine ID, storage device ID, etc.), related information, a network address of a program, a website, etc., a command of a program or a device, metadata about a media object, and Information or other information such as a program executed in response to watermark detection.

The watermark protocol specifies how the watermark message should be embedded and the meaning of the various symbols in the watermark message. This protocol is
It can be specified using one or more parameters. These protocol parameters are
It contains parameters that specify the embedding method, such as a pointer to the embedding module or plug-in used by the rendering device to embed the watermark. There are several different embedding methods per media type. For image signals including video and still images, the embedding method encodes the symbol by adjusting the sample or feature to a spatial or frequency domain spread spectrum watermark embedding means and a quantization level associated with the symbol to be embedded. And a halftone modulation method (changing the halftone dot shape, the display screen, the error diffusion threshold, the size or width of the dot cluster, etc. according to the change related to the message symbol). it can. For audio signals, the method comprises means for temporal or frequency domain spread spectrum watermark embedding, and means for embedding the symbol by adjusting the sample or feature to a quantization level associated with the symbol to be embedded, Watermark embedding means or the like for encoding a masked tone or collection of time / frequency shifted versions of the host signal corresponding to the symbol to be embedded. In some cases, the method may also be left unspecified so that the rendering device or transmission channel may optimize the watermarking method and protocol for this rendering device or channel. In this case, the rendering device or channel has a compatible decoder associated with it for decoding the watermark. Alternatively, a generic watermark signal or metadata can be used to specify the watermark type for decoding.

The protocol parameters may also include more detailed information about the watermark payload, namely the payload specification. The payload specification should include items such as the type of error correction code to use, the type of error detection to use, the number of message symbols (eg binary bits) in the payload, the encryption key for encrypting the payload, etc. You can

This protocol can also specify where to embed the watermark, which is referred to as “embedding location” in FIG. Embedding locations include, but are not limited to, spatial, temporal and transform domain locations for embedding a watermark in a host media signal. Transform region location refers to a transform region coefficient or set of coefficients at a particular block size of content. Examples of transform domains include the Fourier domain, wavelet domain, DCT, etc. The embedding location specifies, for example, that the watermark should be confined within a certain frequency range within the signal. Also, for images and video, the embedding location can specify one or more color planes in which the watermark signal should be embedded, such as a luminance channel, a blue channel, or other color channels.

In some applications, the watermark embedding means will embed different message payloads in different parts (spatial, temporal, frequency, transform domain parts) in the host media signal. In these cases, the watermark embedding command is
Specify these parameters for each of the different message payloads, including their embedding location, strength, vulnerability (for fragile watermarks), robustness parameters, perceptual quality parameters, redundancy, etc. This allows the watermark embedding module (s) to embed different robust watermarks, robust and fragile watermarks, or a combination of vulnerable watermarks of varying vulnerabilities. In some cases, the message payload may be a single bit truncated to the presence or absence of the watermark signal. This single bit can be spread over a signal that covers several embedding locations and can be repeated in several instances of the same signal, or a combination of both.

As previously noted, the embedding location can be specified by a spatial, temporal or transform-domain mask that specifies the area for embedding the watermark. This mask is an array of elements, each corresponding to one implant location. For each element, the mask is associated with other parameters such as payload for that location, robustness for that location, perceptual quality for that location. The mask may be designed by the creator of the media object to specify where the watermark should be embedded, and vice versa, and specify the watermark strength with respect to the area in which the watermark is embedded.

The robustness and perceptual quality parameters allow the user or application inserting the embedded command to control the trade-off between watermark robustness and perceptibility. Robustness parameters include strength (eg, watermark signal gain for a particular embedding location) and redundancy (eg, degree to which message payload is redundantly encoded across the embedding location to improve its robustness).
, And the frequency location (eg, the extent to which the watermark signal concentrates in the lower frequency region where it is more likely to survive the transformation of the host signal). Each of these parameters can be specified as a preferred range to allow the embedding module to optimize the watermark for perceptibility and robustness within a specified robustness and perceptibility range. .

Regarding the robustness parameter, the watermark embedding command can also specify the level of watermark vulnerability at a particular location in the media signal. Such a fragile watermark is embedded according to the embedding command. The presence of a fragile watermark or its measured strength (for example, by the error detection rate of a known embedded symbol set, or by the threshold level of the detected watermark strength) is then watermarked during watermark decoding. Used to detect signal tampering or processing.

This type of robustness and perceptual quality specification allows the watermark embedding means to perform iterative embedding with a feedback path to optimize the embedding for a particular rendering or transmission device. To do. In this iterative method, the embedding means first embeds the watermark payload according to the command parameters with the lowest robustness and the highest perceptual quality, applies a degradation model for a particular rendering device or transmission channel to the watermarked signal, and then To measure the detection error rate on the message payload (eg, detection error is quantified using a measure of the difference between the decoded symbol and the expected symbol before error correction decoding is applied. ). It then repeats another iteration of this process, gradually increasing the robustness with each iteration until the detection error is at a satisfactory level. The degradation model may be a compression operation, or may be a signal conversion that simulates distortion due to digital analog and analog to digital conversion, time scaling, affine transformation, and the like.

Perceptual quality parameters can be specified using automated measurements such as peak signal-to-noise ratio, which quantifies the distortion of the watermarked signal relative to the unwatermarked signal. The perceptual quality parameter can be specified as an acceptable range or as a threshold that must not be exceeded.

The media object creation program inserts a watermark embedding command into the rendering description file 102 as another rendering command. As shown in FIG. 6, this rendering description file refers to the media signal (110, 112) or description of the media signal (eg 114 as in the case of a vector graphics file) to which the rendering command is to be applied. It contains a collection of rendering commands (104, 106, 108). This file is then stored for later use, sent to the rendering device, or distributed via a transmission channel.

For images and documents, PostScript, PCL, EPS, PDF, work sheet,
There are various possible formats for rendering description files such as vector graphics, structured audio and MIDI for audio, and MPEG4 or MPEG7 for video.

FIG. 7 shows the process of embedding a watermark in a media object using the watermark embedding command. The process begins when the user or application program inserts a watermark embedding function as a rendering command (120) into the rendering description file (122). When the media object described later in the rendering description file is prepared for rendering, the rendering process (124, 126, 128) reads the watermark embedding command and follows the parameters specified in the embedding command (120). Watermark embedding module suitable for embedding watermarks (eg 130, 132)
Call. This watermark embedding module is adapted to the particular rendering device (134, 136, 138) rendering the signal or the transmission channel (140) carrying the signal. In order to prevent degradation of the signal by the transmission channel, this module can be sent as a rendering description file over the transmission channel, which can later be rendered on the rendering device and embedded with the watermark.

The rendering process for an image can be realized by a display driver, a printer driver, or a plug-in to this display driver or printer driver. This process can also be implemented in the printer hardware, and especially so that the watermark is particularly adapted to halftone processing, and after the watermark has been or has been rasterized into a halftone image. Can also be integrated into the halftone processing so that it is embedded in the image. This technique is applied to various halftone processes including ordered dithering (eg, blue noise mask, clustered dot halftones, etc.), error diffusion, stochastic halftoning, and the like. Examples of halftone watermark embedding methods include the following methods. That is, 1. Adding a perceptually adapted spread spectrum watermark signal at halftone dot resolution to a pixel multilevel format image before converting the image to a halftone image. The watermark signal is generated by either convolving or multiplying the message payload with a pseudo-random carrier signal and then scaling the carrier signal based on the masking attributes of the image.

2. Modulating the error threshold used for error diffusion halftoning according to a perceptually adapted spread spectrum watermark signal.

[0107]   3. Modulating the line width of halftone dots.

4. Modulating the shape and size of the halftone clusters for embedding the watermark signal in the halftone images, or modulating these halftone screens according to a predetermined relationship between the halftone screens. Detailed information on watermark embedding methods for halftone images is incorporated herein by reference, "Meth
ods and Systems for Watermark Procedures
No. 09 / 074,034 entitled "Sing of Line Art Images," and "Half.
one Watermarking and Related Applies
applications (halftone watermarking and related applications) "
9 / 689,226 and "Halftone Primitive Wate
60/2 entitled "rmarking and Related Applications (halftone primitive watermarking and related applications)"
63,987.

The rendering process for images, audio and video is implemented in a media object generation tool used to convert signals into distribution, broadcast or transmission formats. In these cases, the signal transformation process selects embedding methods and parameters that adapt the robustness of the embedded watermark and the perceptual quality of the rendered watermarked signal to a particular rendering process or transmission channel. For example, an audio processor renders and distributes music signals,
Embed the watermark payload at a robustness level suitable for broadcast or transmission. Similarly, the video processor renders the video signal and embeds the watermark payload at a robustness level suitable for distribution, broadcasting or transmission.

The watermark feature can specify that the watermark be embedded as part of the signal formatting process, such as part of the process of compressing an image, video or audio signal. This watermark feature allows the watermark module to interact with compression processes that embed the watermark to accommodate its format, such as embedding in a compressed data stream or partially compressed stream. The compression ratio of the signal can be set adaptively by determining the maximum compression ratio that the watermarked signal can still cut through based on the error detection level. Similarly, the perceptual quality parameter can be used to adjust the compression process so that a compression ratio is selected that maintains the desired perceptual quality of the signal and the robustness of the watermark signal.

Alternatively, the watermark function may specify that the watermark is embedded after being converted to a particular format for rendering or transmission (eg, embedded after compression or conversion to a broadcast format). The rendering or transmission channel supplies the embedding module with robustness parameters and perceptual quality parameters for that rendering process or transmission channel so that the embedding module can optimize watermark embedding for the particular rendering process or transmission channel. To do. In particular, the watermark function specifies the watermark robustness that the watermark embedding means must protect when embedding the payload specified in the watermark embedding command, for example, strength or quality constraints.

The watermark embedding module queries the rendering process, device or transmission channel for its robustness and perceptual quality attributes. If the quality requirement is lower, the embedding module can increase the watermark robustness within the tolerance specified by the watermark embedding command parameter. On the contrary, if the quality requirement is higher, the embedding module can select the lowest acceptable robustness level specified in the watermark command to embed the watermark so as to minimize the perceptual quality degradation due to the watermark. The process can be applied to adjust the embedding operation based on the rendering process or the robustness attribute of the transmission channel. If the rendering process is expected to substantially degrade the detectability of the watermark, the embedding module can select the most robust level for the watermark within the tolerance of the watermark embedding command.

Rather than querying the rendering device or channel, the watermark embedding command can be designed to automatically select the preferred watermark embedding method for that device or channel.

The watermark embedding function is particularly well suited for controlling the embedding of watermarks in vector graphics used in virtual advertisements for streaming media such as streaming video. A virtual advertisement is a vector graphic, such as a logo, that is superimposed on a video stream when the video being played is rendered on a receiving device, such as a television with a set top box or a personal computer on the Internet. The vector graphics file defining the virtual advertisement can include a watermark embedding command as described above. When rendering a vector graphic, the watermark embedding module on the receiving device side embeds the watermark in the vector graphic. This vector graphic can be used as a trigger for an interactive TV application in which this video runs. For example, a user may order a product or service while playing pre-recorded content or live content to request interactive information such as web pages or via a personal video recorder such as a Tivo machine. To do so (otherwise use the cursor control device to select the logo displayed on the video screen). The watermark in the logo is then decrypted and the payload pointing to the database entry is extracted from the watermark. The database returns interactive information (URLs, HTML, web pages, etc.) or other program code that runs on the user's set-top box or computer and allows the user to purchase the advertised product. return. As shown in this example, the watermark embedding command can be specified for content that includes a combination of different media signals such as video, vector graphics, and audio that are combined at the receiving device during rendering.

Conclusion While the principles of the technology have been illustrated and illustrated with reference to particular embodiments, it will be appreciated that the technology can be implemented in many other different forms. To provide a comprehensive disclosure without unduly prolonging this application, applicants incorporate the above referenced patents and patent applications by reference.

The methods, processes and systems described above may be implemented in hardware, software, or a combination of hardware and software. For example, the auxiliary data encoding process can be realized by a programmable computer or a dedicated digital circuit. Similarly, the decoding of the auxiliary data can be realized by software, firmware, hardware, or a combination of software, firmware and hardware. The methods and processes described above can be implemented by a program executed from a system memory (computer-readable medium such as electronic, optical, or magnetic storage device).

The particular combinations of elements and features of the embodiments detailed above are merely exemplary and are in accordance with the teachings of these and other teachings of the patents / applications incorporated by this and by reference. Replacements and substitutions are also conceivable.

[Brief description of drawings]

FIG. 1a is a diagram for embedding feature-based watermarks to facilitate searching large images for small watermarked regions.

FIG. 1a is a diagram for searching feature-based watermarks to facilitate searching large images for small watermarked regions.

FIG. 2 shows a pseudo-random noise array that can be used to determine auto-correlation image scaling and rotation.

FIG. 3 is a diagram showing a technique for softening a transition between embedding auxiliary data 0 and embedding auxiliary data 1.

Figure 4a   FIG. 7 shows a grid used to embed a watermark based on autocorrelation.

FIG. 4b shows a method of skipping embedding data into several blocks to find the orientation of a watermarked image based on autocorrelation. X represents a watermarked block, and blocks without X have no watermark.

FIG. 5 illustrates a transmarking process in which a first digital watermark in a media signal is transmarked to a second digital watermark in a media signal.

[Figure 6]   It is a figure which shows a watermark embedding function and a rendering description file.

FIG. 7 is a diagram showing a process of embedding a watermark in a media object using a watermark embedding command.

[Explanation of symbols]

20 watermarked media signal 28 First watermark signal 30 First watermark 32 Robustness parameter 34 New environment 44 Second watermark 102 Rendering description file 110,112 Media signal 104, 106, 108 Rendering commands 120 embedded commands 124,126,128 Rendering process 130, 132 watermark embedding module Rendering devices 134, 136, 138 140 transmission channels

─────────────────────────────────────────────────── ─── Continued front page    (31) Priority claim number 60 / 257,822 (32) Priority date December 21, 2000 (December 21, 2000) (33) Priority claiming countries United States (US) (81) Designated countries EP (AT, BE, CH, CY, DE, DK, ES, FI, FR, GB, GR, IE, I T, LU, MC, NL, PT, SE, TR), OA (BF , BJ, CF, CG, CI, CM, GA, GN, GW, ML, MR, NE, SN, TD, TG), AP (GH, G M, KE, LS, MW, MZ, SD, SL, SZ, TZ , UG, ZW), EA (AM, AZ, BY, KG, KZ, MD, RU, TJ, TM), AE, AG, AL, AM, AT, AU, AZ, BA, BB, BG, BR, BY, B Z, CA, CH, CN, CR, CU, CZ, DE, DK , DM, DZ, EE, ES, FI, GB, GD, GE, GH, GM, HR, HU, ID, IL, IN, IS, J P, KE, KG, KP, KR, KZ, LC, LK, LR , LS, LT, LU, LV, MA, MD, MG, MK, MN, MW, MX, MZ, NO, NZ, PL, PT, R O, RU, SD, SE, SG, SI, SK, SL, TJ , TM, TR, TT, TZ, UA, UG, US, UZ, VN, YU, ZA, ZW F term (reference) 5B057 CB19 CE08 CH01 CH11 DC01                 5C063 AB03 AB07 AC01 AC05 CA29                       CA36 DA07 DA13 DB09                 5C076 AA14 BA06

Claims (60)

[Claims] 1. A feature-based watermark embedding method for concealing ancillary data within a media signal, the method comprising: identifying N features in the media signal that are integers greater than one. Embedding a watermark around said N features by modulating a sample value of a group of samples around each feature according to a layer. 2. The method of claim 1, wherein the features include derivative peaks of the media signal. 3. The method of claim 2, wherein the media signal is an image. 4. The method according to claim 1, wherein the watermark signal is a noise layer. 5. A different type of watermark signal is embedded within the media signal, a first type embedded around the N largest features of the media signal and a second type spread around the media signal. The method according to claim 1. 6. The method of claim 5, wherein the media signal is an image. 7. The method of claim 6, wherein the features include peaks in the image. 8. The method of claim 7, wherein the features include derivative peaks of the image. 9. The method of claim 5, wherein the second type carries a message payload. 10. The watermark signal enables a watermark decoder to compensate for scaling distortions by calculating the autocorrelation or cross-correlation of the watermark signal and deriving the scaling from the placement of the peaks in the resulting signal. The method of claim 1, having a correlation property of 11. The watermark comprises a PN sequence of symbol values of either 1 or -1, which symbols are mapped to samples of the media signal and which phase between adjacent groups of samples corresponding to different symbols. The method of claim 1, wherein the transition is slowly changed. 12. A method of steganographically embedding a watermark in a media signal, the method comprising forming a message comprising a PN sequence of either 1 or -1 symbols, the symbols being mapped to samples in the media signal. And smoothly changing the phase transition between adjacent groups of samples corresponding to different symbols. 13. The method of claim 12, wherein the media signal is an audio signal. 14. A method for decoding a feature-based watermark that conceals ancillary data within a media signal, the method comprising identifying N features in the media signal that are integers greater than one, each feature. Decoding the watermark around the N features by correlating the sample values of the samples in one group around the with the watermark signal layer, and N decoding the auxiliary data for one or more of the other features. Using one of the characteristics of the above as a reference. 15. The feature is a peak of the media signal.
The method described in. 16. The peak is a peak of a derivative of the media signal,
The method according to claim 15. 17. The method of claim 14, wherein a correlation peak generated by autocorrelation of the watermark signal is used to compensate for scaling distortion of the watermarked signal before decoding an auxiliary data message. . 18. The media signal is an image and the correlation peak is used to compensate for rotational distortion prior to decoding the auxiliary data message.
The method described in. 19. The method of claim 14, wherein a number of different watermark layers are decoded, including watermarks around the feature and elsewhere. 20. The watermark layer uses a PN sequence carrier signal detected using cross-correlation or auto-correlation on the watermarked signal.
The method described in. 21. A method of transmarking a media signal previously embedded with a first digital watermark using a method of embedding a first digital watermark, said first digital watermark in said media signal. Detecting the message information from the first digital watermark before the media signal undergoes the conversion process so that the second digital watermark is adapted to survive the conversion process. Embedding within the digital watermark of 2). 22. The amplitude of the second digital watermark is increased with respect to the first digital watermark in order to survive the conversion process.
The method described in. 23. The method of claim 21, wherein the second digital watermark is embedded using a steganographic embedding method that is different than the first digital watermark embedding method. 24. The method of claim 21, wherein the first digital watermark is at least partially removed prior to embedding the second digital watermark. 25. The message information includes a message symbol, and the step of decoding the message symbol from the first watermark, the message symbol from the first watermark being re-converted to the second watermark. 22. The method of claim 21, further comprising the step of embedding. 26. The method of claim 25, wherein the message symbol comprises an index to a database entry that stores information about the media signal. 27. The method of claim 25, wherein the message symbol comprises a content identifier. 28. The second digital watermark is embedded using a robustness parameter used to control the embedding so that the second digital watermark adapts to survive the conversion process, and The robustness parameter is intended to process the media signal after the second digital watermark has been embedded in the media signal so that the second digital watermark complies with the robustness constraint of the rendering, editing or transmission process. 22. The method of claim 21, characterized by being specified by the rendering or editing or transmission process that is present. 29. The robustness parameter sets the watermark signal strength and redundancy of the second digital watermark so that the second digital watermark can more reliably pass through the conversion process than the first digital watermark. 29. The method of claim 28, or specifying a frequency domain position. 30. The second digital watermark is embedded using a perceptual quality parameter used to control the embedding such that the second digital watermark has a perceptual quality adapted to the conversion process. , The perceptual quality parameter determines the media signal after the second digital watermark is embedded in the media signal so that the second digital watermark complies with perceptual quality constraints of a rendering process, an editing process, or a transmission process. 22. The method of claim 21, specified by the rendering or editing or transmission process that is about to be processed. 31. The second digital watermark is a feedback process of repeatedly embedding at least a portion of the second digital watermark, and the deterioration of the watermark measured after applying a deterioration process to the watermarked signal. 22. The method of claim 21, wherein the method is embedded using the feedback process that selectively adjusts the strength of the second digital watermark in the portion of the media signal according to, or according to a measure of perceptual quality. 32. A computer-readable recording medium having software for executing the method according to claim 21 stored therein. 33. A method of transmarking a media signal previously embedded with a first digital watermark using a method of embedding a first digital watermark, said first digital watermark detection in said media signal. And converting the media signal to a different format, a message from the first digital watermark so that the second digital watermark adapts to a robustness parameter or a perceptual parameter associated with the new format. Embedding information within a second digital watermark of the transformed media signal, the method of transmarking the media signal. 34. The method of claim 33, wherein the new format is a compressed format of the media signal. 35. The method of claim 33, wherein the second digital watermark is embedded with a greater signal strength than the first digital watermark to survive the transformation of the media signal in the new format. 36. The method of claim 33, wherein the second digital watermark is encoded with less signal strength than the first digital watermark so that it is less perceptible in the new format of the media signal. . 37. The method of claim 33, wherein at least a portion of the first digital watermark is removed before converting the media signal to the different format. 38. A computer-readable recording medium having software for performing the method of claim 33. 39. A transmarker for transmarking a media signal previously embedded with a first digital watermark using a method of embedding a first digital watermark, the transmarker comprising: Means for detecting the digital watermark of the first digital watermark and the second digital watermark adapts the message information from the first digital watermark to the media signal before the media signal undergoes the conversion process so that the second digital watermark is adapted to survive the conversion process. Means for embedding within a second digital watermark of the transmarker for transmarking the media signal. 40. A transmarker for transmarking a media signal previously embedded with a first digital watermark using a method of embedding a first digital watermark, said transmarker comprising: Means for detecting a digital watermark in the first format, means for converting the media signal into a different format, and a second digital watermark to adapt the robustness parameter or the perceptual parameter associated with the new format. Means for embedding message information from a digital watermark in a second digital watermark of the converted media signal, the transmarker for transmarking the media signal. 41. A method of controlling embedding a digital watermark in a media object, the method comprising: receiving a watermark embedding function that specifies a watermark embedding parameter including a watermark strength and a message payload; and rendering the watermark embedding function. Inserting into a description file, reading the watermark embedding function at render time, and steganographically embedding the watermark message payload at the watermark strength in the media object. 42. The method of claim 41, wherein the media object comprises graphic arts that includes a collection of two or more images in different formats. 43. The method of claim 41, wherein the media object comprises a music signal and the steganographic embedding process adapts to a robustness parameter or a perceptual quality parameter selected at rendering. 44. The method of claim 41, wherein the media object comprises a music signal and the steganographic embedding process adapts to a robustness parameter or a perceptual quality parameter selected at rendering. 45. The steganographic embedding process comprises: repeatedly embedding the message payload in two or more iterations; analyzing an error detection rate of the message payload for each iteration; Adjusting the robustness parameter of the embedding process for at least one of the iterations to an acceptable level.
The method described in. 46. The method comprises the steps of providing two or more different watermark embedding modules, each module adapted to a different rendering process or transmission channel, said watermark embedding module being the rendering process or the media object to be provided. 42. The method of claim 41, selected at render time depending on the transmission channel. 47. The method of claim 41, wherein the watermark embedding function specifies a watermark embedding location. 48. The method of claim 41, wherein the steganographic embedding process relies on a rendering process to select a watermark embedding location. 49. The media object includes an image, and the steganographic embedding process embeds the watermark message payload in the image after rasterizing the image into a format compatible with the printer that prints the image. 42. The method of claim 41. 50. The method of claim 41, wherein the watermark message payload is embedded in the image after the image has been converted to a halftone image by a halftoning process compatible with a printer that prints the image. 51. A media object processing system, comprising: a watermark embedding function including a watermark message payload, the parameter embedding the watermark message payload applied to the media object, and controlling the embedding of the watermark message payload in the media object. Input means for allowing a user to specify a rendering command for the media object, means for generating a rendering description file describing a rendering method of the media object, and the watermark message payload for the media object. A media object processing system including a watermark embedding module embedded in a steganographic manner. 52. The system of claim 51, wherein the system is operable to select different embedding modules for different rendering processes. 53. The embedding module is operable to repeatedly embed a watermark message payload in the media object in more than one iteration, and for each iteration adapt the watermark payload to adapt the watermark robustness. 52. The system of claim 51, which analyzes error detection rates. 54. The watermark embedding module performs degradation processing on the watermarked media signal output from one iteration before analyzing the error detection rate.
The system of claim 53. 55. The media object includes graphic arts that are at least partially specified by a rendering command that was not converted to a rasterized image when the watermark embedding feature was inserted into the rendering description file. 52. The system of claim 51. 56. The media object comprises structured audio that is at least partially represented by a rendering command that was not converted into an audio signal when the watermark embedding feature was inserted into the rendering description file. 52. The system of claim 51. 57. The media object includes a rendering command that specifies how to generate an unconverted video stream into a video signal when the watermark embedding feature is inserted into the rendering description file.
The system according to 1. 58. A computer-readable recording medium having a rendering description file, the one or more rendering commands describing a method of rendering a media object, the method comprising: in the media object after the media object has been rendered. A computer-readable recording medium having a rendering description file including a watermark embedding function that specifies a method of embedding a watermark message payload. 59. The rendering description file includes a rendering command that describes a method of generating a rasterized image, and the watermark embedding function specifies a method of embedding the watermark message in the rasterized image. The computer-readable recording medium according to claim 58. 60. The watermark embedding function of claim 59, comprising a watermark strength parameter and an embedding location that specifies where and at which strength of the rasterized image the watermark message should be embedded. Computer-readable recording medium.