JP2014036339A - Moving image processing device - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a moving image processing device capable of enhancing visibility of a desired subject appearing in a moving image.SOLUTION: A post-processing circuit 28 cut partial image data representing a face matching a registered face image out of main image data to be stored in a main file, and enlarges the cut partial image data to generate sub image data. The generated sub image data is stored in a sub file. An LCD driver 30 displays a main moving image based upon the main image data stored in the main file on an LCD monitor 32, and displays a sub moving image based upon the sub image data stored in the sub file on the LCD monitor 32 in an OSD mode. Further, a multiplexing position of the sub moving image is so adjusted to be different according to the cutting position of the partial image data. Consequently, visibility of a face appearing in a moving image can be enhanced.

Description

  The present invention relates to a moving image processing apparatus, and more particularly to a moving image display apparatus that displays a plurality of moving images representing a common scene in different modes.

  An example of this type of device is disclosed in Patent Document 1. According to this background art, when the tele / wide simultaneous shooting mode is set, the zoom lens of the right imaging system and the zoom lens of the left imaging system are set to different zoom positions. The wide side image is taken with the right imaging system, and the tele side image is taken with the left imaging system. The tele-side image is displayed over the entire monitor screen, and the wide-side image is reduced and displayed in a frame assigned to a part of the monitor screen. The user can recognize the shooting range of each of the wide-side image and the tele-side image, and can confirm a detailed portion of the subject.

JP 2011-45039 A

  However, in the background art, since the relative position between the shooting range of the wide-side image and the shooting range of the tele-side image is fixed, the visibility of the subject is limited.

  Therefore, a main object of the present invention is to provide a moving image processing apparatus that can improve the visibility of a desired subject appearing in a moving image.

  The moving image processing apparatus according to the present invention (10: reference numeral corresponding to the embodiment; the same applies hereinafter) includes display means (30a, S111 to S115, S127 to S131) for displaying a moving image, and a partial moving image representing a desired subject. From the moving image to be displayed by the display means (28a, S73, S85), creating means (28c ~ 28f, S75 ~) to enlarge the partial moving image clipped by the cutting means to create an enlarged partial moving image S77, S87), multiplexing means (30c to 30e, S103 to S107, S117 to S119, S123 to S125) for multiplexing the enlarged partial moving image created by the creating means on the moving image displayed by the display means, and clipping means Adjustment means (S121) for adjusting the multiplexing position of the multiplexing means so as to differ depending on the cutout position.

  Preferably, search means (34, S57) for repeatedly searching a partial image representing a desired subject from a frame image forming a moving image to be displayed by the display means is provided, and the clipping means refers to a search result of the search means And defining means (S73, S85) for defining the cut-out area.

  More preferably, the size of the cutout area differs depending on the size of the partial image detected by the search means, and the creation means includes an enlargement ratio adjusting means (S75, S87) for adjusting the enlargement ratio with reference to the size of the cutout area. Including.

  More preferably, the enlargement ratio adjusting means adjusts the enlargement ratio so that the resolution of the enlarged partial image shows a constant value regardless of the variation in the size of the cutout area.

  Preferably, the recording means (S3 to S5, S11 to S19, S33) for recording the moving image to be displayed by the display means prior to the processing of the display means, and the enlarged partial moving image created by the creating means by the recording means An assigning means (S7 to S9, S21, S25 to S29, S53, S65, S79 to S81, S93) for assigning to the recorded moving image is further provided, and the display means of the display means is the moving image recorded by the recording means. Correspondingly, the multiplexing target of the multiplexing means corresponds to the enlarged partial moving image assigned by the assigning means.

  The moving image processing program according to the present invention displays, on the processor (36) of the moving image processing device (10), a display step (S111 to S115, S127 to S131) for displaying a moving image, and a partial moving image representing a desired subject. A cutout step (S73, S85) that cuts out from the moving image to be displayed in steps, a creation step (S75 to S77, S87) that creates a magnified partial moving image by enlarging the partial moving image cut out in the cutout step, Multiplexing step (S103 to S107, S117 to S119, S123 to S125) for multiplexing the created enlarged partial moving image on the moving image displayed by the display step, and the multiple step to be different depending on the cutout position of the cutout step. This is a moving image processing program for executing the adjustment step (S121) for adjusting the multiple positions.

  The moving image processing method according to the present invention is a moving image processing method executed by the moving image processing apparatus (10), and displays the moving image display step (S111 to S115, S127 to S131), which represents a desired subject. A cutout step (S73, S85) for cutting out a partial moving image from a moving image to be displayed by the display step, and a creation step (S75 to S77, S87) for creating an enlarged partial moving image by enlarging the partial moving image cut out by the cutout step ), A multiplex step (S103 to S107, S117 to S119, S123 to S125) for multiplexing the enlarged partial moving image created in the creating step on the moving image displayed in the display step, and the cutout position of the cutout step Thus, an adjustment step (S121) for adjusting the multiplexing position of the multiplexing step is provided.

  An external control program according to the present invention is an external control program supplied to a moving image processing apparatus (10) including a processor (36) that executes processing according to an internal control program stored in a memory (44), A display step (S111 to S115, S127 to S131), a cutout step (S73, S85) to cut out a partial moving image representing a desired subject from a moving image to be displayed by the display step, and a partial video cut out by the cutout step A creation step (S75 to S77, S87) for enlarging an image to create an enlarged partial moving image, and a multiplexing step (S103 to S107) for multiplexing the enlarged partial moving image created in the creating step on the moving image displayed in the display step , S117 to S119, S123 to S125), and an adjustment step (S121) for adjusting the multiplex position of the multiplex step so as to differ depending on the slicing position of the slicing step. It is an external control program for causing a processor to execute in cooperation with a program.

  The moving image processing apparatus (10) according to the present invention executes a process according to an importing means (46) for fetching an external control program, and an external control program fetched by the fetching means and an internal control program stored in the memory (44). The external control program includes a display step for displaying a moving image (S111 to S115, S127 to S131), and a partial moving image representing a desired subject is displayed by the display step. Cutout steps (S73, S85) cut out from the moving image to be created, creation steps (S75 to S77, S87) that create the enlarged partial moving image by enlarging the partial moving image cut out in the cutout step, created by the creation step Multiplexing step (S103 to S107, S117 to S119, S123 to S125) for multiplexing the enlarged partial moving image on the moving image displayed by the display step, and the clipping step Adjustment step of adjusting the multi-position of the multi-step differently depending on the cut position (S121) corresponding to the internal control program in cooperation with a program to be executed.

  According to the present invention, the enlarged partial moving image is created by cutting out a partial moving image representing a desired subject from the moving image and enlarging the extracted partial moving image. Further, the created enlarged partial moving image is multiplexed with the moving image corresponding to different positions depending on the cutout position. Thereby, the visibility of a desired subject appearing in the moving image can be enhanced.

  The above object, other objects, features and advantages of the present invention will become more apparent from the following detailed description of embodiments with reference to the drawings.

It is a block diagram which shows the basic composition of one Example of this invention. It is a block diagram which shows the structure of one Example of this invention. FIG. 3 is an illustrative view showing one example of a mapping state of an SDRAM applied to the embodiment in FIG. 2; It is an illustration figure which shows an example of the scene caught under camera mode. It is an illustration figure which shows an example of the sub image information table applied to the FIG. 2 Example. (A) is an illustrative view showing an example of a structure of a main file created under the camera mode, and (B) is an illustrative view showing an example of a structure of a subfile created under the camera mode. It is an illustration figure which shows an example of the moving image reproduced | regenerated under reproduction | regeneration mode. FIG. 3 is a block diagram illustrating an example of a configuration of a post-processing circuit applied to the embodiment in FIG. 2. It is a block diagram which shows an example of a structure of the LCD driver applied to the FIG. 2 Example. 3 is a block diagram illustrating an example of a configuration of a face detection circuit applied to the embodiment in FIG. 2; FIG. It is a flowchart which shows a part of operation | movement of CPU applied to the FIG. 2 Example. It is a flowchart which shows a part of other operation | movement of CPU applied to the FIG. 2 Example. FIG. 11 is a flowchart showing still another portion of behavior of the CPU applied to the embodiment in FIG. 2; FIG. 10 is a flowchart showing yet another portion of behavior of the CPU applied to the embodiment in FIG. 2; It is a flowchart which shows a part of other operation | movement of CPU applied to the FIG. 2 Example. FIG. 11 is a flowchart showing still another portion of behavior of the CPU applied to the embodiment in FIG. 2; FIG. 10 is a flowchart showing yet another portion of behavior of the CPU applied to the embodiment in FIG. 2; It is a flowchart which shows a part of other operation | movement of CPU applied to the FIG. 2 Example. It is a block diagram which shows the structure of the other Example of this invention.

Embodiments of the present invention will be described below with reference to the drawings.
[Basic configuration]

  Referring to FIG. 1, the moving image processing apparatus of this embodiment is basically configured as follows. The display unit 1 displays a moving image. The clipping unit 2 clips a partial moving image representing a desired subject from the moving image to be displayed by the display unit 1. The creating unit 3 creates an enlarged partial moving image by enlarging the partial moving image cut out by the clipping unit 2. The multiplexing unit 4 multiplexes the enlarged partial moving image created by the creating unit 3 with the moving image displayed by the display unit 1. The adjusting unit 5 adjusts the multiplexing position of the multiplexing unit 4 so as to differ depending on the cutting position of the cutting unit 2.

The enlarged partial moving image is created by cutting out a partial moving image representing a desired subject from the moving image and enlarging the cut out partial moving image. Further, the created enlarged partial moving image is multiplexed with the moving image corresponding to different positions depending on the cutout position. Thereby, the visibility of a desired subject appearing in the moving image can be enhanced.
[Example]

  Referring to FIG. 2, the digital video camera 10 of this embodiment includes a focus lens 12 and an aperture unit 14 driven by drivers 18a and 18b, respectively. An optical image representing a scene is irradiated onto the imaging surface of the image sensor 16 through these members.

  When the camera mode is selected by the mode selection operation directed to the key input device 38, the CPU 36 activates the driver 18c to execute the moving image capturing process under the main image recording task. The driver 18c exposes the imaging surface in response to a periodically generated vertical synchronization signal Vsync, and reads out the charges generated on the imaging surface in a raster scanning manner. The image sensor 16 periodically outputs raw image data that represents a scene captured on the imaging surface and in which R, G, or B color information is assigned to each pixel.

  The preprocessing circuit 20 performs processing such as digital clamping, pixel defect correction, and gain control on the raw image data from the image sensor 16. The raw image data subjected to such preprocessing is written into the raw image area 24a (see FIG. 3) of the SDRAM 24 through the memory control circuit 22.

  The post-processing circuit 26 reads the raw image data stored in the raw image area 24a through the memory control circuit 22, and performs a series of processes of color separation, white balance adjustment, and YUV conversion on the read raw image data. As a result, the generated image data in the YUV format, that is, main image data is written into the YUV image area 24b (see FIG. 3) of the SDRAM 24 through the memory control circuit 22.

  The LCD driver 30 repeatedly reads the main image data stored in the YUV image area 24b, and drives the LCD monitor 32 based on the read main image data. As a result, a real-time moving image (through image) representing the scene captured on the imaging surface is displayed on the monitor screen.

  The preprocessing circuit 20 also simply converts the raw image data into Y data, and provides the converted Y data to the CPU 36. The CPU 36 performs AE processing on the Y data under the parameter adjustment task, and calculates an appropriate EV value. The aperture amount and the exposure time that define the calculated appropriate EV value are set in the drivers 18b and 18c, respectively, and thereby the brightness of the through image is appropriately adjusted. The CPU 36 also performs AF processing on the high frequency component of the Y data when the AF activation condition is satisfied. The focus lens 12 is arranged at the focal point by the driver 18a, and thereby the sharpness of the through image is continuously improved.

  When a recording start operation is performed toward the key input device 38, the CPU 36 accesses the recording medium 42 through the I / F 40 under the main image recording task, and newly creates a main file and a sub file in the recording medium 42. (The created main file and subfile are opened).

  The main file contains MOV ***. A file name of MAIN (*** is an identification number, the same applies hereinafter) is assigned, and MOV ***. A SUB file name is assigned. Here, a common identification number is assigned to the main file and the sub file created at the same time, and the main file and the sub file created at the same time are associated with each other by the identification number.

  When the main file and the sub file are created, the CPU 36 instructs the memory I / F 40 to start the main image recording process. The CPU 36 also increments the main frame number from “1” every time the vertical synchronization signal Vsync is generated. The main image data of each frame to be recorded is identified by the main frame number incremented in this way. The memory I / F 40 reads the main image data stored in the YUV image area 24b through the memory control circuit 22, and writes the read main image data in the main file created as described above on the recording medium 42.

  Further, the CPU 36 issues a face search request to the face detection circuit 34 every time the vertical synchronization signal Vsync is generated in order to execute the face search process under the sub control task. The face detection circuit 34 moves the collation frame from the beginning position to the end position of the YUV image area 24b in a raster scanning manner, and collates a part of the images belonging to the collation frame with the registered face image stored in the dictionary 34e. . The size of the collation frame is changed every time the collation frame reaches the end position, and the collation frame having the changed size is moved again from the top position in a raster scanning manner. The position and size of the detected face image are registered in the register 34e as face information.

  When the predetermined number of raster scans are completed, the face detection circuit 34 returns a search end notification describing the search result to the CPU 36. The search result described in the search end notification indicates “detection” when image data matching the registered face image is found, and indicates “non-detection” when image data matching the registered face image is not found. .

  When a search end notification indicating that the search result indicates “detect” is returned, the CPU 36 regards that the face matching the registered face image has been successfully searched, and executes a reset & start of the timer 36 tm and a sub image recording task. Start up.

  The timer value is set to 2 seconds, for example, and the sub-image recording task is terminated in response to the timer 36tm timeout. The timer 36tm is reset and started every time a face matching the registered face image is detected. Further, the sub-image recording task is not started again during the period from the reset & start of the timer 36tm to the timeout. Therefore, as long as the face matching the registered face image continues to be detected, the common sub-image recording task continues to be activated, and when the face matching the registered face image is not detected, the sub-image recording task is terminated after 2 seconds. .

  Under the sub-image recording task, the CPU 36 refers to the latest face information registered in the register 34e and sets the cut-out area CT1 as the raw image area 24a. The size and position of the cut-out area CT1 are adjusted based on the size and position defining the latest face information so that a face that matches the registered face image can be captured. Therefore, when the scene shown in FIG. 4 is captured in a state where the face image of the boy BY1 shown in FIG. 4 is registered in the dictionary 34e, the cut-out area CT1 is set as shown in FIG.

  Subsequently, the CPU 36 sets a different zoom magnification in the post-processing circuit 28 according to the size of the cut-out area CT1. The zoom magnification is adjusted in a range exceeding “1.0” so that the resolution of the image data after zoom processing, that is, the sub-image data shows a constant value regardless of the variation of the size of the cut-out area CT1. When the zoom magnification adjustment is completed, the CPU 36 activates the post-processing circuit 28, instructs the memory I / F 40 to start the sub-image recording process, and sets the subframe number to “1” every time the vertical synchronization signal Vsync is generated. Increment from ".

  The post-processing circuit 28 reads a part of raw image data belonging to the cut-out area CT1 from the raw image area 24a through the memory control circuit 22, converts the read raw image data into YUV format image data, and is converted. The image data is enlarged according to the zoom magnification set as described above. The enlarged image data, that is, the sub-image data created thereby is written into the YUV image area 24 c of the SDRAM 24 through the memory control circuit 22.

  The memory I / F 40 reads the sub image data stored in the YUV image area 24c through the memory control circuit 22, and writes the read sub image data to the sub file created under the main image recording task.

  In the work area 24d of the SDRAM 24, a sub image information table TBL shown in FIG. 5 is prepared. The CPU 36 accesses the work area 24d through the memory control circuit 22, describes the current main frame number and the current subframe number in the sub-image information table TBL as multiplexed frame numbers, and sets the current position in the cut-out area CT1 as a multiplexed position. Described in the image information table TBL. As a result, the cut position and frame number of the sub image data are associated with the frame number of the cut main image data.

  As long as the timer 36tm does not time out, the CPU 36 updates the setting of the cutout area CT1 and the zoom magnification set in the post-processing circuit 28 every time the vertical synchronization signal Vsync is generated. As described above, the cut-out area CT1 is adjusted based on the size and position that define the latest face information described in the register 34e. Further, the zoom magnification is adjusted in a range exceeding “1.0” so that the resolution of the sub-image data shows a constant value regardless of the variation of the size of the cut-out area CT1.

  When a timeout occurs in the timer 36tm, the CPU 36 stops the post-processing circuit 28 and commands the memory I / F 40 to end the sub image recording process. The post-processing circuit 28 stops reading the raw image data from the cutout area CT1, and the memory I / F 40 ends the recording of the sub image data. The sub-image recording task is then finished.

  When a recording end operation is performed toward the key input device 38, the CPU 36 commands the memory I / F 40 to end the main image recording process under the main image recording task. As a result, reading of the main image data from the YUV image area 24b is completed. Thereafter, the CPU 36 closes the sub file in the open state, writes the sub image information table TBL created under the sub image recording task in the header of the main file, and closes the main file. As a result, the main file and the sub file related to each other are completed as shown in FIGS. 6 (A) and 6 (B). When the scene shown in FIG. 4 is captured, the sub file contains sub image data representing a subject belonging to the cutout area CT1.

  If there is no image data in the subfile, writing of the subimage information table TBL is omitted, and the subfile is deleted.

  When the playback mode is selected by the mode selection operation directed to the key input device 38, the CPU 36 executes the following processing under the image playback task. First, one of a plurality of main files recorded on the recording medium 42 is designated according to a file designation operation directed to the key input device 38.

  When the sub file corresponding to the designated main file exists in the recording medium 42, the CPU 36 acquires the sub image information table TBL described in the header of the main file through the memory I / F 40. The acquired sub image information table TBL is written into the work area 24 d of the SDRAM 24 through the memory control circuit 22.

  When the reproduction start operation is performed toward the key input device 38, the CPU 36 instructs the memory I / F 40 and the LCD driver 30 to start the main image reproduction process, and the main frame number is generated every time the vertical synchronization signal Vsync is generated. Is incremented from “1”.

  The memory I / F 40 reads one frame of main image data corresponding to the current main frame number from the main file specified in the above manner, and reads the read main image data through the memory control circuit 22 in the YUV image area of the SDRAM 24. Write to 24b. The LCD driver 30 reads the main image data stored in the YUV image area 24b through the memory control circuit 22, and drives the LCD monitor 32 based on the read main image data. As a result, the main image of the current frame is displayed on the monitor screen.

  If the current main frame number is described in the sub image information table TBL, the CPU 36 detects the sub frame number assigned to the current main frame number from the sub image information table TBL, and corresponds to the detected sub frame number. One frame of sub image data is read from the sub file. The read sub image data is written into the YUV image area 24 c of the SDRAM 24 through the memory control circuit 22.

  The CPU 36 further detects the multiple position assigned to the current main frame number from the sub-image information table TBL and gives a multiple display command to the LCD driver 30. In the multiple display command, a multiple position detected from the sub-image information table is described.

  The LCD driver 30 reads the sub image data stored in the YUV image area 24c in parallel through the memory control circuit 22, and reads the read sub image data at a timing corresponding to the multiple position described in the multiple display command. Multiplex on image data. As a result, the sub image is multiplexed with the main image on the monitor screen.

  Therefore, when the main file capturing the scene shown in FIG. 4 is reproduced, the sub image data representing the subject belonging to the cutout area CT1 is read from the sub file. The sub image based on the read sub image data is multiplexed with the main image in the manner shown in FIG.

  When the playback end operation is performed or the logical sum condition that the current main frame number reaches the end frame of the main file is satisfied, the CPU 36 notifies the memory I / F 40 and the LCD driver 30 of the end of the main file playback process. Command. As a result, the reproduction of the main image data is finished.

  The post-processing circuit 28 is configured as shown in FIG. The controller 28a reads a part of the raw image data belonging to the cutout area CT1 from the raw image area 24a through the memory control circuit 22. The read raw image data is given to the color separation circuit 28c via the SRAM 28b. The color separation circuit 28c performs color separation on the given raw image data to create interpolated image data in which each pixel has all the R, G, and B color information.

  The white balance adjustment circuit 28d adjusts the white balance of the interpolated image data created by the color separation circuit 28c, and the YUV conversion circuit 28e converts the interpolated image data having the adjusted color balance into image data in the YUV format. The zoom circuit 28f enlarges the image data in accordance with the zoom magnification set by the CPU 36, and writes the enlarged image data, that is, the sub image data in the SRAM 28h. The controller 28g writes the sub image data stored in the SRAM 28h to the YUV image area 24c through the memory control circuit 22.

  The LCD driver 34 is configured as shown in FIG. The controller 30a reads the main image data stored in the YUV image area 24b through the memory control circuit 22. The read main image data is given to the multiplexing circuit 30e via the SRAM 30b.

  The controller 30 c reads out the sub image data stored in the YUV image area 24 c through the memory control circuit 22 when a multiple display command is given from the CPU 36. The read sub-image data is given to the multiplexing circuit 30e via the SRAM 30d.

  The multiplexing circuit 30e multiplexes the sub image data provided from the SRAM 30d with the main image data at a timing corresponding to the multiplexing position described in the multiple display command. From the multiplexing circuit 30e, main image data or multiplexed image data created by multiplexing processing is output. The image output circuit 30f drives the LCD monitor 32 based on the main image data or the multiplexed image data output from the multiplexing circuit 30e.

  The face detection circuit 34 is configured as shown in FIG. The controller 34 a assigns a rectangular collation frame to the YUV image area 24 b and reads out part of the image data belonging to the collation frame through the memory control circuit 22. The read image data is given to the verification circuit 34c via the SRAM 34b.

  The registered face image is stored in the dictionary 34d. The collation circuit 34c collates the image data given from the SRAM 34b with the registered face image stored in the dictionary 34d. When image data matching the registered face image is found, the matching circuit 34c registers face information in which the current position and size of the matching frame are described in the register 34e.

  The collation frame moves by a predetermined amount in a raster scanning manner from the head position (upper left position) to the end position (lower right position) of the YUV image area 24b. The size of the verification frame is updated every time the verification frame reaches the end position. When the predetermined number of raster scans are completed, a search end notification describing the search result is returned from the matching circuit 34c to the CPU 36. The search result described in the search end notification indicates “detection” when image data matching the registered face image is found, and indicates “non-detection” when image data matching the registered face image is not found. .

  The CPU 36 performs the main image recording task shown in FIGS. 11 to 12, the parameter adjustment task shown in FIG. 13, the sub control task shown in FIG. 14, the sub image recording task shown in FIGS. 15 to 16, and FIGS. A plurality of tasks including the image reproduction task shown are processed in parallel under the multitask OS. Note that control programs corresponding to these tasks are stored in the flash memory 44.

  Referring to FIG. 11, in step S1, a moving image capturing process is executed. As a result, a through image is displayed on the LCD monitor 32. In step S3, it is repeatedly determined whether or not a recording start operation has been performed. If the determination result is updated from NO to YES, the process proceeds to steps S5 to S7. In steps S5 to S7, the recording medium 42 is accessed through the memory I / F 40, and an open main file and subfile are newly created in the recording medium 42.

  In step S9, the sub-control task is activated, in step S11, the main frame number is set to “1”, and in step S11, the memory I / F 40 is instructed to start the main image recording process. The memory I / F 40 reads the main image data stored in the YUV image area 24b through the memory control circuit 22, and writes the read main image data in the main file created in step S5.

  In step S15, it is determined whether or not a recording end operation has been performed. If the determination result is NO, the process proceeds to step S17, and the main frame number is incremented after waiting for the generation of the vertical synchronization signal Vsync. When the increment is completed, the process returns to step S15. If the determination result of step S15 is updated to YES, it will progress to step S19 and will command the memory I / F 40 to end the main image recording process. The memory I / F 40 finishes reading the main image data from the YUV image area 24b.

  In step S21, the sub control task is terminated, and in step S25, it is determined whether or not the sub image data is stored in the sub file created in step S7. If a determination result is YES, it will progress to step S27 and will close the subfile in an open state. In step S29, the sub image information table TBL created under the sub image recording task is written in the header of the main file created in step S5. If the determination result of step S25 is NO, it will progress to step S31 and will delete the subfile in an open state.

  In step S33, the recording medium 42 is accessed through the memory I / F 40, and the main file in the open state is closed. When the closing of the main file is completed, the process returns to step S3.

  Referring to FIG. 13, in step S41, the focus, aperture amount, and exposure time are initialized. In step S43, it is determined whether or not the vertical synchronization signal Vsync has been generated. If the determination result is updated from NO to YES, an AE process is executed in step S45. As a result, the brightness of the through image is appropriately adjusted. In step S47, it is determined whether or not the AF activation condition is satisfied. If NO, the process directly returns to step S43. If YES, the AF process is performed in step S49, and then the process returns to step S43. As a result of the AF process, the focus lens 12 is disposed at the focal point, thereby improving the sharpness of the through image.

  Referring to FIG. 14, in step S51, flag FLGsub is set to “0”, in step S53, the subframe number is set to “1”, and in step S55, it is determined whether or not vertical synchronization signal Vsync is generated. . When the determination result is updated from NO to YES, face search processing is executed in step S57. Specifically, a face search request is issued to the face detection circuit 34.

  The face detection circuit 34 moves the collation frame from the start position to the end position of the YUV image area 24b in a raster scanning manner, and collates part of the images belonging to the collation frame with the registered face image. The size of the collation frame is changed every time the collation frame reaches the end position, and the collation frame having the changed size is moved again from the top position in a raster scanning manner. The position and size of the registered face image detected thereby are registered in the register 34e as face information.

  When the predetermined number of raster scans are completed, the face detection circuit 34 returns a search end notification describing the search result to the CPU 36. The search result described in the search end notification indicates “detection” when image data matching the registered face image is found, and indicates “non-detection” when image data matching the registered face image is not found. .

  When the search end notification is returned from the face detection circuit 34, it is determined in step S59 whether or not the search for the face that matches the registered face image is successful. If the search result described in the search end notification indicates “non-detection”, it is determined that the search for the matching face has failed and the process returns to step S55. On the other hand, if the search result described in the search end notification indicates “detection”, it is determined that the matching face has been successfully searched, and the timer 36tm is reset and started in step S61.

  In step S63, it is determined whether or not the flag FLGsub indicates “0”. If the determination result is NO, the process returns to step S55 as it is. If the determination result is YES, the sub-image recording task is started in step S65. To return to step S55.

  Referring to FIG. 15, in step S71, flag FLGsub is set to “1”. In step S73, the cut-out area CT1 is set as the raw image area 24a with reference to the latest face information registered in the register 34e. The size and position of the cut-out area CT1 are adjusted based on the size and position defining the latest face information so that a face that matches the registered face image can be captured.

  In step S75, a different zoom magnification is set in the post-processing circuit 28 depending on the size of the cut-out area CT1. The zoom magnification is adjusted in a range exceeding “1.0” so that the resolution of the sub-image data shows a constant value regardless of the variation of the size of the cut-out area CT1. When the zoom magnification adjustment is completed, the post-processing circuit 28 is activated in step S77, and the memory I / F 40 is commanded to start the sub image recording process in step S79.

  The post-processing circuit 28 reads a part of raw image data belonging to the cut-out area CT1 from the raw image area 24a through the memory control circuit 22, converts the read raw image data into YUV format image data, and is converted. The image data is enlarged according to the zoom magnification set as described above. The enlarged image data, that is, the sub-image data created thereby is written into the YUV image area 24 c of the SDRAM 24 through the memory control circuit 22. The memory I / F 40 reads the sub image data stored in the YUV image area 24c through the memory control circuit 22, and writes the read sub image data in the sub file created in step S7.

  In step S81, the sub image information table TBL formed in the work area 24d of the SDRAM 24 is accessed through the memory control circuit 22, and the current main frame number and the current sub frame number are described in the sub image information table TBL as multiple frame numbers. The current position of the cutout area CT1 is described in the sub image information table TBL as a multiple position.

  In step S83, it is determined whether or not a timeout has occurred in the timer 36tm. If the determination result is NO, the process proceeds to step S85, and the setting of the cut-out area CT1 is updated after waiting for the generation of the vertical synchronization signal Vsync. As described above, the size and position of the cutout area CT1 are adjusted based on the size and position of the latest face information described in the register 34e. In step S87, the zoom magnification set in the post-processing circuit 28 is updated. The zoom magnification is also adjusted in a range exceeding “1.0” so that the resolution of the sub-image data shows a constant value regardless of the variation of the size of the cut-out area CT1. When the zoom magnification update is completed, the subframe number is incremented in step S89, and then the process returns to step S81.

  When the determination result in step S83 is updated from NO to YES, the post-processing circuit 28 is stopped in step S91, and the memory I / F 40 is commanded to end the sub image recording process in step S93. The post-processing circuit 28 stops reading the raw image data from the cutout area CT1, and the memory I / F 40 ends the recording of the sub image data. When the process of step S93 is completed, the flag FLGsub is returned to “0” in step S95, and then the sub-image recording task is ended.

  Referring to FIG. 17, in step S101, one of a plurality of main files recorded on recording medium 42 is designated in response to a file designation operation. In step S103, it is determined whether or not a sub file corresponding to the designated main file exists in the recording medium. If the determination result is NO, the flag FLGadd is set to “0” in step S109, and the process proceeds to step S111. On the other hand, if the determination result is YES, the process proceeds to step S105, and the sub image information table TBL described in the header of the main file is acquired through the memory I / F 40. The acquired sub image information table TBL is written into the work area 24 d of the SDRAM 24 through the memory control circuit 22. In step S107, the flag FLGadd is set to “1”. When the setting is completed, the process proceeds to step S111.

  In step S111, it is determined whether or not a reproduction start operation has been performed. If the determination result is updated from NO to YES, the main frame number is set to “1” in step S113. In step S115, the memory I / F 40 and the LCD driver 30 are instructed to start main image reproduction processing.

  The memory I / F 40 reads out one frame of main image data corresponding to the current main frame number from the main file specified in step S101, and reads the read main image data through the memory control circuit 22 in the YUV image area 24b of the SDRAM 24. Write to. The LCD driver 30 reads the main image data stored in the YUV image area 24b through the memory control circuit 22, and drives the LCD monitor 32 based on the read main image data. As a result, the main image of the current frame is displayed on the monitor screen.

  In step S117, it is determined whether or not the flag FLGadd indicates “1”. In step S119, it is determined whether or not the current main frame number is described in the sub-image information table TBL. If at least one of the determination result of step S117 and the determination result of step S119 is NO, the process directly proceeds to step S127. On the other hand, if both the determination result in step S117 and the determination result in step S119 are YES, the process proceeds to step S127 through steps S121 to S125.

  In step S121, the multiplex position assigned to the current main frame number is detected from the sub-image information table TBL. In step S123, the subframe number assigned to the current main frame number is detected from the subimage information table TBL, and one frame of subimage data corresponding to the detected subframe number is read from the subfile. The read sub image data is written into the YUV image area 24 c of the SDRAM 24 through the memory control circuit 22.

  In step S125, a multiple display command is given to the LCD driver 30 in order to display the sub image in a multiple display. The multiple display command describes the multiple position detected in step S121. The LCD driver 30 reads the sub image data stored in the YUV image area 24c in parallel through the processing of step S123 through the memory control circuit 22, and reads the read sub image data at the multiple positions described in the multiple display command. It is multiplexed with the main image data at a corresponding timing. As a result, the sub image is multiplexed with the main image on the monitor screen.

  In step S127, it is determined whether or not a reproduction end operation is performed or a logical sum condition that the current main frame number reaches the end frame of the main file is satisfied. If the determination result is NO, the process proceeds to step S129, and the main frame number is incremented after waiting for the generation of the vertical synchronization signal Vsync. When the increment is completed, the process returns to step S117. On the other hand, if the determination result in step S127 is YES, the memory I / F 40 and the LCD driver 30 are instructed to end the main file reproduction process. As a result, the reproduction of the main image data is finished. When the process of step S131 is completed, the process returns to step S101.

  As can be seen from the above description, the post-processing circuit 28 cuts out the partial image data representing the face matching the registered face image from the sub-image data to be stored in the main file (S73, S85), and the cut-out image data is extracted. Sub-image data is created by enlarging (S75 to S77, S87). The created sub image data is stored in a sub file. The LCD driver 30 displays the main moving image based on the main image data stored in the main file on the LCD monitor 32 (S111 to S115, S127 to S131), and the sub moving image based on the sub image data stored in the sub file. Are displayed on the LCD monitor 32 in the OSD mode (S103 to S107, S117 to S119, S123 to S125). In addition, the multiplexing position of the sub moving image is adjusted to be different depending on the cutout position of the partial image data (S121).

  In this way, the sub moving image is created by cutting out a partial moving image representing a person's face from the moving image and enlarging the cut out moving image. Further, the created sub moving image is multiplexed with the main moving image corresponding to different positions depending on the cutout position. Thereby, the visibility of the face appearing in the moving image can be enhanced.

  In this embodiment, it is assumed that a single registered face image is stored in the dictionary 34e. However, a plurality of registered face images are stored in the dictionary 34e and selected from these registered face images by the user. A plurality of faces matching the registered face image may be searched. When detecting a plurality of faces in parallel, preferably, a plurality of subfiles corresponding to the plurality of faces are created in advance, and the sub image data representing the detected faces are stored in the corresponding subfile. At the time of reproduction, the sub image of the sub file selected by the user operation is displayed on the LCD monitor 32.

  Further, in this embodiment, the size and position of the cutout area CT1 are adjusted based on the size and position that define the latest face information, and the resolution of the sub image data has a constant value regardless of the change in the size of the cutout area CT1. As shown, the zoom magnification is adjusted. For this reason, the size of the sub-image displayed on the main image at the time of reproduction is fixed regardless of the variation in the size of the face image that appeared on the main image. However, the size of the sub-image to be displayed may be changed in accordance with the change in the size of the cut-out area CT1.

  In this embodiment, the main file and the sub file are created simultaneously in response to the recording start operation. However, only the main file is created in response to the recording start operation, and the main file is created after the recording end operation. A sub file may be created based on the above.

  Furthermore, in this embodiment, a single subfile is assigned to a single main file. However, a subfile is created for each face that is continuously detected, and one or more created in this way. A subfile may be assigned to a single main file.

  In this embodiment, it is assumed that a person's face is searched, but a registered object other than the face may be searched. In this case, the sub image data represents a registered object other than the face.

  In this embodiment, the multitask OS and control programs corresponding to a plurality of tasks executed thereby are stored in the flash memory 44 in advance. However, as shown in FIG. 19, a communication I / F 46 is provided in the digital camera 10 and a part of the control program is prepared as an internal control program in the flash memory 44 from the beginning. May be acquired from an external server. In this case, the above-described operation is realized by cooperation of the internal control program and the external control program.

  Furthermore, in this embodiment, the processing executed by the CPU 36 is divided into a plurality of tasks as described above. However, each task may be further divided into a plurality of small tasks, and a part of the divided plurality of small tasks may be integrated with other tasks. Further, when each task is divided into a plurality of small tasks, all or part of the tasks may be acquired from an external server.

DESCRIPTION OF SYMBOLS 10 ... Digital video camera 16 ... Image sensor 26, 28 ... Post-processing circuit 30 ... LCD driver 34 ... Face detection circuit 36 ... CPU

Claims (9)

Display means for displaying moving images;
Clipping means for cutting out a partial moving image representing a desired subject from the moving image to be displayed by the display means;
Creating means for enlarging the partial moving image cut out by the cut-out means to create an enlarged partial moving image;
Multiplexing means for multiplexing the enlarged partial moving image created by the creating means on the moving image displayed by the display means, and adjustment for adjusting the multiplexing position of the multiplexing means so as to differ depending on the clipping position of the clipping means A moving image processing apparatus comprising means. Further comprising search means for repeatedly searching for a partial image representing the desired subject from a frame image forming a moving image to be displayed by the display means;
The moving image processing apparatus according to claim 1, wherein the cutout unit includes a definition unit that defines a cutout area with reference to a search result of the search unit. The size of the cut-out area varies depending on the size of the partial image detected by the search means,
The moving image processing apparatus according to claim 2, wherein the creating unit includes an enlargement rate adjusting unit that adjusts an enlargement rate with reference to a size of the cutout area.   The moving image processing apparatus according to claim 3, wherein the enlargement ratio adjusting unit adjusts the enlargement ratio so that a resolution of the enlarged partial image shows a constant value regardless of a change in a size of the cutout area. Recording means for recording a moving image to be displayed by the display means prior to processing of the display means, and assigning means for allocating the enlarged partial moving image created by the creating means to the moving image recorded by the recording means In addition,
The display object of the display means corresponds to the moving image recorded by the recording means,
5. The moving image processing apparatus according to claim 1, wherein a multiplexing target of the multiplexing unit corresponds to an enlarged partial moving image allocated by the allocation unit. In the processor of the video processing device,
A display step for displaying moving images,
A step of cutting out a partial moving image representing a desired subject from the moving image to be displayed by the display step;
A creation step of enlarging the partial moving image cut out by the cutout step to create an enlarged partial moving image;
Multiplexing step for multiplexing the enlarged partial moving image created by the creating step on the moving image displayed by the displaying step, and adjustment for adjusting the multiplexing position of the multiplexing step so as to differ depending on the clipping position of the clipping step A moving image processing program for executing steps. A moving image processing method executed by a moving image processing apparatus,
A display step for displaying moving images,
A step of cutting out a partial moving image representing a desired subject from the moving image to be displayed by the display step;
A creation step of enlarging the partial moving image cut out by the cutout step to create an enlarged partial moving image;
Multiplexing step for multiplexing the enlarged partial moving image created by the creating step on the moving image displayed by the displaying step, and adjustment for adjusting the multiplexing position of the multiplexing step so as to differ depending on the clipping position of the clipping step A moving image processing method comprising steps. An external control program supplied to a moving image processing apparatus including a processor that executes processing according to an internal control program stored in a memory,
A display step for displaying moving images,
A step of cutting out a partial moving image representing a desired subject from the moving image to be displayed by the display step;
A creation step of enlarging the partial moving image cut out by the cutout step to create an enlarged partial moving image;
Multiplexing step for multiplexing the enlarged partial moving image created by the creating step on the moving image displayed by the displaying step, and adjustment for adjusting the multiplexing position of the multiplexing step so as to differ depending on the clipping position of the clipping step An external control program for causing the processor to execute steps in cooperation with the internal control program. A moving image processing apparatus comprising: a fetching unit that fetches an external control program; and a processor that executes processing according to the external control program fetched by the fetching unit and an internal control program stored in a memory,
The external control program is
A display step for displaying moving images,
A step of cutting out a partial moving image representing a desired subject from the moving image to be displayed by the display step;
A creation step of enlarging the partial moving image cut out by the cutout step to create an enlarged partial moving image;
Multiplexing step for multiplexing the enlarged partial moving image created by the creating step on the moving image displayed by the displaying step, and adjustment for adjusting the multiplexing position of the multiplexing step so as to differ depending on the clipping position of the clipping step A moving image processing apparatus corresponding to a program for executing steps in cooperation with the internal control program.

Images