JP2012103786A - Test support device, control method, and program - Google Patents

Abstract

PROBLEM TO BE SOLVED: To reduce development costs for a test and an evaluation process by creating a script for test of high test precision and lightening the burden of work needed for the creation of the script for test.SOLUTION: After a generation part 308 for screen image before GUI operation and a generation part 310 for screen image after GUI operation generate an image file (screen image 314 before GUI operation) from a predetermined time before detection of user interface operation (hereinafter operation) until the time when the operation is detected and an image file (screen image 316 after GUI operation) a predetermined time after the time when the operation is detected, a detection image analysis part 320 specifies an operated peripheral image file by comparing values of pixels of the screen image 314 before GUI operation and the screen image 316 after GUI operation so as to generate position coordinates where the peripheral image file corresponds to a window of an application 200 to be tested and operation information associated with the operation as a script 318 for test.

Description

  The present invention relates to a test automation technique for a software application or a software system, and more particularly to a test automation technique related to a user interface operation on a screen.

  In the software development process, the test and evaluation process occupies a large weight in terms of cost such as time and cost. Further, as the software development becomes large, the man-hours and personnel required to be input to the test and evaluation processes increase proportionally, leading to an increase in development cost and postponement of delivery time.

  If the developed software is not fully tested and evaluated, and the software is shipped as a product, and a failure occurs during operation in the system where the software is installed, the software response or modification accompanying it will further As a result, the number of man-hours and personnel increases, resulting in an increase in development costs. If the system is directly connected to the infrastructure of daily life, the impact will be immeasurable.

  Therefore, test automation software is used to suppress the increase in development cost by improving the efficiency of the software testing and evaluation process.

  The existing test automation software mechanism saves user interface operations for screens in the application under test in a script format program and executes the created and saved scripts to reproduce the saved user interface operations for testing and evaluation. Can be made more efficient, and testing and evaluation can be resourceized.

  In addition, a technology has been disclosed that realizes further efficiency by componentizing a script generated in performing an automatic test (see, for example, Patent Document 1).

Japanese Patent Laid-Open No. 2002-14845

  In order to perform an automatic test, it is necessary to create a test script in advance. To create the script, the position where the user interface operation is performed is determined from the coordinate value on the screen, and the coordinate value is set. Created based on the original.

  To determine the coordinates indicating the position where the user interface operation has been performed, the “determine the coordinates of the position where the user interface operation has been performed” method and the “peripheral image of the position where the user interface operation has been performed are displayed from the screen. There are two methods: “Search and derive coordinates”.

  In the method of “determining the coordinates of the position where the user interface operation has been performed directly”, the coordinates may change depending on the environment in which the test script is executed. For example, the display position of the test target application creates the test script. If the time is different from the time of test execution, the coordinates will be misaligned, and the expected operation will not be performed, which is not realistic.

  In the method of “deriving coordinates around the user interface operated position from the screen and deriving the coordinates”, the above-mentioned problems do not occur, and it is realistic to derive the coordinates by this method. Regarding the point, there is no description or suggestion in Patent Document 1.

  Further, when actually creating a test script, the user must specify a target peripheral image (for example, an image of a button) using a user interface, and then scan and generate the peripheral image.

  As a specific example, the procedure is such that the entire screen or the target window is saved as data, the data is pasted into the image editing application software, and the peripheral image is cut out from there as a target. .

  When the test is executed, the cut-out peripheral image is compared with the screen to be tested, the coordinates are determined using the image corresponding to the cut-out peripheral image, and the user interface operation is reproduced.

  Therefore, in the preparatory stage for performing an automatic test, it is very burdensome for the user, and it is inevitable that the amount of operation procedures will increase when large-scale software development is performed. The amount of work to create is also increased, and test automation software for the purpose of reducing development costs may have the potential to result in increased development costs.

  In the present invention, in order to solve the above-mentioned problems, a test script with higher test accuracy is created, and the development cost in the test and evaluation process is reduced by reducing the work load required when creating the test script. An object of the present invention is to provide a test support apparatus capable of performing the above.

  According to a first aspect of the invention for achieving the above object, an operation is detected by the operation detection unit that detects an operation on the screen, and the operation detection unit based on the image data of the screen that is continuously recorded. A detection image analysis unit that acquires, as a detection image, a region where the pixel value of the image data before the timing and the pixel value of the image data after the detected timing are detected, and an operation related to the operation detected by the operation detection unit A script generation unit that generates a test script using information and specific information for specifying a detection image acquired by the detection image analysis unit, and a test target based on the test script generated by the script generation unit The position coordinates corresponding to the image data specified by the specific information with respect to the screen to be determined are determined, and the position coordinates are A script execution unit for performing an operation related to the operation information have, characterized by comprising a.

  The second invention for achieving the above object is to detect an operation by the operation detection step based on an operation detection step of detecting an operation on the screen and image data of the screen obtained by continuous recording. A detection image analysis step of acquiring a region where a pixel value of the image data before the timing and a pixel value of the image data after the detected timing are different as a detection image, and an operation related to the operation detected by the operation detection step A script generation step for generating a test script using information and specific information for specifying the detection image acquired by the detection image analysis step, and a test target based on the test script generated by the script generation step Determine the position coordinates corresponding to the image data specified by the specific information with respect to the screen, And executes and a script execution process for performing operations relating to the operation information by using the serial position coordinates.

A third invention for achieving the above object is a program executed in a test support device, wherein the test support device can be continuously recorded with operation detection means for detecting an operation on a screen. Based on the image data of the screen, an area in which the pixel value of the image data before the timing when the operation is detected by the operation detection unit and the pixel value of the image data after the detected timing are different is used as a detected image. Detection image analysis means to be acquired, script generation means for generating a test script using operation information related to the operation detected by the operation detection means and identification information for specifying the detection image acquired by the detection image analysis means And a screen to be tested based on the test script generated by the script generation means The determined position coordinates of the image data specified falls by the specific information, said the script executing means for performing an operation on the operation information, and program for functioning by using the position coordinates for.

  According to the present invention, testing and evaluation in software development can be performed by automatically generating a test script by obtaining a coordinate value of the position used in a test script from a peripheral image at a position where a user interface operation has been performed. There is an effect that the development cost in the process can be reduced.

It is a block diagram which shows schematic structure of the hardware of the test assistance apparatus which concerns on embodiment of this invention. It is a block diagram which shows the function structure of the module in the test assistance apparatus which concerns on embodiment of this invention. It is a block diagram which shows the structure of the detailed function of the module in the test assistance apparatus which concerns on embodiment of this invention. It is a flowchart which shows the process which produces the script for a test in the test assistance apparatus which concerns on embodiment of this invention. It is a flowchart which shows the process which specifies the periphery image of the position where user interface operation was performed in the production | generation of the script for a test in embodiment of this invention. It is a flowchart which shows the process which selects the candidate of the periphery image of the position where user interface operation was performed in the case of the production | generation of the test script in embodiment of this invention. It is a flowchart which shows the process regarding the production | generation of the script which shows user interface operation in the production | generation of the script for a test in embodiment of this invention. It is a flowchart which shows the process which performs the script for a test in embodiment of this invention. It is explanatory drawing for demonstrating having detected the pressed button in embodiment of this invention. It is explanatory drawing for demonstrating having detected the selected check button in embodiment of this invention. In embodiment of this invention, it is explanatory drawing for demonstrating having detected the selected radio button. In embodiment of this invention, it is explanatory drawing for demonstrating having detected that the record was selected from the list | wrist. In the embodiment of the present invention, it is explanatory drawing for demonstrating having detected that the record was selected from the combo box (in the case of no scroll). FIG. 10 is an explanatory diagram for explaining that a record is selected from a combo box (when scrolling) in the embodiment of the present invention. In embodiment of this invention, it is explanatory drawing for demonstrating having detected that the record was selected from the menu. It is a source code which is an example of the test script in the embodiment of the present invention.

  Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings.

  FIG. 1 is a configuration diagram showing a hardware configuration of a test support apparatus according to an embodiment of the present invention.

  In FIG. 1, reference numeral 100 denotes a test support device that is a computer that executes a test of the created software. The ROM 102 or the external memory 104 includes a BIOS (Basic Input / Output System) or OS that is a control program for the CPU 106, or In addition, various programs to be described later necessary for realizing the functions executed by the test support apparatus 100 are stored.

  The RAM 108 functions as a main memory and work area for the CPU 106. The CPU 106 implements various operations by loading a program necessary for execution of processing into the RAM 108 and executing the program.

  A keyboard controller (KBC) 110 controls input from a keyboard 112, and a point device controller (PDC) 114 controls input from a mouse 116 and other pointing devices. A video controller (VC) 118 controls display on a display device such as the display 120.

  The memory controller (MC) 122 is a hard disk (HDD), floppy disk (registered trademark FD) or PCMCIA card slot for storing a boot program, browser software, various applications, font data, user files, editing files, various data, and the like. Controls access to an external memory 104 such as a compact flash memory connected via an adapter.

  ROM 102, CPU 106, RAM 108, keyboard controller (KBC) 110, in-device controller (PDC) 114, video controller (VC) 118, and memory controller (MC) 122 are connected via a system bus 124, and are mutually connected. Control signals are transmitted and received.

  FIG. 2 is a configuration diagram showing a functional configuration of modules in the test support apparatus 100 according to the embodiment of the present invention.

  The test support apparatus 100 has an environment for executing a test of the created software, and the test support apparatus 100 performs an automatic test on the test target application 200 indicating the created software and the created software. The script generation module 202 automatically generates a test script used at the time, and the script execution module 204 that executes an automatic test of a test target application using the generated test script.

  FIG. 3 is a configuration diagram showing a detailed functional configuration of modules in the test support apparatus 100 according to the embodiment of the present invention.

  The script generation module 202 includes a screen recording unit 300. The screen recording unit 300 uses a screen moving image file continuously displayed on the display 120 shown in FIG. Record on a recording medium.

  In this case, a screen (not shown) for inputting an instruction to start recording is prepared, and recording is started when a recording start button or the like provided on the screen is pressed by a pointing device such as the keyboard 112 or the mouse 116. When recording is stopped, the recording is stopped when the recording stop button or the like is pressed by the pointing device.

  The GUI operation input unit 304 indicates a pointing device such as the keyboard 112 and the mouse 116 illustrated in FIG. 1, and the user uses the GUI operation input unit 304 to generate a test script and execute the generated test script. Thus, an automatic test of the test target application 200 is performed.

  The script generation module 202 includes a GUI operation detection unit 306, a pre-GUI operation screen image generation unit 308, a post-GUI operation screen image generation unit 310, and a script generation unit 312. The GUI operation detection unit 306 includes a GUI operation detection unit 306. The operation input unit 304, the pre-GUI operation screen image generation unit 308, the post-GUI operation screen image generation unit 310, and the script generation unit 312 are connected.

  The GUI operation detection unit 306 includes operation information such as keystrokes on the keyboard 112 and cursor operations with the mouse 116 input by the GUI operation input unit 304 (for example, for the keyboard 112, a return key, “↓” or “↑”). Event information such as key presses or event information such as cursor movement, right-click, and left-click when the mouse 116 is used), screen image generation unit before GUI operation 308 and screen image generation after GUI operation The operation detection information indicating the detection is transmitted to the unit 310 and the operation information is transmitted to the script generation unit 312.

  When receiving the operation detection information, the pre-GUI operation screen image generation unit 308 uses the screen moving image 302 recorded by the screen recording unit 300 from a predetermined time before the reception timing to the reception timing, for example, in the bitmap format. An image file is generated (that is, an image file is generated by acquiring one frame of the screen moving image 302 immediately before the received timing). The generated image file is recorded on a recording medium such as the ROM 102 or the external memory 104 as a pre-GUI operation screen image 314.

  In addition, when the GUI operation post-screen image generation unit 310 receives the operation detection information, the screen moving image 302 recorded by the screen recording unit 300 from the received timing to a predetermined time later is used to generate, for example, an image file in a bitmap format. (That is, an image file is generated by acquiring one frame of the screen moving image 302 immediately after the reception timing). The generated image file is recorded in a recording medium such as the ROM 102 or the external memory 104 as a screen image 316 after GUI operation.

  In order to indicate that the pre-GUI operation screen image 314 and the post-GUI operation screen image 316 are image files related to the operation input by the GUI operation input unit 304, the pre-GUI operation screen image 314 is displayed as 00001_before. Save as .bmp, save the screen image 316 after GUI operation as 00001_after.bmp, set an identification number indicating that the file name is related (in this example, 00001), and save.

  The script generation unit 312 receives the operation information transmitted by the GUI operation detection unit 306, and generates a test script 318 based on the received operation information and specific information for specifying a detection image (details will be described later).

  The script generation module 202 includes a detection image analysis unit 320. The detection image analysis unit 320 is connected to the script generation unit 312, and acquires the pre-GUI operation screen image 314 and the post-GUI operation screen image 316. The peripheral image file at the position where the user inputs using the GUI operation input unit 304 is cut out from the pre-GUI operation screen image 314 obtained and analyzed, and the generated peripheral The image file is recorded as a detected image 322 in a recording medium such as the ROM 102 or the external memory 104.

  A predetermined area (such as a recording destination path) of the recording medium on which the detected image 322 is recorded is specified information, and the specified information is transmitted to the script generation unit 312.

  The script execution module 204 includes a script execution unit 324. The script execution unit 324 is generated by the operation information and specific information of the test script 318 generated by the script generation unit 312 and the detected image analysis unit 320. Based on the recorded detection image 322, an automatic test is executed on the test target application 200.

  In order to execute the test script 318, a screen (not shown) for inputting an instruction for execution is prepared, and a timing when a test start button or the like provided on the screen is pressed by a pointing device such as the keyboard 112 or the mouse 116 Run the test script with.

  Next, it is a flowchart which shows the process which produces | generates the script for a test in the test assistance apparatus 100 which concerns on embodiment of this invention using FIG. Each processing step in this flowchart is executed under the control of the CPU 106.

  In step S100, the GUI operation input unit 304 detects that the test target application 200 has been instructed to start, and starts the test target application 200. In the next step S102, the screen recording unit 300 starts recording. Upon detecting that an instruction has been input, the screen moving image file displayed on the display 120 is recorded as a screen moving image 302 on a recording medium such as the ROM 102 or the external memory 104.

  In step S104, it is determined whether or not the screen recording unit 300 has detected whether or not a recording stop instruction has been input. If detected, the process proceeds to step S116. If not detected, the process proceeds to step S106. Proceed with the process.

  In step S106, the GUI operation detection unit 306 determines whether or not an operation has been performed by the GUI operation input unit 304. If it is detected, the GUI operation pre-screen image generation unit 308 and the GUI operation post-GUI operation are performed. Operation detection information indicating that an operation has been detected is transmitted to the screen image generation unit 310, and operation information related to the operation (for example, if the keyboard 112 is used, such as pressing a return key, “↓” or “↑” key, etc.) Event information or event information such as cursor movement, right click, and left click when the mouse 116 is used) is sent to the script generation unit 312, and the process proceeds to step S108. Proceed to subsequent processing.

  In step S108, the screen image recording unit 300 receives the operation detection information transmitted in step S106 by the pre-GUI operation screen image generation unit 308, and is recorded by the screen recording unit 300 from the received timing to the timing received from a predetermined time before. For example, an image file in bitmap format is generated using 302 (that is, an image file is generated by acquiring one frame of the screen moving image 302 immediately before the received timing), and the generated image file is operated using a GUI operation. The previous screen image 314 is recorded in a recording medium such as the ROM 102 or the external memory 104.

  In step S110, the operation detection information transmitted in step S106 is received by the screen image generation unit 310 after GUI operation, and the screen moving image 302 recorded by the screen recording unit 300 from the received timing until a predetermined time later is used. For example, an image file in a bitmap format is generated (that is, an image file is generated by acquiring one frame of the screen moving image 302 immediately after the reception timing), and the generated image file is used as a screen image 316 after GUI operation. Recording is performed on a recording medium such as the ROM 102 or the external memory 104. In addition, in order to show that the pre-GUI operation screen image 314 and the post-GUI operation screen image 316 are related, the file name is dealt with by assigning an identification number so that the relation is clearly shown.

  In step S112, the detected image analysis unit 320 acquires and analyzes the pre-GUI operation screen image 314 recorded on the recording medium in step S108 and the post-GUI operation screen image 316 recorded on the recording medium in step S110. After that, a peripheral image file at a position where the user has input using the GUI operation input unit 304 is cut out from the pre-GUI operation screen image 314 acquired and analyzed, and the generated peripheral image file is The detected image 322 is recorded in a predetermined area of a recording medium such as the ROM 102 or the external memory 104.

  A predetermined area (such as a recording destination path) of the recording medium on which the detected image 322 is recorded is transmitted to the script generation unit 312 as specific information. Details of the processing in this step will be described later with reference to FIG.

  In step S114, the script generation unit 312 generates a test script using the operation information transmitted in step S106 and the detected image 322 recorded on the recording medium in step S112.

  In step S116, the screen recording unit 300 stops recording the moving image of the screen displayed on the display 120, and in the next step S118, the test target application 200 is stopped.

  Next, details of the processing in the detected image analysis processing will be described using the flowchart shown in FIG. Each processing step in this flowchart is executed by the detected image analysis unit 320 under the control of the CPU 106.

  In step S200, after obtaining and analyzing the pre-GUI operation screen image 314 recorded in the recording medium in step S108 and the post-GUI operation screen image 316 recorded in the recording medium in step S110, the user performs the GUI operation input unit 304. The peripheral image file at the position where the input is performed using is generated by cutting from the pre-GUI operation screen image 314 obtained and analyzed, and candidates that can become the detection image 322 are generated from the generated peripheral image file. The image specified from the pre-GUI operation screen image 314 is specified as a pre-GUI operation range image, and the post-GUI operation screen image 316 is specified as a post-GUI operation range image on a recording medium such as the ROM 102 or the external memory 104. Record. Details of the processing in this step will be described later with reference to FIG.

  Next, in step S202, it is determined whether the range image after GUI operation recorded on the recording medium in step S200 is in a list state. For example, in the list state, when a list is selected with respect to the initial state shown before the GUI operation shown in FIG. 14, a plurality of records are displayed adjacent to each other as shown in 1 after the GUI operation. .

  After that, when one record is selected by detecting that the “↓” key or the like is pressed as shown in 1 after GUI operation for a plurality of records, the selected record (drawing (* .vsd)) is selected. ) Is highlighted, and when the image file before the GUI operation is compared with the image file after the GUI operation, the detected image 322 may be detected because the difference is detected. However, when an operation such as scrolling is performed, Since the records displayed in the list change, a difference is detected when comparing the image file after the GUI operation 1 and the image file after the GUI operation 2. Detection image 322 is generated.

  Therefore, when a plurality of detection images 322 are generated, this step is executed because the detection image 322 is not generated by assuming that no selection is made among the displayed records by the user.

  In step S204, the pre-GUI operation range image recorded in the recording medium in step S200 is recorded as a detected image candidate in a recording medium such as the ROM 102 or the external memory 104.

  In step S206, it is determined whether or not there are a plurality of detection candidate images recorded in the recording medium in step S204. If there are a plurality of detection candidate images, the process proceeds to step S208. A case where there are a plurality of detection candidate images in this step will be described later with reference to FIGS.

  In step S208, the dark image in the range image after GUI operation corresponding to the detected image candidate recorded in the recording medium in step S204 is recorded as a detected image 322 in a recording medium such as the ROM 102 or the external memory 104. Details of the light / dark comparison will be described with reference to FIGS.

  In this step, a predetermined area of the recording medium on which the detected image 322 is recorded is stored as specific information in the RAM 108. When the script generation unit 312 generates the test script 318, the specific information stored in the RAM 108 is stored. Used.

  Next, details of the processing in the detection range image acquisition processing will be described using the flowchart shown in FIG. Each processing step in this flowchart is executed by the detected image analysis unit 320 under the control of the CPU 106.

  In step S300, the value of each pixel of the image file of the pre-GUI operation screen image 314 and the image file of the post-GUI operation screen image 316 recorded in the recording medium in step S110 is compared.

  In this case, the identification numbers set in the file names are compared with those having the same identification number, and in step S302, pixels whose pixel values are different in step S300 are connected by a line.

  In the next step S304, it is determined whether or not the frame connected by the line is a quadrilateral. If it is determined that the frame is a quadrilateral, the process proceeds to step S306, and if it is determined that the frame is not a quadrilateral, The process proceeds to step S308.

  In step S306, the quadrilateral connected by the line is shaped into a rectangle (or square). As for the method of shaping into a rectangle (or square), as shown in FIG. 14, the pixel values differ from the comparison between the image file on the pre-GUI operation screen 900 and the image file on the post-GUI operation screen 904. A frame 902 and a frame 906 in which pixels are connected by lines are shown. Since these frames are not rectangular (or square), they are rectangular (or square) like the frame 908 so as to include a portion determined as a frame. ).

  Next, in step S308, the frame shaped into a rectangle in step S306 is recorded as a pre-GUI operation range image (peripheral image file) in a recording medium such as the ROM 102 or the external memory 104 for the pre-GUI operation screen image 314.

  Next, in step S310, the frame shaped into a rectangle in step S306 is recorded as a post-GUI operation range image (peripheral image file) in a recording medium such as the ROM 102 or the external memory 104 with respect to the screen image 316 after GUI operation.

  Next, details of the processing in the script generation processing will be described using the flowchart shown in FIG. Each processing step in this flowchart is executed by the script generation unit 312 under the control of the CPU 106.

  In step S400, the operation information transmitted from the GUI operation detection unit 306 in step S106 is received, and the received operation information is stored in the RAM 108. In the next step S402, based on the operation information stored in the RAM 108 in step S400. Then, it is determined whether or not image search processing is necessary.

  Note that the image search processing shows a method of obtaining the coordinates by specifying the detected image 322 from the peripheral image file at the position where the user has input using the GUI operation input unit 304, and the keyboard 112 is used as the operation information. If so, buttons and lists arranged on the screen such as event information such as pressing of the return key or “↓” or “↑” key, or event information such as right click and left click when the mouse 116 is used. When the operation used when selecting is performed, the image search process is necessary.

  In step S404, an image search process is added to the test script 318 including a path indicating a predetermined area of the recording medium on which the detected image 322 is recorded in step S112, that is, specific information stored in the RAM 108.

In step S406, a process related to the operation information stored in the RAM 108 in step S400 is added to the test script 318.
An example of the generated test script is shown in FIG.

  Chkimg () of the function Chkimg (“C: \ image.bmp”) shown in the second line indicates the image search process described in step S402. For C: \ image.bmp, the image search process This represents a path indicating a predetermined area of the recording medium in which the detection image 322 necessary for the recording is stored, that is, specific information. Using this function, the detection image 322 recorded in the predetermined area specified by the path is acquired. Thus, the search process is performed.

  Further, the BTN () of the function BTN (LEFT, CLICK, 598, 261, 10) shown in the third line indicates the operation information described in step S 402, and the BTN () is displayed when the mouse 116 is used. BTN (information indicating either right or left, information indicating an event such as a click, the X coordinate for the screen in the test target application 200, the Y coordinate for the screen in the test target application 200,... ) And so on.

  As for the X coordinate and the Y coordinate, the position corresponding to the screen in the test target application 200 is specified from the detection image 322 acquired by Chkimg, and the coordinates of the specified position with respect to the screen are set.

  In other words, according to this example, the script described in the second and third lines acquires the detection image 322 stored in C: \ image.bmp, and the acquired detection image 322 is used in the test target application 200. The corresponding position is specified for the screen, the X coordinate for the specified position is set to 598, the Y coordinate is set to 261, and LEFT, that is, the left button is CLICK, that is, clicked. Show.

  Next, a process for executing an automatic test on a test target application using a test script will be described with reference to a flowchart shown in FIG. Each processing step in this flowchart is executed by the script execution unit 324 under the control of the CPU 106.

  First, in step S500, when the GUI operation input unit 304 detects that the test target application 200 has been instructed to start, the test target application 200 is started, and a test start button provided on a screen (not shown) is displayed. When it is detected that the input has been made by the GUI operation input unit 304, the test script 318 is executed.

Next, in step S502, it is determined whether or not an image search process exists in the test script 318. If it is determined that the image search process exists, the process proceeds to step S504, where it is determined that the test script 318 does not exist. If not all have been executed, the process proceeds to step S502.

  In step S504, the detection image 322 required in the image search process is acquired from the predetermined area of the recording medium recorded in step S112, and the corresponding position is specified with respect to the acquired detection image 322 and the screen in the test target application 200. In step S506, the coordinates for the screen at the position specified in step S503 are acquired.

  In step S508, the user interface operation is automatically executed on the coordinates acquired in step S506 based on the operation information of the test script 318. Note that the processing from step S502 to step S508 is repeated until all processing of the test script 318 is performed.

  In step S510, the test target application 200 is stopped and the processing in this flowchart ends.

  Next, a first example (when the button is pressed) of the detected image analysis process shown in FIG. 5 will be described with reference to FIG.

  When a user interface operation performed on the screen in the test target application 200 is detected, the pre-GUI operation range image is the pre-GUI operation screen 400, the post-GUI operation range image is the post-GUI operation screen 402, and the GUI operation The frame 404 and the frame 406 are specified by comparing the pixel values of the image file on the previous screen 400 and the image file on the screen 402 after GUI operation. Therefore, the peripheral image file of the frame 404 is set as a detection image 408 (referred to as a detection image 322).

  Next, a second example (when the check button is pressed) of the detected image analysis process shown in FIG. 5 will be described with reference to FIG.

  When a user interface operation performed on the screen of the test target application 200 is detected, the pre-GUI operation range image is set as the pre-GUI operation screen 500, and the post-GUI operation range image is set as the post-GUI operation screen 502. The frame 504 and the frame 606 are specified from the comparison between the pixel values of the image file on the previous screen 500 and the image file on the screen 502 after GUI operation. Therefore, the peripheral image file of the frame 504 is set as a detection image 508 (referred to as a detection image 322).

  Next, a third example of detection image analysis processing (when a radio button is pressed) shown in FIG. 5 will be described with reference to FIG.

  When a user interface operation performed on the screen in the test target application 200 is detected, the pre-GUI operation range image is set to a pre-GUI operation screen 600, and the post-GUI operation range image is set to a post-GUI operation screen 602. From the comparison of the pixel values of the image file on the previous screen 600 and the image file on the post-GUI operation screen 602, the pixel values are different in the two combinations of the frame 604 and the frame 608 and the frame 606 and the frame 610. This is detected, and a plurality of detection candidate images are specified.

  Assuming that the detection candidate image in the frame 604 is the detection candidate image 612 and the detection candidate image in the frame 606 is the detection candidate image 614, the frame 614 in which the density of the image after the user interface operation is detected is the detection image 616 (detection image). 322.).

  Next, a fourth example of the detected image analysis process shown in FIG. 5 (when a record displayed in the list is selected) will be described with reference to FIG.

  When a user interface operation performed on the screen in the test target application 200 is detected, the pre-GUI operation range image is the pre-GUI operation screen 700, the post-GUI operation range image is the post-GUI operation screen 702, and the GUI operation From the comparison of the pixel values of the image file on the previous screen 700 and the image file on the post-GUI operation screen 702, the pixel values in the two combinations of the frame 704 and the frame 708 and the frame 706 and the frame 710 are A different thing is detected and a some detection candidate image is specified.

  Assuming that the detection candidate image in the frame 704 is the detection candidate image 712 and the detection candidate image in the frame 706 is the detection candidate image 714, the frame 714 in which the density of the image after the user interface operation is detected is the detection image 716 (detection image). 322.).

  Next, a fifth example of the detected image analysis processing shown in FIG. 5 (when a record displayed in a combo box (without scrolling) is selected) will be described with reference to FIG.

  When a user interface operation performed on the screen in the test target application 200 is detected, the pre-GUI operation range image is set as the pre-GUI operation screen 800, and the post-GUI operation range image is set as the post-GUI operation screen 802. It is detected that the values of the pixels in the frame 808 and the frame 810 are different from the values of the pixels in the image file on the previous screen 800 and the image file on the post-GUI operation screen 802. Therefore, the peripheral image file of the frame 808 is set as a detection image 812 (referred to as a detection image 322).

  When the next user interface operation is detected, the pre-GUI operation range image is set to the post-GUI operation screen 802, the post-GUI operation range image is set to the post-GUI operation screen 804, and the image file of the post-GUI operation screen 802 From the comparison with the value of each pixel with the image file on the screen 804 after GUI operation, it is detected that the pixel value is different in the two combinations of the frame 814 and the frame 818 and the frame 816 and the frame 820. Detection candidate images are identified.

  Assuming that the detection candidate image in the frame 814 is the detection candidate image 822 and the detection candidate image in the frame 816 is the detection candidate image 824, the frame 824 in which the density of the image after the user interface operation is detected is the detection image 826 (detection image). 322.).

  For the GUI operation screen 806, the user interface operation is not detected and only the content displayed in the frame 820 is displayed.

  Next, a sixth example of the detected image analysis process shown in FIG. 5 (when a record displayed in a combo box (when scrolling) is selected) will be described with reference to FIG.

  When a user interface operation performed on the screen in the test target application 200 is detected, the pre-GUI operation range image is set as the pre-GUI operation screen 900, and the post-GUI operation range image is set as the post-GUI operation screen 904. It is detected that the value of each pixel in the frame 902 and the frame 906 is different from the value of each pixel in the image file on the previous screen 900 and the image file on the screen 904 after GUI operation.

  However, since the detected frame does not have a rectangular (or square) shape, it is shaped into a rectangle (or square) in a shape that includes all the frames as indicated by a frame 908. As described above, since the corresponding position of the detected image is specified on the screen of the test target application 200, the position is obtained as a coordinate value such as a set of X coordinates and Y coordinates. (Square). That is, shaping is performed according to the function used in the image search process.

  When the next user interface operation is detected, the pre-GUI operation range image is the post-GUI operation screen 904, the post-GUI operation range image is the post-GUI operation screen 912, and the image file and GUI operation of the post-GUI operation screen 904 are displayed. From the comparison with the value of each pixel of the image file on the back screen 910, the value of each pixel displayed in the list on the screen 904 after GUI operation and the value of each record displayed on the list on the screen 910 after GUI operation are obtained. It is detected that the pixel value is different, and a plurality of detection candidate images are specified.

  In this case, a state where a plurality of detection candidate images are continuous in the vertical direction, that is, the X coordinate of the left end of the detection candidate image with respect to the screen is the same, and a state in which the detection candidate images are continuous without gaps, that is, one detection candidate image The Y coordinate of the lower end and the Y coordinate of the upper end of the detection candidate image displayed immediately below the detection candidate image are the same, and this state is a list state in the embodiment of the present invention. Whether or not it is in the list state is thus determined based on the X coordinate of the left end of the detection candidate image and the Y coordinates of the upper end and the lower end of the detection candidate image.

  In such a list state, it is considered that there is no detected image (the selected record does not exist) like the dotted frame 912. This is because when the record displayed in the list is selected, the user interface operation is detected at the timing when the record is selected, so that the GUI operation is always performed within the time before and after the detected timing. This is because the record displayed in the list on the back screen 904 matches the record displayed on the list on the screen 910 after GUI operation.

  When the next user interface operation is detected, the pre-GUI operation range image is the post-GUI operation screen 910, the post-GUI operation range image is the post-GUI operation screen 916, the image file of the post-GUI operation screen 910, and the GUI operation It is detected from the comparison with the value of each pixel of the image file on the rear screen 916 that the value of each pixel in the frame 914 and the frame 918 is different. Accordingly, the peripheral image file of the frame 914 is set as a detection image 920 (referred to as a detection image 322).

  As for the GUI operation screen 924, the content displayed in the frame 918 is only displayed without detecting the user interface operation.

  When creating the test script 318, in the case of a combo box (when scrolling), the user interface operation (operation information) in the list state is executed with the “↓” key down or the “↑” key down. Then, the test script 318 for searching the detected image 920 is generated.

  Next, a seventh example of detection image analysis processing shown in FIG. 5 (when an item displayed on a menu is selected) will be described with reference to FIG.

  When a user interface operation performed on the screen in the test target application 200 is detected, the pre-GUI operation range image is the pre-GUI operation screen 1000, the post-GUI operation range image is the post-GUI operation screen 1002, and the GUI operation is performed. It is detected that the values of the pixels in the frame 1006 and the frame 1008 are different from the values of the pixels in the image file on the previous screen 1000 and the image file on the screen file 1002 after the GUI operation. Therefore, the peripheral image file of the frame 1006 is set as the detected image 1010.

  When the next user interface operation is detected, the pre-GUI operation range image is the GUI post-operation screen 1002, the post-GUI operation range image is the post-GUI operation screen 1004, and the image file of the post-GUI operation screen 1002 It is detected that the value of each pixel in the frame 1012 and the frame 1014 is different from the value of each pixel of the image file on the screen 1004 after GUI operation. Therefore, the peripheral image file of the frame 1012 is set as a detection image 1016 (referred to as a detection image 322).

  As described above, according to the present invention, a test script in software development can be obtained by automatically generating a test script by obtaining a coordinate value of the position used in a test script from a peripheral image at a position where a user interface operation has been performed. In addition, the development cost in the evaluation process can be reduced.

Although the embodiment has been described in detail above, the present invention can take an embodiment as, for example, a method, a program, a storage medium, or the like. Specifically, the present invention is applied to a system composed of a plurality of devices. The present invention may be applied, or may be applied to an apparatus composed of one device. The program according to the present invention is a program that allows a computer to execute (read) each processing method, and the storage medium according to the present invention stores a program that allows the computer to execute each processing method.

  The program in the present invention may be a program for each processing method of each device.

As described above, a recording medium that records a program that implements the functions of the above-described embodiments is supplied to a system or apparatus, and a computer (or CPU or MPU) of the system or apparatus stores the program stored in the recording medium. It goes without saying that the object of the present invention can also be achieved by performing reading.

  In this case, the program itself read from the recording medium realizes the novel function of the present invention, and the recording medium storing the program constitutes the present invention.

  As a recording medium for supplying the program, for example, a flexible disk, hard disk, optical disk, magneto-optical disk, CD-ROM, CD-R, DVD-ROM, magnetic tape, nonvolatile memory card, ROM, EEPROM, silicon A disk or the like can be used.

  Further, by executing the program read by the computer, not only the functions of the above-described embodiments are realized, but also an OS or the like operating on the computer based on an instruction of the program is a part of the actual processing or It goes without saying that the case where the functions of the above-described embodiments are realized by performing all of the above processing is also included.

Furthermore, after the program read from the recording medium is written to the memory provided in the function expansion board inserted into the computer or the function expansion unit connected to the computer, the function expansion board is based on the instructions of the program code. It goes without saying that the case where the CPU or the like provided in the function expansion unit performs part or all of the actual processing and the functions of the above-described embodiments are realized by the processing.

  Further, the present invention may be applied to a system composed of a plurality of devices or an apparatus composed of a single device. Needless to say, the present invention can be applied to a case where the present invention is achieved by supplying a program to a system or apparatus. In this case, by reading a recording medium storing a program for achieving the present invention into the system or apparatus, the system or apparatus can enjoy the effects of the present invention.

  Furthermore, by downloading and reading a program for achieving the present invention from a server, database, etc. on a network using a communication program, the system or apparatus can enjoy the effects of the present invention. In addition, all the structures which combined each embodiment mentioned above and its modification are also included in this invention.

100 Test support device 102 ROM
104 External memory 106 CPU
108 RAM
110 KBC
112 Keyboard 114 PDC
116 Mouse 118 VC
120 Display 122 MC
124 bus

Claims (5)

An operation detector for detecting operations on the screen;
Based on the image data of the screen obtained by continuous recording, the pixel value of the image data before the timing when the operation is detected by the operation detection unit and the pixel value of the image data after the detected timing A detection image analysis unit that acquires a different region as a detection image;
A script generation unit that generates a test script using operation information related to an operation detected by the operation detection unit and specific information for specifying a detection image acquired by the detection image analysis unit;
Based on the test script generated by the script generation unit, a position coordinate corresponding to the image data specified by the specific information with respect to the screen to be tested is determined, and an operation related to the operation information is performed using the position coordinate. A script execution unit that performs
A test support apparatus characterized by comprising:   2. The test support apparatus according to claim 1, wherein when there are a plurality of different areas, the detection image analysis unit acquires a detection image having a high density in the detection image after the timing as a detection image.   The test support apparatus according to claim 1, wherein the detected image analysis unit does not acquire the detected image when the plurality of different regions are adjacent to each other. An operation detection process for detecting operations on the screen;
Based on the image data of the screen obtained by continuous recording, the pixel value of the image data before the timing when the operation is detected by the operation detection step and the pixel value of the image data after the detected timing A detection image analysis step of acquiring a different region as a detection image;
A script generation step for generating a test script using the operation information related to the operation detected by the operation detection step and the specific information for specifying the detection image acquired by the detection image analysis step;
Based on the test script generated by the script generation step, a position coordinate corresponding to the image data specified by the specific information with respect to a screen to be tested is determined, and an operation related to the operation information is performed using the position coordinate. Script execution process to perform
A method for controlling a test support apparatus, comprising: A program executed in the test support device,
The test support device;
An operation detection means for detecting an operation on the screen;
Based on the image data of the screen obtained by continuous recording, the pixel value of the image data before the timing when the operation is detected by the operation detection means and the pixel value of the image data after the detected timing A detection image analysis means for acquiring a different region as a detection image;
Script generation means for generating a test script using the operation information related to the operation detected by the operation detection means and the specific information for specifying the detected image acquired by the detected image analysis means;
Based on the test script generated by the script generation means, a position coordinate corresponding to the image data specified by the specific information with respect to the screen to be tested is determined, and an operation related to the operation information is performed using the position coordinate Script execution means for performing
Program to make it function.

Images