JP7311696B1 - MULTI-SCREEN DISPLAY SYSTEM, MULTI-SCREEN DISPLAY DEVICE, AND MULTI-SCREEN DISPLAY PROGRAM - Google Patents

Abstract

An object of the present invention is to synchronize the actions of 360-degree images and three-dimensional models that are displayed side by side. A multi-screen display system (10) includes multi-screen display means for displaying a plurality of screens displaying a 360-degree image or a three-dimensional model, and one of the plurality of screens displayed by the multi-screen display means. an operation receiving means for receiving an operation on the screen, an operating means for operating a 360-degree image or a three-dimensional model in the operated screen according to the operation content accepted by the operation accepting means, and a synchronization with the operation in the operated screen a synchronizing means for activating the 360-degree image or the three-dimensional model in the other screens displayed by the multi-screen display means. [Selection drawing] Fig. 1

Description

The present invention relates to a multi-screen display system, a multi-screen display device, and a multi-screen display program.

The following image processing apparatuses are known. In this image processing apparatus, an estimation algorithm is applied to an omnidirectional image called a 360-degree image to generate an omnidirectional image for stereoscopic vision (eg, Patent Document 1).

JP 2021-86494 A

360-degree images including still images and moving images and three-dimensional models are widely used in construction projects, maintenance and management projects, and the like. Unlike flat images such as still photographs and 2D drawings, 360-degree images and 3D models can extract a large amount of information from a single image, making it easier to grasp the overall situation from a bird's-eye view. That is the reason. On the other hand, 360-degree images and 3D models, unlike planar images, serve as spatial information. It is difficult to comprehend. In order to solve this problem, there has been a demand for a mechanism that can display multiple 360-degree images and 3D models side by side and compare information while manipulating each one. It wasn't.

A multi-screen display system according to the present invention comprises multi-screen display means for displaying a plurality of screens displaying 360-degree images or three-dimensional models side by side; operation accepting means for accepting; operating means for operating a 360-degree image or a three-dimensional model within the operated screen according to the operation content accepted by the operation accepting means; and synchronizing means for operating the 360-degree image or the three-dimensional model in other screens displayed by the multi-screen display means.
A multi-screen display device according to the present invention comprises multi-screen display means for displaying a plurality of screens displaying 360-degree images or three-dimensional models side by side; operation accepting means for accepting; operating means for operating a 360-degree image or a three-dimensional model within the operated screen according to the operation content accepted by the operation accepting means; and synchronizing means for operating the 360-degree image or the three-dimensional model in other screens displayed by the multi-screen display means.
A multi-screen display program according to the present invention includes a multi-screen display procedure for displaying a plurality of screens displaying 360-degree images or three-dimensional models side by side, and an operation for one screen among the plurality of screens displayed in the multi-screen display procedure. An operation reception procedure to be received, an operation procedure for operating a 360-degree image or a three-dimensional model in the operated screen according to the operation content accepted in the operation reception procedure, and synchronizing with the operation in the operated screen, A program for causing a computer to execute a synchronization procedure for operating a 360-degree image or a three-dimensional model in other screens displayed in a multi-screen display procedure.

According to the present invention, a screen displaying a 360-degree image and a screen displaying a three-dimensional model are displayed side by side, and an operation is received for the 360-degree image or the three-dimensional model on one screen, and the operation is performed according to the content of the operation. Since the 360-degree image or the 3D model on another screen is moved in synchronization with the movement, the user can compare the information while manipulating the 360-degree image or the 3D model displayed side by side.

1 is a block diagram showing the configuration of an embodiment of a multi-screen display system 10; FIG. 2 is a block diagram showing the configuration of one embodiment of server 100. FIG. 1 is a block diagram showing the configuration of an information terminal 200 according to an embodiment; FIG. FIG. 4 is a diagram schematically showing an example of a multi-display screen; FIG. 4 is a diagram schematically showing an example of synchronization of motions of 3D models; FIG. 10 is a diagram for explaining a method of aligning images; FIG. FIG. 10 is a diagram schematically showing an example of multi-display of n screens; FIG. 4 is a diagram schematically showing a method for automatically extracting image differences; FIG. 4 is a flow chart diagram showing the flow of processing in a link mode executed by the server 100. FIG.

FIG. 1 is a block diagram showing the configuration of one embodiment of a multi-screen display system 10 according to this embodiment. Multi-screen display system 10 according to the present embodiment includes server 100 and information terminal 200, and server 100 and information terminal 200 are connected via a communication line.

FIG. 2 is a block diagram showing the configuration of one embodiment of server 100 in this embodiment. The server 100 has a communication interface 101 , a control device 102 and a storage medium 103 .

The communication interface 101 is an interface for connecting the server 100 to a communication line such as the Internet. For example, a wired LAN module for wired connection to the Internet, a wireless LAN module for wireless connection to the Internet, etc. contains. In this embodiment, server 100 communicates with information terminal 200 via communication interface 101 .

The control device 102 is composed of a CPU, memory, and other peripheral circuits, and controls the server 100 as a whole. The memory constituting the control device 102 is, for example, a volatile memory such as SDRAM. This memory is used as a work memory for expanding programs when the CPU executes the programs and as a buffer memory for temporarily recording data. For example, data read via the communication interface 101 is temporarily recorded in a buffer memory.

The storage medium 103 is a storage medium for recording various data stored in the server 100, data of programs to be executed by the control device 102, and the like. etc. are used. The program data recorded in the storage medium 103 is provided by being recorded on a recording medium such as a CD-ROM or DVD-ROM, or is provided via a network, and the program data acquired by the operator is stored. Installation on the medium 103 enables the control device 102 to execute the program.

As the information terminal 200, for example, a smart phone, a mobile phone, a tablet terminal, a personal computer, or the like is used. FIG. 3 is a block diagram showing the configuration of one embodiment when a personal computer is used as the information terminal 200 in this embodiment.

The information terminal 200 includes an operation member 201 , a communication interface 202 , a control device 203 , a storage medium 204 and a display device 205 .

The operating member 201 includes various devices operated by the operator of the information terminal 200, such as a keyboard and a mouse.

The communication interface 202 is an interface for connecting the information terminal 200 with external devices such as other devices and terminals. For example, the communication interface 202 includes an interface for connecting to a communication line such as a LAN or the Internet. The information terminal 200 communicates with the server 100 via this communication interface 202 .

The control device 203 is composed of a CPU, memory, and other peripheral circuits, and controls the information terminal 200 as a whole. The memory constituting the control device 203 is, for example, a volatile memory such as SDRAM. This memory is used as a work memory for expanding programs when the CPU executes the programs and as a buffer memory for temporarily recording data. For example, data read via the communication interface 202 is temporarily recorded in a buffer memory.

The storage medium 204 is a storage medium for recording various data stored in the information terminal 200, program data to be executed by the control device 203, and the like. ) etc. are used. The program data recorded in the storage medium 204 is provided by being recorded on a recording medium such as a CD-ROM or DVD-ROM, or is provided via a network, and the program data acquired by the operator is stored. Installation on the medium 204 enables the controller 203 to execute the program.

The display device 205 is, for example, a liquid crystal monitor, and displays various display data output from the control device 203 .

In multi-screen display system 10 according to the present embodiment, display files are recorded in storage medium 103 of server 100, and control device 102 displays a multi-display screen in which a plurality of display files are displayed side by side on an information terminal. 200 for output. In this embodiment, the display file is assumed to be, for example, a 360-degree photograph, a three-dimensional model (3D model image), a two-dimensional photograph, a two-dimensional drawing, a PDF, a 360-degree video, or a two-dimensional video file.

360-degree image files, including 360-degree photos and 360-degree videos, are still images and videos taken using, for example, a spherical 360-degree camera that can shoot vertically and horizontally 360 degrees. be. Also, the files of planar photographs and planar moving images may be those captured using a device such as a digital camera capable of capturing planar photographs and planar moving images, for example. Files of 3D models and two-dimensional drawings may be created using software such as CAD that can create 3D models and two-dimensional drawings, for example. As for the PDF file, for example, a file created using software capable of creating a PDF format file may be used.

Although the method of recording the display file in the storage medium 103 is not limited, for example, when the user operates the device to upload the file to the server 100, the control device 102 records the uploaded file in the storage medium 103. methods are assumed. At this time, the control device 102 adds a unique file ID to the uploaded file. As a result, each file can be managed using the file ID, so even if the uploaded files have the same file name, each file can be identified by the file ID.

In recent years, 360-degree images such as 360-degree photographs or 360-degree videos and 3D models have become widely used in construction projects, maintenance and management projects, and the like. This is because 360-degree images and 3D models can extract a lot of information from a single image, unlike flat images such as still photographs and 2D drawings, and it is easy to grasp the overall state from a bird's-eye view. is a factor. On the other hand, 360-degree images and 3D models are spatial information, unlike planar images. It has become difficult to Therefore, the multi-screen display system 10 according to the present embodiment provides a function for comparing information from the same viewpoint based on a plurality of images as a system for solving such problems. A program for using the multi-screen display system 10 is recorded in the storage medium 103 of the server 100, and the user of the information terminal 200 operates the operation member 201 to instruct the execution of the program. A multi-screen display system 10 is available. The multi-screen display system 10 according to this embodiment will be described below.

In the multi-screen display system 10 according to the present embodiment, combinations of files capable of multi-screen display are set in advance. Combinations of files that can be displayed on multiple screens in this embodiment include, for example, 360-degree photos and 360-degree photos, 360-degree photos and 3D models, 360-degree photos and 360-degree videos, 3D models and 3D models, 3D models and Combinations of 360-degree videos, two-dimensional photographs and two-dimensional photographs, two-dimensional drawings and two-dimensional drawings, PDFs and PDFs, and 360-degree videos and 360-degree videos.

A user using the multi-screen display system 10 operates the information terminal 200 to select a file to be displayed on the multi-screen. In this embodiment, for example, when the user instructs file selection via the information terminal 200, the control device 102 of the server 100 receives the selection of the display file recorded in the storage medium 103. A display file selection screen is output to the information terminal 200 . In the information terminal 200 , the control device 203 displays the display file selection screen output from the server 100 on the display device 205 . This allows the user to select a display file on the display file selection screen.

In the server 100, when a display file is selected by the user on the display file selection screen, the control device 102 reads out the file selected by the user from the storage medium 103, and displays the screen displaying it on the information terminal 200. Output to In the information terminal 200 , the control device 203 displays the screen output from the server 100 on the display device 205 .

A button for instructing the display of the multi-display screen is arranged on the screen displayed on the display device 205, and the user can instruct the display of the multi-display screen by clicking the button. When the control device 102 detects that the user has clicked a button for instructing display of the multi-display screen, the control device 102 controls the multi-display screen for accepting the selection of the file to be displayed side by side with the currently displayed file. A file selection screen is displayed on the display device 205 . Thereby, the user can select a display file to be displayed in a multi-display to be displayed side by side with the currently displayed file on the multi-display file selection screen.

In the server 100, when a display file for multi-display is selected by the user on the multi-display file selection screen, the control device 102 reads out the file selected by the user from the storage medium 103 and displays the read multi-display file. A multi-display screen is output to the information terminal 200 in which the display file for the display is displayed side by side with the display file that has already been displayed. In the information terminal 200 , the control device 203 displays the multi-display screen output from the server 100 on the display device 205 . As a result, a multi-display screen as shown in FIG. 4 is displayed on the display device 205. FIG. Note that FIG. 4A shows an example in which two 360-degree photographs are displayed side by side on the multi-display screen, and FIG. Displayed examples are shown, and FIG. 4C shows an example in which a 360-degree photograph and a 3D model are displayed side by side on the multi-display screen.

The user selects between linked and unlinked modes to specify how the multiple files displayed in the multi-view screen should behave. The link mode is a mode for synchronizing the operation of a plurality of files displayed on the screen, and the non-link mode is a mode for operating each of the plurality of images displayed on the screen. In this embodiment, as described above, the link mode is a mode for providing a function for comparing information from the same viewpoint based on a plurality of images. Therefore, the user selects the link mode when he/she wants to compare information from the same viewpoint based on a plurality of images on a multi-display screen in which selected files are displayed side by side. On the other hand, if the user wishes to individually operate each file on the multi-display screen displaying the selected files side by side, the user selects the non-link mode.

In this embodiment, it is assumed that the user can perform the following operations on each display file. Enlargement, reduction, rotation (pitch angle, roll angle, yaw angle) operations for 360-degree photos, enlargement, reduction, rotation (pitch angle, roll angle, yaw angle), horizontal operation for 3D models Operation of movement, vertical movement, operation of enlargement, reduction, and horizontal movement for planar photographs, operation of enlargement, reduction, and horizontal movement for 2D drawings, enlargement, reduction, and manipulation for PDF Horizontal movement, expansion, reduction, and rotation (pitch angle, roll angle, yaw angle) for 360-degree videos, and expansion and reduction for planar videos. .

In the present embodiment, among the above-mentioned combinations of files that can be displayed on a multi-screen, two-dimensional photographs and two-dimensional photographs, two-dimensional drawings and two-dimensional drawings, and PDFs of the same kind are combined in two dimensions. , it is possible to synchronize the operations of a plurality of files multi-displayed by known processing. Therefore, in the present embodiment, 360-degree photos and 360-degree photos, 3D models and 3D models, 360-degree videos and 360-degree videos, 360-degree photos and 3D models, 360-degree photos and 360-degree videos, and 3D models Synchronization processing of operations in the case of a combination of 360-degree moving images will be described. Among these three-dimensional files, when files of the same type are multi-displayed, there is no priority for operations, but for files of different types, the priority for operations is as follows. is set. In the case of 360-degree photos and 3D models, 360-degree photos have higher priority for operations, and in the case of 360-degree photos and 360-degree videos, 360-degree photos have higher priority for operations, and 3D models and 360-degree In the case of moving images, a 360-degree moving image has a higher priority regarding operation.

In the server 100, when the mode selected by the user is the non-link mode, the control device 102 operates each file displayed side by side on the multi-display screen based on the user's operation. For example, the control device 102 accepts an operation such as enlarging or reducing the file displayed on the screen on which the mouse cursor is positioned, and operates the displayed content according to the accepted operation. It should be noted that the method of receiving an operation such as enlarging or reducing an image, video, drawing, etc. displayed on the screen from the user and operating the displayed content in accordance with the content of the received operation is well known, and therefore will not be described here. omitted.

On the other hand, when the mode selected by the user is the link mode, when the user performs an operation on one of the multiple-displayed screens, the control device 102 displays other images according to the operation. Synchronize actions in all screens. In this embodiment, a case is assumed in which a plurality of screens, for example, up to nine screens are displayed in parallel at the same time, and the synchronization of actions based on the operation on the display in each screen is controlled.

FIG. 5 shows that when two screens displaying 3D models are displayed side by side in a multi-display, the 3D model in the other screen is operated in synchronization with the operation according to the user's operation on the 3D model in one screen. FIG. 4 is a diagram schematically showing a state in which In the example shown in FIG. 5, for example, when the user performs an operation such as enlargement, reduction, rotation (pitch angle, roll angle, yaw angle), horizontal movement, or vertical movement of the 3D model in the screen 5a, this This shows an example in which the 3D model is operated within the screen 5a in response to and the 3D model within the multi-displayed screen 5b is operated in synchronization with the movement of the 3D model within the screen 5a. Synchronization processing of operations in the link mode in multi-screen display will be described below.

The control device 102 aligns the images in order to synchronize the operations of the multiple images displayed in the multi display. In this embodiment, viewpoint synchronization can be performed between files of the same type such as 360-degree photographs and 360-degree photographs, and between files of different types such as 360-degree photographs and 3D models. Specifically, in the case of 360-degree photos or 360-degree videos, the control device 102 uses the direction angles (north-south direction and zenith) held as an EXIF file to start from the same direction angle. Align the images. In addition, in the case of a 360-degree image such as a 360-degree photograph or a 360-degree video and a 3D model, the control device 102 calculates the world coordinates of two points of the 3D model with respect to the latitude and longitude information (world coordinates) of the 360-degree image. Positions of images are aligned by calculating positions based on the information and setting them to the same viewpoint.

For example, in the case of a 360-degree photograph or a 360-degree video, as shown in FIG. The starting point is shared by setting the viewpoint vector R=(0, 1, 0) of the starting point in . In addition, in the case of a 3D model, the control device 102 sets the starting coordinate model origin O (0, 0, 0) and the viewpoint vector R = (0, 1, 0) as shown in FIG. Common points.

Further, in the case of a two-dimensional photograph, the control device 102 first aligns the number of pixels and aligns the dots at the center positions to perform alignment. In the case of a drawing, the control device 102 aligns the origin of the drawing by aligning the drawing origins.

For example, in the case of a two-dimensional photograph or a two-dimensional drawing, the control device 102 shares the starting point with the starting coordinate origin O (0, 0, 0) and no viewpoint vector, as shown in FIG. . In the case of planar moving images, as shown in FIG. 6D, the control device 102 shares the starting point with the starting coordinate origin O (0, 0, 0) and no viewpoint vector.

If the viewpoint positions are linked by the alignment described above, it is possible to synchronously display a plurality of images displayed in a multi-display. For this reason, the control device 102 synchronizes the movement angles (pitch angle, roll angle, yaw angle) on the screen with respect to each image after linking the viewpoint positions by the alignment described above. In the case of a 360-degree image or a 3D model, in order to synchronize the rotation angle, a spherical space for movement is set, and by making the amount of movement of the spherical space the same, image display at the same position and at the same angle is realized. Planar photographs and plan views are taken from the same viewpoint by unifying the amounts of movement in the XY-axis directions.

In the case of 3D models, it is necessary to control not only the above-described rotational movement, but also progression in the depth direction and movement in the vertical direction. For this reason, if the scales of the 3D models match, the control device 102 determines the viewpoint movement amount per click time of the control button to be, for example, 10 mm per 1 mm second, as will be described later. to perform the same coordinate movement and display.

FIG. 7 shows an example of multi-display of n screens from screen 1 to screen n. In the example shown in FIG. 7, which of the above-described link mode and non-link mode is selected is indicated using a key-shaped figure 7a or 7b. In FIG. 7, FIG. 7A shows the screen when the link mode is selected, and in this case, the link mode is indicated by using the key figure 7a showing the locked state. FIG. 7B shows the screen when the non-link mode is selected, and in this case, the non-link mode is indicated by using the key figure 7b representing the unlocked state. Therefore, in the multi-display screen shown in FIG. 7A, when the user operates any one of the screens 1 to n, the operation corresponding to the operation is synchronized with the other screens. On the other hand, in the multi-display screen shown in FIG. 7B, the user can individually operate each screen without synchronizing the operations of the screens 1 to n.

Here, synchronization processing of operations when the user operates screen 1 on the screen in the link mode shown in FIG. 7A, for example, will be described. In this embodiment, it is assumed that the user can operate screen 1 by operating operation member 201 of information terminal 200 . It is assumed that the operation of the screen is set as follows, for example, according to the operation of the operation member 201 by the user.

Rotating the mouse wheel enlarges or reduces the display on the screen according to the direction of rotation. At this time, the enlargement ratio or reduction ratio within the screen changes depending on the amount of rotation of the mouse wheel. When the mouse is moved in the horizontal direction, the display in the screen moves in the horizontal direction according to the movement direction. At this time, the amount of movement of the viewpoint in the horizontal direction within the screen changes according to the amount of movement of the mouse in the horizontal direction. When the mouse is moved up and down, the display on the screen moves up and down according to the direction of movement. At this time, the amount of movement of the viewpoint in the vertical direction within the screen changes according to the amount of movement of the mouse in the vertical direction.

If the display file is a 3D model, the user can also operate the screen 1 by operating the keyboard as follows. In this embodiment, "W", "A", "S", and "D" keys of the keyboard are assigned to the control buttons. When the "W" key on the keyboard is pressed, the display on the screen moves forward. At this time, the forward movement amount of the viewpoint changes according to the key click time of the "W" key. When the "A" key on the keyboard is pressed, the display in the screen moves to the left. At this time, the amount of leftward movement of the viewpoint changes according to the key click time of the "A" key. When the "S" key on the keyboard is pressed, the display in the screen moves backward. At this time, the amount of movement of the viewpoint in the backward direction changes according to the key click time of the "S" key. When the "D" key on the keyboard is pressed, the display on the screen moves rightward. At this time, the amount of rightward movement of the viewpoint changes according to the key click time of the "D" key.

In the information terminal 200, when the operation member 201 is operated by the user, the control device 203 transmits to the server 100 information about the operation amount of the mouse and the key click time of the keyboard as information indicating the amount of movement, together with the details of the operation. .

In the server 100, when the control device 102 receives the information indicating the operation content and the movement amount from the information terminal 200, the display content of the screen 1 is operated based on the information. The movement amount of the display in the screen according to the operation amount of the operation member 201 is set in advance, and this setting information is recorded in the storage medium 204 in advance as the setting information of the unit movement amount. In this embodiment, the setting information of the unit movement amount includes, for example, the enlargement ratio and reduction ratio per wheel mouse unit rotation amount, the left-right viewpoint movement amount per mouse left-right movement unit amount, and the mouse up-down movement amount per mouse up-down movement amount. amount, "W" key-click forward movement amount per unit time, "A" key-click leftward movement amount per unit time, "S" key-click backward movement amount, and "D '' key click unit time is set in advance. Note that the left-right viewpoint movement amount per mouse left-right unit movement amount and the vertical viewpoint movement amount per mouse vertical movement amount viewpoint movement amount are the pitch angle, roll angle, and yaw angle for 360-degree images and 3D models. A movement amount is set, and the default value is set to a unit angle α (rad) for a mouse movement of 0.254 mm, for example. In addition, the forward movement amount of the viewpoint per "W" key click unit time, the left movement amount of the viewpoint per "A" key click unit time, the backward movement amount of the viewpoint per "S" key click unit time, and "D" '' key click The amount of movement in the right direction of the viewpoint per unit time is set to the amount of movement on the model (in space) that moves with respect to the 3D model per second. The upward (in space) movement amount is set to 5 m on the scale.

The control device 102 operates the display contents of the screen 1 and synchronizes the display contents of other screens with the operation of the screen 1 as described above. That is, the control device 102 synchronizes the actions in the screens other than the screen 1 with the actions in the screen 1 . In the present embodiment, since it is desired to synchronize screen 1 and other screens at the same time, the movement amount per unit millisecond of screen 1 is calculated as a unit movement amount based on the speed of position change, and the calculated unit movement Drive other screens by sending the value of the amount to the other screen's display frame rate. For example, the unit movement amount per unit millisecond of screen 1 is wheel mouse rotation amount: α/mm second, mouse left/right movement amount: β/mm second, mouse up/down movement amount: γ/mm second, "W" key click Time: Δ/mm seconds, "A" key click time: ε/mm seconds, "S" key click time: ζ/mm seconds, "D" key click time: η/mm seconds. Then, the control device 102 transmits these values to the display frame rate of the screen 2 to the screen n to operate. It should be noted that since the coordinate transformation of the three-dimensional space is handled as Euler angles, addition cannot be performed. For this reason, in the present embodiment, when performing each movement process, rotation movement calculation is performed by sending the order together with the amount of the rotation angle, and movement synchronization is realized.

7(B), when the screen 1 is operated, the control device 102 sets the value of the unit movement amount of the screen 1 to the display frames of the screen 2 to the screen n. Only screen 1 is operated without transmitting to the rate. Further, in this case, the user can individually operate each of the screens 2 to n by operating the operation member 201 .

In this embodiment, the user can attach a comment to a 360-degree photograph and a 360-degree video. When the user inputs a comment on a 360-degree photograph or a 360-degree moving image, the control device 102 performs processing for embedding the input comment in the image. For example, the control device 102 sets a sphere layer separately from the 360-degree image, and attaches comments and files to the layer. In addition, when displaying a 360-degree photograph or a 360-degree moving image to which a comment is attached, the control device 102 also displays the attached comment in the image.

Further, in the present embodiment, image differences can be automatically extracted with respect to images. The control device 102 divides the displayed image into a plurality of image bits, for example, 20 bits vertically and horizontally, compares the differences, and if there is a difference, identifies the location and changes the displayed color. do. FIG. 8 is a diagram schematically showing a method for automatically extracting image differences. In the example shown in FIG. 8, the display image 8a shown in FIG. 8A and the display image 8b shown in FIG. 8B are set with 20-bit vertical and horizontal frames 8c and 8d, respectively. While moving, the images within the frame at each position are compared to automatically extract differences.

FIG. 9 is a flow chart showing the flow of processing in the link mode executed by server 100 in this embodiment. As described above, the processing shown in FIG. 9 is executed by the control device 102 as a program that is activated when the link mode is selected by the user while a plurality of display files are displayed on the multi-display screen.

In step S10, as described above, the control device 102 aligns the images in order to synchronize the operations of the multiple screens. After that, the process proceeds to step S20.

In step S20, the control device 102 determines whether or not the user has operated any one of the multiple screens during multi-display. If a negative determination is made in step S20, the process proceeds to step S50, which will be described later. On the other hand, if the determination in step S20 is affirmative, the process proceeds to step S30.

In step S30, as described above, the control device 102 performs processing for operating the display content within the operated screen according to the content of the user's operation. After that, the process proceeds to step S40.

In step S40, as described above, the control device 102 performs processing for synchronizing the display contents of screens other than the screen operated by the user with the operation of the screen operated by the user. After that, the process proceeds to step S50.

In step S50, the control device 102 determines whether or not the link mode has ended. For example, when the user instructs to end the link mode on the information terminal 200, or when the user instructs to switch from the link mode to the non-link mode, the control device 102 receives an instruction from the user to end the multi-screen display program. It is sufficient to determine that the link mode has ended when the If a negative determination is made in step S50, the process returns to step S20. On the other hand, if the determination in step S50 is affirmative, the process ends.

According to the embodiment described above, the following effects can be obtained.
(1) The control device 102 arranges and displays a plurality of screens displaying a 360-degree image or a 3D model, receives an operation for one of the displayed screens, and performs an operation according to the received operation content. The 360-degree image or the 3D model within the manipulated screen is operated, and the 360-degree image or 3D model within the other screen is operated in synchronization with the operation within the operated screen. This allows the user to compare information while manipulating 360-degree images and 3D models that are displayed side by side.

(2) The control device 102 aligns the images, controls the rotation angle according to the operation content, and rotates the 360-degree images or the three-dimensional models displayed on each of the plurality of screens, and rotates the image vertically according to the operation content. By performing movement control in the direction and the back and front direction, the operation within the operated screen and the other operations within the screen are synchronized. This makes it possible to synchronize display operations in a plurality of screens displayed side by side from the same viewpoint.

(3) The control device 102 receives selection of a 360-degree image file or a 3D model file to be displayed, receives selection of a file to be displayed side by side with the selected file, and selects the selected file. They are read out from the storage medium and displayed side by side. This allows the user to select an image file to be displayed on the multi-screen.

(4) The control device 102 displays a figure indicating that the current mode is the link mode when the link mode is selected from among the link mode in which the operations in a plurality of screens are synchronized and the non-link mode in which they are not synchronized. and when unlinked mode is selected, a graphic is displayed to indicate that the current mode is unlinked mode. This allows the user to graphically grasp whether the current mode is the link mode or the non-link mode.

(5) The control device 102 compares the 360-degree images or 3D models displayed in a plurality of screens, extracts the difference between the displayed images, and changes the display color of the difference. I made it This allows the user to easily grasp the difference between the images displayed in the multi-display.

-Modified example-
Note that the multi-screen display system of the embodiment described above can be modified as follows.
(1) In the embodiment described above, the multi-screen display system 10 includes the server 100 and the information terminal 200, and the server 100 and the information terminal 200 are connected via a communication line. explained. However, the configuration of the multi-screen display system 10 is not limited to the example shown in FIG.

(2) In the above-described embodiment, when the user operates screen 1 on the screen in the link mode shown in FIG. explained. However, in the example shown in FIG. 7A, the screen operated by the user is not limited to the screen 1 . For example, when the user operates screen n, the actions in other screens may be synchronized with the actions in screen n.

It should be noted that the present invention is not limited to the configurations in the above-described embodiments as long as the characteristic functions of the present invention are not impaired. Also, the configuration may be a combination of the above-described embodiment and a plurality of modifications.

10 Multi-Screen Display System 100 Server 101 Communication Interface 102 Control Device 103 Storage Medium 200 Information Terminal 201 Operation Member 202 Communication Interface 203 Control Device 204 Storage Medium 205 Display Device

Claims (18)

multi-screen display means for arranging and displaying a plurality of screens displaying 360-degree images or three-dimensional models;
an operation receiving means for receiving an operation for one screen among the plurality of screens displayed by the multi-screen display means;
an operation means for operating a 360-degree image or a three-dimensional model in the operated screen according to the operation content received by the operation reception means;
a synchronizing means for operating a 360-degree image or a three-dimensional model in the other screen displayed by the multi-screen display means in synchronization with the operation in the operated screen. system. The multi-screen display system according to claim 1,
The synchronization means aligns images between 360-degree images or three-dimensional models displayed on each of a plurality of screens displayed by the multi-screen display means, controls a rotation angle according to operation content, and A multi-screen display system characterized by synchronizing the operation in the operated screen with the operation in the other screen by performing movement control in the vertical direction and in the back and front direction according to the operation content. In the multi-screen display system according to claim 1 or 2,
The operation received by the operation receiving means is an operation of enlargement, reduction, or rotation in the case of the 360-degree image, and an operation of enlargement, reduction, rotation, horizontal movement, or vertical movement in the case of the three-dimensional model. A multi-screen display system characterized by: In the multi-screen display system according to claim 1 or 2,
display file selection receiving means for receiving selection of a 360-degree image file or a 3D model file to be displayed;
a multi-display file selection receiving means for receiving a selection of a file to be displayed side by side with the file selected by the display file selection receiving means;
The multi-screen display means reads the file selected by the display file selection acceptance means and the file selected by the multi-display file selection acceptance means from a storage medium and displays them side by side. display system. In the multi-screen display system according to claim 1 or 2,
A figure having a link mode for synchronizing operations in a plurality of screens displayed by the multi-screen display means and a non-link mode for not synchronizing operations, and indicating that the current mode is the link mode when the link mode is selected. and mode display means for displaying a graphic indicating that the current mode is the non-link mode when the non-link mode is selected. In the multi-screen display system according to claim 1 or 2,
360-degree images or three-dimensional models displayed in a plurality of screens displayed by the multi-screen display means are compared with each other, points of difference between the displayed images are extracted, and display colors of the points of difference are changed. A multi-screen display system, further comprising difference extracting means. multi-screen display means for arranging and displaying a plurality of screens displaying 360-degree images or three-dimensional models;
an operation receiving means for receiving an operation for one screen among the plurality of screens displayed by the multi-screen display means;
an operation means for operating a 360-degree image or a three-dimensional model in the operated screen according to the operation content received by the operation reception means;
a synchronizing means for operating a 360-degree image or a three-dimensional model in the other screen displayed by the multi-screen display means in synchronization with the operation in the operated screen. Device. In the multi-screen display device according to claim 7,
The synchronization means aligns images between 360-degree images or three-dimensional models displayed on each of a plurality of screens displayed by the multi-screen display means, controls a rotation angle according to operation content, and A multi-screen display device characterized by synchronizing the operation in the operated screen with the operation in the other screen by performing movement control in the vertical direction and in the back and front direction according to the content of the operation. In the multi-screen display device according to claim 7 or 8,
The operation received by the operation receiving means is an operation of enlargement, reduction, or rotation in the case of the 360-degree image, and an operation of enlargement, reduction, rotation, horizontal movement, or vertical movement in the case of the three-dimensional model. A multi-screen display device characterized by: In the multi-screen display device according to claim 7 or 8,
display file selection receiving means for receiving selection of a 360-degree image file or a 3D model file to be displayed;
a multi-display file selection receiving means for receiving a selection of a file to be displayed side by side with the file selected by the display file selection receiving means;
The multi-screen display means reads the file selected by the display file selection acceptance means and the file selected by the multi-display file selection acceptance means from a storage medium and displays them side by side. display device. In the multi-screen display device according to claim 7 or 8,
A figure having a link mode for synchronizing operations in a plurality of screens displayed by the multi-screen display means and a non-link mode for not synchronizing operations, and indicating that the current mode is the link mode when the link mode is selected. and further comprising mode display means for displaying a figure indicating that the current mode is the non-link mode when the non-link mode is selected. In the multi-screen display device according to claim 7 or 8,
360-degree images or three-dimensional models displayed in a plurality of screens displayed by the multi-screen display means are compared with each other, points of difference between the displayed images are extracted, and display colors of the points of difference are changed. A multi-screen display device, further comprising difference extracting means. a multi-screen display procedure for arranging and displaying a plurality of screens displaying 360-degree images or three-dimensional models;
an operation reception procedure for accepting an operation for one screen among the plurality of screens displayed in the multi-screen display procedure;
an operation procedure for operating a 360-degree image or a three-dimensional model in the operated screen according to the operation content received in the operation reception procedure;
A multi-screen display program for causing a computer to execute a synchronization procedure for operating a 360-degree image or a three-dimensional model in other screens displayed in the multi-screen display procedure in synchronization with the operation in the operated screen. . In the multi-screen display program according to claim 13,
The synchronization procedure includes aligning the images, controlling the rotation angle according to the operation content, and operating the 360-degree images or three-dimensional models displayed on each of the plurality of screens displayed in the multi-screen display procedure. A multi-screen display program characterized by synchronizing the operation in the operated screen with the operation in the other screen by performing movement control in the vertical direction and in the back and front direction according to the contents. In the multi-screen display program according to claim 13 or 14,
The operation accepted in the operation acceptance procedure is an operation of enlargement, reduction, or rotation in the case of the 360-degree image, and an operation of enlargement, reduction, rotation, horizontal movement, or vertical movement in the case of the three-dimensional model. A multi-screen display program characterized by: In the multi-screen display program according to claim 13 or 14,
a display file selection receiving procedure for receiving a selection of a 360-degree image file or a 3D model file to be displayed;
a multi-display file selection receiving procedure for receiving a selection of a file to be displayed side by side with the file selected in the display file selection receiving procedure;
The multi-screen display procedure is characterized in that the file selected in the display file selection reception procedure and the file selected in the multi-display file selection reception procedure are read from a storage medium and displayed side by side. display program. In the multi-screen display program according to claim 13 or 14,
Provided with a link mode for synchronizing operations in a plurality of screens displayed in the multi-screen display procedure and a non-link mode for non-synchronization, and a figure indicating that the current mode is the link mode when the link mode is selected. and displaying a figure indicating that the current mode is the non-link mode when the non-link mode is selected. In the multi-screen display program according to claim 13 or 14,
Compare the 360-degree images or three-dimensional models displayed in the multiple screens displayed in the multi-screen display procedure, extract the differences between the displayed images, and change the display color of the difference. A multi-screen display program, further comprising a difference extraction procedure.