JP6861353B1 - Programs, methods, and information processing equipment - Google Patents

Abstract

PROBLEM TO BE SOLVED: To provide a program, a method, and an information processing device capable of sharing a screen among a plurality of devices and switching between them by a simple operation. SOLUTION: Information for making an image that can be displayed on a display unit in response to a first operation (for example, scrolling operation) to an input unit among predetermined images identifiable by another information processing device is output. , An image that receives information for making it possible to identify an image that can be displayed on the display unit of the other information processing device in response to the first operation on the input unit of the other information processing device, and displays the information on the display unit. In response to the second operation on the input unit, the first image corresponding to the first operation on the input unit of the information processing apparatus among the predetermined images, and the second image specified from the received information. Switch to one of multiple types of images, including. [Selection diagram] Fig. 5

Description

The present invention relates to programs, methods, and information processing devices.

Conventionally, there has been known a technique for sharing a screen of one terminal device with another terminal device among a plurality of terminal devices. Specifically, the screen of the terminal device A is displayed by performing a screen sharing operation on one terminal device (hereinafter referred to as terminal device A) and performing a consent operation on another terminal device (hereinafter referred to as terminal device B). Some were displayed and shared on the terminal device B (for example, FIG. 3 of Patent Document 1).

JP-A-2017-175638

However, in the conventional technique, when the screen displayed on the terminal device B is displayed on the terminal device A and shared, the operation of canceling the sharing from the terminal device A to the terminal device B or the operation from the terminal device B Screen sharing operation and consent operation in the terminal device A are required. That is, a sharing operation is required each time sharing is performed. Therefore, when the screen is shared in both directions by a plurality of terminal devices, the operation may be complicated.

Further, for example, the user who owns the terminal device A cannot display the image of the terminal device A itself or the image of the terminal device B at the user's will and timing. Therefore, there is a risk that the convenience cannot be improved.

The present invention has been conceived in view of such circumstances, and an object of the present invention is to provide a program, a method, and an information processing device capable of sharing a screen among a plurality of devices and switching between them by a simple operation. To provide.

According to an aspect of one embodiment shown in the present disclosure, a program executed in an information processing device (eg, a user terminal) including a processor, a memory, an input unit, and a display unit (for example, a program included in a WEB page). And
In the information processing device
A step of outputting information for enabling another information processing apparatus to identify an image that can be displayed on the display unit in response to a first operation (for example, scrolling operation) on the input unit among predetermined images (for example). For example, in step S14),
A step of receiving information for making it possible to identify an image that can be displayed on the display unit of the other information processing device in response to the first operation on the input unit of the other information processing device (for example, step S15). ,
An image to be displayed on the display unit is displayed in response to a second operation (for example, a switching operation) on the input unit, and in response to a first operation of the information processing device on the input unit of the predetermined image. The step of switching to one of a plurality of types of images including the image of 1 and the second image specified from the information received by the receiving step (for example, steps S24 to S30) is executed.

According to this configuration, the image corresponding to the first operation displayed on each of the information processing device and the other information processing device can be shared, and the second operation on the information processing device (user's intention). ), The image displayed in the information processing device and the image displayed in another information processing device can be switched. As a result, the image displayed on the other information processing device can be recognized at the timing of the user who owns the information processing device, and smooth communication with the user who owns the other information processing device can be achieved.

Further, the plurality of types of images include a third image which is a video call image between the information processing device and the other information processing device (see FIG. 5 and the like).

According to this configuration, the state of the user who owns another information processing device can be grasped, and smoother communication can be achieved.

Further, the plurality of types of images include a third image which is a chat image between the information processing device and the other information processing device.

According to this configuration, it is possible to chat with a user who owns another information processing device, so that smoother communication can be achieved.

In addition, the plurality of types of images include a predetermined third image that is different from the first image and the second image.

According to this configuration, it is possible to reduce the processing load and prevent the display of one's own video on another information processing device, for example, according to the convenience and preference of the user.

Further, when the predetermined icon image that accepts the operation for permitting the display of the first image in another information processing device is switched to the first image and the second image, it is displayed. Instead, the step of displaying when the image is switched to the third image is executed.

According to this configuration, it is possible to select whether or not to display the first image on another information processing device according to the convenience and preference of the user. Further, since the icon image for selection is not displayed when the first image and the second image are switched to, it is possible to prevent the visibility of the first image and the second image from being lowered. ..

Further, the switching step regulates the switching from the first image to the second image and the switching from the second image to the first image, and the first image and the first image. Switching between the third image and switching between the second image and the third image is allowed.

According to this configuration, it is possible to switch directly from the first image to the second image without going through the third image, or to switch directly from the second image to the first image without going through the third image. Therefore, it is possible to reduce the misrecognition of the first image as the second image and the misrecognition of the second image as the first image.

Further, regardless of the type of the image switched by the switching step, a plurality of types of corresponding images corresponding to each of the plurality of types of images are displayed, and the corresponding image corresponding to the image switched by the switching step is displayed. The step of displaying (for example, reverse display) in a mode different from that of the corresponding image of is executed.

According to this configuration, the type of image currently being switched can be notified to the user in an easy-to-understand manner from the aspect of the corresponding image, and it is possible to prevent erroneous recognition.

Further, the plurality of types of corresponding images are arranged and displayed in a predetermined direction (for example, a horizontal direction), and are displayed.
The input unit is a touch panel.
The second operation is an operation of moving the touch panel along the same direction as the predetermined direction (for example, a swipe operation in the left-right direction).

According to this configuration, the second operation can be intuitively understood from the arrangement of the corresponding images.

Further, the information output by the output step and the information received by the reception step are for specifying a region corresponding to the first operation to the input unit of the predetermined image, respectively. Area information.

According to this configuration, the amount of data of the information to be output or received can be reduced, and the processing load for specifying the image can be reduced.

Further, the input unit is a touch panel.
The first operation includes an operation on the touch panel.
The information output by the output step and the information received by the receiving step are operation mode information for making it possible to specify the mode of operation for the touch panel, respectively.

According to this configuration, the amount of data of the information to be output or received can be reduced, and the processing load for specifying the image can be reduced.

Further, the information output by the output step and the information received by the reception step can each display an image of a region corresponding to the first operation to the input unit of the predetermined image. It is the image information for.

According to this configuration, the processing load for identifying the image can be reduced.

According to an aspect of one embodiment presented in the present disclosure, a method performed in an information processing apparatus including a processor, memory, input unit, and display unit.
A step of outputting information for enabling another information processing apparatus to identify an image that can be displayed on the display unit in response to a first operation (for example, scrolling operation) on the input unit among predetermined images (for example). For example, in step S14),
A step of receiving information for making it possible to identify an image that can be displayed on the display unit of the other information processing device in response to the first operation on the input unit of the other information processing device (for example, step S15). ,
An image to be displayed on the display unit is displayed in response to a second operation (for example, a switching operation) on the input unit, and in response to a first operation of the information processing device on the input unit of the predetermined image. It includes a step (for example, steps S24 to S30) of switching to one of a plurality of types of images including the image of 1 and the second image specified from the information received by the receiving step.

According to this configuration, the image corresponding to the first operation displayed on each of the information processing device and the other information processing device can be shared, and the second operation on the information processing device (user's intention). ), The image displayed in the information processing device and the image displayed in another information processing device can be switched. As a result, the image displayed on the other information processing device can be recognized at the timing of the user who owns the information processing device, and smooth communication with the user who owns the other information processing device can be achieved.

According to an aspect of one embodiment shown in the present disclosure, an information processing apparatus including a processor, a memory, an input unit, and a display unit.
A step of outputting information for enabling another information processing apparatus to identify an image that can be displayed on the display unit in response to a first operation (for example, scrolling operation) on the input unit among predetermined images (for example). For example, in step S14),
A step of receiving information for making it possible to identify an image that can be displayed on the display unit of the other information processing device in response to the first operation on the input unit of the other information processing device (for example, step S15). ,
An image to be displayed on the display unit is displayed in response to a second operation (for example, a switching operation) on the input unit, and in response to a first operation of the information processing device on the input unit of the predetermined image. Information processing including a step (for example, steps S24 to S30) of switching to one of a plurality of types of images including the image of 1 and the second image specified from the information received by the receiving step. apparatus.

According to this configuration, the image corresponding to the first operation displayed on each of the information processing device and the other information processing device can be shared, and the second operation on the information processing device (user's intention). ), The image displayed in the information processing device and the image displayed in another information processing device can be switched. As a result, the image displayed on the other information processing device can be recognized at the timing of the user who owns the information processing device, and smooth communication with the user who owns the other information processing device can be achieved.

It is a figure which shows the hardware configuration example of the communication system. It is a figure which shows the hardware configuration example of a user terminal. It is a figure for demonstrating the flow until starting WebRTC (including for example, P2P communication) between a plurality of user terminals. An example of a touch screen display screen on the user terminal X is shown. An example of a touch screen display screen on the user terminal X during P2P communication and an example of a touch screen display screen on the other terminal user terminal Y during the own terminal mode are shown. An example of a touch screen display screen on the user terminal X during P2P communication and an example of a touch screen display screen on the other terminal user terminal Y during the own terminal mode are shown. It is a flowchart for demonstrating the drawing process. It is a flowchart for demonstrating an image identification process. It is a flowchart for demonstrating the display process.

The communication system according to the present disclosure is a system in which screens are shared among information processing devices owned by a plurality of users and can be switched by a simple operation. Hereinafter, embodiments relating to the communication system will be described with reference to the drawings. It should be noted that the present invention is not limited to the following examples, and is indicated by the scope of claims, and it is intended that all modifications within the meaning and scope equivalent to the scope of claims are included in the present invention. To. In the following description, the same elements are designated by the same reference numerals in the description of the drawings, and duplicate description will not be repeated.

<Hardware configuration of communication system>
FIG. 1 is a diagram showing a hardware configuration example of the communication system 1. The communication system 1 includes a plurality of user terminals 100, a WEB server 200, and a signaling server 300. The plurality of user terminals 100, the WEB server 200, and the signaling server 300 can each communicate and connect via the network 2, and can transmit and receive information (data) in both directions. The network 2 is composed of various mobile communication systems constructed by the Internet and a wireless base station (not shown). Examples of this mobile communication system include so-called 4G and 5G mobile communication systems, LTE (Long Term Evolution), and wireless networks (for example, Wi-Fi (registered trademark)) that can be connected to the Internet by a predetermined access point. Be done.

The user terminal 100 (computer, information processing device) may be a mobile terminal such as a smartphone, a feature phone, a PDA (Personal Digital Assistant), or a tablet computer, and may be a general-purpose computer such as a stationary personal computer. You may. In addition, the user terminal 100 implements (installs) a WEB browser for browsing a WEB page.

FIG. 2 is a diagram showing a hardware configuration example of the user terminal 100. As shown in the figure, the user terminal 100 captures an image taken by a processor 101 constituting a control processing unit, a RAM 102 constituting a storage unit, a ROM 103, an EEPROM 104, and an image sensor (for example, a CMOS (Complementary Metal-Oxide Semiconductor) sensor). A camera 105 that converts a digital signal, a microphone 106 that converts mechanical vibrations such as a diaphragm generated by sound waves into an electric signal, a speaker 107 that outputs sound, and a display unit that displays an image are provided to perform touch operations on the display unit. It includes a touch screen 108 for detection, an operation detection unit 109 for detecting a user pressing operation on a predetermined button (operation unit), a communication IF 110, and an input / output IF 111. These configurations included in the user terminal 100 are electrically connected to each other by a communication bus. The communication IF 110 includes a NIC (Network Interface Controller), a wireless LAN (Local Area Network) chip, and the like. The user terminal 100 may be provided with an input / output IF capable of connecting a display (display unit) configured separately from the user terminal 100 main body instead of or in addition to the touch screen 108.

The processor 101 includes a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a GPU (Graphics Processing Unit), and the like. For example, the processor 101 reads the program stored in the EPROM 104 onto the RAM 102, receives data from the touch screen 108, the operation detection unit 109, the communication IF 110, etc. according to the instruction of the program, and receives the acquired data in the procedure specified in the program. After calculation and processing, the calculated and processed data is output to the EEPROM 104, the touch screen 108, the communication IF 110, and the like.

Returning to FIG. 1, the WEB server 200 and the signaling server 300 (computer, information processing device) may be general-purpose computers such as workstations or personal computers, respectively. The WEB server 200 and the signaling server 300 include at least a processor, a memory, a storage, a communication IF, and an input / output IF, and are electrically connected to each other by a communication bus.

The WEB server 200 is a server that provides a predetermined WEB page. The WEB page may be a page of one layer, or may be a page of a plurality of layers that transitions to the linked page by clicking a predetermined link in the page. The WEB page provided by the WEB server 200 is provided with, for example, a WEB page including a program (for example, a script, JavaScript (registered trademark), etc.) for providing a WebRTC (Web Real-time Communications) service. By accessing the WEB server 200 using the WEB browser mounted on the user terminal 100, the user can provide communication services such as video calling and screen sharing using WebRTC technology based on the program included in the WEB page. Can receive.

The signaling server 300 is a server that mediates the exchange of information for establishing RTC Peer Connection between the WEB browsers implemented in each of the plurality of user terminals 100. By accessing the WEB server 200 using the WEB browser, the user terminal 100 can receive the html file constituting the WEB page and display the WEB page, and can display the WEB page by using the WebRTC technology for video communication / screen. By downloading a program file for sharing or the like, it is possible to make a video call, screen sharing, or the like in real time with another user terminal 100.

The user terminal 100 in the present embodiment is based on a downloaded program, for example, between a plurality of user terminals 100 in which RTC Peer Connection is established via a signaling server 300 and Peer to Peer (P2P) communication is possible. In real time, for example, a video call for transmitting and receiving an image captured by the camera 105 and a voice input to the microphone 106 can be executed. Further, the user terminal 100 in the present embodiment can transmit and receive, for example, information for enabling the WEB page image actually viewed on the user terminal 100 to be displayed on another user terminal 100 by the downloaded program. In addition, a plurality of types of images (for example, an image of the own terminal and an image of the other terminal) are displayed on the touch screen in response to a switching operation (for example, a swipe operation in the left-right direction) from the user who owns the user terminal 100. Screen sharing that switches to either an image, a video call image, etc.) can be executed. In the following, the flow from the establishment of P2P communication to the video call and screen sharing will be described.

FIG. 3 is a diagram for explaining a flow until WebRTC (including, for example, P2P communication) is started between a plurality of user terminals. FIG. 3 shows a flow until P2P communication is established between the user terminal X owned by the user X among the plurality of user terminals and the user terminal Y owned by the user Y. The horizontal axis of FIG. 3 shows the configurations of the user terminal X, the user terminal Y, the WEB server 200, the signaling server 300, and the like, and the vertical axis shows the processing contents in each configuration according to the passage of time. .. Note that FIG. 3 is only an example of the flow until WebRTC (including, for example, P2P communication) is started between a plurality of user terminals, and if communication is possible between the plurality of user terminals, the following is shown below. It is not limited to what is explained. Further, FIG. 4 shows an example of a display screen of the touch screen 108 on the user terminal X.

First, at t1, the Web browser is launched on the user terminal X, and the WEB page request is transmitted to the WEB server 200. The WEB page request includes communicable information (for example, media type (audio, video), codec format, IP address, port number, data transfer) for the user terminal X to enable P2P communication with another user terminal. Information such as the protocol to be used (TCP, UDP, etc.), encryption key, etc. (information for establishing RTC Peer Connection), etc. can be specified). Upon receiving the WEB page request, the WEB server 200 transmits communicable information to the signaling server 300.

Further, at t2, the WEB page response is transmitted from the WEB server 200 to the user terminal X. The WEB page response includes information (html file, image data, etc.) for displaying a predetermined image. As a result, the WEB page provided by the WEB server 200 is displayed on the touch screen 108 of the user terminal X. In addition, the WEB page response includes a program file (script, JavaScript (registered trademark), etc.). As a result, the user terminal 100 that has accessed the WEB server 200 can establish P2P communication and make a video call or screen sharing by executing the program included in the WEB page. The program file included in the WEB page response and saved in the RAM 102 is destroyed when the access to the WEB server 200 is terminated, but the present invention is not limited to this, and the program file is not destroyed according to the user's settings and the like next time. It may be maintained so that it can be used at the time of access.

FIG. 4A shows an example of a screen when the WEB server 200 is accessed and the WEB page is displayed on the touch screen 108. In the screen upper area 108a of the touch screen 108, the current time (for example, "18:21"), the access destination address (for example, "abc.com"), and the accessing user name (here, user X) are displayed. Therefore, "X") and the icon 401 for creating a room, which will be described later, are displayed.

A part of the WEB page (predetermined image) (screen at the start of browsing) is displayed in the screen center area 108b of the touch screen 108. In the present embodiment, the page of the shopping site is illustrated, but the type of the WEB page is not limited to this. The user terminal X sets an arbitrary part of the WEB page provided by the WEB server 200 according to a scroll operation (first operation) such as a swipe operation in the vertical direction or an operation on the scroll bar with respect to the screen center area 108b. It can be displayed in the center area 108b of the screen. As a result, the user can browse the part of the WEB page that he / she wants to browse by scrolling the center area 108b of the screen.

Further, in the upper part of the central area 108a of the screen, the other party terminal notification image 402, the video call notification image 403, and the own terminal notification image 404 are superimposed and displayed on the WEB page. The other party terminal notification image 402, the video call notification image 403, and the own terminal notification image 404 are images for notifying the display mode of the screen center area 108b, respectively. There are multiple types of display modes, including the own terminal mode for displaying the image of the own terminal, the other terminal mode for displaying the image of the other terminal, and the video call mode for displaying the image of the video call. Display mode is provided. The user terminal 100 in the present embodiment is one of the user terminals 100 according to a switching operation (second operation) such as a swipe operation in the left-right direction or an operation to an icon described later, based on a program included in the WEB page. You can switch to the display mode and display the image corresponding to the display mode. When the display of the WEB page is started, the display mode is set to the own terminal mode. Therefore, in the screen center area 108a, the screen of the own terminal (a part of the image of the WEB page) is displayed, and the screen of the own terminal is displayed in reverse by displaying the notification image 404 of the own terminal in reverse. Is notified.

In the lower screen area 108c of the touch screen 108, a call switching icon for switching between a video call in which one's own image can be displayed on the other party's terminal and a voice call in which the default screen is displayed without displaying one's own image on the other party's terminal. 405a, an end icon 405b for terminating P2P communication, an input field 406 for chatting, and various icons 407 to 411 of the WEB browser are displayed. When P2P communication is established with the other party terminal, the call switching icon 405 can be touch-operated to switch from a video call to a voice call or from a voice call to a video call, and the end icon 405b can be touch-operated. It is possible to end P2P communication (video call) and display the image of the own terminal, and by touching the input field 406, it is possible to input characters, stamps, etc., and the input characters, etc. can be input to the other terminal. Can be sent to.

The various icons 407 to 411 are icons displayed by the function of the WEB browser. Of these, the icon 409 is a sharing icon that is operated when sharing a sharing link for starting a video call, screen sharing, or the like with the other party terminal. The sharing link contains, for example, character string information for identifying a user-created room-specific address. By clicking the sharing link, you can access the WEB server 200 and enter the created room. As described above, when accessing the WEB server 200 and displaying the WEB page, a plurality of icons, images, and the like are displayed in addition to the WEB page based on the program included in the WEB page.

Returning to FIG. 3, the signaling server 300 that received the communicable information at t1 performs a process for issuing an SDP (Session Description Protocol) to the user terminal X, and at t3, the issued SDP is the user terminal. It is transmitted to X and stored in a predetermined area of the RAM 102. In SDP, for example, information such as media type (audio, video), codec format, IP address and port number, protocol used for data transfer (TCP, UDP, etc.), encryption key, etc. (RTC Peer Connection established) Information to do) is included.

Further, when the user terminal X is displaying the WEB page provided by the WEB server 200, as shown in t4, the user terminal X satisfies the sharing start condition and transmits the sharing link to another user. It is possible to establish P2P communication with a user terminal owned by the other user.

4 (b) to 4 (d) show an example of a screen from when the sharing start condition is satisfied to when the sharing link is transmitted to another user (user Y in FIG. 4). First, the icon 401 shown in FIG. 4A is touch-operated to display a screen including the room creation icon 420 such as “Create room” shown in FIG. 4B.

Next, the room creation icon 420 is touch-operated to create a unique room to be used for P2P communication connection, and an address unique to the room (for example, an encryption key, encrypted text, etc.) is set to a predetermined URL. A room-specific URL added to the end of the word) is issued, and a sharing link containing information for identifying the address is generated. Further, by touch-operating the room creation icon 420, the display mode can be switched to the video call mode in preparation for the start of P2P communication. As a result, the image of the screen center area 108b is switched from the image of the own terminal to the image of the video call. Along with this, in the screen of FIG. 4C, the screen center area 108b is switched from the own terminal screen displaying the WEB page to a screen including a character string such as "AUDIO". At the stage of FIG. 4C, since P2P communication has not yet been established and the video call has not started, a screen including a character string such as "AUDIO" is displayed as the video call screen. Further, when the video call mode is switched, the video call notification image 403 is highlighted.

P2P communication is established by notifying the other terminal of the sharing link and causing it to be clicked. FIG. 4C describes an example of notifying the other terminal of the sharing link by using the sharing function of the WEB browser. As shown in FIG. 4 (c), the sharing icon 409 of the WEB browser is touch-operated to display the screen shown in FIG. 4 (d) by the function of the WEB browser. In the screen shown in FIG. 4D, the window 430 is superimposed and displayed on the lower half of the screen. In window 430, app icons 431 to 434 corresponding to apps that can be used to send a sharing link containing information for identifying the created room-specific address to other specific users are displayed. The user X activates the application by touching one of the application icons, selects the destination of another user (user Y in this case), and transmits the sharing link. When the transmission of the sharing link is completed by the application, the screen shown in FIG. 4C is displayed again on the touch screen 108, and the user Y waits for a connection. In the connection waiting state, the screen shown in FIG. 4C is displayed once, but during this time as well, the switching operation is performed to switch to the own terminal mode, display the image of the own terminal, and display the WEB page. It may be viewable and automatically switched to the video call mode when the connection of the user Y is completed.

As described above, the user X establishes the sharing start condition by operating the icon 401 → creating a room (see FIG. 4 (b)) → operating the shared icon 409 (see FIG. 4 (c)) → operating the app icon to apply the application. Can be used to send a sharing link to other users. The method of notifying the sharing link is not limited to the one shown in FIGS. 4 (c) and 4 (d), and may be notified by e-mail or the like. When the room is created, the user terminal X owned by the user X is in a state of accessing the URL unique to the room.

In the user terminal Y that has received the sharing link, the sharing link is touch-operated (clicked) to access the WEB server 200 and enter the room created by the user X (access the URL unique to the room). Then, the process for establishing P2P communication with the user terminal X is started. When the shared link is touch-operated, as shown in t5 of FIG. 3, the Web browser is launched on the user terminal Y and the WEB page request is transmitted to the WEB server 200. The WEB page request includes communicable information for the user terminal Y to enable P2P communication with the user terminal X of the sharing address transmission source. Upon receiving the WEB page request, the WEB server 200 transmits communicable information to the signaling server 300.

Further, at t6, the WEB page response is transmitted from the WEB server 200 to the user terminal Y. The WEB page response also includes information for displaying the predetermined image described above and a program file (script, JavaScript (registered trademark), etc.). As a result, the WEB page provided by the WEB server 200 is displayed on the touch screen 108 of the user terminal Y (similar to FIG. 4A), and P2P communication can be established to enable video call and screen sharing. Become.

On the other hand, in the signaling server 300 that has received the communicable information at t5, a process for issuing SDP to the user terminal Y is performed, and at t7, the issued SDP is transmitted to the user terminal Y to determine the RAM 102. Stored in the area. Following t7, in t8, the received SDP is transmitted to the user terminal X of the sharing link transmission source via the signaling server 300, and as shown in t9, the SDP from the user terminal Y is transmitted in the user terminal X. be registered.

At t10 following t9, the user terminal X transmits the SDP of the user terminal X already stored at t3 to the user terminal Y via the signaling server 300, and as shown in t11, at the user terminal Y. , SDP from user terminal X is registered. As a result, the connection confirmation is executed between the user terminal X and the user terminal Y, the pair of the IP address and the port number that can communicate with P2P is determined, and the RTC Peer Connection is established. As a result, P2P communication is established between the user terminal X and the user terminal Y, a video call is started, and screen sharing becomes possible.

FIG. 4E shows a screen in which P2P communication is established and a video call with the other party user Y is started. When the P2P communication is started, the own terminal transmits the image data captured by the camera 105 and the voice data collected by the microphone 106 to the other terminal, and is also transmitted from the other terminal in the same manner. It is possible to receive the incoming image and audio data, display the image from the other terminal in the screen center area 108b, and output the audio from the speaker 107. The transmission and reception of image and audio data is performed at predetermined time intervals (for example, 50 msec). As a result, the video call screen can be displayed. FIG. 4E shows how an image (moving image) including the user Y is displayed on the user terminal X.

Further, when the P2P communication is started, the identification information for making it possible to identify the image (area) of the WEB page that can be displayed on the own terminal in response to the scroll operation in the own terminal mode on the own terminal. Is transmitted to the other terminal, and the specific information also transmitted from the other terminal is received.

The specific information transmitted on the own terminal is, for example, a specific position of the WEB page received when accessing the WEB server 200 (for example, the position of the upper left corner of the WEB page, the position where the coordinates are 0 on the x-axis and the y-axis, etc. ) Is the information for specifying the coordinate position of a predetermined position (for example, the position of the upper left corner) of the image displayed in the center area 108b of the screen in response to the scrolling operation in the own terminal mode. Further, the specific information received by the own terminal is, for example, a specific position of the WEB page received when the other terminal accesses the WEB server 200 (for example, a position where the coordinates are 0 in the upper left corner position, the x-axis and the y-axis). Etc.) as the base point, the coordinate position of the predetermined position (for example, the position of the upper left corner) of the image displayed in the screen center area 108b of the other party terminal in response to the scroll operation in the own terminal mode of the other party terminal. Information that identifies. Transmission and reception of specific information is performed every predetermined time (for example, 100 msec). As a result, not only the image viewed on the own terminal but also the image viewed on the other terminal can be specified in real time on the WEB page.

The user terminal 100 is set by switching to a display mode according to the switching operation based on the program included in the WEB page, and the image corresponding to the display mode can be displayed in the screen center area 108b of the user terminal 100. .. The user terminal 100 during P2P communication transmits at least image / audio data and identification information to the other terminal regardless of the set display mode, and identifies the image / audio data from the other terminal. Receive information and information.

5 and 6 show an example of a display screen of the touch screen 108 in the user terminal X during P2P communication and an example of a display screen of the touch screen 108 in the own terminal mode in the user terminal Y which is the other terminal.

FIG. 5 (a) shows an example of a display screen of the user terminal X, FIG. 5 (a1) is an example of a display screen in the partner terminal mode, and FIG. 5 (a2) is a display in the video call mode. It is an example of a screen, and FIG. 5 (a3) is an example of a display screen in the own terminal mode. When the set display mode is, for example, the video call mode, as shown in FIG. 5A2, the other user Y is imaged based on the image / voice data transmitted from the user terminal Y. The image (moving image) is displayed in the central area 108b of the screen, and the sound is output from the speaker 107. As a result, the video call screen is displayed.

When the display screen in the video call mode is displayed, for example, when a switching operation for swiping from right to left is accepted, the display mode is switched to the own terminal mode as shown in FIG. 5 (a3). The display screen in the own terminal mode is displayed, and the own terminal notification image 404 is switched to the highlighted state. FIG. 5A3 shows an example in which the screen at the start of browsing of the WEB page is displayed.

In the own terminal mode, it is possible to display an arbitrary part of the WEB page according to the scroll operation for the user terminal X which is the own terminal. Further, as described above, the user terminal 100 is a portion of the WEB page corresponding to the scroll operation in the own terminal mode at predetermined time intervals during P2P communication regardless of which display mode is set. (Image in the WEB page) is transmitted to the user terminal Y, which is the other terminal, for identification information that can be specified. As a result, the image of the portion browsed by the user X can be specified on the user terminal Y side in response to the scroll operation in the own terminal mode on the user terminal X. Even in the own terminal mode, the voice from the user Y is output from the speaker 107 based on the voice data transmitted from the user terminal Y.

Further, when the display screen in the own terminal mode is displayed, for example, when a switching operation for swiping from left to right is accepted, the display mode is switched to the video call mode as shown in FIG. 5 (a2). Then, the display screen in the video call mode is displayed, and the video call notification image 403 is switched to the highlighted state. As a result, the image from the other terminal is displayed in the center area 108b of the screen, and the sound is output from the speaker 107.

When the display screen in the video call mode is displayed, for example, when a switching operation for swiping from left to right is accepted, the display mode is switched to the remote terminal mode as shown in FIG. 5 (a1). The display screen in the remote terminal mode is displayed, and the remote terminal notification image 402 is switched to the highlighted state. As described above, the user terminal Y, which is the other terminal, is the user of the WEB page at predetermined time intervals during P2P communication, regardless of which display mode is set, as in the user terminal X. The specific information that can identify the part (image in the WEB page) corresponding to the scroll operation in the own terminal mode in the terminal Y is transmitted to the user terminal X which is the other terminal. Therefore, the image of the portion of the WEB page viewed by the user Y can be specified on the user terminal X side in response to the scroll operation in the own terminal mode of the user terminal Y.

As shown in FIG. 5A, an example is shown in which the screen at the start of browsing of the WEB page is displayed on the user terminal Y, and the user terminal provides specific information that can identify the screen. Send to X. Therefore, when the remote terminal mode is set as the display mode of the user terminal X, as shown in FIG. 5 (a1), the browsing of the WEB page is performed based on the specific information transmitted from the user terminal Y. The start screen is displayed in the screen center area 108b of the user terminal X. As a result, the image of the portion of the WEB page viewed by the user Y can be specified on the user terminal X side according to the operation in the own terminal mode on the user terminal Y. Even in the remote terminal mode, the voice from the user Y is output from the speaker 107 based on the voice data transmitted from the user terminal Y.

Further, when the display screen in the other party terminal mode is displayed, for example, when a swipe operation from right to left is accepted, the display mode is switched to the video call mode, and the display screen in the video call mode is displayed. At the same time, the video call notification image 403 is switched to the highlighted state. As a result, the image from the other terminal is displayed in the center area 108b of the screen, and the sound is output from the speaker 107. As described above, the image displayed in the screen center area 108b of the user terminal 100 is switched to one of the image of the own terminal, the image of the other terminal, and the image of the video call by the switching operation to the user terminal 100. be able to.

It should be noted that even if an operation of further swiping from right to left is performed on the display screen in the own terminal mode, the display mode cannot be switched to another display mode such as the other terminal mode. Further, even if an operation of further swiping from left to right is performed on the display screen in the partner terminal mode, the display mode cannot be switched to another display mode such as the own terminal mode. That is, switching from the display screen in the own terminal mode to the display screen in the other terminal mode and switching from the display screen in the other terminal mode to the display screen in the own terminal mode are regulated.

5 (b) and 5 (b') show an example of a display screen when the part viewed by the user Y changes. In FIG. 5 (b'), when the user terminal Y is in the own terminal mode, the user Y scrolls to display the part of the WEB page including the product image of "Today's push", for example, of the user terminal Y. An example of a display screen displayed in the screen center area 108b is shown. As described above, the user terminal Y provides identification information for identifying an image that can be displayed in the screen center area 108b of the user terminal Y in the own terminal mode for a predetermined time regardless of the set display mode. It is transmitted every time, and the user terminal X receives the specific information. Therefore, by switching the display mode of the user terminal X to the remote terminal mode, the user X sets the screen center area 108b of the user terminal X in the screen center area 108b of the user terminal X as shown in FIG. 5 (b1). The part including the product image and the like is displayed, and the part browsed by the user Y can be grasped in real time.

FIG. 6 shows an example of a display screen when the part browsed by the user X and the part browsed by the user Y change. In FIG. 6 (c'), when the user terminal Y is in the own terminal mode, the user Y scrolls to display a part of the WEB page including, for example, a coupon image of "recommended coupon for you". An example of a display screen displayed in the screen center area 108b of the above is shown. In this case, the user X switches the display mode of the user terminal X to the remote terminal mode, and as shown in FIG. 6 (c1), displays the "recommended for you" in the screen center area 108b of the user terminal X on the WEB page. The part including the coupon image of the "coupon" is displayed, and the part browsed by the user Y can be grasped in real time.

Similarly, in FIG. 6 (c3), when the user terminal X is in the own terminal mode, the user X scrolls and the part of the WEB page including the product image of, for example, "new product!" Is included in the user terminal X. An example of a display screen displayed in the screen center area 108b of the above is shown. As described above, the user terminal X transmits the identification information for specifying the image displayed in the screen center area 108b at predetermined time intervals, and the user terminal Y receives the identification information. Therefore, although not shown, the user Y switches the display mode of the user terminal Y to the other terminal mode, so that the product image of the "new product!" Of the WEB page is displayed in the screen center area 108b of the user terminal Y. The part including the above is displayed, and the part browsed by the user X can be grasped in real time.

Similarly for FIGS. 6 (d) and 6 (d'), the user X switches the display mode of the user terminal X to the remote terminal mode, and as shown in FIG. 6 (d1), the user terminal X of the user terminal X. A part of the WEB page including the product image of the "recently checked product" is displayed in the central area 108b of the screen, and the part browsed by the user Y can be grasped in real time. Further, although not shown, the user Y switches the display mode of the user terminal Y to the other terminal mode, so that the coupon of the "recommended coupon for you" in the WEB page is displayed in the screen center area 108b of the user terminal Y. A part including an image or the like (the same part as in FIG. 6 (d3)) is displayed, and the part browsed by the user X can be grasped in real time.

The user terminal 100 controls the display screen as illustrated in FIGS. 4 to 6 by performing various processes based on the program included in the WEB page described above. In the following, as an example of the process executed based on the program included in the WEB page, the drawing process for drawing the image to be displayed on the touch screen 108 in the RAM 102 and the drawing process for drawing the image to be displayed on the touch screen 108 and the WEB page on each of the plurality of user terminals 100 Of these, the image identification process for specifying the image (location) that can be displayed in the screen center area 108b in the own terminal mode, and the image corresponding to the display mode by switching to the display mode according to the switching operation are displayed in the screen center area 108b. The display process for displaying will be described.

FIG. 7 is a flowchart for explaining the drawing process. From the time when the user terminal 100 accesses the WEB server 200 and receives the WEB page to the end of browsing the WEB page, the user terminal 100 performs drawing processing for a predetermined time based on the program included in the WEB page. For example, it is repeatedly executed every 50 msec).

In step S01, it is determined whether or not it is time to access the WEB server 200 and start browsing in response to the WEB page response. When it is determined that it is time to start browsing, in step S02, the entire WEB page, which is a predetermined image, is drawn in a predetermined first area of the RAM 102 of the user terminal 100. In the own terminal mode, the image of the area corresponding to the scroll operation in the WEB page drawn in the first area can be displayed in the screen center area 108b. On the other hand, when it is not determined that it is time to start browsing, it is already being browsed and the entire WEB page is drawn, so the process proceeds to step S03.

In step S03, it is determined whether or not it is time for P2P communication to be established and the P2P communication to be started. When it is determined that it is time to start P2P communication, a video call image is drawn in the predetermined second area of the RAM 102 in step S04 based on the image data received from the other party terminal. In the video call mode, the image drawn in the second area can be displayed in the center area 108b of the screen.

In step S05, the entire WEB page, which is a predetermined image, is drawn in the predetermined third area of the RAM 102, and the drawing process is completed. In the remote terminal mode, the image of the region corresponding to the specific information received from the remote terminal among the WEB pages drawn in the third region can be displayed in the screen center region 108b.

On the other hand, when it is not determined in step S03 that it is time to start P2P communication, it is determined in step S06 whether or not P2P communication is in progress. When it is determined that P2P communication is in progress, the video call image in the second region is updated to a video call image based on the image data received from the other terminal in step S07, and the drawing process is terminated. This makes it possible to display a moving image as a video call image. On the other hand, when it is not determined that P2P communication is in progress, the drawing process is terminated as it is.

As described above, the drawing process draws a base image for display in the screen center area 108b. In the drawing process of the present embodiment, an example of drawing in the first area used in the own terminal mode has been described, but the drawing in the first area is assumed to be drawn by the function of the WEB browser. In this drawing process, the processes of steps S01 and S02 may not be performed.

FIG. 8 is a flowchart for explaining the image identification process. The user terminal 100 performs image identification processing for a predetermined time based on a program included in the WEB page from the time when the user terminal 100 accesses the WEB server 200 and receives the WEB page to the end of browsing the WEB page. It is repeatedly executed every (for example, 50 msec).

In step S11, it is determined whether or not the display mode is the own terminal mode. The display mode is set in the display process (steps S21, S23 to S25, etc.) described later. When it is determined in step S11 that the mode is the own terminal mode, among the WEB pages drawn in the first area of the RAM 102 in step S12, an area (image) to be displayed in the screen center area 108b according to the scroll operation for the own terminal. Is specified, and the process proceeds to step S13. As a result, the image to be displayed in the screen center area 108b in the own terminal mode is specified. The area (image) specified in step S12 is maintained even after being switched to another display mode, and when the mode is switched to the own terminal mode again, the maintained area (image) is the screen center area 108b. Will be displayed in. On the other hand, when it is not determined in step S11 that the mode is the own terminal mode, the process proceeds to step S13 as it is.

In step S13, it is determined whether or not a predetermined time for transmitting / receiving specific information has elapsed during P2P communication. When it is not determined that the predetermined time has elapsed during P2P communication, the image identification process is terminated as it is. On the other hand, when it is determined that the predetermined time has elapsed during P2P communication, the area (image) specified in step S12 in step S14 is set regardless of the type of the set display mode. The identification information for enabling the identification of the area (image) corresponding to the scroll operation is transmitted to the remote terminal. As a result, the image to be displayed in the screen center area 108b in the own terminal mode can be specified even in the other terminal.

In step S15, among the WEB pages drawn in the third area of the RAM 102 of the own terminal, the area (image) to be displayed in the center area 108b of the screen is specified according to the identification information from the other terminal, and the image identification process is performed. To finish. As a result, the image to be displayed in the screen center area 108b in the partner terminal mode can be specified.

As described above, the image identification process identifies the image to be displayed in the screen center region 108b. In the image specifying process in the present embodiment, an example of specifying the area (image) in the own terminal mode has been described, but the process of specifying the area (image) in the own terminal mode is performed by the function of the WEB browser. It may be performed, and the processing of steps S11 and S12 may not be performed in the image identification processing. In this case, in S14, the area (image) specified by the function of the WEB browser may be used in response to the scroll operation to transmit the identification information for specifying the area (image).

FIG. 9 is a flowchart for explaining the display process. The user terminal 100 executes the display process based on the program included in the WEB page from the time when the WEB server 200 is accessed and the WEB page is received until the browsing of the WEB page is completed.

In step S21, since browsing of the WEB page is started or the like, the own terminal mode is set as the display mode. In the own terminal mode, the process of highlighting the own terminal notification image 404 is also performed. In step S22, it is determined whether or not it is the start of P2P communication. In step S22, it is determined that it is the start of P2P communication when the room is created, but it is determined that it is the start of P2P communication when the P2P communication is actually established. There may be.

When it is not determined in step S22 that it is the start of P2P communication, in step S27, among the WEB pages drawn in the first area of the RAM 102, the WEB page is specified in step S12 according to the scroll operation for the own terminal. The image of the area is displayed in the center area 108b of the screen. As a result, when P2P communication is not in progress or when the own terminal mode is set, an image of a portion of the WEB page corresponding to the scroll operation is displayed in the screen center area 108b and can be viewed (FIG. 4 (a). ) Etc.).

On the other hand, when it is determined in step S22 that it is the start of P2P communication, the video call mode is set as the default display mode at the start in step S23 (see FIG. 4C and the like). In addition, a process of highlighting the video call notification image 403 is also performed. The default display mode is not limited to the video call mode, and may be another display mode or may maintain the own terminal mode.

In step S24, it is determined whether or not the switching operation is detected. When the switching operation is detected, the display mode corresponding to the switching operation is set among the own terminal mode, the other party terminal mode, and the video call mode in step S25, and the process proceeds to step S26. Further, among the other party terminal notification image 402, the video call notification image 403, and the own terminal notification image 404, a process of highlighting the notification image corresponding to the set display mode is performed. It should be noted that switching from the own terminal mode to the other party's terminal mode without going through the video call mode and switching from the other party's terminal mode to the own terminal mode without going through the video call mode are restricted. On the other hand, when it is not determined in step S24 that the switching operation has been detected, the process proceeds to step S26.

In step S26, it is determined whether or not the set display mode is the own terminal mode. When it is determined that the mode is the own terminal mode, the process proceeds to step S27 and the image of the own terminal is displayed as described above. On the other hand, when it is not determined to be the own terminal mode, it is determined whether or not the display mode set in step S28 is the partner terminal mode.

When it is determined in step S28 that the mode is the remote terminal mode, among the WEB pages drawn in the third area of the RAM 102 in step S29, the area specified in step S15 according to the identification information from the remote terminal. The image is displayed in the center area 108b of the screen. As a result, when the partner terminal mode is set during P2P communication, the image displayed on the partner terminal can be grasped in response to the scroll operation in the own terminal mode on the partner terminal.

On the other hand, when it is not determined in step S28 that the other party terminal mode is set, the video call mode is set. Therefore, in step S30, the video call image drawn in step S04 in the second area of the RAM 102 is displayed on the screen. Displayed in the central region 108b. As a result, when the video call mode is set during P2P communication, it is possible to confirm the state of the other user with the image (moving image) captured by the other terminal. As described above, during P2P communication, the voice of the other user can be output based on the voice data from the other terminal regardless of the display mode.

Even when the video call mode is set, if the call switching icon 405a is operated on the other terminal to switch to a voice call without video, in step S30, a predetermined screen (for example, , A screen including a character string such as “AUDIO” shown in FIG. 4C, a screen displaying a user name of the other terminal or a character image set by the user, etc.) is displayed in the center area 108b of the screen.

In step S31, it is determined whether or not the end icon 405b is touch-operated to end the P2P communication. When it is determined that the P2P communication is completed, the process proceeds to step S21 to display the image of the own terminal. On the other hand, when it is not determined that the P2P communication is completed, it is determined in step S32 whether or not the browsing is completed. In step S32, it is determined that the browsing is completed, for example, when the WEB browser is terminated or when the access to the WEB server 200 is terminated.

When it is not determined that the browsing is finished, the process proceeds to step S24 to switch the display mode and display an image according to the display mode. On the other hand, when it is determined that the browsing is finished, the display process is finished.

<Example of effect by this embodiment>
(1) The user terminal 100 (for example, the user terminal X) in the above embodiment has a WEB page as shown in step S14 of FIG. 8 based on the program included in the WEB page response from the WEB server 200. Specific information for making it possible to specify an area (image) that can be displayed in response to a scroll operation on the touch screen 108 of the user terminal X on another user terminal 100 (for example, a user terminal Y during P2P communication). Is output, and as shown in step S15 of FIG. 8, identification information for making it possible to identify an area (image) that can be displayed in response to a scroll operation on the touch screen 108 of the user terminal Y is received. Further, the user terminal X is based on the program included in the WEB page response from the WEB server 200, and as shown in steps S24 to S30 of FIG. 9, the user terminal X touch screen 108 (screen center area 108b) of the user terminal X. ), The image corresponding to the scrolling operation on the user terminal X on the WEB page according to the switching operation on the touch screen 108, and the image specified from the identification information received from the user terminal Y. Switch to one of multiple types of images, including. As a result, a certain user (for example, user X) shares with another user (for example, user Y) which part of the WEB page is being viewed by accessing the WEB server 200 in both directions. The image (location) viewed on the own terminal (user terminal X) and the other terminal (user terminal Y) on the WEB page by a simple operation such as a swipe operation according to the intention of the user X can be performed. It is possible to switch and display the image (location) being viewed. As a result, the user X and the user Y can recognize the image being viewed by another user at their own timing, and can facilitate smooth communication.

(2) As shown in FIG. 5 and the like, the plurality of types of images switched by the switching operation include a video call image between the own terminal and the other party terminal. As a result, in addition to the image (location) being viewed on the other terminal, the state of other users can be grasped, and smoother communication can be achieved. Further, for a plurality of types of images that can be switched by the switching operation, when the call switching icon 405a is operated on the other party terminal to switch to a voice call without video, as shown in FIG. 4C, a video call is made. Predetermined images are included rather than images. This makes it possible to prevent the other terminal from displaying one's own image according to the convenience and preference of the user.

(3) It is restricted to switch from the own terminal mode to the other party's terminal mode without going through the video call mode, or to switch from the other party's terminal mode to the own terminal mode without going through the video call mode. As a result, the screen of the own terminal cannot be directly switched to the other terminal mode or the mode of the other terminal cannot be directly switched to the own terminal mode, so that the screen of the own terminal is erroneously recognized as the screen of the other terminal. It is possible to reduce the fact that the screen of the other terminal is erroneously recognized as the screen of the own terminal.

(4) Regardless of the type of display mode set, a plurality of types of notification images 402 to 404 corresponding to each display mode are displayed, and the notification images corresponding to the currently set display mode are displayed in reverse (others). It is displayed in a mode different from that of the broadcast image of. Therefore, the currently set display mode can be notified to the user in an easy-to-understand manner, and it is possible to prevent erroneous recognition.

Further, the broadcast images 402 to 404 corresponding to each of the plurality of types of display modes are arranged and displayed in a predetermined order in the horizontal direction as shown in FIG. 4A and the like, and together with this, the display modes The switching operation for switching is also defined as an operation of moving along the horizontal direction, that is, a swipe operation to the left or right. As a result, the switching operation can be made intuitively easy to understand.

(5) The specific information transmitted on the own terminal and other terminals is centered on the screen according to the scroll operation in the own terminal mode when the specific position of the WEB page received when accessing the WEB server 200 is used as the base point. It is information for specifying a coordinate position of a predetermined position of an image displayed in the area 108b, and is area information for specifying a display area according to a scroll operation. As a result, the amount of data for the specific information and the processing load for specifying the image can be reduced.

<Modification example>
Examples of modifications of the embodiments described above are listed below.

(1) In the above embodiment, the image of the own terminal, the image of the other party's terminal, and the video call image are exemplified as the plurality of types of images to be switched, but the image of the own terminal and the image of the other party's terminal are illustrated. As long as it includes and, it may not include other images such as a video call image, and even if it contains other images, the other images are not limited to the video call image. Absent. The plurality of types of images to be switched include, for example, a chat image that enables chat between a plurality of user terminals in place of or in addition to a video call image, and displays a message history sent / received by the chat, or during a voice call. It may be a predetermined image in (the video call image is not displayed) or the like.

(2) In the above embodiment, when P2P communication is established, an example of transmitting specific information for making the image of the own terminal identifiable to the other terminal has been described, but the present invention is not limited to this, and the specific information is used. The user of the own terminal may be able to select whether or not to send to the other terminal. For example, by arranging a sharing icon and detecting an operation on the sharing icon, control is performed to send specific information to the other terminal, and the image of the own terminal is allowed to be displayed on the other terminal. When the operation to the shared icon is detected while controlling the transmission of the specific information to the other terminal, the image of the own terminal is displayed on the other terminal by not transmitting the specific information to the other terminal. You may not allow that. As a result, it becomes possible to select whether or not to share the image of the own terminal with the other terminal according to the convenience and preference of the user, and the convenience can be improved. When P2P communication is started, by default, it may be set to a state in which the image of the own terminal is allowed to be displayed on the other terminal, or it is set to a state in which it is not allowed. Alternatively, it may be possible to set (for example, initial setting) whether the state is allowed or not allowed when the P2P communication is started from the stage before the start of the P2P communication.

In addition, the above sharing icon is displayed only on the screen where the WEB page is not displayed (for example, the screen in the video call mode), and is not displayed on the screen in the own terminal mode or the partner terminal mode in which the WEB page is displayed. It may be set to. As a result, it is possible to prevent the screen in the own terminal mode or the partner terminal mode for displaying the WEB page from being obstructed by the share icon. Not limited to this, the above sharing icon may be displayed on the screen in the own terminal mode among the screens on which the WEB page is displayed. As a result, it is possible to improve the convenience when displaying (or displaying) a screen that the other party does not want to see in terms of security or the like on the screen in the own terminal mode.

(3) In the above embodiment, the specific information transmitted / received by the user terminal 100 is displayed in the center area 108b of the screen according to the scroll operation in the own terminal mode when the specific position of the WEB page is used as the base point. An example of information for specifying the coordinate position of a predetermined position of an image, which is area information for specifying a display area according to a scroll operation, has been described. However, the identification information transmitted and received by the user terminal 100 is not limited to the information as long as the image displayed in the screen center area 108b in response to the scroll operation in the own terminal mode can be specified by the other terminal. Absent. The specific information transmitted / received by the user terminal 100 is, for example, a mode of swipe operation (direction, scroll amount, etc.) for the screen center area 108b in the own terminal mode, a touch operation, a pinch-out operation / pinch-in operation, and a scroll bar. It may be operation mode information that can specify the mode (operation content) of the scroll operation such as operation, or it may be image information that can display the image itself displayed in the screen center area 108b in the own terminal mode. Good. In any case, in step S15 of FIG. 8, the area (image) corresponding to the identification information is specified at any time, and in the remote terminal mode, it is displayed on the specified remote terminal in step S29 of FIG. The image of the area is displayed, and the screen of the other terminal can be grasped.

(4) In the above embodiment, the WEB page obtained by accessing the WEB server 200 includes a program for performing the processes shown in FIGS. 7 to 9 and displaying the screens illustrated in FIGS. 4 to 6. The example has been described, but the present invention is not limited to this, and the program may be downloaded and installed in advance from a predetermined server. In this case, images of each user terminal may be displayed between the user terminals on which the program is installed. It may be something that can be shared.

(5) In the above embodiment, a WEB page obtained by accessing the WEB server 200 is illustrated as a predetermined image shared between user terminals, but the predetermined image is not limited to this, and at least one user. It may be an image (for example, a WEB page, a document, a photograph, a document, etc.) stored in the terminal. In this case, among the user terminals undergoing P2P communication, the user terminal that stores the image may transmit image data for displaying the image to another user terminal.

(6) In the above embodiment, an example of sharing an image between two user terminals has been described, but the number of user terminals sharing an image is not limited to this, and may be three or more. For example, when sharing among three users, the plurality of types of images to be switched include, for example, an image of the own terminal, an image of the first partner terminal owned by another first user, and an image owned by another second user. The image of the second partner terminal and other images such as a video call image are included, and the image of the first partner terminal and the image of the second partner terminal are arranged in this order from the left, and then the other images are arranged. , The image of the own terminal may be arranged on the far right (that is, the image of the other terminal and the image of the own terminal are arranged so as to sandwich another image). As a result, by swiping from left to right to display the leftmost image, the image of the first partner terminal is displayed, and each time the swipe operation is performed from right to left from that state, the second partner terminal It is possible to display an image according to the user's intention, such as an image → another image → an image of the own terminal. In addition, the notification images are arranged in the order of the first partner terminal notification image, the second partner terminal notification image, the notification image corresponding to the other image, and the own terminal notification image from the left so as to correspond to the arrangement order of the images. It may be displayed. Further, an example of connecting between a plurality of user terminals by P2P communication has been described, but the communication format is not limited to this, and may be connected via a predetermined server, for example, wireless LAN or wired. It may be connected via a LAN or may be connected by a wireless communication technology capable of short-range data communication.

(7) In the above embodiment, for example, a swipe operation in the left-right direction with respect to the touch screen 108 is illustrated as a switching operation, but a swipe operation in the left-right direction in the screen center area 108b of the touch screen 108 is used as a switching operation. It may be effectively accepted.
Further, the switching operation is superimposed on a predetermined image (WEB page) such as a part of the screen center area 108b (for example, the area in the vertical (vertical) direction is the area including the broadcast image and the area from the left end to the right end). It may be one that effectively accepts a swipe operation into the area (area where an image can be displayed). Further, the switching operation may effectively accept a swipe operation into a specific area other than the screen center area 108b (for example, the screen upper area 108a, the screen lower area 108c, etc.). In this case, the area for accepting the scroll operation for displaying an arbitrary part of the WEB page in the screen center area 108b and the area for accepting the switching operation can be separated, so that the operability can be improved.
Further, the switching operation is not limited to the swipe operation, and may be, for example, a touch operation on the other party terminal notification image 402, the video call notification image 403, and the own terminal notification image 404. In this case, the touch-operated notification image may be used. It may be possible to switch to the display mode corresponding to. Further, the switching operation may be an operation on the operation detection unit 109 mounted on the user terminal, or a sensor for identifying the operation on the user terminal itself (for example, an acceleration sensor, an angular velocity sensor, etc.). The user terminal itself may be shaken to the left or right.

(8) In the above embodiment, an example in which the entire WEB page, which is a predetermined image, is drawn in the third area of the RAM 102 in step S05 has been described in order to display the image of the partner terminal. However, since the shared image is the same image (for example, the image of the same WEB page), the image is drawn in the first area in step S02 even when the image is specified in the other terminal, for example, without performing the process of step S05. Of the WEB pages drawn in the first area, the image of the area corresponding to the specific information received from the partner terminal may be displayed in the screen center area 108b by using the WEB page.

(9) In the above embodiment, an example of displaying a plurality of icons in the screen lower region 108c of the touch screen 108 has been described, but the present invention is not limited to this, and the screen lower portion without displaying the plurality of icons. A predetermined image such as a WEB page may be displayed in the area 108c as well. As a result, the screen area for displaying the predetermined image is widened, and the visibility can be improved. In this case, the plurality of icons displayed in the lower screen area 108c may be displayed by, for example, touching the icon 401.

(10) In the above embodiment, a plurality of types of images including the image of the own terminal and the image of the other terminal are switched along the left-right direction (horizontal direction), and the switching operation is also along the left-right direction (horizontal direction). Although the example defined in the operation is shown, the switching direction is not limited to this, and may be switched along, for example, the vertical direction (vertical direction), and the switching operation in this case is also in the vertical direction (vertical direction). It may be defined as an operation along the vertical direction).

(11) In the above embodiment, a portable smartphone or the like as an information processing device has been mainly described, but even in the case of a stationary personal computer, for example, a plurality of types including an image of the own terminal and an image of the other terminal. One of the images in the above is displayed on the display device connected to the personal computer, but in addition to this, a plurality of types of images including the image of the own terminal and the image of the other terminal are displayed. May be set to be displayed on one screen, and may be switched and displayed by a switching operation or displayed on one screen according to the setting.

1 communication system, 2 networks, 100 user terminals, 200 WEB servers, 300 signaling servers.

Claims (16)

A program executed in an information processing device including a processor, a memory, an input unit, and a display unit.
In the information processing device
A step of outputting information for enabling another information processing apparatus to identify an image that can be displayed on the display unit in response to the first operation on the input unit among the predetermined images.
A step of receiving information for making it possible to identify an image that can be displayed on the display unit of the other information processing device in response to the first operation on the input unit of the other information processing device.
The image to be displayed on the display unit is the first image of the predetermined image corresponding to the first operation of the information processing apparatus on the input unit in response to the second operation on the input unit, and the said image. Any of a plurality of types of images including a second image identified from the information received by the receiving step and a predetermined third image different from the first image and the second image. Steps to switch to and
When the predetermined icon image that accepts the operation for permitting the display of the first image in another information processing device is switched to the first image and the second image, the predetermined icon image is not displayed. A program that executes a step to be displayed when the image is switched to the third image. A program executed in an information processing device including a processor, a memory, an input unit, and a display unit.
In the information processing device
A step of outputting information for enabling another information processing apparatus to identify an image that can be displayed on the display unit in response to the first operation on the input unit among the predetermined images.
A step of receiving information for making it possible to identify an image that can be displayed on the display unit of the other information processing device in response to the first operation on the input unit of the other information processing device.
The image to be displayed on the display unit is the first image of the predetermined image corresponding to the first operation of the information processing apparatus on the input unit in response to the second operation on the input unit, and the said image. Any of a plurality of types of images including a second image identified from the information received by the receiving step and a predetermined third image different from the first image and the second image. to execute the steps of switching to,
The switching step regulates the switching from the first image to the second image and the switching from the second image to the first image, and the first image and the third image. A program that allows switching between images and switching between the second image and the third image. A program executed in an information processing device including a processor, a memory, an input unit, and a display unit.
In the information processing device
A step of outputting information for enabling another information processing apparatus to identify an image that can be displayed on the display unit in response to the first operation on the input unit among the predetermined images.
A step of receiving information for making it possible to identify an image that can be displayed on the display unit of the other information processing device in response to the first operation on the input unit of the other information processing device.
The image to be displayed on the display unit is the first image of the predetermined image corresponding to the first operation of the information processing apparatus on the input unit in response to the second operation on the input unit, and the said image. A step of switching to one of a plurality of types of images including a second image identified from the received information by the receiving step, and
Regardless of the type of image switched by the switching step, a plurality of types of corresponding images corresponding to each of the plurality of types of images are displayed, and the corresponding image corresponding to the image switched by the switching step is supported by another. A program that executes a step of displaying in a manner different from that of an image. The plurality of types of corresponding images are arranged and displayed in a predetermined direction, and are displayed.
The input unit is a touch panel.
The program according to claim 3 , wherein the second operation is an operation of moving the touch panel along the same direction as the predetermined direction. The program according to claim 3 or 4 , wherein the plurality of types of images include a predetermined third image different from the first image and the second image. Wherein the third image is a video call image between the said another information processing apparatus and the information processing apparatus, the program according to any one of claims 1, 2, and 5. Wherein the third image is the chat screen in between the information processing apparatus and the other information processing apparatus, the program according to any one of claims 1, 2, and 5. The information output by the output step and the information received by the reception step are, respectively, area information for specifying an area corresponding to the first operation to the input unit of the predetermined image. The program according to any one of claims 1 to 7. The input unit is a touch panel.
The first operation includes an operation on the touch panel.
Claims 1 to claim that the information output by the output step and the information received by the receiving step are operation mode information for making it possible to specify the mode of operation with respect to the touch panel, respectively. The program described in any of 7. The information output by the output step and the information received by the reception step can each display an image of a region corresponding to the first operation to the input unit of the predetermined image. The program according to any one of claims 1 to 7 , which is image information for the purpose. A method performed in an information processing device including a processor, memory, an input unit, and a display unit.
A step of outputting information for enabling another information processing apparatus to identify an image that can be displayed on the display unit in response to the first operation on the input unit among the predetermined images.
A step of receiving information for making it possible to identify an image that can be displayed on the display unit of the other information processing device in response to the first operation on the input unit of the other information processing device.
The image to be displayed on the display unit is the first image of the predetermined image corresponding to the first operation of the information processing apparatus on the input unit in response to the second operation on the input unit, and the said image. Any of a plurality of types of images including a second image identified from the information received by the receiving step and a predetermined third image different from the first image and the second image. Steps to switch to and
When the predetermined icon image that accepts the operation for permitting the display of the first image in another information processing device is switched to the first image and the second image, the predetermined icon image is not displayed. A method comprising a step of displaying when the image is switched to the third image . A method performed in an information processing device including a processor, memory, an input unit, and a display unit.
A step of outputting information for enabling another information processing apparatus to identify an image that can be displayed on the display unit in response to the first operation on the input unit among the predetermined images.
A step of receiving information for making it possible to identify an image that can be displayed on the display unit of the other information processing device in response to the first operation on the input unit of the other information processing device.
The image to be displayed on the display unit is the first image of the predetermined image corresponding to the first operation of the information processing apparatus on the input unit in response to the second operation on the input unit, and the said image. Any of a plurality of types of images including a second image identified from the information received by the receiving step and a predetermined third image different from the first image and the second image. and a step of switching to,
The switching step regulates the switching from the first image to the second image and the switching from the second image to the first image, and the first image and the third image. A method that allows switching between images and switching between the second image and the third image. A method performed in an information processing device including a processor, memory, an input unit, and a display unit.
A step of outputting information for enabling another information processing apparatus to identify an image that can be displayed on the display unit in response to the first operation on the input unit among the predetermined images.
A step of receiving information for making it possible to identify an image that can be displayed on the display unit of the other information processing device in response to the first operation on the input unit of the other information processing device.
The image to be displayed on the display unit is the first image of the predetermined image corresponding to the first operation of the information processing apparatus on the input unit in response to the second operation on the input unit, and the said image. A step of switching to one of a plurality of types of images including a second image identified from the received information by the receiving step, and
Regardless of the type of image switched by the switching step, a plurality of types of corresponding images corresponding to each of the plurality of types of images are displayed, and the corresponding image corresponding to the image switched by the switching step is supported by another. A method comprising a step of displaying in a manner different from the image . An information processing device including a processor, a memory, an input unit, and a display unit.
A step of outputting information for enabling another information processing apparatus to identify an image that can be displayed on the display unit in response to the first operation on the input unit among the predetermined images.
A step of receiving information for making it possible to identify an image that can be displayed on the display unit of the other information processing device in response to the first operation on the input unit of the other information processing device.
The image to be displayed on the display unit is the first image of the predetermined image corresponding to the first operation of the information processing apparatus on the input unit in response to the second operation on the input unit, and the said image. Any of a plurality of types of images including a second image identified from the information received by the receiving step and a predetermined third image different from the first image and the second image. Steps to switch to and
When the predetermined icon image that accepts the operation for permitting the display of the first image in another information processing device is switched to the first image and the second image, the predetermined icon image is not displayed. An information processing device including a step of displaying when the image is switched to the third image. An information processing device including a processor, a memory, an input unit, and a display unit.
A step of outputting information for enabling another information processing apparatus to identify an image that can be displayed on the display unit in response to the first operation on the input unit among the predetermined images.
A step of receiving information for making it possible to identify an image that can be displayed on the display unit of the other information processing device in response to the first operation on the input unit of the other information processing device.
The image to be displayed on the display unit is the first image of the predetermined image corresponding to the first operation of the information processing apparatus on the input unit in response to the second operation on the input unit, and the said image. Any of a plurality of types of images including a second image identified from the information received by the receiving step and a predetermined third image different from the first image and the second image. and a step of switching to,
The switching step regulates the switching from the first image to the second image and the switching from the second image to the first image, and the first image and the third image. An information processing device that allows switching between images and switching between the second image and the third image. An information processing device including a processor, a memory, an input unit, and a display unit.
A step of outputting information for enabling another information processing apparatus to identify an image that can be displayed on the display unit in response to the first operation on the input unit among the predetermined images.
A step of receiving information for making it possible to identify an image that can be displayed on the display unit of the other information processing device in response to the first operation on the input unit of the other information processing device.
The image to be displayed on the display unit is the first image of the predetermined image corresponding to the first operation of the information processing apparatus on the input unit in response to the second operation on the input unit, and the said image. A step of switching to one of a plurality of types of images including a second image identified from the received information by the receiving step, and
Regardless of the type of image switched by the switching step, a plurality of types of corresponding images corresponding to each of the plurality of types of images are displayed, and the corresponding image corresponding to the image switched by the switching step is supported by another. An information processing device including a step of displaying an image in a mode different from that of an image.

Images