EP3274873A1

EP3274873A1 - Systems and methods for selective incorporation of imagery in a low-bandwidth digital mapping application

Info

Publication number: EP3274873A1
Application number: EP16760231.7A
Authority: EP
Inventors: Daniel Joseph Filip
Original assignee: Google LLC
Current assignee: Google LLC
Priority date: 2015-08-07
Filing date: 2016-08-02
Publication date: 2018-01-31
Also published as: CN109074356A; WO2017027255A1; DE202016007830U1

Abstract

Computer-implemented methods and systems for selectively providing an interactive digital mapping application in a limited bandwidth environment include providing for display on a display of a computing device a three-dimensional representation of a geographic area viewed by a user of an interactive digital mapping application. Download information can be provided for display in the interactive digital mapping application, including a quantifiable parameter (e.g., download size, time, or cost) associated with downloading one or more versions of imagery corresponding to the three-dimensional representation. User instructions requesting that one or more versions of the imagery corresponding to the three-dimensional representation be downloaded for display can be received. The one or more requested versions of the images corresponding to the 3D representation can be downloaded and provided for display.

Description

SYSTEMS AND METHODS FOR SELECTIVE INCORPORATION OF IMAGERY IN A LOW-BANDWIDTH DIGITAL MAPPING APPLICATION

FIELD

[0001] The present disclosure relates generally to digital mapping applications, and more particularly to the selective incorporation of imagery in a low-bandwidth digital mapping application.

BACKGROUND

[0002] Computerized methods and systems for displaying imagery, in particular panoramic imagery, are known. In the context of geographic information systems and digital mapping systems, services such as Google Maps are capable of providing street level images of geographical locations. The images, (e.g., known on Google Maps as "Street View,") typically provide immersive 360° panoramic views centered around a geographic area of interest. The panoramic views allow a user to view a geographic location from a person's perspective, as if the user was located on the street level or ground level associated with the geographic location.

[0003] In some implementations, when a user of a mapping service or other application opts to view a street level image, the client device (e.g., a computing device such as a computer, laptop, handheld device, mobile device, smart phone, etc.) can fetch from a server device image tiles or other data objects needed to render the image for display at the client device. The image tiles or other data objects stored at the server can include high quality images characterized by a relatively large data size. In many geographic locations throughout the world, a user's opportunity to download such large image tiles or other data objects and related data can be limited due to bandwidth constraints, network speeds, and/or download costs. In some cases, it can take a client device a considerable amount of time (e.g., minutes) to fetch and render such an image.

SUMMARY

[0004] Aspects and advantages of embodiments of the present disclosure will be set forth in part in the following description, or can be learned from the description, or can be learned through practice of the embodiments.

[0005] One example aspect of the present disclosure is directed to a computer- implemented method of providing an interactive digital mapping application in a limited bandwidth environment. The method can include providing for display, by one or more computing devices, a on a display of a computing device a three-dimensional representation of a geographic area viewed by a user of an interactive digital mapping application. The method also can include providing, by the one or more computing devices, download information for display in the interactive digital mapping application. The download information can include a quantifiable parameter associated with downloading one or more versions of imagery corresponding to the three-dimensional representation. The method also can include receiving, by the one or more computing devices, user instructions requesting that one or more versions of the imagery corresponding to the three-dimensional

representation be downloaded for display. The method also can include downloading, by the one or more computing devices, the one or more requested versions of the imagery corresponding to the three-dimensional representation. The method also can include providing, by the one or more computing devices, for display the one or more requested versions of the imagery corresponding to the three-dimensional representation.

[0006] Other example aspects of the present disclosure are directed to systems, apparatus, tangible, non-transitory computer-readable media, user interfaces, memory devices, and electronic devices for estimating restaurant wait times and/or food serving times using mobile computing devices.

[0007] These and other features, aspects, and advantages of various embodiments will become better understood with reference to the following description and appended claims. The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present disclosure and, together with the description, serve to explain the related principles.

BRIEF DESCRIPTION OF THE DRAWINGS

[0008] Detailed discussion of embodiments directed to one of ordinary skill in the art are set forth in the specification, which makes reference to the appended figures, in which:

[0009] FIG. 1 depicts an example user interface for presenting three-dimensional representations within a mapping application according to example aspects of the present disclosure;

[0010] FIG. 2 depicts an example user interface for presenting download information related to imagery corresponding to a three-dimensional representation within a mapping application according to example aspects of the present disclosure; [0011] FIG. 3 depicts an example user interface for presenting requested imagery within a mapping application according to example aspects of the present disclosure;

[0012] FIG. 4 provides a flow diagram of an example method for selectively incorporating imagery in a low bandwidth digital mapping application according to example aspects of the present disclosure;

[0013] FIG. 5 provides a flow diagram of additional aspects of an example method for selectively incorporating imagery in a low bandwidth digital mapping application according to example aspects of the present disclosure; and

[0014] FIG. 6 provides an example overview of computer-based system components for selectively incorporating imagery in a low bandwidth mapping application according to example aspects of the present disclosure.

DETAILED DESCRIPTION

[0015] Reference now will be made in detail to embodiments, one or more examples of which are illustrated in the drawings. Each example is provided by way of explanation of the embodiments, not limitation of the present disclosure. In fact, it will be apparent to those skilled in the art that various modifications and variations can be made to the embodiments without departing from the scope or spirit of the present disclosure. For instance, features illustrated or described as part of one embodiment can be used with another embodiment to yield a still further embodiment. Thus, it is intended that aspects of the present disclosure cover such modifications and variations.

[0016] Example aspects of the present disclosure are directed to systems and methods for selectively incorporating imagery in a low-bandwidth digital mapping application.

Computerized methods and systems for displaying imagery, in particular panoramic imagery, are known. In the context of geographic information systems and digital mapping systems, services such as Google Maps are capable of providing street level images of geographical locations. The images, (e.g., known on Google Maps as "Street View,") typically provide immersive 360° panoramic views centered around a geographic area of interest. The panoramic views allow a user to view a geographic location from a person's perspective, as if the user was located on the street level or ground level associated with the geographic location.

[0017] In some implementations, when a user of a mapping service or other application opts to view a street level image, the client device (e.g., a computing device such as a computer, laptop, handheld device, mobile device, smart phone, etc.) can fetch from a server device image tiles or other data objects needed to render the image for display at the client device. The image tiles or other data objects stored at the server can include high quality images characterized by a relatively large data size. In many geographic locations throughout the world, a user's opportunity to download such large image tiles or other data objects and related data can be limited due to bandwidth constraints, network speeds, and/or download costs. In some cases, it can take a client device a considerable amount of time (e.g., minutes) to fetch and render such an image.

[0018] The disclosed embodiments according to aspects of the present disclosure provide a low-bandwidth, three-dimensional (3D) alternative to conventional street level panoramic imagery portions and/or high-resolution satellite imagery portions of a digital mapping application, while selectively incorporating additional imagery when feasible and/or desired by an end user. A 3D representation of a geographic area of interest can include a reduced amount of data providing enough basic features within the geographic area to give the user some idea of where the user is looking. The 3D representation can be presented in a viewport on a display of a computing device operating an interactive digital mapping application, such as Google Maps or the like. In some examples, a 3D representation can be provided when a user requests operation of the interactive digital mapping application in a limited bandwidth mode of operation. The 3D representation can include a wireframe representation using basic lines and shapes to depict the exterior features or outlines of geographic objects (e.g., buildings, roads, signs, trees, etc.) included within the geographic area viewed by the user. The outlines of geographic areas can be formed from 3D point clouds or other data for the geographic area obtained at the same time as the panoramic images and/or satellite images of a geographic area. In some examples, the 3D representation can include text labels identifying such items as road names, building names and the like. Various forms of this data can be in the hundreds of bytes of data as opposed to the many tens of thousands of bytes of data required for conventional high resolution images, including panoramic street level images or satellite images.

[0019] Although a general mode of operation in a digital mapping application can primarily include only low-bandwidth 3D representations, additional features can be provided for downloading actual images when desired. For instance, user selectable interface buttons can be shown within a user interface by which a user can initiate the download and display of various versions of imagery corresponding to the 3D representations. This provides an option for heavy payload data to be downloaded only when specifically requested by a user of the interactive digital mapping application.

[0020] In some examples, download information can be provided for display in the interactive digital mapping application. The download information can correspond to a quantifiable parameter associated with downloading one or more versions of imagery. The download information can indicate to the user a relative amount of that parameter needed to download one or more versions of the one or more images corresponding to the 3D representations. For instance, download information can include a resolution version, bandwidth, file size, download time and/or download cost of the one or more images corresponding to the 3D representations. Download information can be provided for one or more different versions of the images, along with user-selectable interface buttons for the one or more versions by which the user can initiate selection and download of the images. In some examples, download information can be provided for at least a first and second version of the imagery. A first version can correspond to a low-resolution version (e.g., a thumbnail version) of the images, while a second version can correspond to a full resolution or larger resolution version of the images.

[0021] If and when a user desires additional imagery for a requested area, the one or more versions of the images corresponding to the 3D representation can be provided for display to a user within the viewport of the interactive digital mapping application. In some examples, the determination of which version(s) of images to provide when requested or which version(s) of download information provided to a user can be further facilitated by automatically determining a current communication capacity for the communication channel established between a client device and the server from which the interactive digital mapping application data is downloaded and made available to the user. The current communication capacity can include one or more suitable communication characterization parameters, such as but not limited to download speed, channel capacity, signal bandwidth and/or connection latency.

[0022] According to an example embodiment, a computing device user of an interactive digital mapping application desires to receive an immersive view centered around a geographic area of interest that allows the user to view the geographic location from a person's perspective on street or ground level. A viewport on the user's computing device can be used to display a low-bandwidth three-dimensional (3D) representation of the geographic area of interest. The 3D representation can provide basic outlines of roads, buildings and the like along with text labels identifying roads, buildings and other geographic objects. User-selectable buttons can be available within the interactive digital mapping application by which a user can request that one or more versions of imagery (e.g., satellite imagery and/or panoramic street level imagery) corresponding to the 3D representation be downloaded. Additional details, such as the resolution version, file size, download time, or download cost can be provided along with the various image versions and corresponding download buttons. If requested, the additional images can be downloaded and provided for display to a user within the viewport displaying the mapping application.

[0023] Referring now to the drawings, exemplary embodiments of the present disclosure will now be discussed in detail. FIG. 1 depicts an exemplary user interface 100, such as a browser, which can be presented on a display of a computing device, such as a personal computer, smartphone, desktop, laptop, PDA, tablet, mobile device, or other computing device. User interface 100 can include a viewport 102 that presents detailed display information for a user of an interactive digital mapping application. In the particular embodiment of FIG. 1, viewport 102 displays a portion of an immersive three-dimensional (3D) representation 104 of of a geographic area viewed by a user.

[0024] The 3D representation 104 provides a "street level" representation that depicts geographic objects within a geographic area of interest from a perspective at or near the ground level or street level. Although the present disclosure uses the term "street level" representations and "street level" images, the immersive views can depict non-street areas such as trails, rural environments, building interiors, and the like. In some examples, the immersive 3D representation 104 can also depict an overhead view of a geographic area as opposed to a street level view. As discussed below, the 3D representation 104 can be interactive such that the user can navigate within the 3D representation 104 by panning, zooming, rotating, and/or tilting the view of the 3D representation 104. As shown, 3D representation 104 can provide an immersive viewing experience of a geographic area to a user.

[0025] Some interactive digital mapping applications provide immersive 360° panoramic imagery of a geographic area within a viewport similar to viewport 102. Other interactive digital mapping applications provide detailed overhead satellite imagery of a geographic area within a viewport similar to viewport 102. However, a three-dimensional representation of such panoramic street level or overhead satellite imagery can be presented in order to provide a low-bandwidth alternative where download time, cost or other factors are important to a user of the interactive digital mapping application. Providing 3D representations within viewport 102 can be especially advantageous in operating

environments where the infrastructure of available communication networks can be limited.

[0026] The creation of a three-dimensional representation 304 such as depicted in viewport 102 of FIG. 1 can be implemented in a variety of ways. In some examples, three- dimensional representation 304 can be created from object position data and object labeling data gathered for various geographic objects within a geographic area of interest. The object position data can be obtained by or extracted from related data obtained by a mobile data acquisition unit traversing a geographic area of interest. Mobile data acquisition units can include a combination of sensors or data acquisition devices, such as but not limited to cameras capable of acquiring two-dimensional (2D) photos or videos and laser scanners capable of acquiring three-dimensional (3D) data collections for nearby geographic objects. In some examples, the 3D data collections include 3D LIDAR point clouds obtained with a laser range finder or laser sensing components that measure distance to a target by illuminating the target with a laser and analyzing the reflected light. In other examples, stereoscopic systems employing two cameras spaced slightly apart yet looking at the same scene can be utilized. By analyzing the slight differences between images obtained by each camera, it can be possible to estimate the distance at each point associated with a geographic object in a scene. In this manner, combinations of 2D photos can be translated into 3D space and corresponding 3D data points using known photogrammetry and other imaging techniques. Object labeling data can be obtained from databases of available geographic identifiers associated with particular geographic objects (e.g., streets, buildings, etc.) identified within a scene of interest.

[0027] As opposed to including imagery of a geographic area of interest, 3D representation 104 can include a wireframe or outlined representation of various geographic objects within a geographic area of interest. For instance, 3D representation 104 can include 3D wireframe outlines of buildings 114, 116, 118 and street 120. These wireframe outlines use basic lines and shapes to depict the exterior features or outlines of geographic objects (e.g., buildings, roads, signs, trees, etc.) included within the geographic area viewed by the user. Although not depicted in FIG. 1, 3D wireframe outlines of additional objects within the geographic area of interest, including but not limited to people, vehicles, vegetation and the like, also can be depicted when desired. 3D representation 104 also can include object labels such as building text labels 122, 124 and road text label 126. Various forms of the data used to create a 3D representation 104 can be in the hundreds of bytes of data as opposed to the many tens of thousands of bytes of data required for conventional high resolution images, including panoramic street level images or satellite images.

[0028] Referring still to FIG. 1, in addition to 3D representation 104, user interface

100 also can display a map and other information, such as travel directions 106 to a user. The user interface 100 can provide flexibility to the user in requesting immersive 3D

representations associated with a geographic area to be displayed through viewport 102. For instance, the user can enter text in a search field 108, such as an address, the name of a building, or a particular latitude and longitude. The user could also use an input device such as a mouse or touchscreen to select a particular geographic location shown on a map. Yet further, the user interface 100 can provide an icon or other feature that allows a user to request an immersive street level view at a specified geographic location. When providing a 3D representation 104 in a viewport 102, the user interface 100 can indicate the location and orientation of the current view associated with the 3D representation 104 with a viewpoint signifier 110.

[0029] The user interface 100 can include user-selectable controls 112 for navigating the viewpoint associated with the 3D representation 104. The controls can include controls for zooming the view in and out, as well as controls to change the orientation of the view depicted in the 3D representation 104. The user also can adjust the viewpoint of the 3D representation 104 using a user manipulable selecting object, such as a cursor or waffle. For instance, a user can adjust the viewpoint by selecting and dragging the 3D representation to different views, for instance, with a selecting object or through interaction with a touch screen. If the 3D representation 104 was created as an entire 360° representation, changing the direction of the view can necessitate only displaying a different portion of the panorama without retrieving more information from a server. Other navigation controls can be included as well, such as controls in the form of arrows disposed along a street that can be selected to move the vantage point up and down the street.

[0030] Referring now to FIG. 2, another example user interface 130 for an interactive digital mapping application can include similar features to user interface 100 of FIG. 1, including the presentation of 3D representation 104 in viewport 102. However, additional download information 132 can be provided to the user at some predetermined location within user interface 130. Download information 132 is depicted in FIG. 2 as replacing the travel directions 106 of FIG. 1. However, it should be appreciated that download information 132 can be positioned in additional and/or alternative locations within user interface 130, either within or outside viewport 102. In some examples, download information 132 can be provided for display in response to user selection of a selectable user interface button 128 of FIG. 1, such as one designed to trigger a request for "More Image Options."

[0031] Download information 132 can include one or more portions of information defining one or more quantifiable parameters associated with downloading one or more versions of imagery associated with a current view of a geographic area or nearby areas. For instance, a first portion 134, second portion 136 and third portion 138 of download information 132 in FIG. 1 each include three quantifiable parameters for each of three different versions of imagery. First portion 134 can include download information for a first version of an image (e.g., a "High Resolution Version"). Second portion 136 can include download information for a second version of an image (e.g., a "Medium Resolution

Version"). Third portion 138 can include download information for a third version of an image (e.g., a "Low Resolution Version"). Each portion 134, 136, 138 of download information 132 can include information defining quantifiable parameters including the resolution version, download size, download time and download cost. Although three versions of images and three types of quantifiable parameters are depicted in the example of FIG. 2, it should be appreciated that any number of image versions and associated quantifiable parameters can be used.

[0032] For each quantifiable parameter in the portions 134, 136, 138 of download information 132, a relative amount of that parameter needed to download one or more versions of the image(s) corresponding to a given 3D representation can be communicated to a user. For example, first portion 134 of download information 132 can communicate to a user that a download size of 6.4 Megabytes (MB) of data, download time of 1 minute and 15 seconds, and/or download cost of $.60 would be required to download a high-resolution version of imagery corresponding to 3D representation 104. Second portion 136 of download information 132 can communicate to a user that a download size of 1.5 Megabytes (MB) of data, download time of 32 seconds, and/or download cost of $.20 would be required to download a medium-resolution version of imagery corresponding to 3D representation 104. Third portion 138 of download information 132 can communicate to a user that a download size of 250 Kilobytes (KB) of data, download time of 5 seconds, and/or download cost of $.05 would be required to download a low-resolution version of imagery corresponding to 3D representation 104. [0033] Referring still to FIG. 2, user-selectable interface buttons also can be provided for each portion of download information. For example, user-selectable interface buttons 140, 142 and 144 are provided for each of the three different versions of images (high-res, medium-res, and low-res) that can be downloaded by a user of the interactive digital mapping application. User selection by an input device of one of the interface buttons 140, 142, 144 can initiate download of the corresponding version of an image.

[0034] FIG. 3 depicts an example user interface 150 after a user has selected interface button 140 and a high resolution version of an image 152 is downloaded within the interactive digital mapping application. In the example interface 150 of FIG. 3, image 152 corresponds to a panoramic street level image, although it should be appreciated that different types of images, such as overhead satellite images such as offered through Google Earth services provided by Google Inc. also can be provided in example embodiments of the present disclosure. Image 152 can be provided within viewport 102, although it should be appreciated that image 152 does not necessarily need to replace 3D representation 104. In some user interfaces, an image 152 can be provided adjacent to or overlapping a 3D representation 104. In some examples, the location of an image 152 can depend on the requested size of the image. For instance, high resolution versions can be provided within the entirety of a viewport 102 as depicted in FIG. 3, while a low-resolution or thumbnail version of an image can be provided next to or overlaid on top of a three-dimensional representation. When possible, a three-dimensional representation 104 and a corresponding downloaded image 152 are both provided at substantially the same vantage point (e.g., camera

angle/orientation or view angle/orientation). When three-dimensional representation 104 is provided as a street level representation, imagery 152 can be provided as street level imagery having the same vantage point and camera angle. When three-dimensional representation 104 is provided as an overhead map representation, imagery 152 can be provided as overhead satellite imagery having the same vantage point and zoom level.

[0035] Referring now to FIGS. 4-5, example aspects of a method for selectively incorporating imagery in a low bandwidth digital mapping application are depicted in flow diagram form. In FIG. 4, a method (160) for selectively incorporating imagery in a low bandwidth digital mapping application can include identifying (162) a user request to operate an interactive digital mapping application in a limited bandwidth mode of operation. In some examples, a user request identified at (162) can be provided directly within a setting available through the digital mapping application itself. In other examples, a user request identified at (162) can be provided within a general setting available on the computing device (e.g., mobile device, laptop, personal computing device, etc.) that is running the interactive digital mapping application. In some instances, a decision to operate in a limited bandwidth mode of operation can be determined automatically at the computing device operating the interactive digital mapping application by analyzing a current communication capacity for a communication channel associated with the computing device. Operation in a regular mode of operation (without bandwidth limitations) can result in presenting immersive panoramic and/or satellite imagery within a viewport as opposed to low bandwidth 3D representations.

[0036] When operating in a low bandwidth mode of operation, a 3D representation of a geographic area viewed by a user of an interactive digital mapping application is provided for display at (164). The 3D representation provided for display at (164) can include similar features and aspects as the example 3D representation 104 depicted in FIG. 1. More particularly, the 3D representation provided for display at (164) can include a wireframe outline of various geographic objects created from object position data defining exterior surfaces of the geographic objects. Object labels for geographic objects within the 3D representation displayed at (164) also can be provided. The 3D representation can be provided for display at (164) within a viewport or other user interface portion that is presented on a display of a computing device, similar to viewport 102 of FIG. 1.

[0037] Download information also can be provided for display at (166) in the interactive digital mapping application. As described relative to download information 132 of FIG. 2, download information provided for display at (166) can include a quantifiable parameter associated with downloading one or more versions of imagery corresponding to the 3D representation provided for display at (164). Specific quantifiable parameters related to each resolution version of an image can include file download size, download time, download cost and the like. The download information provided for display at (166) can indicate to the user a relative amount of each quantifiable parameter needed to download the one or more versions of imagery corresponding to a 3D representation. In some examples, download information can be provided at (166) for at least a first and second version of the imagery. A first version can correspond to a low-resolution version (e.g., a thumbnail version) of the imagery, while a second version can correspond to a full resolution or larger resolution version of the imagery. Selectable user interface buttons also can be provided for display at (166) as part of the download information for each version of imagery corresponding to the 3D representation provided for display at (164). In this way, a user of the interactive mapping application is provided with an electronic feature for requesting download of one or more versions of an image.

[0038] When a user selects one of the user interface buttons included as part of the download information provided for display at (166), user instructions are received at (168) requesting that one or more versions of imagery corresponding to the 3D representation provided for display at (164) be downloaded for display. The receipt of user instructions at (168) automatically triggers the download at (170) of the requested version(s) of imagery corresponding to the 3D representation. The one or more versions of the imagery requested at (168) corresponding to the 3D representation provided for display at (164) are then provided for display at (172). The imagery can be provided for display at (172) within the same viewport that the 3D representation is provided for display at (164) or at another location within a user interface.

[0039] Referring now to FIG. 5, additional flow diagram features 180 are depicted for optional inclusion in a method for selectively incorporating imagery in an interactive digital mapping application. Additional flow diagram features 180 generally can include using a determination of communication capacity to automatically select different features or versions of features within an interactive digital mapping application. A current

communication capacity can be determined at (182) for a communication channel of the computing device over which information within the interactive digital mapping application is made available to a user. The current communication capacity determined at (182) can be defined in terms of one or more suitable communication characterization parameters, such as but not limited to download speed, channel capacity, signal bandwidth, signal strength and/or connection latency of the communication channel providing mapping information to the client device. In some examples, current communication capacity can be determined relative to the availability of different types of network connections (e.g., a Wi-Fi or other wireless network connection, a cellular network connection, a satellite network connection, etc.).

[0040] Based on the current communication capacity determined at (182), one or more additional features can be implemented. In some examples, a mode of bandwidth operation can be automatically selected at (184) for operating the interactive digital mapping application. Different modes of bandwidth operation can include a low bandwidth mode of operation in which immersive 3D representations are provided for display within a viewport and/or a high bandwidth mode of operation in which full 360° panoramic street level images or high resolution satellite images are provided for display within a viewport. If a current communication capacity is determined at (182) to be limited due to a slow connection, device or network latencies, etc., automatically selecting a low bandwidth mode of operation at (184) can be helpful. In this example, the computing device operating the mapping application does not attempt to download large panoramic or satellite images when download time and cost can be high. In some examples, determining a communication capacity at (182) indicating the availability of a strong Wi-Fi or other wireless network connection can indicate a higher communication capacity, thus resulting in automatic selection at (184) of a full high resolution mode of operation. Different modes of operation can be automatically selected at (184) based on different current communication capacities determined at (182).

[0041] In other examples employing the features (180) of FIG. 5, a method for selectively incorporating imagery can automatically select at (186) one or more specific versions of images for which to provide download information or for automatically downloading based at least in part on the current communication capacity determined at (182). For instance, if the current communication capacity is determined at (182) to be relatively low, then it might be preferred to offer download information at (186) for only lower resolution versions of panoramic street level images and/or satellite images

corresponding to a 3D representation. In other examples, if the current communication capacity is determined at (182) to be relatively high, then it might be preferred to offer download information at (186) for higher resolution versions of images corresponding to a 3D representation. Other versions of images can be automatically selected at (186) based on different current communication capacities determined at (182).

[0042] FIG. 6 depicts an exemplary computing system 200 that can be used to implement techniques for selectively incorporating imagery in a low bandwidth interactive digital mapping application according to exemplary embodiments of the present disclosure. System 200 can include a computing device 210 configured to display 3D representations and corresponding imagery to a user. The computing device 210 can take any appropriate form, such as a personal computer, smartphone, desktop, laptop, PDA, tablet, or other computing device. The computing device 210 can include a display 218 for displaying 3D

representations, corresponding imagery, download information and/or other associated information to a user and appropriate input devices 215 for receiving input from the user. The input devices 215 can be any input device such as a touch screen, a touch pad, data entry keys, a mouse, speakers, a microphone suitable for voice recognition, and/or any other suitable device. [0043] A user can request 3D representations and corresponding imagery by interacting with an appropriate user interface presented on the display 218 of computing device 210. The computing device 210 then can receive 3D representations, corresponding imagery and/or associated data and present at least a portion of the data through a viewport on any suitable output device, such as through a viewport set forth in a browser or other user interface presented on the display 218.

[0044] The computing device 210 can include a processor(s) 212 and a memory 214.

The processor(s) 212 can be any known processing device. Memory 214 can include any suitable non-transitory computer-readable medium or media, including, but not limited to, RAM, ROM, hard drives, flash drives, or other memory devices. Memory 214 can store information accessible by processor(s) 212, including instructions that can be executed by processor(s) 212. The instructions can be any set of instructions that when executed by the processor(s) 112, cause the processor(s) 212 to provide desired functionality. For instance, the instructions when executed by the processor(s) 212 can cause the processor(s) 212 to present interactive 3D representations of imagery geographic area, according to any of the embodiments disclosed herein. The instructions can be software instructions rendered in a computer-readable form. When software is used, any suitable programming, scripting, or other type of language or combinations of languages can be used to implement the teachings contained herein. Alternatively, the instructions can be implemented by hard-wired logic or other circuitry, including, but not limited to application-specific circuits.

[0045] The computing device 210 can include a network interface 216 for accessing information over a network 220. The network 220 can include a combination of networks, such as cellular network, Wi-Fi network, LAN, WAN, the Internet, and/or other suitable network and can include any number of wired or wireless communication links. For instance, computing device 210 can communicate through a cellular network using a WAP standard or other appropriate communication protocol. The cellular network could in turn communicate with the Internet, either directly or through another network.

[0046] Computing device 210 can communicate with another computing device 230 over network 220. Computing device 230 can be a server, such as a web server, that provides information to a plurality of client computing devices, such as computing devices 210 and 250 over network 220. Any number of computing devices 210, 250 can communicate with computing device 230 over the network 220. Computing device 230 receives requests from computing device 210 and locates information to return to computing devices 210 responsive to the request. The computing device 230 can take any applicable form, and can, for instance, include a system that provides mapping services, such as the Google Maps services provided by Google Inc.

[0047] Computing device 230 can provide information, including 3D representations, street level imagery, preview imagery, download information and associated information, to computing device 210 over network 220. The information can be provided to computing device 210 in any suitable format. The information can include information in HTML code, XML messages, WAP code, Flash, Java applets, xhtml, plain text, voiceXML, VoxML, VXML, or other suitable format. The computing device 210 can display the information to the user in any suitable format. In one embodiment, the information can be displayed within a browser, such as the Google Chrome browser or other suitable browser.

[0048] Similar to computing device 210, computing device 230 can include a processor(s) 232 and a memory 234. Memory 234 can include instructions 236 for receiving requests for 3D representations and corresponding geographic imagery from a remote client device, such as computing device 210, and for providing the requested information to the client device for presentation to the user. Memory 234 also can include or be coupled to various databases, such as database 238 that stores information that can be shared with other computing devices. Computing device 230 can communicate with other databases as needed. The databases can be connected to computing device 230 by a high bandwidth LAN or WAN, or could also be connected to computing device 230 through network 220. The databases, including database 238, can be split up so that they are located in multiple locales or they can be all in one location.

[0049] The database 238 can include a map database 240, a street level image database 242, an object position database 244, an object labeling database 246, and a 3D representation database 248. Database 238 also can include other data having information that can be accessed or used by computing device 230.

[0050] Map database 240 stores map-related information, at least a portion of which can be transmitted to a client device, such as computing device 210. For instance, map database 240 can store map tiles, where each tile is an image of a particular geographic area. Depending on the resolution (e.g. whether the map is zoomed in or out), a single tile can cover a large geographic area in relatively little detail or just a few streets in high detail. The map information is not limited to any particular format. For example, the images can include street maps, satellite images, oblique view images, or combinations of these. [0051] The various map tiles are each associated with geographical locations, such that the computing device 230 is capable of selecting, retrieving and transmitting one or more tiles in response to receipt of a geographical location. The locations can be expressed in various ways including but not limited to latitude/longitude positions, street addresses, points on a map, building names, and other data capable of identifying geographic locations.

[0052] The map database 240 also can include points of interest. A point of interest can be any item that is interesting to one or more users and that can be associated with a geographical location. For instance, a point of interest can include a landmark, stadium, park, monument, restaurant, business, building, or other suitable point of interest. A point of interest can be added to the map database 240 by professional map providers, individual users, or other entities.

[0053] The map database 240 also can store street information. In addition to street images in the tiles, the street information can include the location of a street relative to a geographic area or other streets. For instance, it can store information indicating whether a traveler can access one street directly from another street. Street information can further include street names where available, and potentially other information, such as distance between intersections and speed limits.

[0054] The street level image database 242 stores street level images associated with the geographic locations. Street level images comprise images of objects at geographic locations captured by cameras positioned at the geographic location from a perspective at or near the ground level or street level. Although the term "street level" images is used, the images can depict non-street areas such as trails and others. The street level images can depict geographic objects such as buildings, trees, monuments, etc. from a perspective of a few feet above the ground. The street level images can be used to provide an immersive 360° panoramic viewing experience to a user centered around a geographic area of interest.

[0055] The street level image database 242 can store a plurality of different versions of each panoramic image (e.g., low resolution, medium resolution and/or high resolution versions). Street level image database 242 also can store a plurality of preview images associated with each panoramic image. The preview images can be any suitable image stored in any suitable format. The preview image can be provided to a user as the user views a 3D representation of a geographic area of interest to assist the user in deciding whether to download a separate version of corresponding street level imagery corresponding to the 3D representation. [0056] The images stored in street level image database 242 can be captured using any suitable technique. For instance, the street level images can be captured by a camera mounted on top of a vehicle, from a camera angle pointing roughly parallel to the ground and from a camera position at or below the legal limit for vehicle heights (e.g. 7-14 feet). Street level images are not limited to any particular height above the ground. For example, a street level image can be taken from the top of a building. Panoramic street level images can be created by stitching together the plurality of photographs taken from the different angles. The panoramic image can be presented as a flat surface or as a texture-mapped three dimensional surface such as, for instance, a cylinder or a sphere.

[0057] The street level images can be stored in the street level database 242 as a set of pixels associated with color and brightness values. For instance, if the images are stored in JPEG format, the image can be displayed as a set of pixels in rows and columns, with each pixel being associated with a value that defines the color and brightness of the image at the pixel's location.

[0058] Object position database 244 can include position information associated with the geographic objects depicted in the street level images. For instance, the position information can include information concerning the location and/or position of objects in the three-dimensional space defined by the street level imagery, latitude, longitude, and/or altitude of the geographic object, the orientation of the image with respect to user

manipulation, and/or other spatial information.

[0059] As an example, a separate object position data value(s) can be stored in the object position database 244 for each pixel of a street level image stored in street level image database 242, where the object position data value represents the geographic position of the surface of the object illustrated in that particular pixel. For instance, an object position data value representing latitude, longitude, and altitude information associated with the particular surface illustrated in the pixel can be associated with the pixel. In yet another aspect, the object position database 244 can include distance data that represents the distances of the surfaces of the object depicted in the street level imagery relative to the street level perspective. For instance, a value representing the distance from the perspective the image was acquired to a surface of the geographic object depicted in the street level image can be associated with each pixel.

[0060] In another aspect, the object position database 244 can include information associated with the locations of the surfaces depicted in street level images as polygons. In particular, a surface of an object depicted in the street view image can be defined as a polygon with four vertices. Each vertex can be associated with a different geographic object. A surface can be referenced in the object position database 244 as a set of vertices at the various geographic positions associated with the object.

[0061] Other formats for storing object position data or other surface information of the street level images also can be used. For instance, rather than being associated with absolute position values, such as latitude, longitude, and altitude, the values can be relative and in any scale. The locations of the surfaces of objects depicted in the street level images can be saved as polygons. Moreover, even if a first type of information is used (such as storing latitude, longitude, and altitude information for the surface) information of another type can be generated from the first type of information (such as differences between positions to calculate distances).

[0062] A variety of systems and methods can be used to collect the position information to be stored in the object position database 244. For instance, a laser range finder can be used. Alternatively, a three-dimensional model can be generated from a plurality of street view images using a variety of known techniques. For instance, stereoscopic techniques can be used to analyze a plurality of street level images associated with the same scene to determine distances at each point in the images. Once the relative locations of the points in the images are known, a three-dimensional model associated with the geographic area can be generated. The three-dimensional model can include information such as the location of surfaces of objects depicted in the street level imagery. Computing device 230 can access the three-dimensional model to provide position information to one or more client devices, such as computing device 210.

[0063] The database 238 also can include object labeling database 246 that provides text labels or other identifying information (colors, textures, geographic identifiers, and the like) for geographic objects depicted in street level images from street level image database 242 and corresponding object position data for the geographic objects stored in object position database 244. Object labeling database can include text labels such as building text labels 122, 124 and road text label 126 of FIG. 1. The information stored in object position database 244 and object labeling database 246 can be combined to create 3D representations of corresponding street level panoramic images. In some examples, the created 3D representations can be stored in 3D representation database 248. Each 3D representation stored in 3D representation database 248 can have a corresponding panoramic image stored in street level image database 242.

[0064] Additional databases also can be included in the computing system 200 of

FIG. 6 to implement the display of satellite images and corresponding 3D representations when operating a low-bandwidth operating mode of a service such as Google Earth as opposed to a providing street level images and corresponding 3D representations when operating a low-bandwidth operating mode of a service such as Street View within Google Maps. For example, a satellite image database can be provided in addition to or as an alternative to street level image database 242. Object position database 244, object labeling database 246, and 3D representation database 248 then can store information that correlates with corresponding satellite images stored within a satellite image database.

[0065] While the present disclosure is discussed with reference to interactive high- resolution imagery, such as panoramic street level imagery and/or satellite imagery, those of ordinary skill in the art, using the disclosures provided herein, should understand that the present subject matter is equally applicable for use with any type of geographic imagery, such as the imagery provided in a virtual globe application, oblique view imagery, or other suitable imagery.

[0066] It will be appreciated that the computer-executable algorithms described herein can be implemented in hardware, application specific circuits, firmware and/or software controlling a general purpose processor. In one embodiment, the algorithms are program code files stored on the storage device, loaded into one or more memory devices and executed by one or more processors or can be provided from computer program products, for example computer executable instructions, that are stored in a tangible computer-readable storage medium such as RAM, flash drive, hard disk, or optical or magnetic media. When software is used, any suitable programming language or platform can be used to implement the algorithm.

[0067] The technology discussed herein makes reference to servers, databases, software applications, and other computer-based systems, as well as actions taken and information sent to and from such systems. One of ordinary skill in the art will recognize that the inherent flexibility of computer-based systems allows for a great variety of possible configurations, combinations, and divisions of tasks and functionality between and among components. For instance, server processes discussed herein can be implemented using a single server or multiple servers working in combination. Databases and applications can be implemented on a single system or distributed across multiple systems. Distributed components can operate sequentially or in parallel.

[0068] While the present subject matter has been described in detail with respect to specific example embodiments thereof, it will be appreciated that those skilled in the art, upon attaining an understanding of the foregoing can readily produce alterations to, variations of, and equivalents to such embodiments. Accordingly, the scope of the present disclosure is by way of example rather than by way of limitation, and the subject disclosure does not preclude inclusion of such modifications, variations and/or additions to the present subject matter as would be readily apparent to one of ordinary skill in the art.

Claims

WHAT IS CLAIMED IS:

1. A computer-implemented method of providing an interactive digital mapping application in a limited bandwidth environment, comprising:

providing for display, by one or more computing devices, on a display of a computing device a three-dimensional representation of a geographic area viewed by a user of an interactive digital mapping application;

providing for display, by the one or more computing devices, download information in the interactive digital mapping application, wherein the download information comprises a quantifiable parameter associated with downloading one or more versions of imagery corresponding to the three-dimensional representation;

receiving, by the one or more computing devices, user instructions requesting that one or more versions of the imagery corresponding to the three-dimensional representation be downloaded for display;

downloading, by the one or more computing devices, the one or more requested versions of the imagery corresponding to the three-dimensional representation; and

providing for display, by the one or more computing devices, the one or more requested versions of the imagery corresponding to the three-dimensional representation.

2. The computer-implemented method of claim 1, wherein the quantifiable parameter associated with downloading comprises one or more of resolution version, bandwidth, file size, download time or download cost of the one or more versions of imagery corresponding to the three-dimensional representation.

3. The computer-implemented method of claim 1 or 2, further comprising

identifying, by the one or more computing devices, a user request to operate the interactive digital mapping application on the computing device in a limited bandwidth mode of operation.

4. The computer-implemented method of any one of claims 1 to 3, wherein the three- dimensional representation of a geographic area comprises a wireframe representation of the exterior features of geographic objects included within the geographic area viewed by the user.

5. The computer-implemented method of claim 4, wherein the geographic objects for which a wireframe representation are provided for display comprise one or more of buildings or roads.

6. The computer-implemented method of claim 4 or 5, wherein the wireframe representation is constructed from three-dimensional point data of the geographic area obtained at the same time as the images of the geographic area.

7. The computer-implemented method of any one of claims 4 to 6, wherein the three- dimensional representation of the geographic area comprises text labels identifying one or more of road names or building names.

8. The computer-implemented method of any one of claims 4 to 7, wherein the imagery comprises panoramic street level images of the geographic area and wherein the wireframe representation is constructed from three-dimensional point data of the geographic area obtained at the same time as the panoramic street level images of the geographic area.

9. The computer-implemented method of any one of claims 1 to 7, wherein the imagery comprises satellite imagery of the geographic area.

10. The computer-implemented method of any one of claims 1 to 9, wherein a first version of the imagery corresponding to the three-dimensional representation comprises a low-resolution version of the imagery, wherein a second version of the imagery

corresponding to the three-dimensional representation comprises a high-resolution version of the imagery, and wherein the download information is provided in the interactive digital mapping application for both the low-resolution version and the high-resolution version of the imagery.

11. The computer-implemented method of any one of claims 1 to 10, further comprising automatically determining, by the one or more computing devices, a current communication capacity for a communication channel of the computing device over which information within the interactive digital mapping application is made available to the user.

12. The computer-implemented method of claim 11, wherein the current

communication capacity for the communication channel comprises one or more of download speed, channel capacity, signal bandwidth, or connection latency.

13. The computer-implemented method of claim 12, further comprising selecting one or more versions of the imagery corresponding to the three-dimensional representation for which to provide download information based on the determined communication capacity.

14. A computing device, comprising:

one or more processors; and

one or more memory devices, the one or more memory devices storing computer-readable instructions that when executed by the one or more processors, cause the one or more processors to perform a method according to any one of the preceding claims

15. One or more tangible, non-transitory computer-readable media storing computer- readable instructions that when executed by one or more processors cause the one or more processors to perform operations according to any one of claims 1 to 13.