JP7546156B2 - Device recognition communication request - Google Patents

Description

CROSS-REFERENCE TO RELATED APPLICATIONS
This application claims priority to U.S. patent application Ser. No. 17/039,193, entitled "Device-Aware Communication Request," filed on September 30, 2020, U.S. patent application Ser. No. 17/039,405, entitled "Context-Based Communication Request," filed on September 30, 2020, and U.S. patent application Ser. No. 17/039,542, entitled "Sensor-Based Layout Generation," filed on September 30, 2020, the entire contents of which are incorporated by reference herein.

Increasingly, merchants are utilizing electronic devices and associated computing applications to assist in the operation of their businesses. For example, retail stores often use point of sale (POS) devices to process payments, update and/or access inventory levels. Restaurant establishments may also equip servers and other staff with mobile POS devices to order items on behalf of customers, schedule the preparation of items, and the like. As the use of these devices increases, merchants also desire more intelligent applications that run on the devices.

The features of the present disclosure, its nature and various advantages will become more apparent from the following detailed description taken in conjunction with the accompanying drawings.

1 illustrates an exemplary environment in which a first user in an establishment operates a back-of-house (BOH) device to request a server to place an order whose details are presented on the BOH device, in response the BOH device can determine which front-of-house (FOH) device placed the order, establish a communications session with the FOH device, and transmit the server's request to the FOH device.

FIG. 2 illustrates example components of the BOH and/or FOH devices described herein for carrying out the techniques described herein.

FIG. 3 illustrates a flow diagram of an exemplary process for establishing a communications session between a BOH device and an FOH device for transmitting a request to the FOH device regarding an order placed by the FOH device.

FIG. 4 shows a flow diagram of another exemplary process for establishing a communications session between a BOH device and an FOH device for sending an indication to the FOH device that an item associated with an order placed by the FOH device has been prepared.

FIG. 5 shows a flow diagram of another exemplary process for establishing a communications session between a BOH device and an FOH device to send an indication to the FOH device that a user has begun preparation of an item associated with an order placed by the FOH device.

FIG. 6 illustrates a flow diagram of another example process for receiving a request to activate an appliance to process a current capacity of an order.

7 illustrates an exemplary user interface (UI) that a BOH device may display to present current orders to be prepared. FIGURE 7 also illustrates that an operator of the BOH device may select to view details regarding the first displayed order.

Figure 8 illustrates another exemplary UI that a BOH device may display in response to the selection shown in Figure 7, where the selected order has been expanded to present additional details about and options regarding the order.

Figure 9 shows another example UI that the BOH device can display after the UI shown in Figure 8. Here, the UI provides options for the operator of the BOH device, including an option to request the server that placed the order.

FIG. 10 illustrates another exemplary UI that an FOH device may display in response to an operator selection of the BOH device shown in FIG.

11 illustrates another exemplary UI that a BOH device associated with a preparation station may display, where an operator of the BOH device selects items associated with a particular order.

Figure 12 illustrates another exemplary UI that may be displayed by a BOH device associated with a preparation station in response to the selection of Figure 11. As illustrated, the BOH device may present the operator with several options for the operator to select, including an option that requests that the current item be created.

Figure 13 illustrates another exemplary UI that a BOH device associated with a preparation station may display prior to selection at the preparation station shown in Figure 12. As shown, this UI may prioritize the preparation of different orders within the facility.

Figure 14 illustrates another exemplary UI that the BOH device of Figure 13 may display following a selection at the preparation station shown in Figure 12. As shown, this UI includes an indication that the expeditor has requested that the preparation station begin preparation of a particular item.

Figure 15 illustrates another exemplary UI that the BOH device of Figure 13 may display after the device operator selects the option to initiate an order requested by an expedient via the UI of Figure 14. As shown, the displayed orders have been re-prioritized in response to this selection.

16 illustrates another exemplary UI that a BOH device may display, where the UI presents various kitchen performance metrics.

17 illustrates another exemplary UI that a BOH device may display, where the UI presents specific suggestions for facility operations, such as suggestions to activate additional fryer stations. The UI also includes an icon for automatically activating the suggested fryer stations.

FIG. 18 shows another exemplary UI that a BOH device may display after a device operator selects the icon shown in FIG.

Figure 19 illustrates another example UI that the BOH device may display after the UI shown in Figure 18. As shown, the UI indicates that orders previously assigned to other flyer stations have been moved to the recently activated flyer station.

20 illustrates another exemplary UI that a BOH device may display, where the UI shows a particular order and an icon to indicate that the particular order has been initiated.

21 illustrates another exemplary UI that an FOH device may present to inform an operator of the FOH device of the current status of orders the operator has placed with the BOH. For example, the UI may show that one order is ready for delivery, another is nearly complete, and a third is half complete.

FIG. 22 shows another exemplary UI that an FOH device may display to provide information to an operator of the FOH device.

23 illustrates the example environment of FIG. 1, where a first FOH device is proximate a first location within the facility associated with a first order, and thus presents a request or proposal for the server of the first FOH device to perform a particular action associated with the first order, and further, a second FOH device is proximate a second location associated with a second order, and thus presents a request or proposal for the server of the second FOH device to perform a second, different action associated with the second order.

FIG. 24 illustrates a flow diagram of an exemplary process for determining that a particular FOH device is within a threshold proximity of a first location and, in response, sending a UI to the FOH device requesting or suggesting a particular action related to the first location.

FIG. 25 shows a flow diagram of another exemplary process for determining that an FOH device is within a threshold vicinity of a second location and, in response, sending a second UI to the FOH device requesting or suggesting a particular action associated with the second location.

FIG. 26 shows a flow diagram of another exemplary process for determining that an FOH device is within a threshold vicinity of a third location and, in response, sending a third UI to the FOH device requesting or suggesting a particular action associated with the third location.

FIG. 27 shows a flow diagram of another exemplary process for enabling an operator of an FOH device to add items to an order.

FIG. 28 shows a flow diagram of another example process for determining items to suggest adding to a first order and presenting the suggestions on an FOH device in response to the FOH being within a threshold proximity of a first location associated with the first order.

FIG. 29 shows a flow diagram of another example process for determining the identity of an operator of an FOH device, determining whether the identity is associated with previous behavior, and if so, presenting a UI that includes a customized set of actions to the operator based on the previous behavior.

FIG. 30 shows a flow diagram of another example process for determining whether an FOH device is physically coupled to or within a threshold vicinity of another device, and if so, presenting a UI indicating a sequence of actions to be taken.

31 illustrates an exemplary UI that may be present in response to an FOH device being located near a threshold of a particular location within a facility. As shown, the UI suggests that the operator of the FOH device print invoices for customers located near the particular location.

32 illustrates another exemplary UI that may be present in response to an FOH device being located near a threshold of a particular location within a facility. As shown, this UI suggests that the operator of the FOH device request to launch a course ordered by a customer located near the particular location.

33 illustrates another example UI that may be present in response to an FOH device being located near a threshold of a particular location within a facility. As shown, this UI suggests that the operator of the FOH device request to open a table or order associated with the particular location if the table is not currently associated with an open order.

34 shows another exemplary UI that an FOH device may present. As shown, this UI shows items available for ordering and the layout of the establishment. Using this UI, an operator of the FOH device can add items to an order associated with a particular table or seating area by dragging an icon associated with the item to an icon associated with the table or seating area.

FIG. 35 shows another exemplary UI that the FOH of FIG. 34 may present when an operator drags an icon associated with an item towards an icon associated with a table or seating area.

FIG. 36 shows another exemplary UI that the FOH of FIG. 34 may present immediately before the operator "drops" an icon associated with an item onto an icon associated with a table or seating area.

Figure 37 shows another example UI that the FOH of Figure 34 may present after the UI of Figure 36, where an item ("CHIPS & GUAC") has been added to the order associated with Table 2. In addition, the UI presents a timer set for a predefined amount of time, after the expiration of which the FOH device can send the order to the BOH device for preparation of the order.

FIG. 38 shows another exemplary UI that the FOH of FIG. 34 can present, indicating that the operator of the FOH device has added another item to Table 2.

39 illustrates another exemplary UI that the FOH device of FIG. 34 may present after the operator adds a second item, where an item (the "bud bucket") has been added to the order and the timer has been reset in response to the second item being added to the order.

40 illustrates another exemplary UI that the FOH device of FIG. 34 may present after an operator of the FOH device selects the icon associated with Table 2. Here, the UI includes an operation to send the order to the BOH device to prepare the order before the expiration of the timer.

41 illustrates another exemplary UI that an FOH device may present, where the UI includes suggested items or actions that may be presented in response to the FOH device being within a threshold proximity of a particular location within a facility.

42 illustrates another exemplary UI that an FOH device may present, where the UI includes different suggested items or actions that may be presented in response to the FOH device being within a threshold proximity of a particular location within a facility.

43 shows another exemplary UI that an FOH device may present, where the UI includes different actions that an operator can perform.

44 shows another exemplary UI that the FOH device of FIG. 41 may present, where an operator of the FOH device requests to perform a suggested action, which may be customized based on previous behavior of the current operator of the FOH device.

Figure 45 shows another example UI that the FOH device of Figure 44 may be in after the selection shown in Figure 44. Here, the UI shows details regarding hints accepted by the operator of the FOH device, and an icon for viewing details regarding the operator's movements.

Figure 46 shows another example UI of the FOH device after the selection shown in Figure 45. Here, the UI shows shift details and includes an icon to allow the operator to run shift reports.

Figure 47 shows another example UI of the FOH device after the selection shown in Figure 46. Here, the UI shows the FOH device running a shift report for the operator.

FIG. 48 shows another example of the UI of an FOH device after execution of the shift report shown in FIG.

FIG. 49 shows an example UI in which an FOH device may exist prior to being coupled to or within the threshold vicinity of another device.

Fig. 50 illustrates another example UI in which the FOH device of Fig. 49 may reside after being physically docked to another device, such as a cradle. As shown, the FOH device may automatically begin running a shift report in response to being physically docked to the cradle.

FIG. 51 continues the description of FIG. 50 and shows the chip being transferred to an account associated with the operator of the FOH device.

FIG. 52 completes the explanation of FIG. 50 and shows that the operator has been logged out from the FOH device.

53 illustrates another example UI that may be executed in response to an FOH device being coupled to or within a threshold vicinity of another device. In this example, the FOH device detects that it is within a threshold vicinity of a table side device (e.g., another FOH device) and therefore presents a suggestion to open an order.

FIG. 54 shows an example UI that the FOH device of FIG. 53 may present in response to opening an order.

55 and subsequent UIs illustrate different gestures that may be associated with different actions, where the UI shows different options related to a particular item ("Burger"), and the FOH device operator selects one particular option ("Lettuce").

FIG. 56 continues the UI diagram of FIG. 55 and shows the operator of the FOH device swiping right on the selected icon.

FIG. 57 continues the UI description of FIG. 56 and shows that the selected option ("Lettuce") has been added to the item in response to the operator swiping right on the icon.

Figure 58 continues the UI description of Figure 57 and shows the operator swiping left on the icon associated with the selected option. In response, the UI indicates that the selected option ("Lettuce") has been removed from the item.

FIG. 59 shows the operator selecting another exemplary option ("Onion").

FIG. 60 continues the UI description of FIG. 59 and shows the operator swiping upward on an icon associated with an option.

Figure 61 continues the description of the UI of Figure 60, showing that an extra amount of selection option ("onion") has been added to the order in response to an upward swipe. Additionally, the UI shows the operator swiping downward on the icon, in response to which options are provided on the side.

FIG. 62 continues the description of the UI of FIG. 61 and shows that the extra amount of the selection option has been changed to the side of the selection option in response to a downward swipe of the icon.

63 and subsequent UIs show other example UIs and gestures for performing different actions, where the FOH device operator has selected one example option ("bacon").

Figure 64 shows an exemplary UI that an FOH device may present in response to the selections shown in Figure 63, where different modifiers associated with the options are presented.

Figure 65 continues the UI description of Figure 64 and shows the operator dragging an option ("Bacon") onto one of the presented modifiers ("Extras") In response, the icon associated with the selected modifier expands.

FIG. 66 continues the UI description of FIG. 65 and shows that the icon corresponding to a selected modifier displays an optional indication in addition to the modifier itself ("Extra Bacon").

Figure 67 shows an example UI that the FOH can present after the UI of Figure 66. Here, the modifiers are removed from the display, but the initial UI of Figure 63 has been modified to indicate the selection made above.

68 and subsequent UIs show yet other example UIs and gestures for performing different actions, where the FOH device operator has selected one example option ("Lettuce").

Fig. 69 continues the description of the UI of Fig. 68 and shows that several modifiers have been presented adjacent to the selected icon ("Lettuce"), and further shows the operator swiping upward from the selected icon.

Figure 70 continues the description of the UI of Figure 69 and shows that the positions of modifiers have changed in response to the swipe up action of Figure 69. For example, the modifier named "Extra" is now adjacent to the "Lettuce" icon instead of "Add," as it was previously adjacent to this icon.

FIG. 71 continues the UI description of FIG. 70 and shows the example UI after a user swipes downwards, causing the "none" modifier to be adjacent to the "lettuce" icon.

Figure 72 shows an exemplary UI that the FOH device may present following the selection of Figure 71. As shown, the UI indicates that the item should have "no lettuce." Additionally, the operator has selected an icon corresponding to another option ("fried onions").

Fig. 73 shows an example UI that the device may present after the selection of Fig. 72. Again, the modifiers may appear adjacent to the icon of the selected option, and the operator may select one of the modifiers by swiping up and/or down until the desired modifier is adjacent to the selected option.

FIG. 74 illustrates the example environment of FIG. 1, where a merchant operates a merchant device to capture data (e.g., image data) of the establishment. The merchant device then transmits the data to a remote service, which analyzes the data to identify flat surfaces in the environment and transmits a UI to the merchant device to indicate which flat surfaces correspond to seating areas and which do not. After receiving respective instructions from the merchant, the service can generate and present on the merchant device a layout of the available seating areas in the establishment.

FIG. 75 illustrates a flow diagram of an exemplary process for generating a layout of available seating areas within a facility.

FIG. 76 illustrates a flow diagram of another exemplary process for determining a proposed layout of available seating areas within a facility.

FIG. 77 illustrates a flow diagram of another example process for generating an example seating area layout within a facility.

78 illustrates an exemplary UI that a merchant device may present after scanning a merchant's facility. As shown, the UI indicates which surfaces within the facility have been identified as flat surfaces and allows the merchant to select which of the identified flat surfaces are actually available seating areas.

Fig. 79 shows an example UI after the merchant has indicated that all three tables are available seating areas via the UI from Fig. 78. This UI also provides the merchant with the option to continue scanning the facility or generate a layout of the available seating areas.

Fig. 80 illustrates an exemplary UI that a merchant device may present after a merchant requests to view the generated layout via the UI of Fig. 79. As shown, this UI presents the generated layout of the seating areas available at the merchant's establishment.

FIG. 81 is an example network environment in which the described techniques may be implemented.

FIG. 82 is an example hardware and/or software module that may implement the described techniques.

In the figures, the left-most digit(s) of a reference number identifies the figure in which the reference number first appears. Use of the same reference number in different figures indicates similar or identical items or features. The drawings are not to scale.

As mentioned above, more and more merchants are utilizing electronic devices and associated computing applications to assist in running their businesses. For example, retail stores often use point of sale (POS) devices to process payments, update and/or access inventory levels. Restaurant establishments may also equip servers and other staff with mobile POS devices to order items on behalf of customers, schedule the preparation of items, etc.

Despite the use and advancements of these devices, these devices often operate in an independent manner. For example, a server in a restaurant may utilize a mobile front-end of house (FOH) device to take orders from customers and submit them for preparation to a back-of-house (BOH) device that is not generally visible to the customers. Meanwhile, a cook, chef, expedient, or other user of the BOH device may use the BOH device to view current orders in order to prepare the orders. In still other instances, a host or hostess may utilize a separate FOH device to maintain a waiting list for customers waiting to be seated at the establishment. In general, however, these devices provide functionality for a given task and often do not interact with other devices to improve the general operation of the establishment.

This disclosure describes, in part, techniques for enabling these types of merchant devices to interoperate with one another in a more robust manner, such that the overall operation and efficiency within the merchant establishment is improved through at least intelligent, dynamically changing device-to-device and device-to-network connectivity. For example, the techniques described herein may enable intelligent routing of requests between devices within a merchant establishment based on the real-time location of the devices, which devices are associated with which orders, the proximity of the devices to customers or customer locations (e.g., seating areas), and the like. Additionally, such routing may be optimized based on contextual information such as time of day, network usage, device usage, and other such metrics. To provide an example, a first server within a restaurant may operate a first FOH to input orders into a BOH device. Meanwhile, a second server may operate a second FOH to also input orders into the BOH device. The BOH device may receive orders from each of these FOH devices (and potentially many others) and may present details regarding each order that needs to be prepared for the respective customer on the display of the BOH device. For example, the BOH device may display a prioritized list of orders so that one or more BOH employees at a restaurant can view the display and decide how and in what order to prepare the orders.

However, if a cook, chef, food runner, expediter, or other BOH employee has a request regarding an order, the employee may utilize the BOH device to request clarification. For example, the employee may select an order or individual items from an order from the BOH device and enter a request to speak with a server responsible for placing the order. In response to receiving this request, the BOH device may determine which of multiple FOH devices placed the order with the BOH device. In response to identifying the responsible FOH device, the BOH device may establish a dedicated communications session with the FOH device and send the request or query to the FOH device. For example, the BOH device may establish a short-range wireless connection (e.g., Bluetooth, Zigbee, etc.) with the FOH device, may establish a connection over Wi-Fi, may establish a connection over the Internet, etc. to route the request to the FOH device. In some embodiments, the type of connection is based at least on device characteristics, the nature or size of the data packets being transmitted over the channel, etc.

The FOH device may then receive the request from the BOH device and may output data indicative of the request, such as displaying the request on a display of the FOH device. For example, the FOH device may present an indication that a particular BOH device associated with a particular location in the restaurant has requested clarification regarding a particular order placed by the FOH device. The server may then travel to the specified location in response to the request to clarify the order. In another example, a communication session may enable a back-and-forth dialogue between an operator of the BOH device and a server operating the FOH device. For example, the communication session may enable video chat, voice chat, instant messaging chat, or any other type of two-way interaction involving audio, visual, or tactile input from one or both parties.

Thus, the techniques described herein can create real-time or near real-time communication sessions between devices, with the initiating device automatically identifying the device to connect to. As used herein, a real-time communication session can refer to a session that is generated substantially simultaneously with an event (e.g., within 1 second, 100 milliseconds, etc., of a communication request), while a near-real-time communication session can refer to a session that is generated within a larger threshold amount of time after the event (e.g., within a few seconds, within 1 minute, etc., of a communication request). Furthermore, the techniques described herein allow for switching between devices, thereby creating unique sessions one after the other, and sometimes even allowing multiple devices to communicate with each other for a given period of time or a particular order. For example, a BOH device can automatically identify an FOH device to establish a communication session based on the FOH device that previously entered the order in question. In another example, the BOH device can automatically ping the FOH device that is closest to the table where the order in question is placed. The created communication session can remain open for a given amount of time (e.g., 1 minute, 5 minutes, etc.), such as until completion of the request. The automatic establishment of communication sessions in real time provides at least some of the technological advances discussed herein. For example, the creation of these real-time communication sessions creates groups of interoperable devices, rather than individual devices that typically function alone. Furthermore, by automatically identifying the appropriate FOH device to connect to, the present technology eliminates the need to analyze details about individual orders to determine which server or device served a particular order. Instead, the operator of the BOH device may issue a request, and the BOH device may automatically create the appropriate communication channel. These efficiencies may be particularly important in establishments such as restaurants that handle large volumes of orders and move very quickly. Furthermore, using the techniques described herein, FOH devices can identify and initiate connections with other FOH devices and/or BOH devices, while BOH devices can similarly identify and initiate connections with other BOH devices and/or FOH devices.

In other instances, the techniques described herein may utilize device locations within an establishment to issue commands or suggestions on devices. For example, one or more of the devices operating at an example restaurant and/or a remote service coupled to these devices may generate a layout of the orders and their respective locations within the restaurant and may use this layout to determine when to issue commands or suggestions and/or to which devices to issue commands or suggestions. For example, the layout may maintain an association between a first order for a first table associated with a first location within the restaurant and a second order for a second table associated with a second location within the restaurant.

The device(s) and/or remote service may then determine to take an action regarding one of the orders, such as the first order. For example, the device or remote service may determine that a customer associated with the first order is likely ready to receive a bill based on a predetermined amount of time that has elapsed since items associated with the first order were completed and/or delivered to the customer. After making this determination, the device and/or remote service may determine when the FOH is within a threshold vicinity of a first location associated with the first order. For example, the device or remote service may determine when a particular FOH device is within 10 feet of a table associated with the first order based on data from one or more beacons in the restaurant that determine the device's location based on the device's GPS signal, etc.

After determining that the FOH is within a threshold vicinity of the first location, the device and/or remote service may send instructions to the FOH device of a suggestion to print a bill for the customer associated with the first order. For example, the FOH device may present on a display of the FOH device a suggestion to print a bill for the first table and a selectable icon to cause the bill to be printed (e.g., printed on paper via the FOH device or another device, printed on the display of the FOH device, etc.).

In another example, the device or remote service can determine a second action to take with respect to a second order associated with a second location in the restaurant. For example, the second action can include suggesting a particular course to a customer at the second location, initiating a course for preparation at the restaurant's BOH (i.e., sending the order to the BOH device), etc. When the device or remote service determines that the same or another FOH device is within a threshold proximity of the second location, that FOH device can receive a suggestion to perform the action and potentially a selectable icon to perform the action.

By integrating both computational techniques (e.g., for determining proposed actions to take with respect to different orders) and proximity-based techniques (e.g., for determining to which device the proposed actions should be sent), the disclosed method and system provide an intelligent way to detect related pairs of devices for dedicated, possibly timed, communication sessions. That is, these techniques provide at least a technological advancement by sending notifications to appropriate devices based on their current location at the appropriate time, based on the history and timing of individual orders. Thus, the present techniques result in sending communications to devices when communication is feasible, i.e., when the device operator is near a location where communication is feasible, and when communication is feasible.

In still other cases, the techniques described herein can analyze image data of a facility, such as the exemplary restaurant described above, to automatically generate a layout of available seating areas within the facility. For example, the techniques can provide a UI for display on a merchant device and instruct an operator of the merchant device to use his or her camera to walk around and image the interior of the facility. The merchant device can then analyze this data to determine a layout, or can send this data to another device or remote service for analysis. In either case, the device can analyze the data using light detection and ranging (LIDAR), trained machine learning models, etc. to identify relevant surfaces (such as flat surfaces) within the facility. For example, the device can identify one or more surfaces in the image data that are flat, have a predetermined size (e.g., width and length), are oriented approximately parallel to a plane defined by the floor of the facility, and are within a predetermined length above the plane of the floor (e.g., at least 2 feet above the floor). Thus, the device can analyze the data to identify surfaces that may correspond to seating areas for customers, such as tables, bar stools, countertops, etc. In addition to identifying such surfaces, the device performing the analysis may determine a location (e.g., an X, Y location) associated with each respective surface. For example, the device may use LIDAR, computer vision techniques to determine the location of each identified surface within the facility's environment.

After analyzing the data to identify seating or e-commerce surfaces therein, the technology can provide a UI for display on the merchant device. In some examples, the UI includes indicators of the identified surfaces overlaid on the actual surfaces in the image data captured by the merchant device. For example, the UI can overlay a predefined color, cross-hatching, etc., on the identified surfaces overlaid on the image data of the establishment. The UI can further include instructions to an operator of the device to select which of the indicated surfaces correspond to available seating areas. Thus, an operator of the merchant device can use, for example, a touch screen or other input mechanism on the merchant device to select surfaces that correspond to available seating areas and/or surfaces that do not, enabling the virtual layout of the establishment with recommended seating and dining surfaces to be converted into a physical layout.

After selecting the surfaces that correspond to the available seating areas, the operator of the merchant device may indicate that they are finished indicating the available seating areas and request that a layout of the available seating areas be automatically generated. In response to this indication, the merchant device, another device, or a remote service may automatically generate a layout of the available seating areas in the establishment. For example, the device may determine a location associated with each surface indicated to correspond to a seating area and use this location and the estimated size of the surface to generate the layout. In some instances, the merchant may also indicate the number of available seats on each flat surface (e.g., each table), and the layout may indicate the number of these corresponding seats.

After generating the layout, the layout may be presented on a UI on the display of the merchant device. The presented layout may include, for example, a top-down two-dimensional view of the available seating areas in the establishment. In some instances, this UI may also provide tools to allow the merchant to modify or correct any errors in the layout. For example, the tools may allow the merchant to move, add, or remove tables, seats, etc.

While conventional techniques allow a merchant to manually generate a layout of available seating areas, the techniques described above use technology on one or more devices to generate image data of an environment and automatically generate a layout of available seating areas within the environment. Thus, these techniques provide an efficient and coordinated solution for generating facility layouts, and do so through technological advances. For example, the present techniques allow a merchant to generate a layout by capturing image data and allowing one or more devices to analyze the image data to identify flat surfaces, and based on these flat surfaces, automatically create the layout. Thus, the layout is created automatically, not manually, and is done much more quickly and efficiently than through manual methods. Furthermore, such layouts may be tied to devices to provide some of the functionality disclosed herein. For example, one section displayed in the layout may be tied to a pair of FOH and BOH devices, while another section may be associated with another pair of FOH and BOH devices. Such predefined associations may be leveraged, for example, when communicating for an order.

Further technical improvements are described below in the discussion of Figures 1-82. These examples are described with reference to a restaurant environment, but it should be understood that the techniques apply equally to other types of merchant and non-merchant environments.

Device recognition request

FIG. 1 illustrates an example architecture 100 for implementing the techniques described herein. The example architecture 100 includes a merchant facility 102, such as a restaurant, having a front-of-house (FOH) 102(1) and a back-of-house (BOH) 102(2). As is known, the FOH 102(1) corresponds to a portion of the facility 102 where customers interact with the merchant's staff (e.g., ordering items, etc.) and can include both merchant devices accessible by the customer or the merchant, or customer devices accessible by the customer. The BOH 102(2) corresponds to a portion of the facility 102 where staff prepare items on behalf of customers and are accessible by the merchant. In some examples, the merchant's staff uses merchant devices (e.g., devices configured as point-of-sale (POS) devices to conduct transactions with customers, kitchen display devices to display pending orders, etc.) in both the FOH 102(1) and the BOH 102(2). For example, FIG. 1 shows that user 104(1) can operate BOH device 106(1), and users 104(2) and 104(2) can operate respective FOH devices 106(2) and 106(3). While this example and the following examples describe the techniques in the context of a restaurant environment, it should be understood that the techniques may be equally applicable to other retail and non-retail establishments that may utilize different types of devices (e.g., BOH/FOH devices, etc.), such as brick-and-mortar retail establishments, warehouses, grocery stores, and/or other retail and non-retail establishments.

In this example, each of the devices 106(1)-(3) is configured with a facility application to perform an array of functions within the facility. For example, the BOH device 106(1) may be configured to display details regarding orders associated with a respective customer within the FOH 102(1). The BOH device 106(1) may also be configured to establish communication sessions with other BOH/FOH devices, activate appliances therein, and perform other actions, as described in more detail below. Meanwhile, the FOH devices 106(1)-(2) may establish communication sessions with other BOH/FOH devices, generate orders on behalf of customers, transmit orders to the BOH devices, act as POS devices to conduct transactions with customers, display layouts of available seating areas within the facility, and perform other actions, as described in more detail below. In some cases, other devices, such as a remote server, may initiate communication sessions with the BOH and/or FOH devices. For example, the remote server may determine when inventory levels are running low and establish communication sessions with the BOH devices to provide corresponding notifications.

In this example, user 104(1) may use BOH device 106(1) to view details related to an order provided to BOH device 106(1) by a particular FOH device. Further in this example, user 104(1) may have a request regarding the order and therefore may use UI 108(1) to request a server related to the order or proximate to the table where the order was placed. In response to this request, BOH device 106(1) (or another device) may determine the FOH device associated with the order. That is, BOH device 106(1) or another device may determine which FOH device provided the order to BOH device 106(1) from the details related to the order that user 104(1) wishes to clarify.

In this example, the BOH device 106(1) may determine that the FOH device 106(2) operated by the user 104(2) has provided an order and, therefore, may create a connection 110 between the BOH device 106(1) and the FOH device 106(2). That is, the BOH device 106(1) may initiate and/or establish a communication session between the BOH device 106(1) and the FOH device 106(2), or vice versa. This communication session may comprise a connection via a short-range wireless network (e.g., Bluetooth®, Zigbee®, etc.), a connection via Wi-Fi, a connection via the Internet, etc. After establishing the connection 110, the BOH device 106(1) or another device may present on the FOH device 106(2) a UI 108(2) indicating that a request, such as a description regarding a particular order, has been requested by the BOH device. In some instances, this UI 108(2) may indicate the order for which clarification is requested, the BOH device 106(1) that initiated the request, the location of the BOH device 106(1) within the facility (e.g., hot station, cold station, etc.), and/or the like.

...

Using the techniques illustrated in FIG. 1, the BOH device 106(1) automatically identifies the FOH device 106(2) on which the user 104(1) has provided an order with which he has a question, and in response to identifying this FOH device, automatically establishes a communications session with the identified FOH device. In some instances, some or all of these techniques are performed by an application executing on the BOH/FOH device. In other instances, some or all of these techniques are performed by a remote service 112 communicatively coupled to the BOH/FOH device via one or more networks 114. The network(s) may represent any combination of wired and/or wireless networks. Additionally, as described herein, a "merchant" may be any entity that offers items (e.g., goods or services) for purchase or other means of acquisition (e.g., rent, borrow, negotiate, etc.). A merchant may offer items for purchase or other means of acquisition via a brick-and-mortar store, a mobile store (e.g., pop-up shop, food truck, etc.), an online store ("e-commerce"), combinations thereof, and the like. In a peer-to-peer (P2P) scenario, the merchant and customer may be a first P2P customer and a second P2P customer, respectively, who can transfer money to each other via a P2P payment application. In some instances, the merchant may also include a consumer who purchases one or more items (e.g., using a mobile payment application) and procures these items on a personal basis using the techniques described herein.

Furthermore, in some examples, the facility 102 may include one or more network-connected devices 116 (e.g., "smart appliances"), such as smart ovens, microwaves, fryers, dishwashers, etc. In some examples, these appliances may also communicate with the BOH device, the FOH device, and/or the remote service 112. For example, the user 104(1) may, in some instances, use the BOH device 104(1) to turn one or more of the appliances 116 on or off, control the appliances 116, change settings of the appliances 116, and/or the like. In some examples, as described below, the BOH device 106(1) may present one or more UIs to notify the user 104(1) when the capacity of one or more appliances has been reached, and in response may suggest activation of at least one additional appliance.

In some instances, the remote service 112 may act as a payment processing service that receives and processes payment transactions on behalf of the facility 102 as well as on behalf of other merchants 118. In this regard, the remote service 112 may receive transaction information for transactions carried out between multiple merchants and multiple purchasers from respective merchant computing devices associated with the multiple merchants, respectively. For example, the remote service 112 may receive this transaction information from devices within the facility 102, devices associated with the other merchants 118, and/or the like. In addition, each of the merchant computing devices may have a respective instance of a merchant application installed thereon to respectively configure the multiple merchant application devices as multiple point of sale (POS) terminals, with these respective instances of the merchant application configuring the respective POS terminals to communicate transaction information to a payment processing system (i.e., the remote service 112) over one or more networks. In addition, the remote service 112 may use the trained model to generate multiple merchant profiles for each of the merchants. The merchant profile may store information regarding merchant type, merchant location, merchant inventory, and/or other data related to each merchant, while the trained model may be trained using at least one of multiple merchants, multiple purchasers, transaction information, and/or additional information.

FIG. 2 illustrates exemplary components of a BOH and/or FOH device 200 described herein for performing the techniques described herein. Additionally, while this figure illustrates these components as present in the FOH and/or BOH device, it should be understood that some or all of these components may additionally or alternatively reside in the remote service 112 and/or other locations. As illustrated, each device 200 may include one or more processors 202, one or more interfaces 204 for communicating with other devices, and one or more sensors (e.g., cameras, GPS units, etc.) for generating data sensor data. Additionally, each device 200 may include a computer-readable medium 206 for executing an instance of a facility application 210, which may enable the techniques described above and below. In some instances, the FOH device may store and execute an instance of the facility application 210 configured to enable the FOH operations described herein, while the BOH device may store and execute an instance of the facility application 210 configured to enable the BOH operations described herein. In other instances, the facility application 210 may be common across both BOH and FOH devices. The BOH and/or FOH devices may comprise any type of computing device, such as a tablet computing device, a mobile phone, a laptop computer, a desktop computer, a server computer, a dedicated computer, and/or the like. In some instances, the FOH device may comprise a device operated by a merchant, while in other devices the FOH device may be operated by a customer, such as a customer at a table or other seating area. Additionally, in other instances, the FOH device may include a first display operable by the merchant and a second display operable by the customer.

In this example, the facility application 210 includes an order processing component 212, a location determination component 214, a device selection component 216, a communication session component 218, order data 220, location data 222, and analytics data 224. The order processing component 212 may enable an array of features described and illustrated below, such as enabling FOH devices to enter customer orders and enabling BOH devices to view and interact with orders. In general, the order processing component 212 is responsible for presenting some or all of each UI shown and described below.

The location determination component 214 may be used to determine the location of one or more BOH and/or FOH devices in order to determine one or more actions to take with respect to these devices. The location determination component 214 may function to determine the location of the devices using GPS data, using triangulation techniques based on beacons or other devices within the facility 102, or via any other location technique. In some instances, as described below, the facility application 210 may send a UI to specific devices based on the locations of these devices determined by the location determination component 214.

The device selection component 216 may function to determine which devices to establish a communication session between, with this communication session having been initiated and/or established via the communication session component 218. In some instances, the device selection component 216 determines which devices to establish a communication session with based on the location of one or more devices, which devices are associated with a particular order, device characteristics, etc. Additionally, the communication session component 218 may establish a connection between devices in a peer-to-peer manner, over a wide area network, or over a local network, and/or in any other manner. Additionally, the communication session component 218 may customize the communication session based on a request, which may dictate, for example, the timing of the time the session should be active, privacy controls regarding information sharing between the devices, and interaction elements based on a request to enable seamless interaction. Examples of establishing communication between devices for an array of interests are shown and described, for example, at least in FIGS. 3-80.

Order data 220 may comprise details regarding orders placed on a BOH device(s) via one or more FOH devices. For example, order data 220 may include identifiers of items ordered, modifiers for those items, a table or location identifier associated with each order, a seat or person identifier associated with each item on each order, one or more times associated with each order, the current status of each order, etc. This order data 220 may be used for an array of purposes, as shown and described below.

Meanwhile, location data 222 may be determined by location determination component 214 and may correspond to a current, past, and/or predicted location of the mobile terminal within facility 102. In some instances, this location data 222 is used to determine which device to transmit data to and when to transmit the data to the determined device, as described and illustrated in more detail below.

Meanwhile, the analytical data 224 may indicate one or more metrics related to the operation of the facility 102. For example, the analytical data 224 may indicate the average time to prepare an order for a particular appliance in the BOH 102(2) of the facility 102, the current, past, or projected load, capacity of those appliances, and/or other types of metrics that may be tracked within the facility 102.

FIG. 3 illustrates a flow diagram of an exemplary process 300 for establishing a communication session, such as between the BOH device 106(1) and the FOH device 106(2) of FIG. 1. The process 300, as well as additional processes described herein, may be implemented in hardware, software, or a combination thereof. In the context of software, the described operations represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by one or more hardware processors, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, modules, data structures, etc. that perform a particular function or implement a particular abstract data type. Those skilled in the art will readily recognize that certain steps or operations depicted in the above figures may be omitted, combined, or performed in an alternative order. Any steps or operations may be performed sequentially or in parallel. Additionally, the order in which the operations are described is not intended to be construed as a limitation. In some instances, the processes described herein may be performed in whole or in part by modules of the BOH/FOH devices, the remote service 112, other devices, and/or combinations thereof.

Operation 302 represents presenting, on a display of BOH device 106(1), a first UI including details relating to at least a first order and a second order. In this example, FOH device 106(2) provided a first order to BOH device 106(1) and FOH device 106(3) provided a second order to BOH device 106(1). The first UI may show details regarding the first order, the second order, and potentially other order orders provided to BOH device 106(1) by the same or other FOH devices.

7 illustrates an example of a first UI 700. As shown, the UI 700 includes a first order 702(1), a second order 702(2), and six other orders in this example. Each of the illustrated orders may indicate the particular table or other location within facility 102 associated with the order, the name of the server who served the order, and details about the order, such as an identifier for the item, the quantity of the item, etc. Each order may also include additional details, such as the time the order was placed, the time since the order was placed, whether the order is late or nearing delivery, etc. In this example, the FOH device 106(2) of FIG. 1 served the first order 702(1).

3, operation 304 represents determining whether a selection has been received on the first UI. For example, this operation may include determining whether a touch screen or other input mechanism on the BOH device 106(1) has received a selection of a particular order, such as the order displayed on UI 700. If not, process 300 continues to display the first UI in operation 302.
7, however, UI 700 shows that a user 104(1) of BOH device 106(1) has provided a selection 704 of a first order 702(1). In this example, the selection includes a long press (e.g., a press greater than a threshold amount of time) on the touchscreen of the device, while in other examples the selection may include any other type of selection.

3 represents presenting a second UI including details (e.g., additional details) of the selected first order in response to the selection and display of the BOH device 106(1). FIGS. 8-9 show respective exemplary UIs 800 and 900 that the BOH device 106(1) may present in response to the selection 704. For example, UIs 800 and 900 show that the first order 702(1) may expand on the display of the BOH device 106(1) until it reaches the size shown in FIG. 9. In addition to showing details regarding the first order 702(1), the UI 900 may also include additional selectable options such as "print," "snooze," or "request server." The "print" option may be selectable to print a physical ticket associated with the order, while the "snooze" option may be selectable to pause the order and/or notifications regarding the order. Meanwhile, the "request server" option may be selectable to request a server that placed the order.

Returning to FIG. 3, operation 308 represents generating a request corresponding to the first order via the second UI. For example, this operation may include generating the request in response to user 104(1) selecting one of the selectable options 902 of the exemplary UI 900. FIG. 9, for example, depicts selection 904 to make a request to the server that initiated the order. Thus, in this example, operation 308 may generate a request to the server that originated the order.

Operation 310 represents determining that the first FOH device 106(2) originated the first order. For example, the BOH device 106(1) may analyze stored order data associated with the first order 702(1) to determine that the FOH device 106(2) placed the first order 702(1). In some implementations, the identification of the FOH device also includes receiving device characteristics and communication parameters to enable communication with the identified FOH device.

Operation 312 represents establishing a communications session between the BOH device 106(1) and the first FOH device 106(2). As described above, this may comprise a communications session established over a short-range wireless network, Wi-Fi, the Internet, and/or any other wired and/or wireless connection. Also, as described above, the BOH device 106(1) may automatically determine which FOH device 106(2) originated the order, or may automatically establish the communications session in response to receiving a selection from the user 104(1).

In operation 314, an indication of the request is transmitted to the FOH device 106(2) via the established communications session. For example, the indication may comprise a notification that a user 104(1) associated with the BOH device 106(1) has a question regarding the order (or specifically the first order 702(1)), a particular query regarding the order, etc. Additionally, in some instances, the communications session may enable real-time interaction between the respective users 104(1) and 104(2).

10 illustrates an exemplary UI 1000 that the FOH device 106(2) may present in response to receiving a request indication. As illustrated, the UI 1000 may include a notification 1002 indicating that the kitchen has requested clarification regarding a particular table in the facility 102. In addition to identifying the table (or other identifier of the order), the notification 1002 may indicate the physical location of the BOH device 106(1) from which the request originated. For example, in this example, the notification 1002 indicates that the request was initiated from a "hot station." In some instances, the notification 1002 may further indicate a query from the user 104(1) (e.g., "Should the sauce be on the side?") and/or other information.

FIG. 4 illustrates a flow diagram of another example process 400 for establishing a communication session between a BOH device and an FOH device at the facility 102 of FIG. 1. In this example, a communication session may be established to send an indication to the FOH device that an item associated with an order placed by the FOH device has been prepared.

Operation 402 represents presenting a UI on the display of the BOH device 106(1) that includes details about the order. FIG. 11 illustrates an example UI 1100 showing details about, for example, orders 1102(1), 1102(2), etc.

Operation 404 represents determining whether the BOH device 106(1) receives an indication that an item from the order is ready. FIG. 11 shows, for example, that a user 104(1) of the BOH device 106(1) has provided a selection 1104 of a particular item ("Hudson Salad") of the displayed order. In some instances, this selection 1104 may include a long press on the touchscreen of the BOH device 106(1).

12 shows that in response to this selection 1104, the BOH device 106(1) may present a UI 1200. As shown, the UI 1200 includes a menu 1202 of multiple options for making requests regarding the selected item, including "Complete Item", "Make Now", "Send to...", and "Close". Although not shown, selection of the "Complete Item" icon may result in the BOH device 106(1) determining in operation 404 that the BOH device 106(1) has received an indication that an item from the selected order is completed. Meanwhile, selection of the "Make Now" icon may result in a communications session being established with another BOH device, as described below with reference to FIGS. 13-15, and selection of the "Send to..." icon may move the job of preparing the item to another station. Selection of the "Close" icon may close the menu 1202.

In this example, user 104(1) may select a "Complete Item" option in menu 1202, and in response, BOH device 106(1) may determine that it has received an indication that the item is ready. Returning to FIG. 4, operation 406 represents determining that a particular FOH device originated an order associated with the item. Again, this operation may include analyzing order data 220 associated with the order to identify the FOH device that provided the order to BOH device 106(1).

Operation 408 represents establishing a communications session with the FOH device determined in operation 406. As described above, this may comprise a communications session established over a short-range wireless network, Wi-Fi, the Internet, and/or any other wired and/or wireless connection. Again, the BOH device 106(1) may automatically determine the FOH device that originated the order and may automatically establish the communications session in response to receiving a selection from the user 104(1).

In operation 410, an indication that the item is prepared is sent to the FOH device via the established communication session. For example, the indication may comprise a notification that a particular item is completed, that each item in an order is completed, etc. Thus, an operator of the corresponding FOH device may be kept informed of the progress of each order for which he or she is responsible. Figures 21-22 and corresponding discussions illustrate and explain this in further detail below.

FIG. 5 illustrates a flow diagram of another example process 500 for establishing a communication session between a BOH device and an FOH device, in this example, sending an indication that a user has begun preparation of an item associated with an order placed by the FOH device.

Operation 502 represents presenting a UI on the display of the BOH device 106(1) that includes details about the order. For example, FIG. 11 illustrates an exemplary UI 1100 showing details about orders 1102(1), 1102(2), etc.

Operation 504 represents determining whether the BOH device 106(1) has received an indication that a user has begun preparing an item of the order. FIG. 11, for example, shows that a user 104(1) of the BOH device 106(1) has provided a selection 1104 of a particular item ("Hudson Salad") of the displayed order. In some instances, this selection 1104 may include a long press on the touchscreen of the BOH device 106(1).

12 illustrates that in response to this selection 1104, the BOH device 106(1) may present a UI 1200. As illustrated, the UI 1200 includes a menu 1202 of multiple options. Additionally, in addition to the options illustrated, this menu 1202 (or other menus on other UIs illustrated and described herein) may include an option to indicate that an item has been started. In this example, the user 104(1) may select the "Start Item" option, and in response, the BOH device 106(1) may determine that it has received an indication that the user has started preparing a particular item. Returning to FIG. 5, operation 506 represents determining that a particular FOH device originated an order associated with the item. Again, this operation may include analyzing the order data 220 associated with the order to identify the FOH device that provided the order to the BOH device 106(1).

Operation 508 represents establishing a communications session with the FOH device determined in operation 506. As described above, this may comprise a communications session established over a short-range wireless network, Wi-Fi, the Internet, and/or any other wired and/or wireless connection. Again, the BOH device 106(1) may automatically determine the FOH device that originated the order and may automatically establish the communications session in response to receiving a selection from the user 104(1).

In operation 510, an indication that the item is prepared is sent to the FOH device via the established communication session. For example, the indication may comprise a notification that a user in the BOH 102(1) has begun preparation of a particular item. Thus, an operator of the corresponding FOH device may be kept informed of the progress of each order for which he or she is responsible. Again, FIGS. 21-22 and the corresponding discussion illustrate and explain this in further detail below.

FIG. 6 illustrates a flow diagram of another example process 600 for receiving a request to activate an appliance to process current order capacity. For example, this process 600 may occur when capacity of one or more of the appliances 116 of FIG. 1 has been reached or is predicted to be reached shortly.

As shown, operation 602 represents presenting on the display of the BOH device 106(1) (i) a UI indicating that the current or forecasted number of orders being processed by the first appliance differs from the capacity of the first appliance, and (ii) a suggestion to activate/deactivate the second appliance to balance the current or forecasted number of orders being processed by the first appliance. For example, the BOH device 106(1), the remote service 112, and/or another device may determine from the order data 220 and the analytics data 224 that the number of items or orders being processed by the first appliance is greater than the capacity of the first appliance or is expected to be greater than the capacity of the first appliance within a threshold time. In response, the BOH device 106(1) may present a UI indicating the number of orders and a suggestion to activate the second appliance.

17 shows an exemplary UI 1700 that includes, for example, an indication 1702 of several tickets (or orders) currently being handled by a first flyer in facility 102, several tickets currently being handled by a second flyer in facility 102, and a suggestion to activate a third flyer. For example, UI 1700 indicates that the first flyer currently has 32 tickets (approximately its capacity) and the second flyer currently has 23 tickets (and its capacity). Additionally, UI 1700 may include different coloring, hatching, or other visual indicia to indicate the number or relative amount of tickets that are currently late, predicted to be late, timely, and/or the like.

Additionally, the UI 1700 further includes an icon 1704 ("Activate Station") that, when selected, can result in activation of the third flyer. For example, when the icon 1704 is selected, the BOH device 106(1) can send a request to the third flyer via the remote service 112 or in any other manner to activate the third flyer directly. In response to receiving this request, the third flyer can be activated, which can include turning on, changing its settings (e.g., heat), and/or the like. Furthermore, upon activation of the third flyer, tickets currently in queue with the first or second flyer can be automatically moved from those queues to a queue associated with the third flyer. In some instances, each flyer or other appliance can be associated with a corresponding display, and thus the display of the third flyer can be turned on when the flyer is activated. Additionally, the displays of the first, second, and third flyers can be updated upon activation of the third flyer to indicate the tickets that each flyer is currently responsible for. Additionally, while some of the above examples describe suggesting that a user activate an appliance such as a fryer or other IOT device, in other examples the technique may automatically activate one or more of these appliances/devices based on certain criteria being met. For example, once a currently active fryer is associated with a predetermined number of orders or exceeds its capacity, one or more additional fryers may be automatically activated and orders previously assigned to the currently active fryer may be reassigned to the newly activated fryer.

Returning to FIG. 6, operation 604 represents determining whether the BOH device 106(1) receives a selection of an icon to activate a second appliance. For example, this operation may include determining whether the user 104(1) of the BOH device 106(1) selects the icon 1704 to activate a third flyer shown in FIG. 17. If no selection is received, process 600 continues to display the UI at operation 602.

If, however, a selection is received, operation 606 represents causing activation of the second appliance, at least in part in response to the selection. Causing activation of the second appliance may include turning on the second appliance, changing a setting of the second appliance, turning on a display associated with the second appliance, and/or the like.

After activating the second appliance, operation 608 represents presenting a UI on the display of the BOH device 106(1) indicating the number of orders to be prepared using the first appliance and/or the number of orders to be prepared using the second appliance. For example, this operation may include updating one or more displays to indicate the number of orders each appliance is currently serving. In some instances, this may include updating a display on the BOH device at the expediter station, while in other instances it may include updating a display associated with the first appliance and a display associated with the second appliance.

For example, FIG. 18 illustrates UI 1800 that BOH device 106(1) may present in response to user 104(1) selecting icon 1704 on UI 1700 presented on the display of BOH device 106(1). As shown, UI 1800 includes an indication 1802 that icon 1704 has been selected and thus a third flyer has been launched.

19 illustrates a subsequent UI 1900 that the BOH device 106(1) may present after presenting UI 1800. As shown, UI 1900 indicates at 1902 that tickets previously assigned to the first and second flyers have been reassigned to the third flyer. Thus, in this example, the first flyer is now responsible for 21 tickets, the second flyer is responsible for 14 tickets, and the third flyer is responsible for 18 tickets. Importantly, this portion of UI 1900 indicates that none of the three flyers have reached their capacity. By determining when appliances are at or near capacity, suggesting activation of additional appliances, and activating appliances upon receiving a request to do so, the techniques described herein thus enable facilities to avoid becoming overwhelmed and falling behind on their received orders.

As mentioned above, FIG. 7 illustrates an exemplary UI 700 that a BOH device may display to present current orders to be prepared. Again, the UI 700 illustrates eight orders in this example, such as a first order 702(1), a second order 702(2), etc. The UI 700 further displays details about each order, such as the items in the order, modifiers associated with the order, the server that placed the order, and the time associated with the order (e.g., the time since the order was placed). In some examples, the UI 700 may be presented on a BOH device, such as an expediter station of the BOH 102(2), a hot station of the BOH 102(2), a cold station of the BOH 102(2), etc. In this example, the UI 700 illustrates that a user of the BOH device provides a selection 704 of a first order 702(1).

FIG. 8 illustrates an exemplary UI 800 that the BOH device may display in response to a selection 702. As shown, the selected order 702(1) expands to present additional details regarding the order and options regarding the order. FIG. 9 illustrates an exemplary UI 900 that continues the animation that may occur on the display of the BOH device in response to a selection 702. As shown, the UI 900 may display additional details regarding the first order 702(1) and may further display selectable options 904 for the operator of the BOH device, including an option to request the server that placed the order. As discussed above, in this example, the operation of the BOH device may select an option to request the server that originated the order.

FIG. 10 illustrates an exemplary UI 1000 that may be displayed by a FOH device that originated a first order 702(1) in response to a selection 904 from the UI 900. As shown, the UI 1000 may include a notification 1002 that an operator of the BOH device has requested clarification regarding a particular order. In some instances, the notification 1002 may include details regarding the query, an indication of the location of the BOH device, and/or additional details.

FIG. 11 illustrates an exemplary UI 1100 that may be displayed by a BOH device associated with an expeditor station. As shown, the UI 1100 includes details regarding four orders, including a first order 1102(1), a second order 1102(2), etc. Here, an operator of the BOH device provides a selection 1104 regarding a particular item associated with order 1102(1).

12 illustrates an exemplary UI 1200 that a BOH device associated with an expeditor station may display in response to a selection 1104. As illustrated, the UI 1200 presents a menu 1200 that includes several options that an operator may select. These options may include an option to indicate that an item is completed or prepared ("Complete Item"), an option to send a request to another BOH device or station to begin preparation of the item ("Make Now"), an option to send information regarding the preparation of the item to another BOH device or station ("Send to..."), and the like. In this example, an operator of a BOH device at an expeditor station provides a selection 1204 to request that an operator associated with another BOH device and station begin preparation of the item.

For example, selection of this option may cause the BOH device at the expediter station to automatically establish a communications session with the BOH device at the station responsible for preparing this item. In one example, the BOH device shown in FIG. 12 may automatically analyze the order data 220 to determine the BOH station responsible for preparing this item, which may be based on the type of item, the current load at the BOH station in facility 102, etc. In this example, the BOH device of FIG. 12 may determine that because the item is a salad, a request to make the item now should be routed to the cold station in BOH 102(2). Thus, the BOH device of FIG. 12 may automatically establish a communications session with the BOH device at the cold station and send a request to begin preparing the salad.

FIG. 13 is an exemplary UI 1300 that a BOH device at a cold station may display prior to selection 1204 at the expediter station shown in FIG. 12. As shown, this UI 1300 may show the orders and items within the order to be prepared. The UI 1300 may also prioritize the creation of different orders within the facility based on when the order was placed from the FOH device, etc. As shown, the UI 1300 includes details regarding a first order 1302(1), a second order 1302(2), etc.

14 illustrates an exemplary UI that the BOH device of FIG. 13 may display after selection 1204 at the expeditor station shown in FIG. 12. As shown, this UI 1400 includes a notification 1402 that the expeditor has requested that the preparation station begin preparation of a particular item. For example, the notification 1402 may appear at the top of the UI 1400 and may indicate, for example, the table or location of the order for which the item is being requested to be prepared. Additionally, the notification 1402 may include a selectable icon ("Start Order") that the operator of the BOH device may select to begin preparation of the item. As shown below, this may automatically reprioritize the order shown on the BOH device at this preparation station. As shown, in this example, the operator of the BOH device is provided with a selection 1404 of selectable icons.

15 illustrates an exemplary UI 1500 that the BOH device of FIG. 13 may present after the BOH device operator provides the selection 1404 shown in FIG. 14. As illustrated, the displayed orders have been reprioritized in response to this selection 1402. For example, the new order 1502 corresponding to the order that the expediter station requested to be "made now" may be displayed first, order 1302(1) may now be presented second, order 1302(2) may now be presented third, and so on. Thus, the order for which urgency was requested may now be presented in the position of the UI 1500 corresponding to the first inline (i.e., the most urgent order). In addition, the UI 1500 includes an indication 1504 that the order should now be made ("made now").

FIG. 16 illustrates another exemplary UI 1600 that a BOH device may display. Here, the UI 1600 presents various kitchen performance metrics, which may be displayed at an expediter station, a preparation station, or a sequencing station. In some instances, the BOH devices may present different metrics based on the currently assigned role of each BOH device. For example, a BOH device at a hot station may present metrics corresponding to fryer wait time, oven wait time, etc., while a BOH device at a cold station may present metrics corresponding to average time to prepare a salad, etc. In another example, a BOH device at an expediter station may present metrics related to orders as a whole, such as average time to deliver completed orders to a table, orders that are running late or are forecasted to be running late, etc.

In this example, UI 1600 presents exemplary indicators 1602 including the average time to prepare an order, the currently slowest station in the kitchen, and the item currently taking the longest to prepare. Additionally, exemplary indicator 1602 indicates the number of tickets that are overdue by a certain amount of time (e.g., over 10 minutes late, over 5 minutes late, etc.). Exemplary indicator 1602 further indicates the current number of tickets assigned to a first fryer, a second fryer, etc., the current load at a particular station. Additionally, indicator 1602 may indicate the number of tickets currently late at each fryer or other station, the number of tickets late at each fryer and other stations, etc. Of course, while indicator 1602 illustrates exemplary indicators, it should be understood that any other type of indicator may be stored in analytics data 224 and presented on UI 1600 and/or other UIs.

FIG. 17 illustrates an exemplary UI 1700 that a BOH device may display, as described above. Here, the UI 1700 presents specific suggestions 1702 for facility operations, such as suggestions to activate additional flyer stations. This UI 1700 also includes an icon 1704 for automatically activating the suggested flyer station.

FIG. 18 illustrates an exemplary UI 1800 that the BOH device may display after the device operator selects icon 1704. As described above, in response to this selection, UI 1800 may change the display of the icon from "Activate Station" to "Activated." Additionally, the BOH device may respond by automatically sending a request to activate the third flyer station.

19 illustrates an exemplary UI 1900 that the BOH device may display after a third flyer station is activated. As shown, UI 1900 indicates that at 1902, tickets being processed by flyers have been reallocated based on the activation of the third flyer. For example, UI 1900 indicates that the load is balanced across the three flyers such that no individual flyer is currently responsible for a number of orders greater than the flyer's respective capacity. Additionally, while UIs 1700, 1800, and 1900 collectively illustrate the activation of flyers, it should be understood that in other instances any other type of connected appliance may be automatically activated (or offered to be activated).

20 illustrates another example UI 200 that may be displayed by a BOH device in facility 102 of FIG. 1. Here, UI 2000 presents details regarding example orders 2002(1), 2002(2), and 2002(3). In addition, UI 2000 presents a selectable icon 2004 ("Start Order") that, when selected, may reprioritize the order displayed on the UI, may reprioritize orders displayed on other BOH devices, and/or may send a notification to an FOH device associated with the order that preparation of the order has begun.

FIG. 21 illustrates an exemplary UI 2100 that an FOH device may present, for example, to inform an operator of the FOH device of the current status of orders the operator has sent to the BOH. That is, the UI 2100 may present the current status of each order that a server logged into the FOH device has placed on the BOH 102. In this example, the UI 2100 indicates at 2102 that the order associated with table 12 is ready, meaning that the server can deliver or request that another user retrieve the items associated with the order from the BOH 102(2) and deliver the items to table 12 in the FOH 102(1). Additionally, the UI 2100 indicates at 2104 that the order for table 4 is nearly complete (since three of the four items are complete) and at 2106 indicates that half of the order is complete for the order associated with "dining 4". In some instances, the UI 2100 may update periodically as items are marked complete via the BOH device(s). Additionally, in some instances, the FOH device may present pop-up notifications, etc., when an order is completed, when items on an order are completed, etc.

The UI 2100 may include additional details regarding the order placed by the server operating the FOH device. For example, the UI 2100 may show, at 2108, the number and identification of tables currently being processed by the server, at 2110, the items (or "courses") currently being held, etc.

22 illustrates another exemplary UI 2200 that an FOH device may display to provide additional information to a server operating the FOH device. For example, in addition to the information described above with reference to FIG. 21, UI 2200 may present information regarding current inventory at facility 102. For example, UI 2200 may show at 2202 items that are currently unavailable (e.g., because they have sold out), and at 2204 items that are currently low. Of course, while some exemplary inventory details are shown in UI 2200, it should be understood that any other details, indicators, etc. may be presented.

Routing requests based on context information

23 illustrates an exemplary architecture 100 introduced in FIG. 1. In this example, a request to perform a particular action may be sent to the most relevant FOH device, where relevance may be determined based on context information such as the location of the FOH device within the facility 102, as well as the current status of orders within the facility 102 and the locations associated with those orders. In another context information may be derived from time of day, past history of communication sessions, location of the FOH device relative to the location of the orders, ranking of the FOH device relative to other devices (in the case of multiple devices, the devices may be ranked in priority order), and historical data for the first device (e.g., for a particular request, a particular FOH device may be activated for assistance), etc.

23 illustrates, for example, that FOH device 106(2) operated by user 104(2) is currently located a particular distance 2302 from a location 2304 within facility 102 associated with a particular order associated with exemplary customer 2316. For example, when user 104(2) opens an order for customer 2316 using FOH device 106(2), user 104(2) may indicate that the customer (and thus the associated order) is seated at a particular bar seat associated with the illustrated location 2304. Thus, in some instances, FOH device 106(2) may present information regarding customer 2316's order when FOH device 106(2) is determined to be within a threshold distance of location 2304.

In the illustrated example, for example, the distance 2302 between the FOH device 106(2) and the location 2304 may be within a threshold vicinity. Additionally, the FOH device, the BOH device, the remote services 112, and/or another device may determine that there are one or more current actions that may need to be taken regarding the order of the customer 2316. For example, based on a time associated with the order (e.g., time elapsed, time since courses have been served, etc.), one or more of the devices may determine that it is time to provide the customer 2316 with a bill, ask if the customer 2316 would like dessert, check on the customer 2316, etc. In the illustrated example, it is determined that it may be time to print a bill for the customer 2316. Thus, in response to determining that the FOH device 106(1) is within a threshold vicinity of the location 2304, the FOH device 106(1) may present an example 2306 suggesting that the user 104(1) print a bill for the user. Additionally, the UI 2306 may include a selectable icon for printing the bill.

In another example, the FOH device 106(2) may currently be at a distance 2308 from a location 2310 associated with an order for a particular table 2318 within the facility. Further, the FOH device, the BOH device, a remote service, and/or another device may determine that the distance 2308 is less than a threshold distance. Further, one or more devices may determine an action that may need to be taken with respect to the order associated with the table 2318. Thus, in response to determining that the FOH device 106(2) is within a threshold distance of the location 2310, the FOH device 106(2) may present an exemplary UI 2312 suggesting that the user 104(2) perform a particular action. In this example, the UI 2312 may suggest that the user 104(2) launch the course ordered for the user at the table 2318. For example, a predetermined amount of time may be determined to have elapsed since a particular event (e.g., since serving an appetizer) and therefore it is time to launch the next course to the BOH 102(2).

The location determination component 214 described above may determine the current location of the FOH devices in any suitable manner. In some examples, the FOH devices utilize GPS sensors that periodically or continuously report their current location. In other examples, the facility 102 includes one or more beacons, such as exemplary beacons 2314(1), 2314(2), and 2314(3), that may be used by the FOH devices to triangulate their current location within the facility 102. For example, each FOH device may measure the respective signal strength between the FOH device and each of the beacons 2314(1)-(3) to triangulate the current location of the FOH device, and each of the beacons may be associated with a predefined location. In one example, the beacons may be wireless beacons. Of course, while several exemplary location determination techniques are described, it should be understood that the respective locations of the FOH devices may be determined in any other manner.

Thus, the above techniques can suggest specific actions to the FOH device based on the current status of the order within the facility 102 and based on the current or projected location of the FOH device within the facility. For example, as a user traverses the facility 102, the display of each of the FOH devices can be updated to suggest specific actions related to the order associated with the location to which the equipment operator is currently near. For example, while the above example describes suggesting to print a bill and launch a course, other suggested actions can include suggesting to open a table when the operator is near a table not currently associated with an order, suggesting to check on the customer when near a table that received an entree five minutes ago, etc.

In another example, the positioning component 214 can determine if a table moves within the facility and, in response, can update a UI showing the current layout of the facility. For example, if a customer or employee of the facility moves a first table to which a first beacon is coupled so that it is closer to a second table to which a second beacon is coupled, the positioning component can determine that the first table has moved closer to the second table based on the changed signal strength. In response, the UI showing the current layout of the facility can be updated to indicate this change.

FIG. 24 illustrates a flow diagram of an example process 2400 for determining that a particular FOH device is associated with a particular order based on context information, such as the device being within a threshold vicinity of a first location associated with the order, the device being ranked higher with respect to the order than other devices, and/or historical data regarding the device (e.g., the device has previously interacted with the order). In response, the process can send a UI to the FOH device requesting or suggesting a particular action associated with the first location.

Operation 2402 represents generating a layout of the establishment indicating that a first order is associated with a first location and a second order is associated with a second location. In some instances, the FOH device, the BOH device, another device in the establishment, and/or a remote service may generate and maintain this layout. For example, the device may store associations between tables and other seating areas in the establishment, as well as the respective locations of these tables and other seating areas in the establishment. For example, a first table (T1) may be associated with a particular X, Y location in the establishment, while a second table (T2) may be associated with another X, Y location, and a bar seat (B1) may be associated with yet another X, Y location.

Operation 2404 represents determining an action to be taken with respect to the first order based at least in part on the current status of the first order. For example, the order processing component 212 may determine trigger data from the order data 220 indicating that a particular action may need to be taken with respect to the first order. For example, the order processing component 212 may determine from the order data 220 that a threshold time has elapsed since an event (e.g., opening the order, serving of a previous course, etc.), which may trigger a suggestion for an action to occur (e.g., suggesting printing a bill, suggesting a next course, etc.). Thus, the order processing component 212 may determine an action that may need to be taken with respect to the first order based at least in part on the current status of the first order.

Operation 2406 represents determining whether an FOH device in the facility is within a threshold vicinity of the first location associated with the first order. If not, process 2400 may wait until an FOH device is within the threshold vicinity. In some instances, this operation may comprise determining whether a particular FOH device associated with the first order is within a threshold vicinity of the first location, while in other instances, this operation may comprise determining whether any FOH device is within a threshold vicinity.

If operation 2406 determines that the FOH device is within a threshold vicinity of the first location, operation 2408 represents generating a UI indicating an action for the first order, while operation 2410 represents transmitting the UI for display on the FOH device. For example, FIG. 31 illustrates an example UI 3100 that may be present in response to the FOH device being located within a threshold vicinity of a first location associated with the first order. As illustrated, the UI 3100 may include a menu that identifies a table or order as one or more suggested actions 3102 for the order. In this example, the menu 3102 suggests that the operator of the FOH device continue to print a bill for the customer associated with the first order. The UI 3100 may also include a selectable icon 3104 for performing the suggested action. In some instances, the FOH device may generate the UI, while in other instances, the remote service 112, the BOH device, or another device may generate the UI for presentation on a display of the FOH device.

Meanwhile, FIG. 25 illustrates a flow diagram of an example process 2500 that may occur with respect to a second order associated with a second location, as described above with reference to operation 2402. Here, operation 2502 represents determining an action to take with respect to the second order based at least in part on a current status of the second order. For example, the order processing component 212 may determine trigger data from the order data 220 that indicates that a particular action may need to be taken with respect to the second order.

Operation 2504 represents determining whether an FOH device in the facility is within a threshold vicinity of a second location associated with the second order. If not, process 2500 may wait until an FOH device is within the threshold vicinity. Also, in some examples, this operation may comprise determining whether a particular FOH device associated with the second order is within a threshold vicinity of the second location, while in other examples, this operation may comprise determining whether any FOH device is within a threshold vicinity.

If operation 2504 determines that the FOH device is within a threshold vicinity of the first location, operation 2406 represents generating a second UI indicating an action to be taken with respect to the second order, while operation 2508 represents transmitting the UI for display on the FOH device. FIG. 32, for example, illustrates an exemplary UI 3200 that may be present in response to the FOH device being located within a threshold vicinity of a second location associated with the second order. As illustrated, the UI 3200 may include a menu that identifies a table or order as one or more suggested actions 3202 for the order. In this example, the menu 3202 suggests that the operator of the FOH device "launch course 2" for the customer associated with the second order. The UI 3200 may also include a selectable icon 3204 for performing the suggested action. In some instances, the FOH device may generate the UI, while in other instances, the remote service 112, the BOH device, or another device may generate the UI for presentation on a display of the FOH device.

26 illustrates a flow diagram of another example process 2600, where operation 2602 represents determining whether the FOH device is within a threshold vicinity of a third location in the establishment that is associated with a table or other seating area that is not currently associated with an open order. That is, this operation may include determining whether a user associated with the FOH device is currently walking around or otherwise near a table or seating area that is not associated with an open order or ticket.

Otherwise, operation 2604 represents generating a third UI suggesting that a third order associated with the third order be opened, at least in part in response to determining that the FOH device is within a threshold vicinity of the third location. Operation 2606 represents transmitting this UI for presentation on a display of the FOH device. For example, FIG. 33 illustrates an example UI 3300 that may be present in an FOH device in response to being within a threshold vicinity of a third location that is not currently associated with an open order. As illustrated, the UI 3300 may include a menu that identifies a table or order as one or more suggested actions 3302 for the order. In this example, the menu 3302 suggests that an operator of the FOH device open a table so that the operator can begin placing orders for customers who are or will be sitting at the table or other seating area. The UI 3300 may also include an icon 3304 that is selectable to perform the suggested action. In some instances, the FOH device may generate the UI, while in other instances, the remote service 112, the BOH device, or another device may generate the UI for presentation on the display of the FOH device.

27 illustrates a flow diagram of another exemplary process 2700 for enabling an operator of an FOH device to add items to an order. Operation 2702 represents presenting, on a display of the FOH device, a fourth UI including (i) respective icons corresponding to items offered in the establishment, and (ii) a layout of tables and other seating areas in the establishment. For example, FIG. 34 illustrates an exemplary UI 3400. As shown, the UI 3400 includes exemplary icons 3402 of items offered in the establishment and a layout 3404 of tables and other seating areas in the establishment.

Returning to FIG. 27, operation 2704 includes determining whether user input has been received to add a first item to a first order (associated with a first seating area on the layout). For example, this operation may include determining whether user input has been received beginning at a location on the display corresponding to an icon associated with the first item and ending at a location on the display corresponding to a location on the layout associated with the first order.

FIG. 34 illustrates an operator with an FOH device providing a selection 3406, such as a press on a touch screen of the FOH device, while FIG. 35 illustrates a UI 3500 with an operator dragging an icon of an item ("CHIPS & GUAC") toward a layout 3404 of the establishment. FIG. 36 illustrates an exemplary UI 3600 in which the operator "drops" the icon onto an icon 3602 on the UI 300 that corresponds to a particular table ("TABLE 2") in the layout 3404.

Returning to process 2700, operation 2706 represents adding a first item to the first order in response to receiving user input in operation 2704. That is, as shown in FIG. 36, the first item can be added to the first order after the user drops an icon corresponding to the item onto icon 3602 corresponding to the table.

Further, operation 2708 represents setting a time for a predetermined amount of time. In some instances, this may include setting a timer for a period of time after which the items associated with the order are "fired" (i.e., sent to the BOH device for preparation). That is, after adding a first item to a first order, a timer can be set and upon expiration of the timer, the first order can be sent for preparation. For example, FIG. 37 illustrates an exemplary UI 3770 including a timer 3702 set for a predetermined period of time (20 seconds in this example) in response to an item icon being dragged and dropped onto an icon 3602 associated with a table.

Meanwhile, operation 2170 represents determining whether user input has been received to add a second item to the first order. For example, FIG. 38 shows an operator of the FOH device selecting an icon 3802 corresponding to a second item (the "bud bucket") and dragging and dropping the icon 3802 onto an icon associated with the same table. FIG. 38 also shows a timer 3702 counting down to 15 seconds after adding the first item to the order.

Operation 2712 represents adding a second item to the first order in response to the second user input. Further, operation 2714 represents resetting the timer to a predetermined amount of time. For example, FIG. 39 illustrates an exemplary UI 3900 showing timer 3702 reset to the original 20 seconds.

Meanwhile, operation 2716 represents determining whether user input was received at a location on the display corresponding to a first location associated with the first order. For example, this operation may determine whether user input was received on an icon corresponding to a table or other seating area associated with the first order. For example, FIG. 39 shows an operator providing selection 3902 of an icon corresponding to a table associated with an exemplary order.

If operation 2716 determines that user input has been received, operation 2718 represents presenting an icon on the display of the FOH device for submitting the first order, including the first and second items, to the BOH prior to the expiration of the predetermined time. FIG. 40 illustrates an exemplary UI 4000 including a menu 4002 including an icon 4004 ("Fire Now") for enabling an operator to directly send the order to the BOH 102(2) prior to the expiration of timer 3702. Menu 4002 also includes an icon for canceling each item in this example.

28 illustrates a flow diagram of another example process 2800 for determining items to suggest for addition to a first order and presenting suggestions on an FOH device in response to the FOH being within a threshold vicinity of a first location associated with the first order. Operation 2802 represents determining items to suggest for addition to the first order. For example, the FOH device, the BOH device, the remote service 112, or another device may determine the items to suggest for the first order based on any number of criteria, such as items currently on order, a history of items ordered by a customer associated with the first order, the current time, current specials at the facility, etc.

Operation 2804 represents determining whether the FOH device is within a threshold vicinity of a first location associated with the first order. As described above, this operation can be based on the location determination component 214 using the location data 222 to determine whether any FOH device (or FOH devices associated with the first order) is within a threshold vicinity of the first location. Operation 2806 then represents sending a fifth UI to the FOH device including a suggestion to add the suggested item to the first order. In some instances, this suggestion, and other suggestions described herein, may be based on one or more customer prior behaviors, such as the purchase history of the particular customer for whom the suggestion is being presented, prior purchases of other customers, etc. In some instances, the remote service 112 described above may generate some or all of these suggestions.

For example, FIG. 41 shows an exemplary UI 4100 that includes an indication of a table at 4102 and suggested items and other actions for that table at 4104. For example, in this example, for an order currently associated with "Chef's Table," a suggested item of "Bug Light Bucket" and a suggested item of "Fries" are determined. Thus, when the FOH device is located within a threshold proximity of the chef's table, UI 4100 may be presented on the FOH device to suggest that the operator of the FOH device suggest adding items to the order.

Meanwhile, FIG. 42 illustrates another exemplary UI 4200 for presenting suggestions to an operator of a FOH device. As shown, the UI 4200 includes, at 4202, an indication of a table, and, at 4204, one or more suggested items and actions. In this example, the UI 4200 suggests that the operator suggest to the customers at the chef's table that they repeat the last round of drinks. The suggested actions may further include a suggestion to add an employee discount (perhaps because the current customer is an employee of the establishment 102), a suggestion to split the bill seating, etc.

FIG. 29 illustrates a flow diagram of another example process 2900. In this case, the process 2900 can determine the identity of an operator of an FOH device, determine whether the identity is associated with a previous behavior, and if so, present a UI that includes a customized set of actions to the operator based on the previous behavior.

Operation 2902 represents determining an identifier of a user currently operating the FOH device. For example, this operation may include determining an identifier associated with a user currently signed in to the FOH device. Operation 2904 represents determining whether a previous behavior is associated with the determined identifier. For example, analytics data 224 may store patterns of behavior associated with different operators of devices within a facility. These patterns of behavior may indicate a particular sequence of actions that a user tends to repeat when operating an FOH or other merchant device. For example, analytics data 224 may determine that an operator associated with a first identifier often checks her tip before running a shift report, while another operator associated with a second identifier often skips this step.

If it is determined that the identifier is associated with a previous behavior, operation 2906 represents generating a sixth UI indicating a customized action to be performed based on the previous behavior of the user currently operating the FOH device. Operation 2908 represents transmitting the sixth UI for display on the FOH device. For example, the sixth UI may present a sequence of actions that match a sequence of actions previously performed by a particular operator. Continuing with the example from above, for example, the sixth UI for the first user may start with a screen for checking her tip, while the sixth UI for the second user may not include this screen at all.

FIG. 43 illustrates an example UI 4300 that an FOH device may present based on, for example, the previous behavior of a first operator of the FOH device. At 4302, the UI shows suggested actions to perform, such as printing the latest bill for table 5 (T5), splitting the bill by seat, etc. However, FIG. 44 illustrates an example 4400 that an FOH device may present based on the previous behavior of a second operator of the FOH device. At 4402, the UI 4400 suggests that the operator review tip adjustments, review shift details, and run shift reports. In some instances, the UI may include a set of suggested actions based on information added to the operator's previous behavior, such as the current time and the time the operator is expected to log out of the FOH device (e.g., because their shift is ending).

FIG. 44 further illustrates that the operator of the FOH device may provide a selection 4404 to navigate a sequence of actions customized for the operator based on the operator's previous behavior. In response to selection 4404, FIG. 45 may present a UI 4500 that begins by allowing the operator to review her tip adjustment at 4502. Again, this screen may be presented to an operator who often views this screen at the end of a shift. The UI 4500 also includes a selectable icon 4504 to move from the tip adjustment screen to a shift details screen. As shown, the UI 4500 receives a selection 4506 of this icon 4504.

FIG. 46 presents a UI 4600 that may be presented in response to selection 4506 of icon 4504. As shown, the UI may include details at 4602 regarding a shift for an operator of the FOH device. These shift details may include hours worked, time until the shift occurs, number of breaks taken by the operator, total sales, number of open orders, number of tips received, etc. Additionally, the UI 4600 may include a selectable icon 4604 to navigate from the shift details screen and run a shift report for the operator. The operator provides selection 4606 in this example.

Figure 47 shows an example UI 4700 that an FOH device may present in response to selection 4606 of icon 4604 from UI 4600. As shown at 4702, this UI 4700 indicates that the FOH device is running a shift report for an operator and potentially logging the operator out of the device. Figure 48 shows an example UI 4800 that may exist after the FOH device has run a shift report and logged out the previous operator.

Although FIGS. 44-47 illustrate and describe a customized sequence of actions that may be presented based on a previous behavior of an operator of the FOH device, in some instances an operator may not be associated with such previous behavior and therefore may not receive a customized sequence of actions. Returning to FIG. 29, if no previous behavior is associated with the determined identifier (or a threshold amount of previous behavior is not associated with the identifier), operation 2910 may generate a seventh default UI that indicates the behavior, while operation 2912 may transmit the seventh UI for presentation on a display of the FOH device. This default UI may present actions that are not customized to the current operator of the FOH device, and rather, corresponds to a default, non-customized sequence of actions.

FIG. 30 illustrates a flow diagram of another exemplary process 3000 for determining whether an FOH device is physically coupled to or within a threshold vicinity of another device, and if so, presenting a UI indicating a sequence of actions to be performed. Operation 3002 represents determining whether an FOH device is physically coupled to or within a threshold vicinity of another piece of equipment. This may include determining whether the FOH device is in a cradle, determining whether the FOH device is near another FOH device or a BOH device (e.g., via Bluetooth), etc. If so, action 3004 represents providing, on the display of the FOH device, eight UIs for performing one or more predefined actions. For example, this operation may include presenting a UI for performing a shift report if the FOH device is plugged into a cradle or within a particular device in facility 102.

Figure 49 shows an example UI 4900 that may exist, for example, before or within an FOH device that is coupled to a threshold vicinity of another device. That is, this UI 4900 may show that the operator of the FOH device has navigated to a screen to view shift details in 4902, but has not yet selected the icon 4904 to run a shift report.

50, however, illustrates an exemplary UI 5000 that may exist after the FOH device of FIG. 49 is physically coupled to another device, such as a receiver 5002. As shown, the receiver 5002 may connect to one or more other FOH/BOH devices 200, remote services 112, etc., via wired and/or wireless connections. Additionally, the hardware-to-hardware connections may enable certain software workflow actions. For example, the FOH device may automatically begin running a shift report in response to being physically coupled to the receiver 5002. While in some embodiments the receiver and FOH device may be a power source and a power receiving device, respectively, in other embodiments the receiver and FOH device may not provide power to the other, but instead trigger other actions such as shift report generation, clocking out from a server, transferring chips, etc. The receiver 5002, when communicatively coupled to the FOH device, may initiate certain actions preconfigured, for example, by the merchant or remote services, via rules. For example, a merchant through the BOH can configure the receiver to finalize a shift report or clock out from the FOH server when the FOH device is docked to the receiver 5002. Custom rules can also allow financial flows from a merchant's financial account to an account at the FOH server in response to detection of docking of an FOH device in the receiver 5002. The receiver 5002 can allow computing devices to communicate with other devices communicatively coupled to the receiver (e.g., sync with remote services) and other merchants to refine the data displayed on the FOH device. In some embodiments, control can automatically transfer from an FOH device docked to the receiver 5002 to another device, e.g., another FOH device or another customer device, to move the displayed content to the other device. Alternatively, this allows another server to pick up where the previous server left off.

Figure 51 continues the illustration of Figure 50 and shows the tip being transferred, via UI 5100, to an account associated with the operator of the FOH device. Figure 52 completes the illustration of Figure 50 and shows the operator being logged out of the FOH device, via UI 5200. In some instances, the tip may be seamlessly and automatically transferred to the operator's checking account, the operator's mobile payment application, and/or the like. Thus, the series of UIs 50-52 illustrate that the FOH device may perform a predetermined series of operations in response to being physically coupled to another device, such as the cradle 5002.

53 and 54, on the other hand, show that a predefined set of actions may occur in response to the FOH device being within a threshold proximity of another device in the facility 102. FIG. 53 illustrates an example UI 5300 in which the illustrated FOH device detects, and is therefore within a threshold proximity of, a "table-side device," which may comprise a device that is coupled to or otherwise associated with a particular table or other seating area in the facility. The UI 5300 includes an indication 5302 that the device has detected a table-side device, and also includes an icon 5304 that is selectable to open a new order in response to detecting the table-side device. The UI 5300 shows that the operator of the device provides a selection 5306 of the icon 5304. In some implementations, creating an order on the table-side device may tag such an order and may also send an alert regarding the order to the BOH device, thus enabling real-time communication regarding the order.

FIG. 54 illustrates an example UI 5400 that the FOH device may present in response to selection 5306. As shown, the UI 5400 indicates that a new order has been opened, items have been added to the new order, etc. Of course, although several example sequences of UIs have been described in response to a merchant device being physically coupled to or within a threshold vicinity of another device, other UI sequences may be utilized.

Gesture examples

55 and subsequent UIs illustrate different gestures that may cause the facility application 210 to perform different actions. While these figures illustrate example gestures and corresponding actions, it should be understood that additional gestures may be associated with additional actions on the facility application 210 in other instances. In some instances, a server, individual devices, and/or the like may store rules that associate respective gestures with respective actions. In some instances, these rules may be user configurable such that different devices and/or different operator profiles may be associated with different rules.

55-62 illustrate, for example, that an operator of an FOH device may add or remove different items (e.g., lettuce, tomato, cheese, etc.) from an example item (here, a burger) based on selecting respective modifier icons and swiping those icons in a particular direction. For example, selecting an icon and swiping right may cause the venue application 210 to add a modifier to the item, while selecting an icon and swiping left may cause the venue application 210 to remove a modifier from the item. Additionally, selecting an icon and swiping upward may cause the settings application 210 to add an additional amount of modifier to the item, while selecting an icon and swiping downward may cause the venue application 210 to indicate that the modifier should be placed "on the side." Of course, while the example directions are associated with example actions in these examples, it should be understood that these are merely example and that other directions may be associated with other actions.

In one embodiment, a surface of a gesture-accepting device, such as a sensor pad, can include various sensor points identified by (x,y) coordinates, and the sensor pad can identify a user activating one or more of these sensor points that ultimately generate sensor data, and in the case of multiple sensor points, the sensor data also records the sequence of activated sensor points. The sensor points can be touch or touchless. When registering an action, the user can register a specific sensor point activation to trigger a specific action. For example, hovering over an item can mean "find more information about the item" and tapping can mean "add an order". The methods and systems herein allow the user to define what each gesture can mean in the context of this order, or in general, in their experience with the application. Thus, as the user uses the application, the embedded rules are invoked in response to each gesture to determine the underlying customized action associated with the gesture. As shown in FIG. 64, the server can also embed options for each action to provide sub-actions.

For example, FIG. 55 shows UI 5500 displaying an exemplary change at 5502. UI 5500 also shows that the operator of the FOH device provides a selection 5504 corresponding to an icon associated with an exemplary modifier (here, lettuce). FIG. 56 shows exemplary UI 5600 after the operator has made selection 5504. Here, the operator begins to drag the icon to the right side of the display. Meanwhile, FIG. 57 shows UI 5700 indicating that the operator has continued to drag the icon further to the right, and in response, UI 5700 indicates at 5702 that lettuce has been added to the order. Meanwhile, FIG. 58 shows UI 5800 in which the user drags the icon back to the left. In response, UI 5800 indicates at 5802 that the modifier is removed ("no lettuce").

FIG. 59 illustrates an exemplary UI 5900 in which an operator of the FOH device is offered a selection 5902 of another modifier (in this case, "onion"). FIG. 60 illustrates that, via an exemplary UI 6000, the operator may drag the icon corresponding to the modifier significantly upwards (e.g., greater than a threshold distance from the icon's initial position and/or at a velocity greater than a threshold velocity). In response, the UI 6000 illustrates that the name of the icon has changed from "onion" to "extra onion". FIG. 61 illustrates, via UIs 6100 and 6102, that the item is now associated with "extra onion", but in addition, the UI 6100 illustrates that the operator has changed from swiping up on the icon to swiping below the icon's initial position on the display. In response, the UI 6100 updates the name of the modifier from "extra onion" to "side onion", assuming that a downward swipe is associated with adding a modifier to the side in this example. FIG. 62 shows an example UI 6200 indicating that the modifier has changed from "extra onion" to "side onion" at 6202.

63-67 show additional example UIs and corresponding gestures for adding or removing modifiers to an item such as an example burger. For example, FIG. 63 shows an example UI 6300 listing some of the same example modifiers described above with reference to FIGS. 55-62. Here, the operator provides a selection 6302 corresponding to the modifier "bacon." FIG. 64 shows that the FOH device may respond by presenting a UI 6400. As shown, the UI 6400 includes an icon 6402 corresponding to the selected modifier and is surrounded by a menu of options 6404 for the modifier. For example, the modifier ("bacon") is surrounded in this example by examples of "allergy," "none," "sub," "extra," "additional," and "side." Of course, while several example options are listed, it should be understood that other embodiments may utilize additional and/or different options.

FIG. 65 illustrates an exemplary UI 6500 after an operator selects an icon 6402 and begins dragging it toward an option ("Extra") 6502 in the menu of options 6404. As shown, the size and/or color of the option 6404 may change as the icon 6402 approaches the option 6502. For example, the size of the option 6502 may increase as the icon 6402 approaches the option 6502. FIG. 66 illustrates an exemplary UI 6600 after an operator releases the icon 6402 while over the option 6502. As shown, the text of the option 6502 has been changed to add an indication of the modifier. For example, here, the option 6502 has been changed from "Extra" to "Extra Bacon". Additionally, FIG. 67 illustrates an exemplary UI 6700 in which the FOH device may present the UI 6600 a predetermined amount of time after the menu of options 6404 is presented. As shown, the menu of options 6404 has disappeared. Additionally, the previously selected modifier has been changed from its initial text ("Bacon") to indicate the selected option ("Extra Bacon").

68-73 illustrate yet another example of gestures for adding or removing modifiers from an item, such as the exemplary burger. For example, FIG. 68 illustrates an exemplary UI 6800 illustrating an array of modifiers as described above. Further, this UI 6800 illustrates that an operator of the device has provided a selection 6802 corresponding to one of the modifiers (here, "lettuce"). FIG. 69 illustrates a UI 6900 responsive to the selection 6802. As shown, the UI 6900 presents a menu of options 6902 along with an icon corresponding to the selected modifier. Again, these exemplary options may include "Extra," "Additional," "Side," "Sub," "None," and the like. Additionally, the UI 6900 illustrates that the particular options adjacent to the selected modifier icon may change based on the direction of the user input 6904. For example, FIG. 69 illustrates that after selecting the icon corresponding to "lettuce," the operator may swipe upwards.

In response to the operator swiping upward at 6904, FIG. 70 illustrates that, via the exemplary UI 7000, options from the menu of options 6902 have also moved downward based on the swipe upward gesture. Thus, while “Add” was previously adjacent to “Lettuce” in FIG. 69 prior to the upward swipe gesture, “Extra” appears next to “Lettuce” after the upward swipe. Meanwhile, FIG. 71 illustrates the UI 7100 after the operator provides a downward swipe. As shown, the menu 6902 has moved upward based on the downward swipe, such that the option “None” is adjacent to the selected modifier icon “Lettuce”. Meanwhile, FIG. 72 illustrates the exemplary UI 7200 after the operator removes touch input from the touch screen of the FOH device while the option “None” is adjacent to “Lettuce”. In response, UI 7200 has changed the modifier to include the option at 7202 (from "lettuce" to "no lettuce"). In addition, although this example describes providing visual feedback to the user (e.g., by changing the icon from "lettuce" to "no lettuce"), the techniques may additionally or alternatively provide other types of feedback, such as audible feedback, haptic feedback, and/or the like.

FIG. 72 further shows that the operator has provided a selection 7204 of another modifier ("fried onions"). FIG. 73 shows an example UI 7300 to which the FOH device may respond. As shown, the UI 7300 includes a menu of options 7302, which in this instance appears to the left of the selected modifier. Again, the operator can swipe up or down to move the menu to the desired location and select the desired option. Additionally, while the instances in FIGS. 68-73 show the menu of options moving in opposition to the trajectory of the operator's swipe gesture, in some instances the menu can move in response to (i.e., in the same direction as) the swipe gesture.

While the above examples describe some examples for modifying an item, such as swiping up and down, swiping at a threshold velocity, other examples may allow for modification of an item based on one or more additional or alternative gestures. For example, the device may include a touch sensor configured to determine the amount of force being applied to the display and associate a particular modifier with the item based on the level of force. For example, a user lightly touching the "lettuce" icon may cause the icon to display "no lettuce," pressing harder may cause the icon to change to "extra lettuce," etc. Additionally, the device may be configured to modify the item based on any combination of inputs, such as force, velocity, audible commands, etc. Additionally, the device may be configured to output numerous types of output signals to indicate the addition of a modifier, such as an audible sound, a tactile response, a visual response, etc.

Automatic generation of seating area layouts

FIG. 74 illustrates the example architecture 100 of FIG. 1. Here, a merchant, such as the example user 104(2), operates a merchant device, such as the example FOH device 106(2), to capture image data of the facility 102. For example, the FOH device 106(2) may utilize one or more LiDAR sensors 200 (e.g., cameras, light sources, etc.) to generate sensor data of the environment. For example, the device's LiDAR sensor 200 may include a light source that illuminates a portion or portions of the environment and a camera that generates data indicative of the amount and timing of reflection of the light emitted by the light source. The merchant device can then analyze this sensor data or transmit this sensor data to the remote service 112 or another device for analysis. The analysis may be useful in identifying one or more surfaces (e.g., flat surfaces) within the facility 102 that meet certain criteria. Additionally, the merchant device can present a UI 7400 to allow the merchant to indicate which identified surfaces correspond to seating areas and which surfaces do not. After receiving the respective instructions from the merchant, the merchant device, the remote service 112, or another device can generate and present on the merchant device a layout of the available seating areas in the facility. Of course, while this example describes automatically generating a seating layout for a facility, it should be understood that the techniques can be applied to automatically generating any other type of facility layout. For example, the techniques can be used to generate layouts for clothing stores (e.g., showing where clothing items are displayed), grocery stores (showing where food items are located), classrooms (identifying where students will sit), and/or any other type of facility layout.

Further, it should be appreciated that the generated layout(s) may be used in conjunction with the techniques described above with reference to Figures 1-73. For example, after generating a layout of a facility, the layout may be used to determine which FOH devices are associated with (e.g., responsible for) which seating areas within the facility. This information may then be used to determine which FOH and/or BOH devices should establish communication sessions to clarify orders, etc., as described above. In other words, the techniques described below with reference to Figures 74-80 may be used in conjunction with the techniques described above with reference to Figures 1-73 to help inform devices within the facility as to which communication sessions to establish with which devices.

In some instances, the remote service 112 may be configured to analyze image data of the facility 102 and generate a layout of available seating areas. Thus, the user 104(2) may operate the facility application 210 on the FOH device 106(2) (or other merchant device) to generate image data of the interior of the facility 102 and transmit this image data over the network 114 to the remote service.

As shown, the remote service may include one or more processors 7402 and a computer readable medium 7404 that stores an instance of a facility application 7406. In this example, the facility application 7406 may include a data analysis component 7408, a UI component 7410, a layout component 7412, and a proposal component 7414, as described below, along with sensor data 7416 received from the merchant devices, and one or more models 7418 for identifying relevant surfaces and determining a proposed layout. Here, flat surfaces are identified as relevant surfaces, but depending on the facility (e.g., identified via an inventory of items offered), surfaces may be defined in other ways, such as retail surfaces (e.g., a retail clothing store example), or e-commerce surfaces (e.g., a grocery store example, where shelves are located).

The data analysis component 7408 may function to receive imagery or other sensor data received from the FOH device 106(2) and analyze the data to identify candidate planes within the environment. In some instances, the data analysis component 7408 uses computer vision (CV) techniques and trained machine learning models to identify planes. In other examples, the data analysis component 7408 uses LiDAR techniques to identify these surfaces.

In one embodiment, a LiDAR sensor performs a scan to collect scanner data comprising multiple scan points. Each scan point specifies a range to a surface imaged by the LiDAR sensor when emitting a laser pulse in a particular direction. The sensor data is then mapped to known surface data to determine the sensor data that corresponds to the known surface data within a threshold error. In one implementation, an ML model, such as an unsupervised or supervised training model, can be used to identify surfaces, such as dining surfaces, retail surfaces, etc., and a server or merchant can provide a definition of such surfaces. A dining surface can be any surface that is substantially flat and has certain dimensions that allow dishes to be laid out. A dining surface for another section of the establishment can have sufficient dimensions to accommodate only certain items, such as beverages. The sensor data can also be used to identify furniture or other stationary objects in the environment. In some implementations, an architectural sketch of the environment, or a comparable environment from a similar merchant, can be provided that the layout API feeds into the data analysis component 7408. The layout API can also include data, such as restaurant menus, inventory, most requested items, etc., for use as training data in the ML model.

In one example, the data analysis component 7408 can analyze the received image data to identify surfaces that meet certain criteria. In some examples, the criteria may include surfaces that are substantially parallel to a horizontal plane, surfaces that have an area greater than a threshold area, surfaces that have a width and/or length greater than a respective threshold width and/or length, surfaces that are elevated a certain distance (e.g., greater than 2 feet) from the floor, surfaces that are within a certain range of the floor (e.g., 2 feet to 6 feet), and/or the like. These criteria help ensure that the identified candidate flat surfaces correspond to potential seating areas, as opposed to non-seating area flat surfaces such as cabinet tops or surfaces. Additionally or alternatively, the data analysis component 7408 can analyze the received image data to determine a level of glare or reflection, which can then be used to determine whether an otherwise flat surface corresponds to a seating location. For example, if a table has some glare while a carpet does not, this data can be used by the trained model to obscure available seating areas from other flat surfaces such as the floor.

After the data analysis component 7408 identifies candidate flat surfaces, the UI component 7410 can generate a UI for presentation on the merchant device to allow the merchant to select which candidate flat surfaces correspond and/or do not correspond to the available seating area. FIG. 78 illustrates an example UI 7800 that the UI component 7410 can generate and send to the merchant device, for example. As illustrated, the UI 7800 can include image data 7802 captured by a camera on the merchant device overlaid with representations 7804(1), 7804(2), and 7804(3) of each of the flat surfaces in the environment that the data analysis component 7408 identified as meeting the specified criteria. In some instances, the layout generated as a result of the image 7802 can also show depths, heights, and other measurements to create a layout of the entire facility with the defined surfaces.

In one example, augmented reality (AR), virtual reality, mixed reality, or a combination thereof can be used to record a "scene" and insert virtual objects into the real environment. In one implementation, virtual objects can be overlaid on a live camera feed with flat surfaces identified via a LiDAR sensor. In another implementation, the LiDAR modeled environment can act as a virtual canvas onto which virtual objects are overlaid without interacting with the live camera feed. Additionally or alternatively, relevant aspects of the establishment's actual environment can be embedded in the virtual layout (e.g., restaurant canvas, retail store canvas, brick-and-mortar grocery store canvas, etc.) or the virtual world can be restyled to better match real-time conditions. Elements of the real-time environment can be incorporated into the scene generated by the augmented layout. Elements can be weather conditions, time parameters, essential elements of the establishment (POS devices, furniture, table-side devices, servers, etc.) and can be utilized to augment the layout. In another example, moving elements (e.g., servers taking orders, clothing racks, etc.) can be detected and incorporated into the scene using the data analysis component 7408. This may include overlaying content onto the real world image via the computing device (e.g., in near real-time) such that a user of the computing device seamlessly sees a combination of the real world image with the AR content. Additionally, the UI 7800 includes a request 7806 to "select which flat surfaces are available seating areas." In some instances, the merchant may use a touch screen display or other input mechanism on the merchant device to select which of the identified candidate flat surfaces 7804(1)-(3) represent available seating areas in the establishment. For example, the UI 7800 may request that the merchant touch a flat surface that corresponds to an available seating area. Additionally, the UI 7800 may allow the merchant to specify the number and/or location of available seats in each seating area.

For example, FIG. 79 illustrates an exemplary UI 7900 after a merchant has selected which of the candidate flat surfaces correspond to the available seating area. In this example, the merchant has indicated that each of the three flat surfaces 7804(1)-(3) corresponds to an available seating area. After providing these selections and/or indicating that the merchant has performed, the UI 7900 may present a menu 7902 that requests the merchant to indicate whether they are ready to create a layout or whether they wish to continue scanning the facility 102 (e.g., walking through the facility and generating image data of the facility 102).

Returning to FIG. 74, after the merchant has provided input back to the facility application 7406 regarding which of the identified candidate flat surfaces correspond to available seating areas in the facility 102 and requested that a layout be generated, the layout component 7412 can generate a layout of the available seating areas in the facility. For example, FIG. 80 illustrates an example UI 8000 that the layout component 7412 can generate. As shown, the UI 8000 illustrates that in this example, the layout of the facility includes an area 8002 of tables and an area 8004 of bar seats. In some instances, the UI 8000 can provide editing tools that allow the merchant to add, remove, or modify portions of the layout. Additionally, in some instances, the UI 8000 can include AR objects overlaid on the layout to allow the merchant to see what the facility would look like under normal operating conditions. For example, the UI 8000 may allow the device operator to overlay layers including servers, other employees, appliances, items, and/or any other type of AR object, allowing the operator to better understand the generated layout as it would appear in the facility when presented on the UI 8000.

Returning again to FIG. 74, the suggestion component 7414 may function to generate one or more layout suggestions and present these suggestions to the merchant. For example, after the layout component 7412 generates a layout, the suggestion component 7414 may analyze the generated layout to determine whether additional tables can be placed in the facility, whether a more efficient layout can be created, etc. In some examples, the suggestion component 7412 may use one or more trained machine learning models that receive the layout as input and output one or more alternative layouts that may be suggested to the merchant.

75 illustrates a flow diagram of an example process 7500 for generating a layout of available seating areas within a facility. In some instances, process 7500, as well as processes described below, may be performed in whole or in part by a merchant device, a remote service 112, and/or a combination thereof. Operation 7502 represents generating image data of the merchant's physical facility using a camera of the merchant device. Operation 7504 represents determining whether any flat surfaces and corresponding locations have been identified based on an analysis of the image data. If not, operation 7506 represents presenting (e.g., on the merchant device) an indication that no flat surfaces have been identified.

However, if one or more flat surfaces and their respective locations within the facility have been identified, operation 7508 represents presenting, on a display of the merchant device, image data superimposed with a representation of each of the identified flat surfaces. For example, this operation may include presenting UI 7800 on the merchant device. Operation 7510 represents receiving a selection of a first flat surface, the selection indicating that the first flat surface represents an available seating area within the physical facility. Again, this selection may be received via UI 7800 or in any other manner. Meanwhile, operation 7512 represents receiving an indication of a number of available seats associated with available seating locations at a first location associated with the first flat surface. In some instances, the merchant uses the merchant device to provide this indication of the number of available seats.

Operation 7514 represents receiving a selection of a second flat surface, where the selection of the second flat surface indicates that the second flat surface is not an available seating area. For example, the merchant may use UI 7800 to provide a selection indicating that the second flat surface is not an available seating area. Operation 7516 represents generating a layout of available seating areas within the physical facility based at least in part on the first and second selections, where the seating area layout indicates that the first flat surface is an available seat at the first location. Operation 7518 represents presenting the layout on a display of the merchant device. For example, this operation may include presenting UI 8000 on the merchant device.

76 illustrates a flow diagram of another example process 7400 for determining a proposed layout of available seating areas in a facility. Operation 7402 represents analyzing image data to determine a proposed layout of available seating areas at a merchant's facility. For example, this operation may include a proposal component 7414 that inputs the image data and/or the generated layout into a trained machine learning model. Operation 7604 represents determining whether a proposed layout has been determined. If not, processing may end at operation 7608. However, if a proposed layout has been determined, operation 7606 represents presenting the proposed layout of available seating areas on a display of the merchant device.

Once the layout has been derived and the AR content objects have been generated based on the derived view of the layout, the AR objects may be overlaid on the layout. Thus, operation 7610 represents determining whether a request has been received (e.g., from a user operating the device) to add an AR object to the proposed layout, as described above. If so, operation 7612 represents embedding one or more AR objects into the proposed layout. If not, processing may end at operation 7614. Operation 7612 may include generating the AR object based on one or more characteristics of the physical facility, such as a type of facility, objects within the facility, etc. For example, if the facility includes a restaurant, the generated AR object may include servers, food, etc. If the facility includes a clothing store, the generated AR object may include clothing items and/or the like. Operation 7612 may further include associating the AR object with image data (e.g., the presented layout) and modifying the presented layout to include the generated AR object. Modifying the presented layout may comprise showing the AR objects in the same view as the presented layout in response to a user request to do so (e.g., a request to present a layer associated with the AR object), or in some examples in response to modifying the layout itself that was generated based on the AR object. For example, a surface identified in the layout may be modified based on the position of the AR object.

The set of AR content objects may be obtained in any suitable manner, for example, based on a search query of an AR object database (e.g., searching for AR content objects associated with one or more descriptors associated with one or more views of the layout), based on characteristics of the initial map (e.g., dimensions, architectural sketches, indications of types of facilities, etc.), based on user selections, recommendations, or requests (e.g., advertisers, merchants, etc.), or based on the user's intended context of use (e.g., based on the activity the user wishes to capture (e.g., shopping, dining, gaming, etc.)).

FIG. 77 illustrates a flow diagram of another example process 7700 for generating an example seating area layout within a facility. Operation 7702 represents receiving data from devices within the environment. For example, this operation may include receiving image data from a merchant device, such as the example FOH device 106(2).

Operation 7704 represents determining whether any flat surfaces and corresponding locations have been identified. If not, process 7700 ends at operation 7706. However, if so, operation 7708 represents generating a UI showing the identified flat surfaces, and operation 7710 represents sending the UI to a device, such as a merchant device. In some instances, identifying flat surfaces and their corresponding locations may include inputting data into one or more trained machine learning models to identify flat surfaces and their X,Y or X,Y,Z locations, etc., within the facility using LIDAR techniques.

Operation 7712 represents receiving a selection of a first planar surface from the device indicating that the first planar surface represents an available seating area in the environment. For example, the selection may be received via the exemplary UI 7900. Operation 7714 represents generating a layout of available seating areas in the environment based at least in part on the selection, the layout of the seating areas indicating that the first planar surface is an available seat at the first location, and operation 7716 represents transmitting the layout of the available seating areas to the device. For example, the operation may comprise transmitting the UI 8000 to the merchant device. The generated layout may then be used to perform any of the techniques described above with reference to the operation of the facility application 210 executing on the BOH and/or FOH device.

Examples of environments and systems

FIG. 81 illustrates an exemplary environment 8100. The environment 8100 includes a server computing device 8102 that can communicate with user devices 8106 (which in some examples may be merchant devices 8108 (individually, 8108(A)-8108(N)) and/or a server computing device 8110 associated with a third-party service provider) via a network 81104. The server computing device 8102 can be associated with a service provider 8112 that can provide one or more services for the benefit of a user 8114, as described below. Actions attributed to the service provider 8112 can be executed by the server computing device 8102.

In some instances, the merchant device 8108 and/or the user device 8106 may correspond to the BOH and/or FOH device 200 described above. Additionally, the server computing device 8102 of the service provider 8112 may correspond to the server computing device of the remote service 112, as described in detail above.

Referring to FIG. 81, the server computing device 8102 may correspond to a server computing device of the remote service 112. Additionally, the user device 8106 may correspond to an FOH/BOH device, or other merchant and/or customer device. In at least one example, the service provider 8112 may correspond to the remote service 112 described above.

The environment 8100 may include a plurality of user devices 8106, as described above. Each of the plurality of user devices 8106 may be any type of computing device, such as a tablet computing device, a smartphone or mobile communication device, a laptop, netbook or other portable or semi-portable computer, a desktop computing device, a terminal computing device or other semi-stationary or stationary computing device, a dedicated device, a wearable computing device or other body-worn computing device, an augmented reality device, a virtual reality device, an Internet of Things (IoT) device, etc. In some examples, individual ones of the user devices may be operable by a user 8114. The user 8114 may be referred to as a customer, a buyer, a merchant, a seller, a borrower, an employee, an employer, a payer, a payee, a courier, etc. The user 8114 may interact with the user device 8106 via a user interface presented via the user device 8106. In at least one example, the user interface may be presented via a web browser or the like. In other examples, the user interface may be presented via an application, such as a mobile or desktop application, which may be provided by the service provider 8112 or may be another dedicated application. In at least one example, the user 8114 may interact with the user interface via touch input, voice input, or any other type of input.

As mentioned above, in at least one example, the user 8114 can include merchants 8116 (individually, 8116(A)-8116(N)). In one example, the merchants 8116 can operate respective merchant devices 8108, which can be user devices 8106 configured for use by the merchants 8116. For purposes of this discussion, a "merchant" can be any entity, or employee/associate of an entity, that offers items (e.g., goods or services) for purchase or other means of acquisition (e.g., rent, borrow, bargain, etc.). The merchants 8116 can offer items for purchase or other means of acquisition via a brick-and-mortar store, a mobile store (e.g., pop-up store, food truck, etc.), an online store, a combination of the foregoing, etc. In some examples, at least some of the merchants 8116 can be associated with the same entity but can have different merchant locations and/or can have franchise/franchisee relationships. In additional or alternative examples, the merchants 8116 can be different merchants. That is, in at least one example, merchant 8116(A) is a different merchant than merchant 8116(B) and/or merchant 8116(C).

For purposes of this description, "different merchants" may refer to two or more unrelated merchants. Thus, "different merchants" may refer to two or more merchants that are different legal entities that do not share accounting, employees, branding, etc. As used herein, "different merchants" have different names, employer identification numbers (EINs), lines of business (in some instances), inventory (or at least a portion thereof), etc. That is, use of the term "different merchants" does not refer to merchants that have various merchant locations or franchise/franchisee relationships.

Each merchant device 8108 may have an instance of a POS application 8118 stored thereon. The POS application 8118 may configure the merchant device 8108 as a POS terminal, allowing a merchant 8116(A) to interact with one or more customers 8120. As discussed above, the user 8114 may include a customer, such as customer 8120 shown interacting with merchant 8116(A). For purposes of this discussion, a "customer" may be any entity that obtains items from a merchant. Although only two customers 8120 are shown in FIG. 81, any number of customers 8120 may interact with a merchant 8116. Additionally, while FIG. 81 shows customer 8120 interacting with merchant 8116(A), customer 8120 may interact with any of the merchants 8116.

An interaction between a customer 8120 and a merchant 8116, involving an exchange of funds (from a customer 8120) for an item (from a merchant 8116) in at least one example may be referred to as a "POS transaction" and/or a "transaction", and in at least one example the POS application 8118 may determine transaction data associated with the POS transaction. The transaction data may include payment information, user authentication data, purchase amount information, point of purchase information (e.g., item purchased, purchase date, purchase time, etc.), which may be obtained from a reader device 8122 associated with the merchant device 8108(A). The POS application 8118 may transmit the transaction data to the server computing device 8102. Additionally, the POS application 8118 may present a UI to enable the merchant 8116(A) to interact with the POS application 8118 and/or the service provider 8112 via the POS application 8118.

In at least one example, merchant device 8108(A) may be a dedicated computing device configured as a POS terminal (via execution of POS application 8118). In at least one example, the POS terminal may be connected to a reader device 8122 capable of accepting a variety of payment instruments, such as credit cards, debit cards, gift cards, near-field communication-based payment instruments, as described below. In some examples, reader device 8122 may read information from alternative payment instruments, including, but not limited to, wristbands, etc.

The POS terminal can be paired with a reader device 8122 that constitutes the POS system 8124. The reader device 8122 can physically interact with a payment instrument, such as a magnetic stripe payment card, an EMV payment card, and/or a near field communication (e.g., Near Field Communication (NFC), Radio Frequency Identification (RFID), Bluetooth, Bluetooth Low Energy (BLE), etc.) payment instrument. The POS terminal can provide a rich user interface, communicate with the reader device 8122, and communicate with a server computing device 8102, which can provide, among other services, payment processing services. A service provider 8112 can communicate with the server computing device 8110, as described below. In this manner, the POS terminal and the reader device 8122 can collectively process transactions between the merchant 8116 and the customer 8120. In some examples, the POS terminal and the reader device can be configured in a one-to-one pairing. In other examples, the POS terminals and reader devices may be configured in a many-to-one pairing (e.g., one POS terminal coupled to multiple reader devices or multiple POS terminals coupled to one reader device).

Although the POS terminal and reader device 8122 of the POS system 8124 are shown as separate devices, in additional or alternative examples the POS terminal and reader device 8122 may be part of a single device. In some examples, the reader device 8122 may have a display integrated therein for presenting information to the customer 8120. In additional or alternative examples, the POS terminal may have a display integrated therein for presenting information to the customer 8120. POS systems such as the POS system 8124 may be mobile such that the POS terminal and reader device may process transactions in different locations around the world. POS systems may be used to process card present and card not present (CNP) transactions as described below.

A card presence transaction may be processed by a swipe, dip, tap, or any other interaction between a physical payment instrument (e.g., card) and the reader device 8122, whereby the reader device 8122 can obtain payment data from the payment instrument. A swipe is a card presence transaction in which the customer 8120 slides a card or other payment instrument having a magnetic strip through the reader device 8122, which captures the payment data contained on the magnetic strip. A dip is a card presence transaction in which the customer 8120 first inserts a payment instrument having an embedded microchip (i.e., chip) into the reader device 8122. The dipped payment instrument remains in the payment reader until the reader device 8122 prompts the customer 8120 to remove the card or other payment instrument. While the payment instrument is in the reader device 8122, the microchip can be sent from the POS system 8124 to the server computing device 8110 (which can be associated with an acquirer bank, an issuer, and/or a third party service provider providing payment services, including but not limited to a card payment network (e.g., Mastercard®, VISA®, etc.)) to create a time code that matches the same time code. A tap is a card presence transaction in which a customer 8120 can tap or hover their payment instrument (e.g., a card, an electronic device such as a smartphone running a payment application) over the reader device 8122 to complete the transaction via short-range communication (e.g., NFC, RFID, Bluetooth®, BLE, etc.). The short-range communication allows the payment instrument to exchange information with the reader device 8122. A tap is sometimes referred to as a contactless payment.

A CNP transaction is one in which a card or other payment instrument is not physically present at the POS, and as a result, payment data must be manually keyed in (e.g., by the merchant, customer, etc.) or payment data must be recalled from a card-on-file data store to complete the transaction.

The POS system 8124, the server computing device 8102, and/or the server computing device 8110 can exchange payment information and transaction data to determine whether a transaction is authorized. For example, the POS system 8124 can provide encrypted payment data, user authentication data, purchase amount information, point of purchase information, etc. (collectively, transaction data) to the server computing device 8102 over the network 81104. The server computing device 8102 can transmit the transaction data to the server computing device 8110. As mentioned above, in at least one example, the server computing device 8110 can be associated with a third party service provider that provides payment services, including, but not limited to, an acquirer bank, an issuer, and/or a card payment network (e.g., Mastercard®, VISA®, etc.).

For purposes of this discussion, a "payment service provider" may be acquiring bank ("acquirer"), issuing bank ("issuer"), card payment network, etc. In one example, an acquirer is a bank or financial institution that can process payments (e.g., credit card or debit card payments) and assume risk on behalf of the merchant. An acquirer may be a registered member of a card association (e.g., Visa®, MasterCard®) and may be part of a card payment network. An acquirer (e.g., its associated server computing device 8110) may send a funds transfer request to a server computing device of a card payment network (e.g., MasterCard®, VISA®, etc.) to determine whether the transaction is authorized or delinquent. In at least one example, service provider 8112 may act as an acquirer and connect directly to the card payment network.

The card payment network (e.g., a server computing device 8110 associated therewith) can forward the funds transfer request to the issuing bank (e.g., an "issuer"). The issuer is a bank or financial institution that provides a financial account (e.g., a credit or debit card account) to a user. The issuer can issue a payment card to a user and can make payments to an acquirer for purchases made by a cardholder to whom the issuing bank issued the payment card. The issuer (e.g., a server computing device 8110 associated therewith) can make a determination as to whether a customer has the ability to absorb associated fees associated with a payment transaction. In at least one example, a service provider 8112 can act as an issuer and/or can partner with an issuer. The transaction is approved or rejected by the issuer and/or the card payment network (e.g., a server computing device 8110 associated therewith), and a payment authorization message is communicated from the issuer to the POS device via a path opposite to that described above or via an alternative path.

As described above, the server computing device 8110, which may be associated with a payment service provider, may determine whether a transaction is authorized based on the transaction data and information about the parties to the transaction (e.g., the customer 8120 and/or the merchant 8116(A)). The server computing device 8110 may send an authorization notification to the server computing device 8102 over the network 81104, which may in turn send an authorization notification to the POS system 8124 over the network 81104 to indicate whether the transaction is authorized. The server computing device 8102 may also send additional information, such as a transaction identifier, to the POS system 8124. In one example, the server computing device 8102 may include a merchant application and/or other functional modules for communicating with the POS system 8124 and/or the server computing device 8110 to authorize or deny the transaction.

Based on the authorization notification received by the POS system 8124 from the server computing device 8110, the merchant 8116(A) may indicate to the customer 8120 whether the transaction was approved. In some examples, the approval may be indicated at the POS system 8124, for example, on a display of the POS system 8124. In other examples, for example, if a smartphone or watch is operating as a near-field communication payment instrument, information regarding the approved transaction may be provided to the near-field communication payment instrument for presentation via a display of the smartphone or watch. In some examples, additional or alternative information may be further presented along with the approved transaction notification, including, but not limited to, a receipt, special offers, coupons, or loyalty program information.

As mentioned above, the service provider 8112 may provide payment processing services, inventory management services, catalog management services, business banking services, financial services, web development services, payroll services, employee management services, reservation services, restaurant management services, order management services, delivery/home delivery services, peer-to-peer payment services, ride-hailing services, identity verification (IDV) services, etc., among other services. In some examples, the user 8114 may have access to all of the service provider's 8112 services. In other examples, the user 8114 may have tiered access to services that may be based on risk tolerance, IDV output, subscription, etc. In at least one example, access to such services may be available to the merchant 8116 via the POS application 8118. In additional or alternative examples, each service may be associated with its own access point (e.g., application, web browser, etc.).

The service provider 8112 may provide payment processing services to process payments on behalf of the merchant 8116, as described above. For example, the service provider 8112 may provide payment processing software, payment processing hardware, and/or payment processing services to the merchant 8116 to enable the merchant 8116 to receive payments from the customer 8120 when conducting a POS transaction with the customer 8120, as described above. For example, the service provider 8112 may enable the merchant 8116 to receive cash payments, card payments, and/or electronic payments from the customer 8120 for a POS transaction, and the service provider 8112 may process the transaction on behalf of the merchant 8116.

When the service provider 8112 processes a transaction on behalf of the merchant 8116, the service provider 8112 may maintain an account or balance for the merchant 8116 in one or more ledgers. For example, the service provider 8112 may analyze transaction data received for the transaction to determine an amount of funds to be paid to the merchant 8116(A) for the transaction. In at least one example, such amount may be the total purchase price less a fee charged by the service provider 8112 for providing payment processing services. Based on determining the amount of funds to be paid to the merchant 8116(A), the service provider 8112 may deposit the funds into an account for the merchant 8116(A). The account may have a stored balance, which may be managed by the service provider 8112. The account may differ from a traditional bank account at least because the stored balance is managed by the service provider's 8112 ledger and the associated funds are accessible through a variety of withdrawal channels, including, but not limited to, scheduled deposits, same-day deposits, immediate deposits, and linked payment instruments.

A scheduled deposit can occur when the service provider 8112 transfers funds associated with the stored balance of the merchant 8116(A) to the merchant's 8116(A) bank account held at a bank or other financial institution (e.g., associated with the server computing device 8110). The scheduled deposit can occur at a pre-set time after the POS transaction is made, i.e., on a business day after the POS transaction is made, or sooner or later. In some examples, the merchant 8116(A) can access the funds before the scheduled deposit. For example, the merchant 8116(A) can access a same-day deposit (e.g., in some examples before the POS transaction is funded, the service provider 8112 deposits funds from the stored balance to the merchant's linked bank account on the same day as the POS transaction) or an immediate deposit (e.g., the service provider 8112 deposits funds from the stored balance to the merchant's linked bank account on demand, upon request). Additionally, in at least one example, the merchant 8116(A) may have a payment instrument linked to the stored balance that allows the merchant to access funds without first transferring the funds from an account managed by the service provider 8112 to the merchant's 8116(A) bank account.

In at least one example, the service provider 8112 may provide inventory management services. That is, the service provider 8112 may provide inventory tracking and reporting. The inventory management services may enable the merchant 8116(A) to access and manage a database that stores data related to the quantity (i.e., inventory) of each item available. Additionally, in at least one example, the service provider 8112 may provide catalog management services to enable the merchant 8116(A) to maintain a catalog, which may be a database (i.e., a catalog management service) that stores data related to items available for acquisition to the merchant 8116(A). In at least one example, the catalog may include a plurality of data items, where the data items of the plurality of data items may represent items available for acquisition to the merchant 1461(A).

In at least one example, service provider 8112 may provide business banking services that allow merchant 8116(A) to track deposits (from payment processing and/or other funding sources) to merchant's 8116(A) account, payments from the account (e.g., payments to employees of merchant 8116(A)), payments to other merchants directly from the account or from a linked debit card, withdrawals made via scheduled and/or instant deposits, etc. Additionally, the business banking services may allow merchant 8116(A) to check how much money they are earning (e.g., via a presentation of available earned balance), understand where their money is going (e.g., via deposit reports (which may include a breakdown of fees), expense reports, etc.), access/use the money they have earned (e.g., via scheduled deposits, instant deposits, linked payment instruments, etc.), feel in control of their money (e.g., via management of deposit schedules, deposit speeds, linked products, etc.), etc. Additionally, business banking services can enable merchants 8116 to visualize their cash flow to track their financial health, set aside money for upcoming obligations (e.g., savings), organize their finances around goals, and the like.

In at least one example, the service provider 8112 may offer financial services and products via business loans, consumer loans, fixed term loans, flexible term loans, etc. In at least one example, the service provider 8112 may utilize one or more risk signals to determine whether to extend funding and/or terms associated with such funding.

In at least one example, the service provider 8112 may provide financing services to provide and/or lend loans to borrowers that in some examples may be used to fund the borrower's short-term operational needs (e.g., capital loans). For example, a potential borrower who is a merchant may obtain a capital loan via a capital loan product to finance various operational costs (e.g., rent, payroll, inventory, etc.). In at least one example, the service provider 8112 may offer different types of capital loan products. For example, in at least one example, the service provider 8112 may offer a daily repayment loan product, where the capital loan is repaid daily, e.g., from a portion of transactions processed by the payment processing service on behalf of the borrower. Additionally and/or alternatively, the service provider 8112 may offer a monthly repayment loan product, where the capital loan is repaid monthly, e.g., via the borrower from a bank account linked to the payment processing service.

Additionally or alternatively, the service provider 8112 may provide financing services to provide and/or lend loans to borrowers that, in some examples, will be used to fund the borrower's consumer purchases (e.g., consumer loans). In at least one example, a request for a loan may be submitted to enable the borrower to purchase an item from a merchant, which may be one of the merchants 8116. The service provider 8112 may generate a loan based, at least in part, on determining that the borrower has purchased or is about to purchase an item from the merchant. The loan may be associated with a balance based on the actual purchase price of the item, and the borrower may repay the loan over time. In some examples, the borrower may repay the loan via installments, which may be paid via a stored balance associated with the service provider 8112. The service provider 8112 may offer certain financial products, such as payment instruments, that are specifically tied to the loan product. For example, in one implementation, the service provider 8112 associates capital with a merchant's or customer's debit card, and the use of the debit card is defined by the terms of the loan. In some examples, the merchant may only use the debit card to make certain purchases. In other instances, "installments" related to a loan product are credited directly via the payment instrument. Thus, the payment instrument is customized to the loan and/or the parties associated with the loan.

The service provider 8112 may provide web development services that enable users 8114 who are not familiar with HTML, XML, Javascript, CSS, or other web design tools to create and maintain professional, aesthetically pleasing websites. Some of these web page editing applications allow users to build web pages and/or modify web pages (e.g., change, add, or remove content associated with a web page). Furthermore, in addition to websites, the web development services may create and maintain other online omnichannel presences, such as social media posts.

Additionally, the service provider 8112 may provide payroll services that allow employers to pay employees for work performed on behalf of the employer. In at least one example, the service provider 8112 may receive data including hours worked by employees, sales made by employees, tips received by employees, and the like. Based on such data, the service provider 8112 may make payroll payments to employees on behalf of the employer via payroll services. For example, the service provider 8112 may facilitate a transfer from the employer's bank to the service provider 8112's bank of a total amount due for the employee's payroll to be used to make the payroll payment. In at least one example, when funds are received at the service provider 8112's bank, the service provider 8112 may pay the employee by check or direct deposit, etc., when the work is actually performed by the employee, often a day, a week, or more later. In an additional or alternative example, the service provider 8112 may enable the employee to receive payment via same-day or immediate deposit, based at least in part on risk and/or reliability analysis performed by the service provider 8112.

Further, in at least one example, service provider 8112 may provide employee management services for managing employee schedules. Further, service provider 8112 may provide scheduling services for setting schedules for users 8114 to schedule appointments and/or for enabling users 8114 to schedule appointments.

In some examples, service provider 8112 may provide restaurant management services to enable user 8114 to make and/or manage reservations, monitor front of home and/or back of home operations, etc. In at least one example, service provider 8112 may provide order management services and/or delivery services to enable a restaurant to manage open tickets, split tickets, etc., and/or manage delivery services. In some examples, such services may be associated with a restaurant merchant, as described above. In additional or alternative examples, such services may be any type of merchant.

As described above, the service provider 8112 may provide a courier service, where the courier may travel between multiple locations to provide delivery services, photo services, etc. The courier may be a user 8114 who may travel between locations to perform a service (e.g., deliver an item, capture an image, etc.) for the requesting user 8114. In some examples, the courier may receive compensation from the service provider 8112. The courier may use one or more vehicles, such as a car, bicycle, scooter, motorcycle, bus, airplane, helicopter, boat, skateboard, etc. However, in other examples, the courier may travel on foot or in another manner without a vehicle. Some examples discussed herein allow people to participate as couriers in a type of crowdsourced service economy. Here, essentially, any person with a mobile device may quickly become a courier or cease to be a courier in a courier network that provides the services described herein. In at least one example, the courier may be an unmanned aerial vehicle (e.g., a drone), an autonomous vehicle, or any other type of vehicle that may receive instructions to travel between locations.

In at least one example, a service provider 8112 can provide a peer-to-peer payment service enabling peer-to-peer payments between two or more users 8114. In at least one example, the service provider 8112 can communicate with an instance of a payment application (or other access point) installed on a device 8106 configured for operation by a user 8114. In one example, an instance of a payment application running on a first device operated by a payer can send a request to the service provider 8112 to transfer an amount of funds (e.g., fiat currency or non-fiat currency such as cryptocurrency, securities, and related assets) from a payer's account to a payee's account (e.g., a peer-to-peer payment). The service provider 8112 can facilitate the transfer and can send a notification to an instance of a payment application running on a second mobile device operated by the payee that the transfer is in process (or has been completed). In some examples, the service provider 8112 can send additional or alternative information to the instance of the payment application (e.g., a low balance to the payer, a current balance to the payer or payee, etc.). In some examples, the payer can transfer funds to an unclaimed payer. In other examples, the payee may send a request for funds to the payer before the payer initiates the transfer of funds. The funds transferred may be associated with any digital currency type, including but not limited to cash, cryptocurrency, etc.

In at least one example, the user 8114 may be new to the service provider 8112, such that the user 8114 has not registered (e.g., subscribed to receive access to one or more services offered by the service provider) with the service provider 8112. The service provider 8112 may provide an onboarding service for registering the potential user 8114 with the service provider 8112. In some examples, onboarding may involve presenting the potential user 8114 with various questions, prompts, etc., to obtain information that may be used to generate a profile for the potential user 8114. In response to the potential user 8114 providing all the necessary information, the potential user 8114 may be onboarded to the service provider 8112.

The service provider 8112 may be associated with an IDV service that may be used by the service provider 8112 for compliance purposes and/or may be provided as a service, for example, to a third party service provider (e.g., associated with the server computing device 8110). That is, the service provider 8112 may provide an IDV service to verify the identity of a user 8114 who uses or attempts to use their services. Identity verification requires a customer (or potential customer) to provide information used by a compliance department to prove that the information is associated with the identity of a real person or entity. In at least one example, the service provider 8112 may perform a service to determine whether the identification information provided by the user 8114 accurately identifies the customer (or potential customer) (i.e., whether they are the customer they say they are).

The service provider 8112 may provide additional or alternative services, and the services described above are provided as a sampling of services. In at least one example, the service provider 8112 may exchange data with a server computing device 8110 associated with a third-party service provider. Such a third-party service provider may provide information that enables the service provider 8112 to provide services such as those described above. In additional or alternative examples, such a third-party service provider may have access to the services of the service provider 8112. That is, in some examples, the third-party service provider subscribes to or otherwise has access to the services of the service provider 8112.

The techniques described herein can be configured to operate in both real-time/online and offline modes. An "online" mode refers to a mode when the device is able to communicate with the service provider 8112 (e.g., server computing device 8102) and/or server computing device 8110 over the network 81104. In some examples, the merchant device 8108 is unable to connect with the service provider 8112 (e.g., server computing device 8102) and/or server computing device 8110, e.g., due to network connectivity issues. In additional or alternative examples, the server computing device 8102 is unable to communicate with the server computing device 8110, e.g., due to network connectivity issues. In such examples, the device may operate in an "offline" mode in which at least some payment data is stored (e.g., on the merchant device 8108 and/or server computing device 8102) until connectivity is restored and the payment data can be transmitted to the server computing device 8102 and/or server computing device 8110 for processing.

82 illustrates an exemplary block diagram illustrating a system 8200 for performing the techniques described herein. The system 8200 includes a user device 8202 that communicates with a server computing device (e.g., a server 8204) via a network 8206 (e.g., the Internet, a cable network, a cellular network, a cloud network, a wireless network (e.g., Wi-Fi), and a wired network) and a short-range communication such as Bluetooth, Bluetooth Low Energy (BLE). Although a single user device 8202 is shown, in additional or alternative examples the system 8200 can have multiple user devices, as described above with reference to FIG. 81. For example, the user device 8202 can correspond to any one of the merchant computing device 104, the photographer courier computing device 106, or the image capture device 108, as described above with reference to FIG. 1.

In at least one example, the user device 8202 may be any suitable type of computing device, e.g., portable, semi-portable, semi-stationary, or stationary. Some examples of the user device 8202 may include, but are not limited to, a tablet computing device, a smartphone or mobile communication device, a laptop, netbook or other portable or semi-portable computer, a desktop computing device, a terminal computing device or other semi-stationary or stationary computing device, a dedicated device, a wearable or other body-worn computing device, an augmented reality device, a virtual reality device, an Internet of Things (IoT) device, and the like. That is, the user device 8202 may be any computing device capable of transmitting communications and performing functions in accordance with the techniques described herein. The user equipment 8202 may include equipment such as a payment card reader or a module capable of accepting payments, as described below.

In the illustrated example, the user device 8202 includes one or more processors 8208, one or more computer-readable media 8210, one or more communication interfaces 8212, one or more input/output (I/O) devices 8214, a display 8216, and sensors 8218.

In at least one example, each processor 8208 may itself comprise one or more processors or processing cores. For example, the processor 8208 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and/or any device that manipulates signals based on operational instructions. In some examples, the processor(s) 8208 may be one or more hardware processors and/or logic circuits of any suitable type that are specifically programmed or configured to execute the algorithms and processes described herein. The processor(s) 8208 may be configured to fetch and execute computer-readable processor-executable instructions stored on the computer-readable medium 8210.

Depending on the configuration of the user device 8202, the computer readable medium 8210 may be an example of a tangible, non-transitory computer storage medium and may include volatile and non-volatile memory and/or removable and non-removable media implemented in any type of technology for storage of information such as computer readable processor executable instructions, data structures, program modules, or other data. The computer readable medium 8210 may include, but is not limited to, RAM, ROM, EEPROM, flash memory, solid state storage, magnetic disk storage, optical storage, and/or other computer readable media technologies. Additionally, in some examples, the user device 8202 may have access to external storage devices such as a RAID storage system, a storage array, network attached storage, a storage area network, cloud storage, or any other medium that may be used to store information and that may be accessed by the processor 8208 directly or through another computing device or network. Thus, the computer readable medium 8210 may be a computer storage medium capable of storing instructions, modules, or modules that may be executed by the processor 8208. Additionally, when referred to, non-transitory computer-readable media excludes media such as energy, carrier signals, electromagnetic waves, and the signals themselves.

The computer-readable medium 8210 may be used to store and maintain any number of functional modules executable by the processor 8208. In some implementations, these functional modules are executable by the processor 8208 and comprise instructions or programs that, when executed, implement operational logic for performing the actions and services described above on the user device 8202. The functional modules stored on the computer-readable medium 8210 may include a user interface 8220 to allow a user to interact with the user device 8202 and thus the server 8204 and/or other networked equipment. In at least one example, the user interface 8220 may be presented via a web browser or the like. In other examples, the user interface 8220 may be presented via an application, such as a mobile application or desktop application, that may be provided by the service provider 8112 associated with the server 8204 or may be another dedicated application. In some examples, the user interface 8220 may be an e-commerce user interface 124, a courier user interface 128, or an imaging user interface 130. In at least one example, a user can interact with the user interface via touch input, voice input, gestures, or any other type of input. The word "input" is also used to describe "contextual" input that may not be directly provided by a user via the user interface 8220. For example, a user's interaction with the user interface 8220 may be analyzed, for example, using natural language processing techniques to determine the user's context or intent, which may be processed in a similar manner as "direct" user input.

Depending on the type of user device 8202, the computer readable medium 8210 may also optionally include other modules and data 8222, which may include programs, drivers, and the like, as well as other functional modules and data, such as data used or generated by the functional modules. In some instances, the other modules and data 8222 may comprise the same functions and data as described above with respect to the merchant device 114 and other client devices. Additionally, the computer readable medium 8210 may also store data, data structures, and the like used by the functional modules. Additionally, the user device 8202 may include many other logical, program, and physical modules, the description of which is merely an example relevant to the description herein.

In at least one example, the computer-readable medium 8210 may include additional functional modules, such as an operating system 8224 for controlling and managing various functions of the user device 8202 and enabling basic user interaction.

The communication interface 8212 may include one or more interfaces and hardware modules for enabling communication with various other devices, such as through the network 8206 or directly. For example, the communication interface 8212 may enable communication over one or more networks 8206, which may include any type of network known in the art, such as, but not limited to, a local area network or a wide area network such as the Internet, a wireless network such as a cellular network, a cloud network, a local wireless network such as Wi-Fi, and/or a near field communication such as Bluetooth, BLE, NFC, RFID, a wired network, or any other such network, or any combination thereof. Thus, the network(s) 8206 may include both wired and/or wireless communication technologies, including Bluetooth, BLE, Wi-Fi, and cellular communication technologies, as well as wired or fiber optic technologies. The modules used for such communication may depend at least in part on the type of network, the environment selected, or both. Protocols for communicating over such networks are well known and will not be described in detail herein.

Embodiments of the present disclosure may be provided to users through a cloud computing infrastructure. Cloud computing refers to the provision of scalable computing resources as a service over a network, enabling convenient, on-demand network access to a shared pool of configurable computing resources that can be rapidly provisioned and released with minimal management effort or service provider interaction. Cloud computing thus enables users to access virtual computing resources (e.g., storage, data, applications, and complete virtualized computing systems) in the "cloud" regardless of the underlying physical systems (or the location of those systems) used to provide the computing resources.

The user device 8202 may further include one or more input/output (I/O) devices 8214. The I/O devices 8214 may include speakers, microphones, cameras, and various user controls (e.g., buttons, joysticks, keyboards, keypads, etc.), tactile output devices, etc. The I/O devices 8214 may also include attachments that utilize accessories (audio jacks, USB-C, Bluetooth, etc.) to connect with the user device 8202.

In at least one example, the user device 8202 may include a display 8216. Depending on the type of computing device used as the user device 8202, the display 8216 may use any suitable display technology. For example, the display 8216 may be a liquid crystal display, a plasma display, a light emitting diode display, an OLED (organic light emitting diode) display, an electronic paper display, or any other suitable type of display on which digital content can be presented. In at least one example, the display 8216 may be an augmented reality display, a virtual reality display, or any other display on which digital content can be presented and/or projected. In some examples, the display 8216 may have a touch sensor associated with the display 8216 to provide a touch screen display configured to receive touch input to enable interaction with a graphical interface presented on the display 8216. Thus, implementations herein are not limited to any particular display technology. Alternatively, in some examples, the user device 8202 may not include a display 8216, and information may be presented by other means, such as audibly, tactilely, etc.

In addition, the user device 8202 may include sensors 8218. The sensors 8218 may include a GPS device capable of indicating location information. Further, the sensors 8218 may include, but are not limited to, an accelerometer, a gyroscope, a compass, a proximity sensor, a camera, a microphone, and/or a switch.

In addition, the user device 8202 may include various other modules not shown, examples of which include removable storage, a power source such as a battery and power control unit, a barcode scanner, a printer, a cash drawer, etc.

Additionally, in some examples, the user device 8202 may include, be connectable to, or otherwise coupled to a reader device 8226 to read a payment instrument and/or identifier associated with a payment purpose. In some examples, as described above, the reader device 8226 may plug into a port in the user device 8202, such as a microphone/headphone port, a data port, or other suitable port. In additional or alternative examples, the reader device 8226 may be coupled to the user device 8202 via another wired or wireless connection, such as via Bluetooth, BLE, or the like. The reader device 8226 may include a read head for reading the magnetic strip of a payment card and may further include encryption technology for encrypting information read from the magnetic strip. Additionally or alternatively, the reader device 8226 may be an EMV payment reader, which in some examples may be embedded in the user device 8202. Additionally, numerous other types of readers may be used with the user device 8202 herein, depending on the type and configuration of the user device 8202.

The reader device 8226 may be a portable magnetic stripe card reader, an optical scanner, a smart card (card with an embedded IC chip) reader (e.g., an EMV compliant card reader or a near field communication enabled reader), an RFID reader, etc., configured to detect and acquire data from any payment instrument. Thus, the reader device 8226 may include hardware implementations such as slots, magnetic tracks, and rails with one or more sensors or electrical contacts to facilitate detection and acceptance of the payment instrument. That is, the reader device 8226 may include hardware implementations that enable the reader device 8226 to interact with a payment instrument via a swipe (i.e., a card-present transaction in which a customer slides a card having a magnetic strip through a payment read that captures the payment data contained on the magnetic strip), a dip (i.e., a card-present transaction in which a customer first inserts a card having an embedded microchip (i.e., chip) into the payment read until the payment read prompts the customer to remove the card), or a tap (i.e., a card-present transaction in which a customer taps or hovers their electronic device, such as a smartphone running a payment application, to complete the transaction via short-range communication). Additionally or optionally, the reader device 8226 may also include a biometric sensor for receiving and processing biometric characteristics and processing them as a payment instrument, provided that such biometric characteristics are registered with the payment service system and connected to a financial account using a bank server.

The reader device 8226 may include a processing unit, a computer-readable medium, a reading chip, a transaction chip, a timer, a clock, a network interface, a power supply, and the like. The central processing unit of the reader device 8226 may execute one or more modules and/or processes to cause the reader device 8226 to perform various functions, as described above and in further detail in the following disclosure. In some examples, the processing unit may include a central processing unit (CPU), a graphics processing unit (GPU), a CPU and a GPU, or processing units or modules known in the art. In addition, each of the processing unit(s) may possess its own local memory, which may also store program modules, program data, and/or one or more operating systems. Depending on the exact configuration and type of the reader device 8226, the computer-readable medium may include volatile memory (such as RAM), non-volatile memory (such as ROM, flash memory, small hard drives, memory cards, etc.), or some combination thereof. In at least one example, the computer-readable medium of the reader device 8226 may include at least one module for performing various functions described herein.

The reading chip can perform functions to control the operation and processing of the reader device 8226. That is, the reading chip can perform functions to control a payment interface (e.g., contactless interface, contact interface, etc.), a wireless communication interface, a wired interface, a user interface (e.g., a signal state device (FPGA)), etc. Additionally, the reading chip can perform functions to control a timer, which may provide a timer signal indicating an amount of time that has elapsed after a particular event (e.g., an interaction, a power-down event, etc.). Additionally, the reading chip can perform functions to control a clock, which may provide a clock signal indicating time. Additionally, the reading chip can perform functions to control a network interface, which may interface with the network(s) 8206, as described below.

In addition, the reader chip can perform functions to control the power source. The power supply can include one or more power supplies, such as a physical connection to AC power or a battery. The power supply can include power conversion circuitry to convert the AC power and generate multiple DC voltages for use by the modules of the reader device 8226. When the power supply includes a battery, the battery can be charged via a physical power connection, via inductive charging, or via any other suitable method.

The transaction chip may perform functions related to processing payment transactions, interfacing with payment instruments, encryption, and other payment specific functions. That is, the transaction chip may access payment data associated with the payment instrument and provide the payment data to the POS terminal, as described above. The payment data may include, but is not limited to, the customer's name, the customer's address, the type of payment instrument (e.g., credit, debit, etc.), a number associated with the payment instrument, a verification value (e.g., PIN Verification Key Indicator (PVKI), PIN Verification Value (PVV), Card Verification Value (CVV), Card Verification Code (CVC), etc.), expiration date data associated with the payment instrument, a Primary Account Number (PAN) corresponding to the customer (which may or may not match the number associated with the payment instrument), restrictions on what types of charges/debts may be made, etc. Additionally, the transaction chip may encrypt the payment data upon receiving it.

It should be understood that in some examples, the reading chip may have its own central processing unit(s) and computer readable media, and/or the transaction chip may have its own central processing unit(s) and computer readable media. In other examples, the functionality of the reading chip and the transaction chip may be embodied in a single chip or multiple chips, each including any suitable combination of central processing units and computer readable media for collectively performing the functions of the reading chip and the transaction chip described herein.

Although the user device 8202, which may be a POS terminal, and the reader device 8226 are shown as separate devices, in additional or alternative examples the user device 8202 and the reader device 8226 may be part of a single device, which may be a battery-operated device. In such examples, both the user device 8202 and the reader device 8226 modules may be associated with a single device. In some examples, the reader device 8226 may have a display integrated therewith, which may be in addition to (or as an alternative to) the display 8216 associated with the user device 8202.

The server 8204 may include one or more servers or other types of computing devices that may be embodied in any number of ways. For example, in the server example, the modules, other functional modules, and data may be implemented on a single server, a cluster of servers, a server farm or data center, a cloud-hosted computing service, a cloud-hosted storage service, etc., although other computer architectures may additionally or alternatively be used. In at least one example, the server 8204 may correspond to the server computing device 102 described above with reference to FIG. 1.

Furthermore, although the diagram depicts the modules and data of server 8204 as residing in a single location, these modules and data may alternatively be distributed in any manner across different computing devices and different locations. Thus, functionality may be implemented by one or more server computing devices, and the various functions described above may be distributed in various manners across the various computing devices. Multiple servers 8204 may be located together or separately, and may be organized, for example, as virtual servers, server banks, and/or server farms. The described functionality may be provided by a single merchant or business server, or may be provided by multiple different customer or business servers and/or services.

In the illustrated example, the server 8204 may include one or more processors 8228, one or more computer-readable media 8230, one or more I/O devices 8232, and one or more communication interfaces 8234. Each processor 8228 may be a single processing unit or several processing units and may include single or multiple computing units or multiple processing cores. The processor 8228 may be implemented as one or more microprocessors, microcomputers, microcontrollers, digital signal processors, central processing units, state machines, logic circuits, and/or any device that manipulates signals based on operational instructions. For example, the processor 8228 may be one or more hardware processors and/or logic circuits of any suitable type that are specifically programmed or configured to execute the algorithms and processes described herein. The processor 8228 may be configured to fetch and execute computer-readable instructions stored on the computer-readable media 8230, which may program the processor 8228 to perform the functions described herein.

The computer readable medium 8230 may include volatile and non-volatile memory and/or removable and non-removable media implemented in any type of technology for storing information such as computer readable instructions, data structures, program modules, or other data. Such computer readable media 8230 may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, optical storage, solid state storage, magnetic tape, magnetic disk storage, RAID storage systems, storage arrays, network attached storage, storage area networks, cloud storage, or any other medium that can be used to store the desired information and that can be accessed by a computing device. Depending on the configuration of the server 8204, the computer readable medium 8230 may be a type of computer readable storage medium and/or may be a tangible non-transitory medium to the extent that, when referred to, non-transitory computer readable media excludes media such as energy, carrier signals, electromagnetic waves, and signals themselves.

The computer-readable medium 8230 may be used to store any number of functional modules executable by the processor 8228. In many implementations, these functional modules are executable by the processor 8228 and comprise instructions or programs that, when executed, specifically configure one or more processors 8228 to perform the actions described above for the service provider 8112 and/or the payment processing service. The functional modules stored on the computer-readable medium 8230 may optionally include a merchant module 8236, a training module 8238, and one or more other modules and data 8240.

The merchant module 8236 may be configured to receive transaction data from a POS system, such as the POS system 8124 described above with reference to FIG. 81. The merchant module 8236 may send requests (e.g., authorization, capture, settlement, etc.) to a payment services server computing device to facilitate a POS transaction between a merchant and a customer. The merchant module 8236 may communicate success or failure of the POS transaction to the POS system. In some examples, the payment processing module 116 and the merchant module 8236 described above may perform the same or similar functions.

The training module 8238 may be configured to train the model using machine learning mechanisms. For example, the machine learning mechanisms may analyze the training data to train a data model that generates an output that may be a recommendation, a score, and/or another indicator. The machine learning mechanisms may include, but are not limited to, supervised learning algorithms (e.g., artificial neural networks, Bayesian statistics, support vector machines, decision trees, classifiers, k-nearest neighbors, etc.), unsupervised learning algorithms (e.g., artificial neural networks, association rule learning, hierarchical clustering, cluster analysis, etc.), semi-supervised learning algorithms, deep learning algorithms, etc.), statistical models, etc.
In at least one example, the machine-trained data model may be stored in a data store associated with the user device 8202 and/or the server 8204 for use once (e.g., at run-time) after the data model has been trained.

The one or more other modules and data 8240 may include programs, drivers, etc., as well as data used or generated by the functional modules. Additionally, the server(s) 8204 may include many other logical, program, and physical modules, of which the above-described are merely examples relevant to the description herein. In some instances, the other modules and data 8222 may comprise the same functionality and data as those described above with respect to the remote service 112.

One or more "modules" referenced herein may be implemented as more or fewer modules, and the functionality described for a module may be redistributed depending on the implementation details. The term "module" as used herein broadly refers to software stored on a non-transitory storage medium (e.g., volatile or non-volatile memory for a computing device), hardware, or firmware (or any combination thereof) module. A module typically functions such that it can generate useful data or other output using specified inputs. A module may or may not be self-contained. An application program (also called an "application") may include one or more modules, or a module may include one or more application programs that may be accessed over a network or downloaded onto a device as software (e.g., executable code that causes a device to perform an operation). An application program (also called an "application") may include one or more modules, or a module may include one or more application programs. In some examples, a module may include one or more application programming interfaces (APIs) to perform some or all of its functions (e.g., operations). Additionally and/or alternatively, the modules may be implemented as computer-readable instructions, various data structures, etc., via at least one central processing unit to configure a computing device described herein to execute instructions and perform operations described herein.

The computer-readable medium 8230 may further include an operating system 8242 for controlling and managing various functions of the server 8204.

The communication interface 8234 may include one or more interfaces and hardware modules for enabling communication with various other devices, such as through the network 8206 or directly. For example, the communication interface 8234 may enable communication over one or more networks 8206, which may include any type of network known in the art, such as, but not limited to, a local area network or a wide area network such as the Internet, a wireless network such as a cellular network, a local wireless network such as Wi-Fi and/or a near field communication such as Bluetooth, BLE, NFC, RFID, a wired network, or any other such network, or any combination thereof. Thus, the network(s) 8206 may include both wired and/or wireless communication technologies, including Bluetooth, BLE, Wi-Fi and cellular communication technologies, as well as wired or fiber optic technologies. The modules used for such communication may depend at least in part on the type of network, the environment selected, or both. Protocols for communicating over such networks are well known and will not be described in detail herein.

The server 8204 may further include various I/O devices 8232. Such I/O devices 8232 may include a display, various user interface controls (e.g., buttons, joystick, keyboard, mouse, touch screen, biometric or sensor input devices, etc.), audio speakers, connection ports, etc.

In at least one example, the system 8200 can include a data store 8244 that can be configured to store accessible, manageable, and updatable data. In some examples, the data store 8244 can be integrated with the user device 8202 and/or the server 8204. In other examples, as shown in FIG. 82, the data store 8244 can be located remotely from the server 8204 and can be accessible to the server 8204. The data store 8244 can comprise multiple databases and/or servers connected locally and/or remotely via the network 8206.

In at least one example, data store 8244 may store user profiles, which may include merchant profiles, customer profiles, carrier profiles, etc., as described above. Additionally, in at least one example, data store 8244 may store inventory and/or catalog databases. As described above, inventory may store data related to the quantity of each item that is available to the merchant. Additionally, catalogs may store data related to items that are available for acquisition to the merchant. Data store 8244 may store additional or alternative types of data (e.g., images 120, etc.), as described herein.

In at least one example, the data store 8244 can be associated with a brand asset database that can store information associated with brand assets associated with a merchant. The brand assets can be logos (e.g., text logos, graphic logos, favicons, etc.), fonts, colors, images, icons, button styles, effects, spacing, etc. used in connection with the merchant's brand. In some examples, the brand assets can be associated with multiple content items that can be applicable to different end users and/or intended audiences. For example, a brand asset corresponding to a logo can be associated with a logo presented to a customer and a logo presented to a merchant. In some examples, images captured via the techniques described herein can be added to the brand asset database and/or data associated with the brand asset database can be used to inform guidance as described herein.

Phrases such as "in some examples," "according to various examples," "in the illustrated example," "in one example," "in other examples various examples," "some examples," and the like generally refer to the particular feature, structure, or characteristic that follows the phrase and is included in at least one example of the present invention, and may be included in more than one example of the present invention. In addition, such phrases do not necessarily refer to the same or different examples.

If the specification states that a module or feature "may" or "can" be included or have the feature, that particular module or feature is not required to be included or have the feature.

Furthermore, the above description is directed to devices and applications related to payment technology. However, it will be understood that the technology can be extended to any device and application. Moreover, the techniques described herein can be configured to operate regardless of the type of payment object reader, POS terminal, web application, mobile application, POS topology, payment card, computer network, and environment.

Various techniques described herein refer to "requests" and/or "prompts." For purposes of this description, a "request" or "prompt" may be a data item transmitted between computing devices described herein with instructions to perform and/or take an action (e.g., a "request") and/or a recommendation to perform and/or take an action (e.g., a "prompt"). In some examples, a "request" and/or "prompt" may be associated with a graphical user interface (GUI) element to be presented via a user interface to communicate instructions to a user of the respective user device.

The various figures included herein are flow charts illustrating example methods incorporating the techniques described herein. The illustrated methods are described for convenience and ease of understanding. However, the illustrated methods are not limited to being performed using the modules described in the illustrated figures, and such modules are not limited to performing the methods illustrated herein.

Furthermore, the methods described above are illustrated as a collection of blocks in a logical flow graph, which represent sequences of operations that may be implemented in hardware, software, or a combination thereof. In the context of software, the blocks represent computer-executable instructions stored on one or more computer-readable storage media that, when executed by a processor, perform the recited operations. Generally, computer-executable instructions include routines, programs, objects, modules, data structures, etc. that perform particular functions or implement particular abstract data types. The order in which the operations are described is not intended to be construed as a limitation, and any number of the described blocks may be combined in any order and/or in parallel to perform a process. In some embodiments, one or more blocks of a process may be omitted entirely. Furthermore, the methods may be combined in whole or in part with each other or with other methods.

Section

A. A method

presenting a first user interface (UI) on a display of a back-of-house (BOH) device, the UI including details related to a first order originating from a first front-of-house (FOH) device and details related to a second order originating from a second FOH device;

Receiving a selection of the first order via a first UI;

Presenting a second UI including details about the first order in response to the selection and display of the BOH device;

Generating a request corresponding to the first order via the second UI;

Determining that the first FOH device has sent the first order;

Establishing a dedicated communications session with the first FOH device in response at least in part to the determining;

and transmitting an indication of the request via the dedicated communication session to the first FOH device.

B. The method according to A,

Generating the request includes generating a query regarding the first order,

Transmitting the instruction includes transmitting an instruction of the query to the first FOH device.

C. The method of claim A or B, wherein transmitting the indication of the request further includes transmitting an indication of a physical location within a facility where the BOH device is located.

D. A method according to any one of paragraphs A to C,

Presenting a third UI on the display of the BOH device that includes details about the second order;

receiving an indication via the third UI that an item in the second order is prepared;

Determining that the second FOH device has submitted the second order including the item;

Establishing a communication session with the second FOH device;

and transmitting an indication that the item is prepared to the second FOH device via the communication session with the second FOH device.

E. A method according to any one of paragraphs A to D,

Presenting a third UI on the display of the BOH device that includes details about the second order;

receiving an indication via the third UI that the user has begun preparing items in the second order;

Determining that the second FOH device has submitted the second order including the item;

Establishing a communication session with the second FOH device;

and transmitting an indication via and to the second FOH device that the user has begun preparing the item.

F. The method of any of paragraphs A-E, further comprising presenting a third UI on the display of the BOH device indicating (i) that a current or forecasted number of orders to be prepared using a first appliance is greater than a capacity associated with the first appliance, and (ii) a suggestion to activate a second appliance to prepare at least a portion of the current or forecasted number of orders.

G. The method according to paragraph F, wherein the third UI further includes an icon for launching the second appliance;

Receiving a selection of the icon via the third UI;

and causing activation of the second appliance at least in part in response to receiving the selection of the icon.

H. The method of clause G, further comprising presenting, on the display of the BOH device, a fourth UI indicating at least one of a number of orders to be prepared using the first appliance and a number of orders to be prepared using the second appliance in response to the selection of the icon.

I. A method implemented at least in part by a first device,

presenting on a display of the first device a first user interface (UI) including details related to a first order originating from a second device and details related to a second order originating from a third device;

Receiving a selection of the first order via the first UI;

Presenting a second UI including details about the first order in response to the selection and display of the first device;

Generating a request corresponding to the first order via the second UI;

Determining that the second device has sent the first order;

Establishing a communications session with the second device at least in part in response to the determining.

and transmitting an indication of the request via the communication session to the second device.

The method described in Section J.I,

Generating the request includes generating a query regarding the first order;

Transmitting the instruction includes transmitting an instruction for the query to the second device.

K. The method of clause I or J, wherein transmitting the indication of the request further includes transmitting an indication of a physical location within a facility where the first one is located.

A method according to any one of paragraphs L. I to K,

presenting a third UI on the display of the first device, the third UI including details about the second order;

receiving an indication via the third UI that an item in the second order is prepared;

Determining that the third device has submitted the second order including the item;

Establishing a communication session with the third device;

and transmitting an indication that the item is prepared to the third device via a communication session with the third device.

M. A method according to any one of paragraphs I to L,

presenting a third UI on the display of the first device, the third UI including details about the second order;

receiving an indication via the third UI that the user has begun preparing items in the second order;

Determining that the third device has submitted the second order including the item;

Establishing a communication session with the third device;

and transmitting an indication to the third device via the communication session with the third device that the user has begun preparing the item.

N. The method of clauses I-M, further comprising presenting a third UI on the display of the first device indicating (i) that a current or forecasted number of orders to be prepared using a first appliance is greater than a capacity associated with the first appliance, and (ii) a suggestion to activate a second appliance to prepare at least a portion of the current or forecasted number of orders.

O. The method according to paragraph N, wherein the third UI further includes an icon for launching the second appliance,

Receiving a selection of the icon via the third UI;

Causing activation of the second appliance at least in part in response to the receiving of the selection of the icon.

P. The method of paragraph O, further comprising presenting, on the display of the first appliance, a fourth UI indicating at least one of a number of orders to be prepared using the first appliance and a number of orders to be prepared using the second appliance in response to the selection of the icon.

Q. A first device,

Display and,

One or more processors,

One or more computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to:

presenting on a display of the first device a first user interface (UI) including details related to a first order originating from a second device and details related to a second order originating from a third device;

Receiving a selection of the first order via the first UI;

Presenting a second UI including details about the first order in response to the selection and display of the first device;

Generating a request corresponding to the first order via the second UI;

Determining that the second device has sent the first order;

Establishing a communications session with the second device, at least in part in response to the determining.

and transmitting an indication of the request via the communication session to the second device.

R. A first device as described in section Q,

Generating the request includes generating a query regarding the first order;

Transmitting the instruction includes transmitting an instruction for the query to the second device.

S. A first device according to clause Q or R,

The first device comprises a back-of-house (BOH) device and the second device comprises a front-of-house (FOH) device, or

Or at least one of the first device having the FOH device and the second device having the BOH device.

T. A first device as described in any one of clauses Q-S, wherein the operation is

presenting a third UI on the display of the first device, the third UI including details about the second order;

receiving an indication via the third UI that an item in the second order is prepared;

Determining that the third device has submitted the second order including the item;

Establishing a communication session with the third device;

and transmitting an indication that the item is prepared to the third device via a communication session with the third device.

A method including AA,

generating a layout of the facility, the layout indicating that a first order is associated with a first location within the facility and a second order is associated with a second location within the facility;

determining an action to be taken with respect to the first order based at least in part on the current status of the first order;

receiving an indication that a front-of-house (FOH) device is associated with the first order based on a context, the context being determined based on at least one of: (a) a location of the FOH device relative to the first location; (b) a ranking of the FOH device relative to other devices; and (c) historical data relating to the FOH device;

generating a user interface (UI) indicative of the action to be taken with respect to the first order, at least in part in response to receiving the instruction;

and transmitting the UI to the FOH device for presentation on a display of the FOH device.

AB. The method of claim AA, wherein the action includes at least one of requesting preparation of an item associated with the first order, providing an invoice for the first order, checking in one or more users at the first location for the first order, or providing an item or course to the one or more users.

AC. A method according to paragraph AA or AB, wherein the instruction includes a first instruction, the context includes the first context, and the UI comprises a first UI;

determining an action to be taken with respect to the second order based at least in part on a current status of the second order, the action to be taken with respect to the second order being different from the action to be taken with respect to the first order;

receiving a second indication that the FOH device is associated with the second order based on the second context, the second context being determined based on at least one of: (a) a location of the FOH device relative to the second location; (b) a ranking of the FOH device relative to other devices; and (c) historical data relating to the FOH device;

generating a second UI indicating the action to be taken with respect to the second order, at least in part in response to receiving the second instruction;

and transmitting the second UI to the FOH device for presentation on the display of the FOH device.

AD. A method according to any one of Sections AA-AC, wherein the UI includes a first UI,

receiving an indication that the FOH device is located near a threshold of a third location that is not currently associated with the order;

generating a second UI suggesting that a third order associated with the third location be opened, at least in part in response to receiving the indication that the FOH device is located within the threshold vicinity of the third location;

and transmitting the second UI to the FOH device for presentation on the display of the FOH device.

AE. A method according to any one of Sections AA-AD, wherein the layout includes a layout of tables in the facility including a first table at the first location, the UI includes a first UI, and the method includes:

Presenting on the display of the FOH device a second UI including (i) icons corresponding to items offered within the facility and (ii) a layout of the tables within the facility;

receiving user input on the display beginning at a location on the display corresponding to the icon and ending at a location on the display corresponding to the first table of the table layout;

and adding the item to the first order.

AF. The method of clause AE, further comprising setting a timer for a predetermined amount of time, the FOH device transmitting an indication of the item to a back-of-house (BOH) device upon expiration of the predetermined amount of time.

AG. A method according to clause AF, wherein the icon comprises a first icon, the item comprises a first item, and the user input comprises a first user input, the method comprising:

receiving a second user input on the display beginning at a location on the display corresponding to a second icon corresponding to a second item and ending at a location on the display corresponding to the first table of the table layout;

Adding the second item to the first order;

and resetting the timer for the predetermined amount of time.

AH. A method according to paragraph AF, wherein the user input includes a first user input, the method comprising:

receiving the second user input on the display at a location on the display that corresponds to the first table in the layout of the tables;

and presenting an icon on the display for submitting the first order including the item to the BOH device prior to the expiration of the predetermined amount of time.

AI. A method according to any one of Sections AA-AH, wherein the UI includes a first UI,

Determining items proposed to be added to the first order;

generating a second UI including a suggestion that the item be added to the first order, at least in part in response to receiving an indication that the FOH device is located within the threshold vicinity of the first location;

and transmitting the second UI to the FOH device for presentation on the display of the FOH device.

A method according to any one of Sections AA-AI of AJ., wherein the UI includes a first UI,

Determining an identifier associated with a user currently operating the FOH device;

Using the identifier to determine the user's previous behavior when operating the FOH device or another FOH device;

generating a second UI indicating one or more actions to be performed based at least in part on the previous behavior;

and transmitting the second UI to the FOH device for presentation on the display of the FOH device.

AK. A method according to any one of Sections AA-AJ, wherein the UI includes a first UI, and the method includes:

Determining at least one of that the FOH device is physically coupled to another device or is within a threshold vicinity of the other device;

Further comprising: presenting a second UI on the display of the FOH device and at least in part in response to at least one determination that the FOH device is physically coupled to the other device or within the threshold vicinity of the other device for performing one or more predetermined actions.

AL. system,

One or more processors,

One or more computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to:

generating a layout of the facility, the layout indicating that a first order is associated with a first location within the facility and a second order is associated with a second location within the facility;

determining an action to be taken with respect to the first order based at least in part on the current status of the first order;

receiving an indication that the first device is associated with the first order based on a context, the context being determined based on at least one of: (a) a location of the first device relative to the first location; (b) a ranking of the first device relative to other devices; and (c) historical data relating to the first device;

generating, at least in part, a user interface (UI) indicative of the action to be taken with respect to the first order in response to receiving the instruction;

and transmitting the UI to the first device for presentation on a display of the first device.

A system as described in Section AL of A.M., wherein the action includes at least one of requesting preparation of an item associated with the first order, providing an invoice for the first order, checking in one or more users at the first location for the first order, or providing an item or course to the one or more users.

AN. A system according to clause AL or AM, wherein the context includes a first context, the instruction includes a first instruction, the UI includes a first UI, and the action is

Determining an action to be taken with respect to the second order based at least in part on a current status of the second order, where the action to be taken with respect to the second order is different from the action to be taken with respect to the first order;

receiving a second indication that the first device is associated with the second order based on the second context, the second context being determined based on at least one of: (a) a location of the second device relative to the second location; (b) a ranking of the second device relative to other devices; and (c) historical data regarding the second device;

generating a second UI indicating the action to be taken with respect to the second order, at least in part in response to receiving the second instruction;

and transmitting the second UI to the first device for presentation on the display of the first device.

AO. A system according to any one of Sections AL-AN, wherein the UI includes a first UI, and the operation is

receiving an indication that the first device is located near a threshold of a third location that is not currently associated with the order;

generating a second UI suggesting that a third order associated with the third location be opened, at least in part in response to receiving the indication that the first device is located within the threshold vicinity of the third location;

and transmitting the UI to the first device for presentation on the display of the first device.

AP. A system according to any one of Sections AL-AO, wherein the layout includes a layout of tables in the facility including a first table at the first location, the UI includes a first UI, and the operation includes:

Presenting on the display of the first device a second UI including (i) icons corresponding to items offered in the facility and (ii) a layout of the tables in the facility;

receiving user input on the display beginning at a location on the display corresponding to the icon and ending at a location on the display corresponding to the first table of the table layout;

and adding the item to the first order.

AQ. The system described in clause AP, wherein the operations further comprise setting a timer for a predetermined amount of time, and the first device sending an indication of the item to a second device upon expiration of the predetermined amount of time.

AR. A system as described in Section AQ, wherein the icon comprises a first icon, the item comprises a first item, the user input comprises a first user input, and the action is

receiving a second user input on the display beginning at a location on the display corresponding to a second icon corresponding to a second item and ending at a location on the display corresponding to the first table of the table layout;

Adding the second item to the first order;

and resetting the timer for the predetermined amount of time.

AS. The system described in Section AQ, wherein the user input includes a first user input, and the operation is:

receiving the second user input on the display at a location on the display that corresponds to the first table in the layout of the tables;

and providing an icon on the display for submitting the first order including the item to the second device prior to the expiration of the predetermined amount of time.

AT. A system according to any one of Sections AO-AS, wherein the UI includes a first UI, and the operation is

Determining items proposed to be added to the first order;

generating a second UI including a suggestion that the item be added to the first order, at least in part in response to receiving an indication that the first device is located within the threshold vicinity of the first location;

and transmitting the second UI to the first device for presentation on the display of the first device.

CA. A method comprising:

generating image data of the merchant's physical premises using a camera on the merchant device;

(i) identifying at least a first surface and a second surface within the physical facility; and (ii) analyzing the image data to determine a first location associated with the first surface within the physical facility and a second location associated with the second surface within the physical facility;

presenting, on a display of the merchant device, (i) a first indication that the first surface has been identified, and (ii) the image data overlaid with a second indication that the second surface has been identified;

receiving a selection of the first surface, the selection indicating that the first surface represents an available seating area within the physical facility;

generating a layout of available seating areas within the physical facility based at least in part on the selection, the layout of the seating areas indicating that the first surface is an available seat at the first location;

and presenting the layout of the available seating areas on the display of the merchant device.

CB. The method of claim 41, further comprising receiving an indication of a number of available seats associated with the available seats at the first location, and wherein the presenting comprises presenting the layout together with the indication of the number of available seats associated with the available seats at the first location.

CC. The method of any one of clauses CA or CB, wherein generating the image data includes acquiring sensor data from one or more LiDAR sensors.

CD. A method as set forth in any of Sections CA-CC, wherein generating the image data includes generating the image data of the physical facility of the merchant using the camera of the merchant device as the merchant device moves through the physical facility of the merchant.

CE. A method according to any one of Sections CA-CD, comprising:

Analyzing the image data to determine a proposed layout of available seating areas within the merchant's establishment;

Presenting a proposed layout of the available seating areas on the display of the device;

generating an augmented reality object based on at least one characteristic of the physical facility;

Associating the augmented reality object with the image data;

and modifying the layout based on the augmented reality object.

CF. A method as set forth in any of Sections CA-CE, wherein analyzing the image data includes analyzing the image data using at least one trained machine learning model to identify at least the first surface and the second surface within the physical facility.

CG. A method as set forth in any of Sections CA-CF, wherein analyzing the image data includes using LIDAR on the merchant device to identify at least the first surface and the second surface within the physical premises.

CH. A method comprising:

Receiving data from devices in the environment;

(i) analyzing the data to identify a first surface within the environment, and (ii) a first location associated with the first surface within the environment;

generating a user interface (UI) showing the first surface;

Sending the UI to the device;

receiving from the device a selection of a first surface indicating that the first surface represents an available seating area within the environment;

generating a layout of available seating areas in the environment based at least in part on the selection, the layout of the seating areas indicating that the first surface is an available seat at the first location;

and transmitting the layout of the available seating areas to the device.

CI. The method of clause CH, further comprising receiving from the device an indication of a number of available seats associated with the available seats at the first location, the layout of the available seating area indicating a number of available seats associated with the available seats at the first location.

CJ. A method according to section CH or CI,

(i) analyzing the data to further identify a second surface within the environment, and (ii) a second location associated with the second surface within the environment;

and receiving a selection of a second surface indicating that the second surface is not an available seating area.

CK. The method of claim CH-CJ, wherein receiving the data includes receiving sensor data from the device, the sensor data being generated by the device as the device moves through the environment.

CL. A method according to clause CH-CK, wherein the UI includes a first UI, and the method comprises:

Analyzing the data to determine a proposed layout of available seating areas within the environment;

generating a second UI showing a proposed layout of the available seating areas within the environment;

and transmitting the second UI to the device.

CM. The method of clause CH-CL, wherein analyzing the data includes analyzing the data using at least one trained machine learning model to identify the first surface within the environment.

CN. The method of clause CH-CM, wherein analyzing the data includes using one or more LiDAR sensors to identify the first surface in the environment and distinguish the first surface from a second surface.

CO. system,

One or more processors,

One or more computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to:

Receiving data from devices in the environment;

(i) analyzing the data to identify a first surface within the environment, and (ii) a first location associated with the first surface within the environment;

generating a user interface (UI) showing the first surface;

Sending the UI to the device;

receiving from the device a selection of a first surface indicating that the first surface represents an available seating area within the environment;

generating a layout of available seating areas in the environment based at least in part on the selection, the layout of the seating areas indicating that the first surface is an available seat at the first location;

Performing operations including transmitting the layout of the available seating areas to the device.

CP. The system of claim CO, wherein the operations further include receiving from the device an indication of a number of available seats associated with the available seats at the first location, and the layout of the available seating area indicates a number of available seats associated with the available seats at the first location.

CQ. A system according to clause CO or CP, wherein the operation is:

(i) analyzing the data to further identify a second surface within the environment, and (ii) a second location associated with the second surface within the environment;

and receiving a selection of a second surface indicating that the second surface is not an available seating area.

CR. A system as described in any of Sections CO-CQ, wherein receiving the data includes receiving image data from the device, the image data being generated by the device as the device moves through the environment.

CS. A system according to any one of Sections CO-CR, wherein the UI includes a first UI, and the operation is:

Analyzing the data to determine a proposed layout of available seating areas within the environment;

generating a second UI showing a proposed layout of the available seating areas within the environment;

Further includes transmitting the second UI to the device.

CT. A system as described in any of Sections CO-CS, wherein analyzing the data includes at least one of: (i) analyzing the data using at least one trained machine learning model to identify the first surface within the environment; or (ii) using LIDAR to identify the first surface within the environment.

Claims (20)

generating a layout of tables within an establishment indicating that a first order is associated with a first table at a first location within the establishment and a second order is associated with a second table at a second location within the establishment;
determining a first action to take with respect to the first order based at least in part on a current status of the first order;
receiving an indication that a front-of-house (FOH) device is associated with the first order based at least in part on the context;
in response at least in part to receiving the instruction, and causing presentation on a display of the FOH device of a first user interface (UI) indicating the first action to be taken with respect to the first order;
determining whether to cause the display to switch from presenting the first UI to presenting a second UI including (i) icons corresponding to first items offered within the establishment and (ii) a layout of the tables within the establishment;
generating the second UI;
causing presentation of the second UI on the display;
receiving a user input on the display beginning at a location on the display corresponding to the icon and ending at a location on the display corresponding to the first table of the table layout, the user input causing the first item to be added to the first order;
generating a third UI in response to the user input causing the first item to be added to the first order, the third UI including a visual representation of a layout of the tables at the establishment and a timer proximate the first table;
causing presentation of the third UI on the display.
method. the first action includes at least one of: requesting preparation of an item associated with the first order; providing an invoice for the first order; checking in one or more users at the first location for the first order; or providing the item or course to the one or more users.
The method of claim 1 . the instructions include a first instruction and the context includes a first context;
determining an action to be taken with respect to the second order based at least in part on a current status of the second order, wherein the action to be taken with respect to the second order is different from the first action to be taken with respect to the first order;
receiving a second indication that the FOH device is associated with the second order based on another context ; and
generating a fourth UI indicating the action to be taken with respect to the second order, at least in part in response to receiving the second instruction;
causing presentation of the fourth UI on the display.
The method of claim 1 . determining that the FOH device is located near a threshold of a third location that is not currently associated with an order;
generating a fourth UI suggesting that a third order associated with the third location be opened, at least in part in response to receiving the indication that the FOH device is located within the threshold vicinity of the third location;
causing presentation of the fourth UI on the display.
The method of claim 1 . setting the visual indication of the timer for a predetermined amount of time, the FOH device transmitting the first item to a back-of-house (BOH) device upon expiration of the predetermined amount of time.
The method of claim 1 . the icon comprises a first icon , the user input comprises a first user input, and the method further comprises:
receiving a second user input on the display beginning at a location on the display corresponding to a second icon corresponding to a second item and ending at the location on the display corresponding to the first table of the tabular layout;
adding the second item to the first order;
resetting the visual indication of the timer for the predetermined amount of time.
The method according to claim 5 . The user input includes a first user input, the icon includes a first icon, and the method further includes:
receiving the second user input on the display at the location on the display corresponding to the first table of the table layout;
presenting a second icon on the display for transmitting the first order including the first item to the BOH device prior to expiration of the predetermined amount of time.
The method according to claim 6 . determining a second item to propose for addition to the first order ;
generating a fourth UI including a suggestion that the second item be added to the first order, in response at least in part to determining that the FOH device is located within a threshold vicinity of the first location within the facility ;
causing presentation of the fourth UI on the display.
The method of claim 1 . determining an identifier associated with a user currently operating the FOH device;
using said identifier to determine previous behavior of said user when operating said FOH device or another FOH device;
generating a fourth UI indicating one or more actions to be performed based at least in part on the previous behavior;
causing presentation of the fourth UI on the display.
The method of claim 1 . determining at least one of the FOH device being physically coupled to or within a threshold vicinity of another device;
and causing presentation of a fourth UI on the display and at least in part in response to determining at least one of the FOH device being physically coupled to the other device or being within the threshold proximity of the other device for performing one or more predetermined actions.
The method of claim 1 . one or more processors;
One or more computer-readable media storing computer-executable instructions that, when executed, cause the one or more processors to:
generating a layout of tables within an establishment indicating that a first order is associated with a first table at a first location within the establishment and a second order is associated with a second table at a second location within the establishment;
determining a first action to take with respect to the first order based at least in part on a current status of the first order;
receiving an indication that a first device is associated with the first order based at least in part on the context ;
in response at least in part to receiving the instruction and causing presentation on a display of the first device of a first user interface (UI) indicating the first action to be taken with respect to the first order;
determining whether to cause the display to switch from presenting the first UI to presenting a second UI including (i) icons corresponding to first items offered within the establishment and (ii) a layout of the tables within the establishment;
generating the second UI;
causing presentation of the second UI on the display;
receiving a user input on the display beginning at a location on the display corresponding to the icon and ending at a location on the display corresponding to the first table of the table layout, the user input causing the first item to be added to the first order;
generating a third UI in response to the user input causing the first item to be added to the first order, the third UI including a visual representation of a layout of the tables at the establishment and a timer proximate the first table;
causing presentation of the third UI on the display.
system. the first action includes at least one of: requesting preparation of an item associated with the first order, providing an invoice for the first order, checking in one or more users at the first location for the first order, or providing an item or course to the one or more users;
The system of claim 11 . The context includes a first context, the instructions include a first instruction , and the action includes:
determining an action to be taken with respect to the second order based at least in part on a current status of the second order, wherein the action to be taken with respect to the second order is different from the first action to be taken with respect to the first order;
receiving a second indication that the first device is associated with the second order based on another context ; and
generating a fourth UI indicating the action to be taken with respect to the second order, at least in part in response to receiving the second instruction;
causing presentation of the fourth UI on the display.
The system of claim 12 . The operation includes :
determining that the first device is located near a threshold of a third location that is not currently associated with an order;
generating a fourth UI suggesting that a third order associated with the third location be opened, at least in part in response to receiving the indication that the first device is located within the threshold vicinity of the third location;
causing presentation of the fourth UI on the display.
The system of claim 12 . The operations further include setting the visual indication of the timer for a predetermined amount of time, the first device transmitting an indication of the first item to a second device upon expiration of the predetermined amount of time.
The system of claim 12 . The icon comprises a first icon , the user input comprises a first user input, and the action comprises:
receiving a second user input on the display beginning at a location on the display corresponding to a second icon corresponding to a second item and ending at the location on the display corresponding to the first table of the tabular layout;
adding the second item to the first order;
resetting the visual indication of the timer for the predetermined amount of time.
The system of claim 15 . The user input comprises a first user input, the icon comprises a first icon, and the action comprises:
receiving a second user input on the display at the location on the display corresponding to the first table of the table layout;
presenting a second icon on the display for transmitting the first order including the first item to the second device prior to expiration of the predetermined amount of time.
The system of claim 15 . The operation includes :
determining a second item to propose for addition to the first order;
generating a fourth UI including a suggestion that the second item be added to the first order, at least in part in response to receiving the indication that the first device is located within a threshold vicinity of the first location within the facility ;
causing presentation of the fourth UI on the display.
The system of claim 12 . determining an identifier of the first item prior to the setting of the visual indication of the timer, the setting of the visual indication of the timer being based at least in part on the determining of the identifier of the first item.
The method according to claim 6. The context is determined based on at least one of: (a) a location of the FOH device relative to the first location of the first table within the facility; (b) a ranking of the FOH device relative to other devices; and (c) historical data regarding the FOH device.
The method of claim 1.

Images