1. Introduction: Evolution of Dashboard Architectures to Support for the Internet of Things
With the uprise of Internet of Things in industry, appliances and industrial machines are equipped with sensors for remote monitoring. The resulting amount of sensor data is overwhelming and hence difficult to interpret as-is. Therefore, dashboards are often used for aggregating the sensor data into visualizations that present a clear overview of a limited number of operational parameters (KPIs) in an instant.
Traditional dashboards, such as those in References [
1,
2,
3], consist of a set of visualizations chosen in advance and are designed for a specific set of data sources; for example, a dashboard composed of gauges displaying the current temperatures in the five ovens in an industrial bakery. Such fixed-structure dashboards make the sensor data inspection easier, but they also restrain the user to the use cases of the dashboard that were agreed upon during its development phase.
Appliances or industrial machines may be part of a so-called fleet [
4] of similar devices. Such fleets are often dynamic in nature—devices can be added to or removed from the fleet; sensors can be added to the devices, removed or replaced; and new types of sensors can be installed in the fleet. Such changes in the fleet must be registered in a dashboarding application, by modifying its code or its configuration, in order to keep the visualizations up to date. The introduction of new sensors in an industrial fleet may also require development of new aggregations and/or visualizations. As a result, traditional fixed-structure dashboarding applications require continuous investments in development and configuration effort such that they remain a practical monitoring tool for the evolving fleet. Changes to a dashboard require interventions by skilled people that know the internals of the fleet and the dashboarding software.
To overcome this issue, a next generation of dashboards was introduced, known as self-service dashboards or dynamic dashboards. They allow their end users to add, update and remove their personal visualizations in an intuitive way.
A good example of this approach is the Dynamic Dashboard Based on Activity and Reporting Traces (DDART) [
5], a dashboarding platform for people that learn a skill by working on assigned challenges (project-based learning, PBL). With the dashboard, the participants can, for example, investigate the time spent per week on writing a project report. To do so, they can drag and drop (and further configure) scales (e.g., a datetime scale) and parameters (e.g., time spent per day) to the visualization’s axes, an aggregation (e.g., summing hours per day to hours per week) and an appropriate visualization. The basic process for composing widgets on a dashboard is similar in all dynamic dashboards, such as QlikView [
6], Tableau [
7] and Microsoft PowerBI [
8].
The self-service visualization flexibility of these dashboarding platforms is delivered using an internally programmed myriad of if-then-else logic that expresses which visualizations fit with selected input data types. The flexibility comes at a cost—in some dashboarding platforms, such as DDART, the end user must have detailed knowledge about the data required for the visualization (e.g., know the function of each table in a database and indicate the data types of the columns to be visualized); in other platforms, an initial setup phase is required, in which such metadata is entered about all available data sources, after which end users can freely create visualizations.
Handcrafting visualizations however remains a tedious process that often draws the focus away from the problem at hand, since there often are a vast amount of data entities, filters, calculation functions and visualizations to select from. This may lead to choice overload for the end user, or to the creation of nonsensical or even non-functional visualizations. For example, on a dashboard monitoring air quality in offices, an end user might wish to visualize levels on a bar chart, not knowing that these levels have been saved with textual descriptions such as “high”, “medium” and “low”. A standard bar chart implementation will not be able to handle such input data.
To try and solve this issue, commercial dashboarding platforms adopted machine learning to recommend widgets automatically; for example, QlikSense [
9] introduced natural language processing (NLP)—the platform creates the desired visualization based on the description of it, entered by the end user in his own words.
Such machine learning-based visualization recommenders require the creation of large supervised data sets (examples of user input and corresponding visualizations that were correct, as reported by the user) in order to train their skill. In the end, their results on new, never-seen user input is not guaranteed to always be exactly what the user wanted. Also, for visualization queries to be well understood, such as "average office temperature per floor", the setup phase in which the data must be annotated with its meaning, is still required, unless when queries are simple and data is stored in nicely structured database tables with clear column names. This approach meets its limits when all sensor observations are stored in different data formats on a message bus topic, or when observations must be queried on the fly from a set of web services.
Since a metadata annotation setup phase is required anyway, it is possible to save this metadata in a format that machines can understand without resorting to machine learning, using the Resource Description Framework (RDF) [
10], a World Wide Web Consortium (W3C) standard for expressing knowledge in a machine-understandable format. The intelligent sensor-visualization matchmakings can then be suggested using semantic reasoners. After studying 23 dashboarding platforms, to the best of our knowledge, none of the commercial dashboards currently semantically annotate data sources, algorithms and visualizations with RDF. However, a lot of research has been done already on semantic annotation of data, some of which also focused on annotating and recommending visualizations. We will overview relevant research in this area in
Section 3.
Another challenge to dynamic dashboarding software is the vast amount of sensors on the Internet of Thing to monitor in real-time. This requires extensive engineering in terms of scalability for data storage (e.g., long-term queryable backend storage of all sensor measurements) and software (e.g., efficient data aggregation). Some dashboarding platforms fail to adequately meet these requirements, which has led to open-source efforts that specialize in it, such as Mainflux [
11].
Lastly, heterogeneousness of sensors deployed in industrial fleets also is a major challenge to dashboarding platforms. On the one hand, new types of sensors introduced in the fleet may require that a driver for yet another communication protocol and data exchange format must be added in the dashboarding software. Some dashboarding platforms, such as IBM Cognos [
12], try to automate data format handling for web services using web service description languages, such as OpenAPI (formerly Swagger) [
13] or JSONSchema [
14] for REST services [
15], and WSDL [
16] for SOAP services [
17]. On the other hand, all additions and removals of sensors and assets in the industrial fleet must be registered in the dashboarding platform, either manually on the user interface, programmatically by integration with an external, company-owned asset management system or by publishing changes to an API, for example, the ThingsBoard [
18] REST API. It is, however, clear that, for any of these fleet management solutions, a lot of human effort is still required. Therefore, automatically discovering the presence of devices, such as sensors, and potentially even metadata about the assets they monitor, will evidently be of great added value. This subject has been covered in many research studies.
Section 2 will discuss this in more detail.
In conclusion, a number of challenges for dynamic dashboards remain unsolved. Our literature overview showed that semantic annotation of sensors, data processing functionality and visualizations, combined with matchmaking rules, allow a semantic reasoner to suggest visualizations. We also argued that the annotation of expert knowledge may be more promising than using machine learning-based recommendations. In this paper, we therefore build a dynamic dashboarding approach using semantic Web of Things technologies.
We define the following goals to be tackled in this paper:
Build a functional dynamic dashboarding application
- (a)
Enable continuous monitoring of an industrial fleet
- (b)
Enable interactive creation of visualizations
Reduce human effort with automated industrial asset management
- (a)
Automate sensor discovery
- (b)
Automate asset discovery
Reduce human effort with visualization suggestion, using semantic reasoning
- (a)
Semantically annotate sensors, data processing functionality and visualizations
Annotate data types of sensor observations
Annotate functionality of visualizations and aggregations
Do not require more effort and technical expertise for metadata annotation
- (b)
Suggest visualizations for (aggregated) sensor data, using semantic reasoning
Avoid visualization choice overload and creation of nonsensical and non-functional widgets
Keep the required time for reasoning acceptable for user experience (<1 second).
Do not require technical expertise or detailed knowledge about the fleet, sensors, data processing services and visualizations.
Some first steps have been taken to deliver dynamic dashboards with Semantic Web of Things technologies, as discussed in this section (and in more detail in
Section 3), but to the best of our knowledge, we are the first to combine dynamic dashboarding, sensor-visualization suggestion using semantic reasoning and automated sensor discovery.
Figure 1 explains our approach on a high level, in five steps. First, a component is able to fetch or generate semantic annotations of the available Web Things, such as sensors, aggregations and visualizations. Then, a component discovers the presence and semantic annotations of these Web Things. This is achieved by crawling (walking over linked URLs) a Web Thing Model compliant API. This service discovery component is then able to send the dashboarding user interface, upon request, the list of discovered Web Things. With this, the user interface can present the sensors and aggregations that can be selected while building a new widget to be added to a dashboard. Once a user selected one or more sensors to visualize, and possibly an aggregation function to apply, a semantic reasoning component suggests appropriate visualizations for the selected sensor(s) and aggregation, given their semantic annotations, which were discovered earlier. Then, once the user selected one of the suggested visualization options, a widget will be created on the dashboard for the chosen visualization. Finally, the dashboard interface will start communicating with the Web Thing Model compliant API—it will fetch the visualization’s layout and code, load that into the created widget and start to continuously fetch the latest sensor observations, possibly process it using the aggregation service and eventually add the resulting data points to the visualization.
We note that one of our research goals is to evaluate the dynamic dashboard on an industrial fleet monitoring use case. We had the chance to do this on data produced by sensors on a train bogie, delivered by one of the partners within the imec.icon DyVerSIFy project.
In the remainder of this paper, we first discuss the three main drivers of our solution in detail—Web Thing discovery in
Section 2, semantic annotation of Web Things in
Section 3 and semantic reasoning about appropriate sensor visualizations in
Section 4. Combining these sections, we present the conceptual architecture of our dynamic dashboarding software in
Section 5. We present and discuss our results in
Section 6 and also list some future work.
Section 7 summarizes the added value of the presented solution.
2. Discovering Sensors and Services and Agreeing on Data Exchange Protocols
Dynamic dashboarding applications for real-time monitoring of fleets of industrial assets must discover a vast amount of sensors at run-time and integrate with a wide range of sensor communication protocols and data formats. In this section, we will discuss these challenges and work towards a solution.
The Industry 4.0 interconnection principle [
19] dictates the industry to interconnect sensors, appliances and industrial machines over the Internet of Things (IoT) and encourages to reduce the number of sensor intercommunication protocols (e.g., MQTT, CoAP) by only using standard Web protocols (e.g., HTTP over TCP/IP) and providing RESTful [
15] web APIs as a uniform interface [
20,
21,
22]. Such web-enabled sensors are often called Web Things, and the web of interconnected Web Things is then called the Web of Things. The encapsulation of a sensor in a RESTful web API means that, for example, a simple HTTP GET request suffices to retrieve one or more of the sensor’s observations. This reduces the heterogeneity of sensor protocols that an integrator application, such as a dashboard, is faced with.
Encapsulating sensors as RESTful web APIs does not solve the lack of widely accepted service description languages, which is a frequently recurring issue in emerging IoT applications. When discovering the sensors, either encapsulated as RESTful web API or not, it is unclear what API endpoints exist (e.g.,
/data or
/observations/latest), which requests must be sent to them, what the response will be and what the format of the content should be. If this information is documented, it is only documented for humans, therefore requiring human effort to integrate the API with a dashboard. Moreover, the so-called bootstrap problem [
20,
22,
23] remains as well—for an application to retrieve the service definition and to communicate with the service, it must first be told or find out what sensor web APIs are available and what their address is (e.g.,
http://company.com/sensors/1).
These issues are particularly dramatic in the search for a dynamic dashboarding platform.
2.1. Sensor Discovery
Discovering available sensors is key in avoiding that sensor addresses must be configured one by one, by the dashboard user at run time, or by the developer at design time. In a local network, the bootstrap (discovery) problem can be solved using network discovery methods like DNS-SD, mDNS, UPnP, or protocol suites like DLNA or zeroconf. For these methods, devices added to the network generally obtain their IP address automatically from a DHCP server and broadcast it to other devices present on the local network using one of these announcement protocols.
However, these local network discovery methods don’t scale up to the Internet [
23], nor do they work for dashboards that are deployed outside the network in which the fleet monitoring sensors are deployed (for example, the dashboarding software may run in the cloud but monitor a sensor network in a remote factory). One approach could be to securely expose the DHCP table [
23,
24] to the dashboard, however, for companies that maintain vast sensor networks, this means the software of many DHCP servers must be modified. Building a public search engine or private software component that crawls the Web for linked sensor APIs is also an option, but this will only work for RESTful sensor APIs that link to other APIs with REST’s building block HATEOAS [
23]. So the most pragmatic solution is to just manually register a “root” address or address inside each sensor network in the dynamic dashboarding software. Then, upon HTTP GET request to that IP address and a predetermined TCP port, a discovery service responds with a list of the available services by using a local network discovery method.
2.2. The Web Thing Model
This Internet-scale approach, so far, does not inform about the vast amount of heterogeneous communication protocols and data formats used by the discovered Web Things. To avoid the need to implement a plethora of such definition languages, the Web Thing Model [
22] was proposed, which is a W3C Submission formulating a contract that both RESTful sensor APIs and software clients can agree upon in order to make sensor services discoverable and to exchange data over the TCP, IP and HTTP protocols and with a agreed-upon JSON data format.
The Web Thing Model imposes three different integration patterns, namely direct connectivity, gateway-based connectivity, and cloud-based connectivity. Regardless of the chosen connectivity pattern, a set of RESTful API routes needs to be provided, each with their own goal.
Table 1 gives an overview of these Web API routes. A sensor API that complies with the Web Thing Model contract is also called an Extended Web Thing.
As
Table 1 shows, the API "contract" specifies web resources to find the sensors (Things) and sensor properties that are available. It also imposes a JSON data model to convey information about the sensor properties.
Note that a Web Thing API can expose sensor properties directly on the API’s root ({wt}/properties) when the direct connectivity integration pattern is applied (the Web Thing API represents a single sensor), or expose sensor properties on a Web Thing resource ({wt}/things/{thingId}/properties) when the other integration patterns are applied (the API serves as a proxy for one or more remote sensors).
Listing 1 shows a shortened JSON response, retrieved from the
{wt}/things resource (
http://localhost/web-thing-gateway/ in this particular case), that announces the availability of three Web Things on the Web Thing Model API—a latitude sensor, a concatenation service and a map visualization.
The standardized web resources on a Web Thing API allow software, such as dynamic dashboards, to construct a list of the provided sensors and the properties they observe, by crawling the available web resources on the API.
Annotating Web Things semantically, describing what their purpose is, allows a dynamic dashboarding application to also understand how to combine multiple Web Things. The Web Thing Model defines semantic extensions for this purpose, however proposes that the available web resources should have a HTTP Link header, referring to a JSON-LD context. We argue that content negotiation on the Web Thing’s URL also allows for semantically annotating the Web Thing, and thus provide a separate annotation in Turtle, using standard and custom ontologies, as will be described in
Section 3. We semantically annotate things and properties. A crawler can then fetch these annotations.
As will also be discussed in
Section 3, in the annotations of sensors, we also refer to the system that a sensor is part of, in terms of an industrial system hierarchy; for example an engine oil temperature sensor may be deployed in the engine of the left wing of an air plane. We assume that a web resource will also be provided for these systems, hosted by the Web Thing Model compliant Web API and semantically annotated, for example on a custom route
{wt}/systems/1. This URL can then be linked to from within the semantic annotation of the sensor.
2.3. Discovering Visualisations
In the dynamic dashboarding application, visualization components are also web resources provided under /things by a Web Thing Model compliant web API. However, no properties are made available through the Web Thing Model API for these visualization Web Things, since they do not observe anything and therefore do not generate data. Additionally, we added content negotiation on the visualization web resources, by using the proper Accept header during the request, to support retrieving (i) the JavaScript file to inject into HTML-based dashboarding applications, (ii) the visualization’s layout (CSS) file, and (iii) the semantic annotation of the component.
2.4. Advantages and Disadvantages
In summary, the main advantage with the Web Thing Model is that it facilitates dynamic dashboards with, on one hand, automatic sensor and semantic annotation discovery, and, on the other hand, unifying the dashboard’s access to Web Things such as sensors with a single protocol and data format.
Because of the introduction of the Web Thing Model, the responsibility of configuring the presence of a new Web Thing within a known network, annotating its data types and implementing protocol bindings and data format mappings is moved from within the dashboarding software to the Web Thing API.
This has the advantage that the end user of the dashboarding software must not know the technical details about the sensors in the fleet and must not be a semantics expert in order to visualize sensor data.
A drawback is that Web Thing registration, semantic annotation, protocol driver implementation and Web Thing Model API implementation must still be performed in the Web Thing API by a technical expert such that a dynamic dashboard can discover the Web Things. However, the Web Thing Model API is a light-weight specification and therefore does not require a lot of implementation effort. Communication protocol drivers are already implemented in dashboarding software, they can be copied to the Web Thing API. Web Thing registration on the Web Thing gateway can be automated using the local network discovery methods discussed in this section. Semantic annotation effort can also be reduced, see
Section 3 and
Section 6.
4. Matching Sensors and Visualizations
The previous two sections explained that sensors, aggregations and visualizations are all encapsulated as RESTful Web services that can be discovered and that their metadata can be fetched by sending a GET request (with a certain Accept header) to the URL of the service. Using this information, the dynamic dashboarding software collects knowledge about all available Web services, in order to combine them, and, this way, accomplish a visualization task for the user when he/she selects one or more sensors in the graphical interface. For example, one can intuitively see that a thermometer visualization only fits with temperature sensors.
This matchmaking between sensors, visualizations and aggregations is done by combining the gathered service metadata with internally known ontologies, reasoning theorems and logical rules, for example, “a column chart visualizes only quantitative data”. This metadata is expressed in semantic format and passed to a semantic reasoner, which derives new knowledge—the visualizations to suggest. In our application, we use the EYE Reasoner [
39,
40].
A first input to the reasoning process is the sensor property selection made by the user in the dashboarding user interface. Note that we decided to match available visualizations with a selection of sensor properties, not the sensors, nor the observations they make over time. The reason for that is twofold. The first argument is user-friendliness—a sensor can observe multiple properties, whereas a user may not want to visualize all of them on a single widget, or worse, the visualization may fail to display one or more of the properties linked to a sensor, because it was simply never built for that type of sensor property. The second argument is related to performance. An observed property will make a vast amount of observations over time. Semantifying these and reasoning on them would be nefast for the execution time. As a sensor property usually leads to observations of the same type all the time, and therefore results in the same reasoning conclusions every time, there is no need to execute the reasoning on observation level. In summary, matching visualizations are found when a sensor’s observable property produces a metric with identical characteristics (e.g., data type and metric type) as the one accepted by a visualization component. This required link in the metadata was also indicated in grey on
Figure 2.
4.1. Use Case—Monitoring the Position of a Train
Section 1 introduced a use case on real-time sensor data provided by a partner in the railway industry. In order to evaluate our research, we simulated the creation of four dashboard widgets that an end user would be interested in, when monitoring the fleet of trains. In these widget creation scenarios, the user respectively visualizes the following in real-time:
Wheel acceleration, observed by the SPM sensor on one train
Accelerations of a train carriage along three axes, observed by the IMU sensor
Average wheel acceleration across three trains, calculated at every incoming timestamp
Position of a train
We will demonstrate the semantic reasoning for visualization suggestion on use case 4 below. The complete files involved for this use case, and the others, can be found at
http://purl.org/dynamic-dashboard.
The collected data contains latitude and longitude sensor properties, observed by a GPS sensor on the train bogie. Since these are two different properties and the available map visualizations only accept pairs of latitude and longitude observations as input, a concatenation operation must be performed on all latitude and longitude observations to merge them into one single observation. As a result, the observations can be shown on a map of the train’s position or route.
The user’s selection of sensors and aggregation is expressed semantically as in Listing 6.
Listing 6. User selection: concatenation of two sensor properties—latitude and longitude. |
|
A query file, generated on the fly in this use case, instructs the semantic reasoner about the triples that are expected as result, if they can be derived, see Listing 7. This query file will instruct the reasoner to return the URL and name of visualizations that were found to support the concatenation aggregate as data input and were deemed to be “candidate visualizations”, out of which the user can make a final choice. Note that the serialization format of the query file, and some of the others that will be discussed, is actually N3 [
41,
42], because of its ability to express logic—for example, inferencing that, if a certain visualizations is proposed as a candidate for the new widget and if that visualization has a name, the implication should be that the name of the visualization is returned, as is the case in Listing 7.
Listing 7. The query file instructs the semantic reasoner to return triples containing the URL and name of the visualizations that support the concatenation input and were marked to be displayed as candidate visualizations to the user. |
|
In order for the reasoner to be able to answer the above query, also, the semantic annotation of the concatenation aggregate (see Listing 5) and latitude and longitude sensor properties (see Listing 2) are passed to the reasoner.
Additionally, the metrics taxonomy (see Listing 3 for the relevant part) and annotations of all available visualizations (Listing 4 lists a few) are taken into account.
A reasoning theory about RDFS subclassing is also passed to the semantic reasoner, such that it can derive that instances of specific visualization classes (e.g.,
dashb:Map) are actually typed with the parent classes (e.g.,
dashb:Visualization), see Listing 8. We note that this theorem, and many others, are included with the EYE reasoner documentation at
http://eulersharp.sourceforge.net/.
Listing 8. RDFS subclassing theorem used during the visualization suggestion task. |
|
Finally, two sets of logic rules are inputted to the semantic reasoner, one to support aggregations, the other to support visualization. Using all of the above, the reasoner derives new knowledge. Listing 9 presents logic rules that will add support for aggregations as well. The relevant part of the aggregation rules in the train bogie monitoring use case is the following. Since a concatenation service was found, and was presented a list of sources (sensor properties), referring to a metric describing latitude and longitude and their accepted data type, it is concluded that the concatenation service actually produces a list of latitude and longitude metrics.
Listing 9. Logic rule involved in the reasoning process to support aggregations on lists of data sources. |
|
Then, the semantic reasoner uses inference rules for data visualization. In a first step, the reasoner essentially filters the visualizations on their ability to process historical data or real-time data. The widget requirements stated that observations would be provided by the Web Thing per time instant, not in batches (see Listing 6), therefore, the reasoner concludes that the time instants are supported by all RealtimeDataVisualizations, as shown in Listing 10.
Listing 10. Logic rule informing the reasoner that only visualizations that display data in real-time can visualize the user’s input. |
|
Given that the visualization accepts a list of data inputs and the aggregation service produces a lists of data outputs, a set of rules checks the data inputs in both lists one by one and if they match, the data input is marked as partially supported. If all data inputs were partially supported by the visualization component, the component states that it fully supports the list of data inputs. Listing 11 shows this approach.
Listing 11. Inference rules for deciding if a visualization component supports a data source, a concatenation service in this use case, that provides a list of data outputs. |
|
Finally, the semantic reasoner decides that a visualization is an appropriate suggestion if the visualization’s components support a number of data sources (the concatenation in this use case), less than a configured maximum and greater than or equal to a configured minimum amount. The logic rules involved in this can be found in Listing 12.
As a result, the semantic reasoner outputs two map visualizations suggestions—one that can show positions and one that can show routes, see Listing 13. Note that the reasoner was aware of many visualizations (see Listing 4) and will now only suggest two of them to the user, which really aids to prevent creation of nonsensical widgets on a dashboard.
Listing 12. Logic rules used for suggesting aggregations. |
|
Listing 13. Output of the semantic reasoner, stating that two visualizations can display the latitude and longitude sensor input, selected by the user—one that displays subsequent positions as markers on a map, and one that displays routes on a map. |
|
With the deduced visualization suggestions, the user can finish building a new widget for the dashboard. The next section will detail the architecture of our dynamic dashboarding application. After that, we will discuss the results achieved with this application.
5. Design of the Dynamic Dashboarding Application
In the previous sections, we presented the discovery of Web Things such as sensors, aggregations and visualizations, using the Web Thing Model. We also demonstrated how these Web Things can be semantically annotated, such that a semantic reasoner can suggest interesting visualizations, when given a selection of sensor(s) and possibly an aggregation service. Here, we will explain how these three drivers are combined into a dynamic dashboarding application.
Figure 3 shows the conceptual architecture of our dynamic dashboarding application. It is a more detailed overview than the workflow that was described in the introduction section with
Figure 1. The three main promises of the application to accomplish dynamic dashboarding, namely service discovery, semantic annotations and semantic reasoning, are drawn in the center of the figure. These promises will be fulfilled by a collaboration of the user interface (GUI), Broker and Web Thing gateway components. The goal of these main components and their interaction with the components around them will be detailed below.
5.1. Web Thing Gateway
A Web Thing gateway is used as a proxy for the available data services (sensors), aggregation services and visualizations for sensor data display, as shown on the right of
Figure 3.
The Web Thing gateway essentially offers web resources for each of the available Web Things – sensors, aggregation services and visualizations – through a Web API that is compliant with the Web Thing Model that was discussed in
Section 2. This approach makes provided Web Things discoverable, a task that the Broker will manage, with its internal service discovery component.
Our Web Thing gateway implementation supports all three integration patterns (direct, gateway-based and cloud-based connectivity, see
Section 2.2) and automatically makes web resources available on the Web Things API, according with the guidelines of the Web Thing Model, for each of the configured Web Things.
A Web Thing gateway also provides, using content negotiation, semantic annotations about the available Web Things on their respective URL. These semantic annotations express the abilities of the provided Web Things, see
Section 3.
Because a Web Thing gateway makes sensors, aggregations and visualizations findable and provides the semantic annotations of their abilities in a machine-readable format, the dashboard’s broker component (or any other application that can interpret semantic annotations) can detect the available Web Things and understand how to combine them, for example, setting up a chain of a sensor data source, an aggregation service and finally a visualization.
The end user of the dashboarding application can build dashboards using the knowledge that the Broker gathered during its service discovery. Once the user has added a widget to a dashboard, the dashboard will start communicating directly to the Web Thing gateways involved, such that the chain of required services can be continuously used—first for fetching the latest observation of one or more sensors, then possibly passing the observations to an aggregation service, and loading the visualization into the dashboarding interface and presenting it the processed sensor observations.
Noteworthy to mention is that most of these Web Thing gateways will make use of a cache and/or long-term storage, as drawn in the upper right corner of
Figure 3, in order to reduce heavy request traffic to certain Web Things.
The available web resources on a Web Thing gateway can be extended at any moment. The presence and annotations of Web Things must be configured in the Web Thing gateways, or the gateways can discover the available Web Things and their semantic annotations automatically (i.e., decentralized approach), when they reside in the same local network as the gateway (as discussed in
Section 2). In an alternative approach, the gateway can just read from a central data publishing repository, for example, a publish/subscribe message bus such as Apache Kafka, which all Web Things write to and read from (i.e., centralized approach).
In the case of fleet monitoring, often many services listen and write data and do this at high frequency. This creates large data volumes at high velocity, two motivators that typically cause scalability problems in big data systems. In such cases, the decentralized approach (service discovery and communication, through a Web Thing gateway) may not meet expectations. The centralized approach however will, using technology that scales, better guarantee a timely delivery of sensor observations to the sensor gateway and thus also in the dashboarding GUI.
For scaling reasons, we used a centralized approach where the aggregation services expect streaming observations on a Kafka message bus, as indicated on in the bottom left corner of
Figure 3. Note that only our aggregation services use the centralized approach in our architecture at the moment, since the aggregation operation is costly on large volumes of high-velocity sensor data. Discovery of the aggregation services is done with the Web Thing’s API though.
5.2. Broker
The broker is the core of the dynamic dashboarding application. It provides a REST API that covers three functionalities, as indicated on
Figure 3.
Firstly, the broker discovers available sensor services, aggregation services and visualization components. The only requirement is that the base URL of a Web Thing gateway is added in the graphical user interface (GUI). The broker then finds the available Web Things by crawling (walking over) all URLs exposed by the Web Thing gateway conform the Web Thing Model, as discussed in
Section 2. During this process, the broker gathers the semantic annotations of all found Web Things (as discussed in
Section 3) and updates its knowledge base (semantic database, drawn in the center of
Figure 3) with it.
Secondly, the broker also reasons, upon request of a user, about what visualizations are interesting matches with the available sensor services. To do so, it reads semantic annotations about the discovered Web Things from its knowledge base and uses, as drawn on
Figure 3, EYE Server as the semantic reasoner that must complete the reasoning process, which was described in
Section 4.
Finally, through its REST API, the Broker also allows for persisting and serving the state of the dashboards of all users (e.g., created widgets) that users generate when using the dynamic dashboarding application’s user interface. This state is preserved in a database, displayed on the bottom of
Figure 3.
5.3. Dashboarding GUI
The dashboarding graphical user interface (GUI) component allows a user to build and view a near real-time summary of the operational status of a fleet of appliances or industrial assets. The dashboarding GUI is also called the dashboarding interface; or client, when discussing it in relation to the broker component.
With a pop-up meant for creating widgets, the user can build a dashboard interactively, widget by widget, by selecting sensors and sensor properties to be displayed in the new widget. The dashboarding software will then present visualization options that the broker has reasoned about. Finally, the user can select one of the visualization suggestions and the widget will be added to the dashboard. All data in the pop-up and its drop down menus is automatically filled as a result of crawling and reasoning, so nothing of this GUI is hard-coded nor preconfigured.
During this process, the GUI consecutively makes use of all three functionalities provided by the broker’s API, as described in
Section 5.2. Firstly, the GUI retrieves data about the sensors that the broker discovered. Secondly, when the user has selected a sensor and a property observed by the sensor, the GUI fetches several sensor visualization suggestions, made by the broker’s sensor-visualization reasoning component. And lastly, once the user has confirmed all visualization choices for the widget to be created, the parameters are again sent to the broker, which then persists the changes to the dashboard composition in its dashboarding database.
Section 6 will show screenshots of the pop-up that allows a user to create dashboard widgets, and a dashboard that has been constructed with it.
7. Conclusions
Dashboards present and communicate the condition of monitored environments to supervising experts. In the case of monitored industrial fleets, the often vast and rapidly evolving fleet motivates the choice for dynamic dashboards—they allow users to freely and interactively create visualizations. The proposed platform enables dynamic data visualization. State-of-the-art dynamic dashboarding platforms however require a lot of human effort to manage changes to a fleet and a plethora of available visualization types may lead to choice overload for the user or to the creation of nonsensical or non-functional visualizations.
In this paper, we presented a dynamic dashboarding platform for continuous monitoring of industrial fleets that tackled both aforementioned problems, by adopting Semantic Web of Things technologies.
Sensors, linked assets, data processing services such as aggregations and visualizations are discovered automatically, because they are encapsulated as web resources (Web Things), exposed through a REST API that is compliant with the Web Thing Model. Because of the imposed standard API resources, the dynamic dashboard can crawl the API to discover the resources it exposes, once a user registers the address of he API’s root resource in the dashboard’s user interface. Manually registering sensor changes in the dashboard or another system (which publishes changes to a sensor management API) is no longer required. However, sensor protocol and data format drivers and the Web Thing Model specifications must be implemented in the Web Thing API, but we argued that the extra work nicely trades off with the elimination of continuously registering sensor changes manually for large industrial fleets.
Also, semantic annotations are provided on the Web Thing API about the data types that sensor properties produce and the data types that aggregations and visualizations support. Annotations were made with a data types taxonomy, the OM-2 ontology, a dashboarding ontology and N3 rules. Semantic reasoning about these annotations allows the platform to suggest appropriate visualizations for a selection of sensor properties and aggregations. As a result, the large set of visualization options is reduced and construction of nonsensical dashboard widgets is avoided. In contrast with current commercial dashboarding platforms, visualizations and aggregations must also be annotated, which requires slightly more human effort and technical skill. We however suggested two ideas to overcome this—a user interface that allows users to interactively create semantic annotations without knowing semantic technologies, and annotation by example (annotating new sensors automatically when similar sensors with annotations already exist). We demonstrated that the suggested visualizations make sense and reduce the tediousness in handcrafting dashboard widgets, that reasoning times were below 1 second and that the end user of the dashboarding interface does not need to have technical expertise or detailed knowledge about the sensor data.
All files involved in the semantic reasoning for visualization suggestion are available online at
http://purl.org/dynamic-dashboard, including the dashboarding ontology.
We used the proposed dynamic dashboarding platform on two projects with industrial partners and showcased it on a train monitoring use case in this paper.
To the best of our knowledge, we are the first to present a dynamic dashboarding platform that combines automated Web Thing discovery and semantic reasoning for visualization suggestion. Such a dashboarding platform proves to be a very powerful monitoring tool for complex, hard to access and/or critical environments.