i-DarkVec: Incremental Embeddings for Darknet Traffic Analysis

Figure 2 gives an overview of the traffic the darknet observes along the (i) service, (ii) space, and (iii) time dimensions. In detail, Figure 2(a) reports the Empirical Cumulative Distribution Function (ECDF) of the number of packets received by each port in one month.³ As already observed, all ports get some unsolicited packets, and most traffic is directed to specific ports, as detailed in the inset showing the top-14 ports—each easily linked to well-known services or vulnerabilities.⁴

Figure 2(b) showcases the activity of each sender over time. On the \(y\)-axis, each line represents a given sender IP address, sorted by the timestamp of the first appearance in the trace. The \(x\)-axis represents time. A dot is a packet received from a sender at a given time. In the first 10 days, we observe more than 220,000 senders sending about 1M packets. As expected [31], we witness a continuous growth of the number of senders over time. Some senders are persistently present (long horizontal lines); some senders appear sporadically, then disappear and reappear (horizontal segments); some senders are seldom visible (sparse dots).

To complete the overview, Figure 3(a) reports the ECDF of the total number of packets received from each sender. The large majority of senders hits the darknet with few packets—36% are seen just once in a month. These senders are likely victims of attacks with spoofed addresses—i.e., the backscatter phenomenon [37, 49]. Yet, there exist many senders that are quite active. We keep in our analysis only senders that send 10 or more packets to the darknet in the considered period. We call them active senders, for which we have enough evidence to perform a reliable analysis. These senders (20% of the total) are responsible for the majority of the darknet traffic.

At last, Figure 3(b) shows the count of distinct IP addresses seen over an increasing period of time. On the first day, we observe about 40,000 distinct senders. This figure quickly grows over time so after 30 days, we observe more than 500,000 unique senders. About 20% of them are active, i.e., after 30 days, we collect enough data to characterise 100,000 active senders.

One of the main difficulties in automating the analysis of darknet traffic is the lack of ground truth for evaluating the results. i-DarkVec’s aim is to learn meaningful representations that allow us to identify senders performing similar activities over time. We validate i-DarkVec using our domain knowledge to label senders that perform coordinated activities that can be observed in the darknet. In particular, we exploit two sources of data: (i) the presence of the widely known Mirai-like malware(s) fingerprint [26, 32] in packets; (ii) our knowledge about security search engines and research projects such as Shodan [20] and Sonar [17] that make publicly available the IP addresses they use.

As said, in our 30-day-long dataset, we observe about 100,000 unique active senders. The manual labelling of all of them is unfeasible. We thus focus on the most active senders that are present on the last day of our collection, which we use as a labelled dataset in our validation. Here, we observe 22,399 active senders in total. Among these senders, we identify nine ground truth (GT) classes, summarised in Table 2. We identify senders that are part of the Mirai-like botnet(s) with more than 7,300 hosts targeting a limited number of ports and services, i.e., Telnet (23/TCP and 2323/TCP) or ADB (5555/TCP). Next, we identify senders that are part of well-known projects performing Internet scans. The largest group includes 336 active senders of the Censys project [8] that target more than 11,000 unique destination ports. The smallest groups include 10 senders of the Engin-umich project [15] that perform scans focusing on DNS (port 53/UDP) only. About 2/3 of the active senders remain Unknown. These senders may belong to other classes or even be part of some of the known classes, which, however, we could not identify.

To motivate the need for advanced representations of the darknet traffic, we set up a simple classification task that directly leverages traffic-related features to classify senders reaching the darknet. We consider simple features, such as top-destination ports, numbers of packets, and others. Intuitively, one could argue that classifying senders using such features could already lead to the identification of coordinated groups without the need of projecting senders in a latent space with our proposed embeddings.

To illustrate and check this intuition, Figure 4 shows the fraction of daily packets sent by senders of each ground truth class to generic services. Here, we identify a service (\(y\)-axis) from the ports typically used by the service, e.g., we label traffic with destination ports \(\lbrace 25, 110, 143, 587, 993, 995\rbrace\) as “Mail,” while ports \(\lbrace 80, 443, 8080\rbrace\) identify HTTP. Fractions in the heatmap are normalised by column. The heatmap clearly shows that a naive port-based approach would not be able to classify all traffic. Some services can be characterised by their focus on a single service, such as the Engin-umich group, which generates only DNS traffic. Yet, other groups produce DNS traffic, too, and these overlaps are hard to solve based only on such features.

Indeed, to check whether it would be possible to group senders with a supervised approach, we design a classifier that uses as features the fraction of traffic each sender generates to top destination ports. We take the last day of traffic and label senders according to the 10 classes in Table 2, i.e., the 9 GT classes and the “Unknown” class. For each class, we extract the top-5 ports in terms of packets and merge the obtained ports as features in a single set. We then project each sender to the feature space by computing the percentage of traffic it sends to each selected port.⁵

We use a \(k\)-Nearest-Neighbor (\(k\)-NN) classifier to assign a label to each sender according to the labels of the majority of its \(k\) neighbours. We use the cosine distance to identify the \(k\) nearest neighbours. Using a Leave-One-Out approach, for each sender, we compare the \(k\)-NN classifier prediction with the original label and evaluate the accuracy of the classifier. We test values of \(k\in \lbrace 1,3,5,7,9,11\rbrace\), with best performance with \(k=7\). Table 3 details the results. Overall, the performance is quite unsatisfactory. Only for the most popular class (Mirai-like) the classifier can provide good results, but already for the Censys class—the second most popular one—the recall is very low.

We have also tested other classification models, with similar unsatisfactory results [33]. This negative result strengthens the intuition that an approach based on simplistic features is insufficient. We will show later that i-DarkVec approach, which exploits the temporal information present in the data, can provide a better representation of the darknet traffic and enable the clustering of senders on the obtained latent space.

To process the aggregate traffic received by a darknet, i-DarkVec leverages word embeddings. Ubiquitously used in modern NLP tasks, word embeddings leverage the frequency and —most important—the co-occurrence of words in sentences to project them in a high-dimensional latent space, associating a rich feature vector to each word. Although they were built exploiting the co-occurrence of words, these vectors end up encoding the semantics and syntactical properties of words. They are used as input to other ML algorithms to perform various NLP tasks. We build on the same idea to embed senders’ IP addresses into a latent space, thus mapping each sender’s IP address to a vector in the embedding space.

Unlike natural language where a large corpus with all known words is used to pre-build word embeddings in a single pass, i-DarkVec does not see all IP addresses at once, but incrementally discovers them as they hit the darknet (e.g., new senders that start sending traffic or new threats that emerge). In Reference [34], we introduced DarkVec, a single-iteration algorithm that builds a single, static, and immutable embedding space from all the data observed in a darknet. This is what is commonly done in natural language problems. The incremental version of DarkVec presented in this article (i-DarkVec) instead builds and continuously updates the embeddings as new traffic arrives. Figure 5 provides the high-level overview of both methods. In both cases, we first collect packets from a darknet (the top part of the plots). Given packets observed during a time period, we extract separate sequences of senders, considering the services they target. We identify senders by their source IP addresses and define services based on destination ports in the packets. Next, we use such sequences of senders per service to train a Word2Vec model. The original version of DarkVec (Figure 5(a)) uses all data to define a corpus that is then used to build the embeddings from scratch using the Word2Vec algorithm. The new i-DarkVec conversely receives as input new batches of traffic as time evolves. After building the corpus for each new batch, it updates the previous model to obtain the current embeddings (Figure 5(b)).

In both cases, we map the categorical IP addresses into a multidimensional space using traditional one-hot encoding to create independent input vectors. Sequences of senders thus become sequences of vectors that define the corpus of the Word2Vec input to create the embeddings. The resulting embeddings can be used to solve semi-supervised and unsupervised machine-learning tasks that exploit the correlations in the embeddings.

More in detail, Figure 5(a) illustrates the case of a unique batch used for training DarkVec, which aggregates the history of all the observation period \(T = \lbrace t_0, t_1, \ldots , t_n\rbrace\). To generate the embeddings, DarkVec (i) generates a corpus that spans the entire period \(T\) and (ii) learns a single embedding space with this entire dataset. Accordingly, generating an embedding for a new sample observed in \(t_{n+1}\) would require retraining from scratch using the entire dataset. Figure 5(b) illustrates instead the incremental strategy we adopt in i-DarkVec. In this case, at each timestep \(t_{i+1}\), we obtain a new batch of data from which we generate the corpus and incrementally update the embeddings starting from the weights computed at the \(t_i\) timestep.⁶ This latter strategy has two main advantages: (i) it significantly reduces the corpus size and thus speeds up the embeddings learning, and (ii) it lets the system weigh newer information automatically.

We outline the pseudo-code of the incremental algorithm of i-DarkVec in Algorithm 1. Details are explained in Sections 4.2 and 4.3.

To build the embeddings, we need to create ordered sequences of words (i.e., documents) as originally done in Word2Vec [41, 43]. Here, we aim at finding similarities among senders’ activity considering packets they send to a darknet. We then consider each source IP address associated with an incoming packet to be a word \(w\) and create documents as the sequence of IP addresses as they appear in time. We define as \(V\) the vocabulary containing all sender IP addresses targeting darknets.

We also leverage the definition of services to coarsely separate senders into different semantic groups. Note that the definition of the services is helpful to guide the training to consider the different aims of the senders. Given a destination port \(p\), with \(p \in P, P=\lbrace 0,\, \dots ,\, 65,535\rbrace\), we characterise a service \(s\) by the set of ports used by services \(s = \lbrace p_{1}, \ldots ,\,p_{n} \rbrace\). Hence, \(s\) results in a partition of \(P\), i.e., \(s \subseteq P\), and we call \(S\) the set of all the services.

For each observation timing \(t_i\) and corresponding sender sequence \(W(t_i)\) of senders \(V(t_i)\), we split its incoming packets into multiple sequences, one for each service. In detail, taking the sequence of sender IP addresses appearing in a given time interval, the sequence \(W^s(t_i)\) of the service \(s\) is the sequence of IP addresses sending packets to ports in \(s\) in observation time \(t_i\).

Finally, for each observation time \(t_i\), we build a corpus \(C(t_i)\) as the union of the per-service sequences \(W^s(t_i)\):We use the final corpus \(C(t_i)\) to train Word2Vec embeddings. We outline in the pseudo-code of Algorithm 1 the corpus definition in the function GenerateCorpus.

Figure 6 shows an example of the corpus definition. On the left, we have the sequence of packets in a time window. There are only two services, 22/TCP (red) and 445/TCP (blue). IP addresses of senders targeting those ports form sequences. Notice that the same sender IP address may appear in different services, as “10.0.0.1” in the example that appears both in the port 22 and 445 services.

As we will see later, the definition of services (GetServices in line 3 of Algorithm 1) is fundamental for the construction of good embeddings. Here, we consider three alternatives:

Given the final corpus \(C(t_i)\) for observation time \(t_i\), we build our embeddings. We employ the skip-gram model (see Section A.1), which provides excellent results in NLP when looking for embeddings that efficiently predict the context of a word in a sentence. Given a specific word in the middle of a sentence (the input word), look at the words nearby and pick one at random. The network is going to tell us the probability for every word in our vocabulary of being the “nearby word” that we chose.

Given a sequence \(W=[w_1,w_2,w_3,\ldots ,w_n]\) of senders and a context window of size \(c\), we define the context of each \(w_i\) as the sub-sequence of the \(c\) previous and \(c\) following words of \(w_i\), i.e., \([w_{i-c},\ldots , w_{i-1}, w_{i+1},\ldots ,w_{i+c}]\).

The central part of Figure 6 shows the skip-gram model for the sequences of the Port 22/TCP service. The blue circles represent the target sender’s IP address, while the yellow circles represent the context IP addresses to predict.

We build the embeddings using off-the-shelf Word2Vec where we consider also some negative sampling [43]. It consists of giving an additional set of words that are never found in any context window of the given word \(w_i\) (thus, “negative” words). As said, the training of Word2Vec consists in training a Neural Network (NN) that has the task of predicting the context window words for a given target word. The NN is formed by three fully connected layers, and the weights of the neurons that connect the input to the (hidden) second layer of \(e\) neurons build the embedding space that maps each input word into an \(e\)-dimensional vector, i.e., \(g(w_i)=u_i \in \mathbb {R}^e\). The dimension \(e\) of the projection is a parameter that impacts the quality of the embeddings. Notice that the embedder is a function \(g\) from the space of observed senders \(w\in V(t_i)\) to \(\mathbb {R}^e\). We call the set of all embeddings \(U(t_i)=\lbrace g(w_i), w_i \in V(t_i)\rbrace\).

We report the pseudo-code outline of the incremental embedding training in Algorithm 1. Refer to Appendix A.1 for more details.

In our case, Word2Vec leverages the co-occurrence of the sender IP addresses targeting the same port/service in a given time window. The resulting embeddings map those IP addresses that frequently appear in the same context window into the same region in the \(e\)-dimensional space, i.e., senders that perform similar patterns at a given time are mapped into a compact region.

For our experiments, we use the skip-gram-based Word2Vec Python implementation of Gensim [12], which we modify to support the incremental updates of the weights. We make all our source code and anonymised datasets available to the community to allow others to reproduce results.⁷

We consider the traffic observed in our darknet for a period of 5 days (first scenario) and 30 days (second scenario). For each scenario, we select the subset of active senders and create the embeddings for each case. We then run the same test done with the baseline classifier to check how senders in the ground truth are projected into the embedding space. Intuitively, good embeddings shall project IP addresses of the same ground truth class to compact regions in the latent space so a \(k\)-NN classifier can recover the correct label.

Following a Leave-One-Out approach, we consider each IP address \(w_i\) for which we have a label and that results active in the considered dataset. Using the cosine similarity, we measure the distance between \(w_i\) and other IP addresses, i.e., \(cosine(u_i,u_j)\) measures the similarity between the projection vectors \(u_i,u_j\) of \(w_i\) and \(w_j\). Then, we extract the \(k\)-Nearest Neighbours of \(w_i\) in the embedding space and use majority voting to assign the predicted class to \(w_i\). At last, we compare the predicted class against the \(w_i\) ground truth class. If the predicted and actual class match, then we have a correct prediction. By repeating the procedure for all labelled IP addresses, we compute the average accuracy, i.e., the probability that an IP address gets associated with the correct class. We consider only senders that belong to some ground truth class, i.e., GT1-GT9, skipping all IP addresses of the Unknown class, since we do not know if they eventually belong to any of the GT classes.

Given the large amounts of data to process, scalability is vital for practical implementations. Thus, we compare the corpus size and the training time required to build the embeddings. We consider DarkVec with best parameters as in Reference [34], i.e, with the domain knowledge service definition, context window \(c=25\), and embedding dimension \(e=50\). We consider i-DarkVec with the auto-defined services, context window \(c=5\), and embedding of \(e=200\) dimensions. We will discuss the choice of i-DarkVec parameters in Section 5.2.

We summarise results in Table 4. Consider first the 5-day dataset. With i-DarkVec, we can predict the correct class with a macro accuracy of 0.97, while with DarkVec single training approach, we can reach an accuracy of 0.93. IP2VEC reaches only 0.67. Here, both the incremental training and the flexible definition of services in i-DarkVec play a role, and we will discuss that in detail next. Considering scalability, because of the incremental training approach, i-DarkVec is the most scalable algorithm with a total training time of only 18 seconds—this is the time to process 5 batches and model updates of about 4M samples each. Conversely, DarkVec produces a corpus of over 17M sequences that require more than 14 minutes to complete the Word2Vec training. IP2VEC requires about four times more time to complete the training. The additional complexity is also due to the large usage of negative sampling suggested by authors. This negative sampling, together with a custom definition of context, increases the number of training samples to 38 M, increasing the training time.

DANTE does not scale, and after more than 10 days, we could not complete the training. This happens because DANTE generates around 7B sequences during sequence creation. Recall that DANTE associates each port to a word and generates a different sentence for each IP address. This approach results unfeasible with hundreds of thousands of sources, each generating independent sequences. The original paper presenting DANTE confirms the scalability problem. For the sake of completeness, we perform an additional test, reducing the dataset size by considering only one day of data. DANTE was finally able to complete the embeddings computations in about 2 minutes, resulting in 0.47 average accuracy. i-DarkVec completed the training in just 1 second, reaching 0.64 in accuracy. On the one hand, this result testifies to the scalability issues of DANTE; on the other hand, it shows the need to consider a big dataset to let the embeddings converge to a useful representation.

Consider now the 30-day dataset. In this case, the number of active IP addresses grows by a factor of 5, reaching about 100,000 IP addresses (cf. Figure 3(b)). More data allows us to build embeddings that cover more IP addresses. In fact, the number of active senders found on the last day and covered by the embeddings grows from 17,463 to 22,399. Restricting to those senders for which we have a label, the embeddings built on 5 days of traffic cover 82% of senders and, by construction, the coverage is 100% when using the 30-day dataset.

The increase in the data produces a sizeable increase in the number of sequences in the corpus. Thanks to the incremental approach, i-DarkVec processes 29 batches of data, each containing 1 day of traffic, each time updating the \(i\)th embeddings to get the \((i+1)\)-th embeddings. i-DarkVec training over the whole 30 days completes in 2 minutes, and we obtain 0.97 accuracy. When trained using a single batch of 30 days, we obtain almost the same accuracy (0.96) using the original DarkVec embeddings, but DarkVec takes 1.2 hours to complete the training. IP2VEC cannot complete the word sequence creation process after more than 10 hours, producing more than 200M training samples.

In sum, the incremental training approach and the simple word sequence creation of i-DarkVec increase scalability. Its flexible service definition and incremental approach result also in embeddings that can be used to train more precise classifiers than the original DarkVec.

We perform a sensitivity analysis on i-DarkVec hyper-parameters. We again focus on the classification task, verifying the accuracy following the same Leave-One-Out approach on the IP addresses that are active on the last day of the trace. Here, we use the macro F-score as the main performance indicator. It is a conservative metric that averages the per-class F-score to avoid the class unbalance problem.

We vary the number of days used to train the Word2Vec model and the number of neighbours \(k\), as well as we test the different service definition strategies. We then study strategies to reduce the corpus size, as this is a key parameter for the scalability of i-DarkVec. Finally, we study the impact of the embedding size \(e\) and the context window size \(c\). Given the number of parameters to test, we cannot perform a complete grid search. Instead, we follow a greedy optimisation by varying and choosing one parameter at a time. When not otherwise specified, we set \(e = 50\) and \(c = 25\), as they are the best choices for the parameters in Reference [34]—we later optimise these parameters for i-DarkVec, too. We train i-DarkVec on 30 batches, each corresponding to one day of traffic. At each batch, we update the embeddings by performing a 1-epoch iteration.

So far, we have reported only the F-score to summarise the results. The macro F-score, however, weights the performance independently on the class support, thus giving equal importance to all classes. In our case, we have a considerably unbalanced dataset with few classes with thousands of senders (e.g., Mirai-like) and others with just a handful of senders (e.g., Engin-umich). To appreciate in detail the per-class results, Table 6 summarises the Precision, Recall, and F-score for each GT class. At the bottom, we summarise the average performance using both the macro average (i.e., dividing each metric by the number of classes) and weighted average (i.e., weighing the metric by the support of each class). For the sake of completeness, we report results for all four service definitions, highlighting in red those results that are particularly unsatisfying (<0.5) and in bold the best performance in the F-score.

The Single Service embeddings result is particularly poor for this task. They work well for the Mirai-like botnet(s) but fail in most other classes. Being the largest class, Mirai dominates the weighted metrics. The AS and DKS help Word2Vec to obtain descriptive embeddings even for minority classes. IP addresses of the same class are projected into the same portion of the space so the majority of the \(k=5\) neighbours results of the same class.

Interestingly, the 10 Engin-umich senders (which target port 53 only) are projected into the same portion of the embedding space so the \(k\)-NN correctly classifies all of them. Notice that there are a lot of other senders that target port 53. Yet, the skip-gram model can perfectly capture the coordinated and very impulsive action of the 10 Engin-umich senders, which we depict in Figure 9(a).

The Stretchoid class results in the lowest performance metrics. Looking into its temporal pattern in Figure 9(b), we observe a very irregular pattern, with few packets from each sender arriving at irregular time intervals. Not having a similar context, Word2Vec likely projects these points at random in the embedding space.

Finally i-DarkVec allows us to assign labels to previously unlabeled senders by adopting a semi-supervised approach. Given the set of “Unknown” IP addresses classified as one GT class, we sort them by increasing the average distance to their \(k\)-NN and manually check if the assigned label could be correct. We stop when the average distance becomes higher than the maximum average distance among senders of the given GT class. With this simple process, we identify new senders performing scan patterns very similar to Shodan servers (confirmed by manual investigation), other senders being very likely part of the Censys network, and so on. While qualitative, this analysis lets us extend our knowledge about already known GT classes.

In the next section, we apply this rationale using an unsupervised approach to identify new classes sharing similar activities.

Given the good properties exhibited by \(k\)-NN for classification, we design a graph-based clustering for the unsupervised exploration of the embeddings. Given the space of all possible senders \(V\), we define the set of senders for which we have an embedding \(V^{\prime }\subseteq V\). Then, we build a directed graph \(G(V^{\prime },E)\), where we connect each vertex to its \(k^{\prime }\) nearest neighbours:We assign to each edge \((w_i,w_j)\) a weight equal to the cosine similarity \(cosine(u_i,u_j)\) between the embeddings of the vertices \(w_i\) and \(w_j\). Note that each edge \((w_i,w_j)\) is directed, since \(w_j\) can be among the \(k^{\prime }\) nearest neighbours of \(w_i\), but \(w_j\) can have \(k^{\prime }\) different neighbours.

Given the graph \(G\), we use the Louvain algorithm [25] to extract non-overlapping communities or clusters. The algorithm maximises the modularity score of clusters, where the modularity—in the range [-0.5,1]—quantifies the quality of an assignment of vertices to clusters. In a nutshell, the algorithm evaluates how much more densely connected the vertices within a cluster are when compared to how connected they would be in a random network with the same degree distribution. The Louvain algorithm has been successfully used for cluster detection in social networks [45] and even for darknet traffic analysis [52].

Among its advantages, the algorithm does not require a pre-defined number of clusters. Moreover, there exist several open-source and scalable implementations of the algorithm.

The only parameter for the graph-based clustering is \(k^{\prime }\), the number of neighbours to each vertex is connected to. Since we follow a completely unsupervised approach, the selection of \(k^{\prime }\) cannot be guided by our GT. As such, the \(k^{\prime }\) leading to good clusters may be different from the \(k=7\) used for supervised analysis.

We study the impact of \(k^{\prime }\) considering the 30-day dataset and the corpus obtained through the auto-defined services. We run i-DarkVec to build the embeddings, build the graph, and extract clusters for different \(k^{\prime }\). We show the number of clusters (left \(y\)-axis, solid red curve) and the modularity (right \(y\)-axis, blue dotted curve) in Figure 10. Intuitively, when \(k^{\prime }=1\), we connect each IP address to only the nearest point. This creates many disconnected components in the graph, resulting in thousands of tiny clusters. With \(k^{\prime }\gt 1\), disconnected components start to get connected, resulting in fewer clusters. With \(k^{\prime }=3\) (suggested by the elbow method [51]), we obtain 70 clusters with high modularity. Larger values of \(k^{\prime }\) have little impact, only slightly decreasing the overall modularity. As such, we use \(k^{\prime }=3\) in the results that follow.

Next, we compare the results of the Louvain clustering with traditional clustering algorithms. For this analysis, we use silhouette. The silhouette measures how similar a sample in a cluster is to the other samples in the same cluster (cohesion), compared to how dissimilar the sample is to samples in the other clusters (separation). It takes values in the [\(-\)1,1] range. Positive values reflect good cohesion, while negative values suggest possible wrong assignments.

We consider a simple \(k-\)Means algorithm and a hierarchical agglomerative algorithm [23]. In both cases, we select parameters (the number of clusters for \(k-\)Means and the maximum linkage threshold for the agglomerative clustering) by using the same elbow-based parameter selection.¹⁰

Figure 11 shows the average per-cluster silhouette for the three algorithms. All clusters tend to have a positive silhouette. The hierarchical agglomerative clustering detects only 3 clusters, while \(k-\)Means detects 12 clusters. Given we already know there are at least 10 groups of coordinated sources in this dataset (9 GT classes + 1 unknown), we would expect a higher number of clusters if the clustering algorithm is able to capture such behaviour from the embeddings. Manual inspection shows that these clusters are very large, with samples from the GT classes that are spread in multiple clusters. These results hint at poor clusters, which do not reflect groups of coordinated senders. Instead, the Louvain algorithm on 3-NN Graph offers detects 70 clusters, and 80% of them have positive silhouette values. In the following, we inspect these clusters to show how they represent compact and homogeneous groups of senders.

Figure 12 shows the average silhouette per cluster with respect to the cluster size on the x-axis. In general, we observe that the size of the clusters varies a lot (note the logarithmic x-scale). The small clusters tend to have a high silhouette, resulting in compact and well-defined groups. For instance, senders in the Shadowserver (green squares) or Censys (red points) classes clearly emerge as well-separated clusters. This confirms that senders belonging to known classes are well-separated from other senders in the embeddings space generated by i-DarkVec.

Interestingly, there are also some large clusters aggregating thousands of senders. For instance, three clusters aggregate a lot of senders in the Mirai class with a positive silhouette (highlighted with blue triangles). As several variants of the Mirai class exist, the identification of several clusters of senders having the same Mirai signature is a sign that i-DarkVec can detect sub-classes of the generic Mirai class. In the following, we analyse clusters with high silhouettes looking for insights and possibly previously unknown classes of scanners.

We summarise the results of our manual analysis in Table 7. Similarly to the steps used to build the GT, we rely on manual inspection, searching for explanations for the senders’ activity in each group. To find evidence of the type of activity they perform, we collect reverse DNS hostnames, consult the whois database, check public security repositories, and fire HTTP requests to senders’ IP addresses to check for “abuse” pages redacted by people running legitimate scanners. Here, the ability to work on groups of senders dramatically simplifies the analysis, showing how i-DarkVec eases the work of the security analyst. We make publicly available the clustering report¹¹ i-DarkVec automatically generates, providing a brief characterisation of each cluster.

All in all, i-DarkVec identifies (i) well-known Internet scan projects, including those listed in our ground-truth, some not reported for brevity, (ii) Internet scanners from security services, which were previously unknown to us, (iii) distributed scan events for which the observed patterns suggest coordination from botnets.

Word2Vec [41, 43] is an NLP technique based on artificial neural networks. It allows mapping words (tokens) of text sentences (corpora) into a latent space as a real-valued array (the embedding), such that words belonging to similar contexts have similar embeddings.

The core element of the Word2Vec model is the context. It is defined as the sequence of words surrounding the one for which the embedding must be generated. The number of words to consider in the context is specified by the context window size \(c\) (see Section 4.3). For example, by considering the sentence “Chicago is a great city,” if the word “a” is the target one and \(c = 2\), then the context for “a” is the list of the 2 previous and 2 following words of “a”:

To generate the embeddings, Word2Vec relies on two possible architectures: skip-grams and Continuous Bag Of Words (CBOW). Since we work with the skip-gram architecture, for the sake of simplicity, we omit the description of CBOW. By considering a corpus with \(N\) distinct words, the model aims at predicting the probability of finding each one of the \(N\) words within the context window of a given target word. In Figure A.1, we report an overview of the skip-gram architecture. Each word of the sentences is fed as input to the model through a one-hot-encoded input layer. The \(e\)-dimensional hidden layer links all the \(2c\) context words to the target one. After the model training, the embeddings are obtained from the weights matrix \(\mathbf {W} \in \mathbb {R}^{N\times e}\). Each of the \(i\in \lbrace 1, \dots , N\rbrace\) entries of \(\mathbf {W}\) is the embedding in \(\mathbb {R}^{1\times e}\) associated to the ith word.

In this section, we provide further details about the data in Table 7 by reporting more examples of the most notable clusters learned with i-DarkVec.

Here, we complete the description of the clusters uncovered by i-DarkVec that are formed by scanners that we could not explain based on online security databases (cf. Section 6.4.3).

Kubernetes exploiter: All the 29 unknown senders target port 10250/TCP with \(\gt\)24k packets (32.5% of the cluster traffic). The port is usually targeted to exploit Kubernetes vulnerabilities.

Sequential ports scanner: 28 unknown senders. All of them are active in the same period and target the same sequential ports 18081/TCP, 18082/TCP, 18083/TCP.

HTTP/Telnet scanner: More than 900 senders. The 80% of cluster traffic is directed to ports 81/TCP (99.6% of senders) and 23/TCP (94.6% of senders).

NSR Mixed scanner: Half of the 95 scanners belong to the Net System Research (NSR) project [16] extending the ground truth, half of them are unknown. All the senders target the same 416 ports with evident activity patterns.

Massive scanner: Ten unknown senders belonging to two contiguous /24 networks and generating 8k packets. They are active in the same period every 4/6 days targeting the same set of 40 ports.