Privacy preserving existence recognition and construction of hypertree agent organization

Decentralized probabilistic reasoning, constraint reasoning, and decision theoretic reasoning are some essential tasks of cooperative multiagent systems. Several frameworks for these tasks organize agents into a junction tree (JT). We show that existing techniques for JT existence recognition and construction leak information on private variables, shared variables, agent identities and adjacency, that can potentially be protected. We present a scheme to quantify these privacy losses. We develop two novel algorithms for JT existence recognition and for JT construction when existing, that provide strong guarantee of agent privacy. Our experimental comparison shows that the proposed algorithms out-perform existing techniques, one of them having the lowest privacy loss and the other having no privacy loss, while being more efficient than most alternatives.

1. WebWeavr is a Java-based toolkit for graphical models and is available at http://www.socs.uoguelph.ca/~yxiang/.

We thank anonymous reviewers for their very helpful comments. Financial support through Discovery Grant from NSERC, Canada is acknowledged.

Authors and Affiliations

1. School of Computer Science, University of Guelph, Guelph, Canada

   Yang Xiang & Kamala Srinivasan

Corresponding author

Correspondence to Yang Xiang.

This article significantly extends Xiang and Srinivasan [41, 42] as part of the Canadian AI 2013 Proceedings.

Appendix: Proofs

Proof of Proposition 1

We show that running intersection holds in \(T'\). Let \(C'\) and \(Q'\) be non-adjacent clusters in \(T'\) such that \(C' \cap Q' \not = \emptyset \), and \(X'\) be a cluster on the path between \(C'\) and \(Q'\). Let \(C\), \(Q\), \(X\) be clusters in \(T\) corresponding to \(C'\), \(Q'\), \(X'\), respectively. We show that \(C' \cap Q'\) is contained in \(X'\), namely, \(C' \cap Q' \subseteq X'\).

Since \(T\) is a JT, \(C \cap Q\) is contained in \(X\), i.e., \(C \cap Q \subseteq X\). Because \(C\) and \(Q\) are boundaries (made of shared variables), it follows \(C' \cap Q' = C \cap Q\). That is, \(C' \cap Q' \subseteq X\). Since \(X\) is a boundary, we have \(X' = X \cup Y'\), where \(Y'\) is the set of private variables in subenvironment \(X'\). Hence, \(X \subseteq X'\). From \(C' \cap Q' \subseteq X\) and \(X \subseteq X'\), it follows that \(C' \cap Q' \subseteq X'\). \(\square \)

Proof of Proposition 3

By assumption, the transmission of each message takes at most one time unit. Combining Eqs. (7) and (9) and substituting \(t_4\) by \(t\), the result follows. \(\square \)

Proof of Proposition 5

Suppose that no JT exists with boundaries in \(W\) as clusters, but a JT \(T'\) exists with subenvironments in \(\Omega \) as clusters. Then for every pair of nonadjacent clusters in \(T'\) such that \(C' \cap Q' \not = \emptyset \), and a cluster \(X'\) on the path between \(C'\) and \(Q'\), it holds that \(C' \cap Q' \subseteq X'\).

Let \(T\) be a cluster tree with boundaries in \(W\) as clusters such that it is isomorphic to \(T'\) with each boundary mapped to the corresponding subenvironment. Let \(C\), \(Q\), \(X\) be clusters in \(T\) corresponding to \(C'\), \(Q'\), \(X'\), respectively. Since \(C\) and \(Q\) are boundaries, \(C \cap Q = C' \cap Q' \subseteq X' = X \cup Y'\), where \(Y'\) is the set of private variables in subenvironment \(X'\). From \(C \cap Q \subseteq X \cup Y'\) and \(C \cap Q \cap Y' = \emptyset \), we obtain \(C \cap Q \subseteq X\). That is, running intersection holds in \(T\) and \(T\) is a JT: a contradiction to the assumption. \(\square \)

Proof of Lemma 1

From subcondition 1, a JT \(T\) exists and no two clusters in \(T\) are comparable. We prove the claim by contradiction. Suppose there exists a cluster \(Q\) in \(T\) such that for every boundary \(W_i \in W\), \(Q \not = W_i\).

From subcondition 2, there exists a boundary \(W_i\) such that \(Q \subseteq W_i\). Since \(Q \not = W_i\), it follows that \(Q \subset W_i\). Because \(BG\) is the boundary graph, \(W_i\) is complete in \(BG\). Therefore, there exists a clique \(C_i\) in \(BG\) such that \(W_i \subseteq C_i\). Since clusters of \(T\) are cliques in \(BG\), \(C_i\) must be a cluster in \(T\). From \(Q \subset W_i\) and \(W_i \subseteq C_i\), we have \(Q \subset C_i\). That is, \(T\) contains two comparable clusters: a contradiction. Hence, every cluster in \(T\) is a boundary. \(\square \)

Proof of Theorem 1

[Necessary Condition] Suppose a JT \(H\) exists whose clusters are subenvironments. Extending Proposition 1, if private variables are removed from each cluster, the resultant cluster tree \(T\) is still a JT, whose set of clusters is \(W\). Since \(T\) is a JT, the boundary graph \(BG\) is chordal and subcondition 1 holds. Since each clique of \(BG\) is a cluster in \(T\) and each cluster of \(T\) is a boundary, subcondition 2 follows.

[Sufficient Condition] Suppose both subconditions hold. We prove by construction. Since \(BG\) is chordal, a JT \(T\) exists whose clusters are cliques of \(BG\). By Lemma 1, every cluster in \(T\) is a boundary. Hence, for every cluster \(C\) in \(T\) such that \(C = W_i\) for some \(W_i \in W\), we associate \(C\) with agent \(A_i\).

It is possible that not every agent has been associated with a cluster in \(T\) yet. In that case, consider such an agent \(A_i\) whose boundary is \(W_i\). Since \(W_i\) is complete in \(BG\), there exists a cluster \(C\) in \(T\) such that \(W_i \subseteq C\). Add a new cluster \(W_i\) to \(T\), making it adjacent to cluster \(C\) only, and associate the new cluster with \(A_i\). Repeat this for each remaining agent, until each agent is associated with a cluster in \(T\).

Next, for each agent, add its private variables to its associated cluster in \(T\). The resultant \(T\) is a JT agent organization with each cluster being a subenvironment. \(\square \)

Proof of Theorem 2

Based on Proposition 1, it suffices to prove the theorem relative to a boundary based JT.

[Necessary Condition] Suppose a boundary based JT \(T\) exists whose clusters are boundaries in \(W\). We show that boundaries in \(W\) can be eliminated iteratively using \(T\) as a reference structure.

Since \(T\) is a tree, there exists a leaf cluster \(W_i\). Let \(W_j\) be the adjacent cluster of \(W_i\). Since \(T\) is a JT, for every \(W_k\) such that \(W_k \not = W_i\) and \(W_k \not = W_j\), we have \(W_i \cap W_k \subseteq W_j\). Hence, \(W_i \subseteq W_j\) and \(W_i\) can be eliminated from \(W\) relative to \(W_j\) to obtain \(W'_j\) and the reduced boundary set \(W'\). Cluster \(W_i\) can also be removed from \(T\), with cluster \(W_j\) replaced by \(W'_j\) and resultant cluster tree denoted by \(T'\).

Now the set of clusters of \(T'\) is \(W'\). Since \(W'_j\) differs from \(W_j\) only in terms of variables that \(W_j\) shares uniquely with \(W_i\), \(T'\) is still a JT. Hence, the above operation can be applied repeatedly, until two boundaries are left. Since the reduced \(W'\) is a well-defined boundary set, the two boundaries must be identical and the last elimination results in \(\{\emptyset \}\).

[Sufficient Condition] Suppose \(W\) can be eliminated iteratively into a singleton. Denote the sequence of reduced boundary sets as

where \(\eta \) is the number of agents, \(W^{\eta } = W\), \(W^1 = \{\emptyset \}\), and superscript counts the number of boundaries.

Each \(W^i\) (\(i>1\)) is a well-defined boundary set. The elimination process can be viewed as follows. To produce \(W^x\) from \(W^{x+1}\), eliminate a \(W_i \in W^{x+1}\) relative to a \(W_j \in W^{x+1}\). Remove from \(W_j\) variables that it shares with \(W_i\) but not with any other \(W_k \in W^{x+1}\). The result is \(W^x\). We show below that boundaries in each \(W^x\), for \(x = 2, \ldots , \eta \), can be organized into a JT.

It is trivially true for \(x=2\), as can be seen from Example 9. We assume that it is true for \(x = n\), and consider the case \(x=n+1\). Suppose when \(W^n\) is derived from \(W^{n+1}\), \(W_i \in W^{n+1}\) is eliminated relative to \(W_j \in W^{n+1}\) and \(W_j\) is reduced to \(W'_j \in W^n\). By inductive assumption, boundaries in \(W^n\) can be organized into a JT \(T^n\). If the cluster \(W'_j\) in \(T^n\) is replaced by \(W_j\), the resultant cluster tree is still a JT, since \(W_j{\setminus } W'_j\) is not shared by any other cluster in \(T^n\). Next, add cluster \(W_i\) to \(T^n\), and make it adjacent to \(W_j\). Since \(W_i \subseteq W_j\), the resultant cluster tree is still a JT, and its clusters are exactly the boundaries in \(W^{n+1}\). Hence, boundaries in \(W^{n+1}\) can be organized into a JT.

From the above induction, boundaries in \(W = W^{\eta }\) can be organized into a JT. \(\square \)

Proof of Theorem 3

It suffices to show that the sender-receiver relations define a boundary based JT for \(W\). Assume that \(W\) has a boundary based JT \(T\). Since \(T\) is a cluster tree, there exists a leaf cluster \(W_i \in W\) in \(T\), that is adjacent to a single cluster \(W_j \not = W_i\) in \(T\). Because \(T\) satisfies running intersection, \( W_i \cap W_k \subseteq W_j \) holds for every \(W_k \in W\), where \(k \not = i\). It can be equivalently written as \( W_i \cap W_k \subseteq W_i \cap W_j = I_{ij}\) for every \(W_k \not = W_i\). It follows that

Since \(W_i\) is the boundary of \(A_i\), \(\bigcup _{k \not = i} (W_i \cap W_k) = W_i\) and the above becomes \(W_i \subseteq I_{ij}\). As \(I_{ij}\) is the border between \(A_i\) and \(A_j\), we have \(W_i \supseteq I_{ij}\) which yields \(W_i = I_{ij}\).

Let \(Q\) be the set of leaf clusters in \(T\) and \(S\) be the set of links between clusters in \(T\). Let \(T'\) be another boundary based JT of \(W\) with the corresponding \(Q'\) and \(S'\). In general, \(Q \not = Q'\). However, it is always true that \(S = S'\) (see Proposition 8.3 in [35]).

Let \(\mathcal{T}_1\) be the set of all boundary based JTs of \(W\), \(S_1\) be the set of links in any such JT, and \(\mathcal{Q}_1\) be the union of leaf cluster sets over all such JTs (one \(Q\) per JT). Then, after \(HTBS\) starts from a leader agent, the first agent \(A_i\) that runs \(DoDFT\) and passes the test in line 1 must have its boundary \(W_i \in \mathcal{Q}_1\). Suppose \(A_i\) sends the first inter-agent \(StartNewDFT\) to \(A_j\). Hence, the sender-receiver relation \(\langle A_i, A_j \rangle \) identifies the link \(\langle W_i, W_j \rangle \) in \(S_1\).

When \(A_j\) responds to \(StartNewDFT\), \(W_i\) is eliminated and \(W_j\) is updated (as \(Y_j\), Proc. 7, line 5). The reduced boundary set \(W'\), without \(W_i\) and \(W_j\) but with \(Y_j\), is a well-defined boundary set. Hence, \(\mathcal{T}_2\), \(S_2\), and \(\mathcal{Q}_2\) can be defined accordingly from \(W'\), where each \(T \in \mathcal{T}_2\) has one cluster less than \(\mathcal{T}_1\) and \(S_2 = S_1 {\setminus } \{\langle W_i, W_j \rangle \}\) (one link less).

Applying the argument for \(\mathcal{T}_1\) similarly to \(\mathcal{T}_2\), another \(StartNewDFT\) will be sent, resulting in \(\mathcal{T}_3\), \(S_3\), and \(\mathcal{Q}_3\). Since \(|W| = \eta \) and \(|S_1| = \eta - 1\), after \(\eta -1\) \(StartNewDFT\) messages, the reduced boundary set has \(|W'| = 1\) and all links in \(S_1\) are identified, which specifies one of the JTs in \(\mathcal{T}_1\). \(\square \)

Reprints and permissions