Modelling heterogeneity: on the problem of group comparisons with logistic regression and the potential of the heterogeneous choice model

The comparison of coefficients of logit models obtained for different groups is widely considered as problematic because of possible heterogeneity of residual variances in latent variables. It is shown that the heterogeneous logit model can be used to account for this type of heterogeneity by considering reduced models that are identified. A model selection strategy is proposed that can distinguish between effects that are due to heterogeneity and substantial interaction effects. In contrast to the common understanding, the heterogeneous logit model is considered as a model that contains effect modifying terms, which are not necessarily linked to variances but can also represent other types of heterogeneity in the population. The alternative interpretation of the parameters in the heterogeneous logit model makes it a flexible tool that can account for various sources of heterogeneity. Although the model is typically derived from latent variables it is important that for the interpretation of parameters the reference to latent variables is not needed. Latent variables are considered as a motivation for binary models, but the effects in the models can be interpreted as effects on the binary response.

Authors and Affiliations

1. Ludwig-Maximilians-Universität München, Akademiestraße 1, 80799, Munich, Germany

   Gerhard Tutz

Corresponding author

Correspondence to Gerhard Tutz.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

1.1 Proof Proposition 3.1

(a) Let us assume that the general heterogeneous choice model with predictor

holds, where for \(S=\{j_1,\ldots ,j_m\}\) one has \(({\varvec{x}}_i^{S})^T=(x_{ij_1},\ldots ,x_{ij_m})\), and interaction effects \((\varvec{\alpha }^{S})^T=(\alpha _{0j_1},\ldots , \alpha _{0j_m})\). One can define new parameters

When using these parameters as parameters in the interaction model

one obtains that the linear predictor in (14) is the same as the linear predictor in the general heterogeneous choice model (13). Thus, the interaction model holds.

In addition, for \(j \notin S\) the relation \(\beta _{0j}/\beta _j= (1-e^{\gamma })/(e^{\gamma })\) holds. Let \(\{1,\ldots ,p\}\) be partitioned into the disjunct subsets S and \({\tilde{S}}=\{1,\ldots ,p\} {\setminus } S\). Then for pairs \(j,s \in {\tilde{S}}\) the constraints

hold. Of course it is only a constraint if \({\tilde{S}} \ge 2\).

(b) Let us now assume that the interaction model (14) with constraints (15) holds.

Case 1 If \(|S|=p, |{\tilde{S}}|=0\) one obtains with the parameters defined by \(\alpha _{00}=\beta _{00}, \alpha _{0}=\beta _{0}\)\(\alpha _{j}=\beta _{j}\), \(\alpha _{0j}=\beta _{0j}\), \(j=1,\ldots , p\) that the linear predictor \(\eta _i = ({\alpha _{00}+x_{i0}\alpha _0+{\varvec{x}}_i^T\varvec{\alpha }}+ x_{i0}{\varvec{x}}_i^T \varvec{\alpha })/\exp (x_{i0}\gamma )\) is equivalent to the predictor in (14), which means that the heterogeneous choice model holds with \(\gamma \) fixed by \(\gamma =1\) since it is not identified.

Case 2 Let \(|S|=p-1, |{\tilde{S}}|=1\) hold and parameters be defined by

Using these parameters in the predictor \(\eta _i = ({\alpha _{00}+x_{i0}\alpha _0+{\varvec{x}}_i^T\varvec{\alpha }}+ x_{i0}(x_{i1}\ldots , \ldots , x_{i,p-1}) \varvec{\alpha }^{S})/\exp (x_{i0}\gamma )\) yields the predictor in (14). Thus the interaction model is represented as a heterogeneous chioce model, in which \(\alpha _{0p}=0\). It should be noted that one could have omitted another interaction parameter. Without loss of generality we chose the parameter \(\alpha _{0p}\).

Case 3 Let \(|S| \le p-2, |{\tilde{S}}|=m \ge 2\) hold. Without loss of generality let \({\tilde{S}}=\{p-m+1,\ldots ,p\}\). Let parameters be defined by

In addition, \(\gamma \) is defined by

which is possible since \((1-e^{\gamma })/e^{\gamma }= \beta _{0j}/\beta _{j}\), and \(\beta _{0j}/\beta _{j}\) has the same value for all \(j \in {\tilde{S}}\). Using these parameters in the predictor \(\eta _i = ({\alpha _{00}+x_{i0}\alpha _0+{\varvec{x}}_i^T\varvec{\alpha }}+ x_{i0}(x_{i1}\ldots , \ldots , x_{i,p-m}) \varvec{\alpha }^{S})/\exp (x_{i0}\gamma )\) yields the predictor in (14). Therefore, it is shown that the heterogeneous choice model with interactions \(\alpha _{0,p-m+1}=\cdots =\alpha _{0,p}=0\) holds.

1.2 Proof Proposition 3.2

Let us consider the model (10) and assume that one of the interaction parameters is zero. Without loss of generality we assume \(\alpha _{0p}=0\). Then one has the model

Let \(\alpha _{00},\ldots ,\alpha _{0,p-1},\gamma \) and \({{\tilde{\alpha }}}_{00},\ldots ,{{\tilde{\alpha }}}_{0,p-1},{{\tilde{\gamma }}}\) be two parameterizations of the model. It has to be shown that the two parameterizations are identical.

Let \(\pi (x_{ij})\) denote the probability of observing \(Y_i=1\) when the jth covariate has value \(x_{ij}\) and \(\pi (x_{ij}+1)\) denote the probability if the jth covariate has value \(x_{ij}+1\); all other variables are kept fixed. In addition we let \(\pi (x_{ij}, x_{i0}=g)\) denote the probability of observing \(Y_i=1\) when the jth covariate has value \(x_{ij}\) and \(x_{i0}=g\), correspondingly \(\pi (x_{ij}+1)\) denotes the probability if the jth covariate has value \(x_{ij}+1\) and \(x_{i0}=g\); all other variables are kept fixed.

(1) One obtains immediately

and therefore, provided \(\alpha _p \ne 0\),

Since the equations hold for both parameterizations one obtains \(e^{\gamma }=e^{{{\tilde{\gamma }}}}\) and therefore \(\gamma ={{\tilde{\gamma }}}\).

(2) For all variables \(j \ne p\) one has

This yields for \(x_{i0}=0\) that \(\alpha _j={{\tilde{\alpha }}}_j\) holds, and for \(x_{i0}=1\) that \(\alpha _{0j}={\tilde{\alpha }}_{0j}\) holds.

(3) The only left parameters, which still to be investigated, are \(\alpha _{00}\) and \(\alpha _0\). By using for \(x_{i0}=0\)

and for \(x_{i0}=1\)

one obtains \(\alpha _{00}={\tilde{\alpha }}_{00}\) and \(\alpha _{0}={\tilde{\alpha }}_{0}\), which concludes the proof.

Reprints and permissions