Performance analysis of generalized QAM modulation under η−μ and κ−μ fading

This paper presents new closed-form expressions for the symbol error probability (SEP) of θ-QAM modulation with maximum ratio combining (MRC) receiver under η−μ and κ−μ fading. The SEP formulae, obtained from the definite integrals of the moment generating function (MGF) of the signal-to-noise ratio (SNR) at the input of the MRC receiver, are written in terms of Lauricella functions. The numerical evaluation of the expressions is carried out for the η−μ distribution, which includes important distributions as special cases, such as Hoyt, Nakagami-m, Rayleigh, and one-sided Gaussian, as well as for the κ−μ distribution, which includes Rice, Nakagami-m, Rayleigh, and one-sided Gaussian as special cases.

Two-dimensional signal constellations with suppressed carrier, such as quadrature amplitude modulation (QAM) schemes, are widely used in communication systems in a variety of applications, such as modems with asynchronous transmission, digital television, and cooperative systems [1, 2].

In [3], the rectangular QAM modulation schemes are evaluated with maximum ratio combining (MRC) receivers and η−μ correlated fading, while in [4], these schemes are employed to evaluate the performance of the MRC receiver considering imprecise knowledge of the state of the Rician fading channel. In applications involving asymmetric digital subscriber lines and digital video broadcasting-cable, cross QAM schemes have been adopted. The error probability of cross QAM with MRC reception over generalized η−μ fading channel was presented in [5].

Before 2002, the evaluation of the fading effects in the reception of the M-QAM schemes was performed by means of approximate mathematical expressions. In 2002, Cho and Yoon [6] found exact expressions for the bit error probability (BEP) of QAM schemes under additive white noise. An extension of that work for Nakagami fading was presented in [7]. Then, the evaluation of structures such as the MRC receiver with M-QAM was performed by means of exact expressions. The MRC receiver has been proposed in coherent demodulation schemes for environments with Rayleigh and Nakagami fading [8–10].

Since 1962, when the square QAM (SQAM) modulation was introduced [11], different geometries for the QAM constellation have been proposed. Motivated by aspects such as minimization of the symbol error probability (SEP) and implementation complexity, a new constellation geometry was proposed in [12], the so-called triangular quadrature amplitude modulation (TQAM), for which the symbols associated with the transmitted signals are located in the vertices of contiguous triangles [13].

As an attempt to improve the power gain, Park and Byeon presented in [14] an alternative configuration for the triangular constellation and proposed constellations with irregularly distributed symbols while preserving the triangular structure. By means of theoretical analysis and simulations, they showed how to obtain a 0.62-dB power gain with respect to the SQAM constellation and a 0.20-dB power gain with respect to the TQAM regular constellation for a 10⁻⁶ symbol error rate with 64-symbol constellations. The mathematical analysis of the effects of changing the QAM constellation angle θ gave rise to a modulation scheme referred to as parametric θ-QAM, which includes the SQAM and TQAM schemes as special cases [15, 16].

The calculation of exact expressions for the SEP or BEP of a modulation scheme under fading means, in many cases,that the expressions for additive white Gaussian noise are weighted by the probability density function of the signal-to-noise ratio for a given fading. One of the most used structures to improve the BEP and SEP of QAM schemes under different types of fading is the MRC receiver [17]. In [18], the effect of the diversity provided by that structure was evaluated, through the BEP parameter, for square M-QAM and rectangular R-QAM and different types of fading, modeled by distinct distributions.

The analysis of BEP and bit error (BER) in optical wireless links, also known as free-space optics (FSO), has received much attention in recent years. In [19], the authors show that the turbulence-induced fading caused by atmospheric conditions can be modeled as multiplicative random process which follows the K distribution. The authors also present approximated closed-form expressions for the average BER of single-input multiple-output (SIMO) FSO systems. Alternatively, for FSO channels, the fading can be modeled by log-normal or gamma-gamma distributions [20, 21].

This paper proposes the use of the MRC receiver jointly with θ-QAM modulation and gives new mathematical expressions for the SEP under η−μ and κ−μ fading, modeled by κ−μ and η−μ distributions. The MRC receiver provides an additional degree of freedom for the performance control of the systems, and the probability distributions used to model the fading provide a unification of the mathematical analysis since they characterize different types of fading. One of the contributions of this paper is the derivation of exact expressions for the SEP, written in terms of hypergeometric Lauricella functions.

The paper is organized as follows: In Section 2, the average SEP for an MRC receiver with N branches is presented in terms of the moment generating function (MGF) for η−μ and κ−μ fading. In Section 3, the MGF of the signal-to-noise ratio (SNR) per branch of the MRC receiver is adjusted to the problem under consideration. In Sections 4 and 5, new SEP expressions are provided in terms of Lauricella functions. In Section 6, curves of SEP are presented for different sets of parameters. Section 7 presents the conclusions of the paper.

This section presents the calculation of the average SEP of a θ-QAM scheme under η−μ and κ−μ fading and MRC diversity. The main objective is to show that this probability can be written in terms of definite integrals of the MGF of SNR per branch of the MRC receiver. In the family of θ-QAM schemes, presented in [15], the authors obtained the expression given in Equation 1 to evaluate the SEP under additive white Gaussian noise:

In Equation 1, the parameters c ₁, c ₂, c ₃, c ₄, c ₅, and c ₆ are related to the geometry of the constellation θ-QAM and are given by the following [16]:

in which M represents the number of constellation symbols s _m,n , whose coordinates (x _m ,y _n ) are

with $m=1,\cdots ,\sqrt{M}$, $n=1,\cdots ,\sqrt{M}$, a=2d cosθ, and b=2d sinθ, and m o d(x,y) denotes the modulus operation after division of x by y. Half of the Euclidean distance between adjacent symbols of the constellation is given by

and is related to the parameter δ and to the average energy per symbol, E _av, through the expression

In order to determine the SEP of the θ-QAM modulation scheme under η−μ and κ−μ fading, for an MRC receiver, it is necessary to average the symbol error probability under additive white Gaussian noise conditioned to the instantaneous SNR γ at the MRC input. This conditional probability, denoted by P(E|γ), corresponds to P _s (γ,θ,M) for $\gamma =\sum _{k=1}^N{\gamma }_k$ at the MRC output, in which the random variables γ _k represent the instantaneous SNR in each of the N branches of the MRC receiver. The MRC structure with N branches makes decisions based on the signals α _i e ^−jθi s(t)+n _i (t), for i=1, 2, ⋯,N, in which α _i and θ _i represent, respectively, the fading attenuations and phase variations in the i th receiver branch, s(t) represents the transmitted signal modulated with θ-QAM, and n _i (t) represents the complex AWGN with zero mean and variance N ₀/2. The fading attenuation α _i can be modeled as η−μ distribution as well as κ−μ distribution. The phase shifts θ _i are uniformly distributed in the interval (0,2Π].

Therefore, the SEP, represented by P _as, can be calculated averaging P _s (γ,θ,M) by the joint probability density function $p_{{\gamma }_1,{\gamma }_2,\cdots ,{\gamma }_N}({\gamma }_1,{\gamma }_2,\cdots ,{\gamma }_N)$ of the variables γ ₁, γ ₂, ⋯, γ _N , using the multiple integral

The probability P _s (γ,θ,M) can be rewritten as follows:

Substituting Equation 7 into Equation 6, the SEP P _as can be written as follows:

in which $M_{{\gamma }_k}\left(s\right)\triangleq E\left[e^{-s{\gamma }_k}\right]$ denotes the MGF of the k th random variable γ _k .

From Equation 8, one can notice that if the variables γ _k are independent and identically distributed with corresponding MGF M _γ (s), then the SEP P _as can be written as follows:

Under η−μ fading, an exact expression for P _as can be obtained by adjusting the integration intervals to the interval [0,1] in Equation 9. This is equivalent to modify the integrands of the integral representation of the Lauricella hypergeometric function as follows:

in which ℜ{c}>ℜ{a} and ℜ{x} denote the real part of x.

On the other hand, for κ−μ fading, the calculation of P _as can be obtained using a series representation of the paremeter s. This representation is taken into account in $M_{{\gamma }_{\kappa ,\mu }}\left(s\right)$, as shown in the next section.

In [15], the BEP was computed using the product of constants and integral expressions. The integrals are related to the SEP of each decision region of the θ-QAM constellation, and the constants are concerned with the number of different bits between decision regions. It was assumed that the decision errors occur very close to the borders of these regions. In [15], these constants were presented as follows:

The BEP approximation obtained so far is highly accurate for medium and high values of SNR. In [16], corrected constants for the BEP were presented

Using these new constants in Equation 9, one can obtain the BEP with MRC diversity effect.

The expression presented in Equation 9 is evaluated in this article for the fading models characterized by η−μ and κ−μ distributions. Both these fading models are extensively treated in [22].

The MGFs of these distributions were presented in [23] in a more compact form than their expressions presented in [24],

To evaluate the SEP in Equation 9, one needs to calculate M _γ (s) for s=δ ²csc²(ϕ), 4δ ²sin²(θ)csc²(ϕ) and ${\delta }^2{\text{sec}}^2\left(\frac{\theta }{2}\right){\text{sin}}^2\left(\theta \right){\text{csc}}^2\left(\varphi \right)$ and perform the integrations in Equation 9.

The parameters h and H of Equation 13 can be written in terms of the parameter η, respectively, as h=(2+η ⁻¹+η)/4 and H=(η ⁻¹−η)/4 for format 1 and h=1/(1−η ²) and H=η/(1−η ²) for format 2. In format 1, the parameter η, 0<η<∞, represents the scattered wave power ratio between in-phase and quadrature components of clusters of multipath. In format 2, the parameter η, −1<η<1, represents the correlation coefficient between scattered wave in-phase and quadrature components of clusters of multipath. The parameter μ can be written as $\mu =\frac{1}{2V\left({\alpha }^2\right)}\left[1+{\left(\frac{H}{h}\right)}^2\right]$ for η−μ distribution and $\mu =\frac{1}{V\left({\alpha }^2\right)}\frac{1+2\kappa }{{(1+\kappa )}^2}$ for κ−μ distribution. The parameter κ represents the ratio between the total power of the dominant components and the total power of the scattered waves.

Since the Nakagami-m distribution can be obtained from the distribution κ−μ for κ→0 and μ=m, it is observed that the MGF M _κ−μ (s) coincides exactly with the MGF expression for the Nakagami-m fading implicit in [15]. For this case, in which κ→0 and μ=m, the expression of the SEP coincides exactly with expression 13 of [15]. The κ−μ distribution also includes the Rice distribution for μ=1 and κ=k. Hence, the Rayleigh distribution can be obtained from the κ−μ distribution for κ=0 and μ=1.

The Nakagami-m distribution can also be obtained from η−μ distribution for μ=m and η→0 in format 1 or η→±1 in format 2 [22]. It can also be obtained for μ=m/2 and η→1 in format 1 or η→0 in format 2. Given that the Nakagami-m is obtained, the Rayleigh distribution can be obtained for m=1.

Using a similar procedure, the Hoyt distribution can be obtained from the η−μ distribution for μ=0.5. In this case, the parameter q of Hoyt is related to η by q ²=η in format 1 or by q ²=(1−η)/(1+η) in format 2. The Rayleigh distribution can be obtained from that result for μ=0.5 and η=1 in format 1 or η=0 in format 2.

Applying the expression of the MGF of Equation 13 in Equation 9 and adjusting the ranges of the integrals to Lauricella’s function integrals, it is possible to write the SEP expression as follows:

where ${\gamma }_1=\frac{2(h-H)\mu }{\overline{\gamma }{\delta }^2}$, ${\gamma }_2=\frac{2(h+H)\mu }{\overline{\gamma }{\delta }^2}$, $\beta =2\mathrm{\mu N}+\frac{1}{2}$, $\xi =\frac{{\left(4{\mu }^{2}h\right)}^{\mathrm{\mu N}}}{{\left({\delta }^2\overline{\gamma }\right)}^{2\mathrm{\mu N}}}$,

and

The expression of the SEP for κ−μ fading can be obtained by writing the exponential function of Equation 14 in terms of its series representation and following the same procedure applied for η−μ fading. In this case, after adjusting the interval of integration to [0,1], it is possible to represent each of the integrals of P _as by means of sums of Lauricella functions:

where $\beta =\mathrm{\mu N}+\frac{1}{2}$, ${\gamma }_1=\frac{\mu (1+\kappa )}{\overline{\gamma }{\delta }^2}$, ${\gamma }_2=\frac{\mu (1+\kappa )}{4\overline{\gamma }{\delta }^2}$, ${\xi }_1=e^{-\mathrm{\kappa \mu N}}{\gamma }_1^{\mathrm{\mu N}}$, ${\xi }_2=e^{-\mathrm{\kappa \mu N}}{\gamma }_2^{\mathrm{\mu N}}$,

and

Firstly, two important aspects of Equations 15 and 19 are highlighted. Despite the expression of SEP in terms of Equations 15 and 19 seems at a first glance more complex than Expression 9, it is not true since the Lauricella functions can be easily calculated by the integral representation in the interval [0,1]. For real numbers, Lauricella functions, $F_D^r(a,b_1,\cdots ,b_r,c,x_1,\cdots ,x_r)$, can be written in terms of a sum of a finite number of terms using the quadrature Gauss-Legendre, for instance. In this case,

in which

ξ _l , for l=1,2,⋯,N _t , are the roots of the Legendre polynomials $P_{N_t}$ of order N _t , and

The second aspect regards Equation 19. Although it is written in terms of an infinite series, its convergence is fast. In the present work, it was observed that n ranging from 0 to 10 suffices. Indeed, for the first term, one has

since

and A ₁(n) is directly proportional to $\frac{\Gamma \left(2\right(\beta +n\left)\right)}{2^{2n}\mathrm{B}(\beta +n,1)n!{\Gamma }^2(\beta +n)}$.

6.1 SEP as a function of θ

Figure 1 presents SEP curves as a function of the angle θ for the η−μ model. Due to the symmetry of the geometric regions of the θ-QAM constellation, it suffices to take the angle θ between 45° and 90°. The red curves were obtained considering the diversity order of the MRC receiver equal to 2, while the blue curves were obtained for diversity order equal to 3. In both cases the SNR was maintained equal to 10 dB. The assignment of parameter values η and μ in Figure 1 provided the following types of fading: Nakagami-m for η=0 and μ=1.5, Rayleigh for η=0 and μ=0.5, one-sided Gaussian for η=0 and μ=0.25, and Hoyt for η=0.65 and μ=0.5.

SEP curves for θ -QAM with M=16 under η−μ fading.

Full size image

It is observed in Figure 1 that the spatial diversity of the MRC has a stronger influence in the SEP when compared to that of the optimum angle θ, for which the curves attain a minimum. This is an important aspect since it is not possible to perform an optimum Gray mapping for every value of θ. All curves attain their minimum value for θ close to 60°, and as one can see, there is a small difference in the SEP values when θ ranges from 45° to 60° compared to the difference obtained increasing the diversity order by one.

In Figure 2, the curves were obtained from the model κ−μ. Similarly to Figure 1, red curves were plotted for N=2 and blue curves for N=3. The SNR was set at 10 dB, and the choice of the parameters κ and μ provided the following types of fading: Nakagami-m for κ=0 and μ=3.0, Rayleigh for κ=0 and μ=1.0, one-sided Gaussian for κ=0 and μ=0.5, and Rice for κ=0.8 and μ=1.0.

SEP curves for θ -QAM for M =16 under κ−μ fading.

Full size image

The parameters used in the curves of Figure 2, except for Rice fading, provide the same curves presented in Figure 1. These parameters were properly chosen to show that both the results of Equations 15 and 19 can be used in the calculation of the SEP under the main categories of fading. The difference is that Hoyt (Nakagami-q) fading can only be modeled by the η−μ distribution, and Rice fading can be modeled by the κ−μ distribution.

6.2 SEP as a function of SNR for a fixed θ

The curves in Figure 3 show the behavior of the SEP as a function of SNR, under η−μ fading, for the parameter values used in Section 6.1. The curves were obtained for the order of the constellation M=64 and diversity orders N=1 and N=3.

SEP curves for θ -QAM with M =64 under η−μ fading.

Full size image

As can be seen in Figure 3, regarding Nakagami fading obtained from the η−μ model with η=0 and μ=1.5, the increase of the diversity order from N=1 to N=3 provides a gain in SNR of about 7.5 dB for a SEP fixed at 10⁻². For all types of fading under consideration (Nakagami for η=0 and μ=1.5, Rayleigh for η=0 and μ=0.5, one-sided Gaussian for η=0 and μ=0.25, and Hoyt for η=0.65 and μ=0.5), one can note that a diversity order N=3 does not assure the SEP lower than 10⁻³ for SNR <20 dB. The curves in Figure 4 present the behavior of the SEP as a function of SNR under κ−μ fading for the constellation order M=64 and diversity orders N=1 and N=3.

SEP curves for θ -QAM with M =64 under κ−μ fading and θ=62° .

Full size image

The equivalence of η−μ and κ−μ models, for appropriated choices of the corresponding parameters, can be observed in Figures 3 and 4, for which the SEP curves for Nakagami, Rayleigh, and one-sided Gaussian fading are exactly the same.

In Figure 4, the results for Rice fading are obtained using κ=0.8 and μ=1.0. For all curves, a diversity order N up to 3 does not guarantee a SEP lower than 10⁻³ for SNR <20 dB.

The curves in Figure 5 are obtained for η−μ fading, constellation order M=64, and diversity orders N=1 and N=3. The angle θ was considered 62° near the angle 60° of the triangular QAM constellation. Considering only the dashed curves for η=0.1, one can observe that the SEP increases when μ decreases from 2.5 to 1.5. This is expected because the corresponding fading intensity also increases. A similar behavior occurs for η=0.8. For the parameters ranging from (η=0.1, μ=2.5) to (η=0.80, μ=1.5) and N=3, about 3 dB of SNR is necessary to maintain the SEP at 10⁻⁴.

SEP curves for θ -QAM with M=64 under η−μ fading and θ=62° .

Full size image

The curves in Figure 6 were obtained for κ−μ fading, modulation order M=256 and diversity order N=3 and N=4. The angle θ was maintained at 62°. As one can note, even for diversity order of N=4, the curves of SEP remain above 10⁻² for values of SNR lower than 20 dB. This occurs because the higher-order θ-QAM constellations are more susceptible to fading effects. For the parameters ranging from (κ=2.50,μ=2.50) to (κ=0.60,μ=1.50) and N=4, about 1.25 dB must be invested in terms of the SNR to keep the SEP at 10⁻².

SEP curves for θ -QAM with M =256 under κ−μ fading.

Full size image

This paper presents new and exact expressions for the SEP of θ-QAM modulation considering the MRC receiver under η−μ and κ−μ fading, modeled by η−μ and κ−μ distributions. An important aspect of those distributions is the fact that they can provide a unified analysis of the influence of different types of fading in the performance of the communications system, such as Hoyt, Rice, Nakagami-m, Rayleigh, and one-sided Gaussian. The SEP expressions, obtained from the definite integrals of the MGF function of the SNR at the input of the MRC receiver, are written in terms of Lauricella functions. Expressions obtained in the paper show that the spatial diversity introduced by the MRC receiver has a strong influence for reducing the SEP, even for high-order constellations. As an example, considering 256-QAM constellation and a SEP of 10⁻², savings of about 1.7 dB can be obtained when the diversity order is inscreased from N=3 to N=4. The SEP expressions can be useful to design and evaluate the performance of wireless communication systems, since they alleviate the need for Monte Carlo simulations. Future works include the effects of fading correlation and the impact of channel estimation errors on the performance of wireless systems.

The authors would like to express their thanks to CNPq for the financial support on this research.

Authors and Affiliations

1. Department of Electrical Engineering, Federal University of Campina Grande, Campina Grande, Paraíba, Brazil

   Wamberto J. L. Queiroz, Waslon T. A. Lopes & Marcelo S. Alencar

2. Polytechnic School of Pernambuco, University of Pernambuco, Recife, Pernambuco, Brazil

   Francisco Madeiro

3. Institute for Advanced Studies in Telecommunications, Campina Grande, Paraíba, Brazil

   Wamberto J. L. Queiroz, Waslon T. A. Lopes & Marcelo S. Alencar

Corresponding author

Correspondence to Francisco Madeiro.

Competing interests

The authors declare that they have no competing interests.

Open Access This article is distributed under the terms of the Creative Commons Attribution 2.0 International License ( https://creativecommons.org/licenses/by/2.0 ), which permits unrestricted use, distribution, and reproduction in any medium, provided the original work is properly cited.

Reprints and permissions