# Spectrally Efficient Multiplexing - Nyquist-WDM

This is a continuation from the previous tutorial - ** laser amplification explained in detail**.

This tutorial discusses the basics of the Nyquist signaling theory and various aspects regarding its application in coherent optical communication systems to improve the transport spectral efficiency (SE).

An alternative, and dual, approach to achieve a high SE is based on the use of orthogonal frequency-division multiplexing (OFDM), which is described in the following tutorial.

## 1. Introduction

After the advent of digital coherent technology, there has been a significant interest in using advanced multilevel modulation formats, in combination with advanced multiplexing techniques and efficient digital signal processing (DSP) algorithms, to increase the spectral efficiency (SE) in fiber-optic communication systems.

This tutorial focuses on wavelength-division multiplexing (WDM) optical systems, for which the SE is defined as the information capacity of a single channel (in bit/s) divided by the frequency spacing \(\Delta{f}\) (in Hz) between the carriers of the WDM comb:

\[\tag{4.1}\text{SE}=\frac{R_s}{\Delta{f}}\frac{\log_2(M)}{(1+r)}\]

where \(R_s\) is the symbol rate, \(M\) is the number of constellation points of the modulation format, and \(r\) is the redundancy of the forward-error correction (FEC) code, for example \(r=0.07\) for an FEC with overhead (OH) equal to 7%.

The total system capacity (defined as the maximum information in bit/s that can be transmitted by the WDM comb) is obtained as the product between the SE and the available bandwidth. The maximization of the SE thus plays an important role in the maximization of the overall system capacity.

In the past years, the SE of optical systems has significantly increased, mainly due to the advent of coherent-detection technologies, which enabled the use of high-order modulation formats based on polarization-division multiplexing (PDM), such as PDM-QPSK (quadrature phase-shift keying), with \(M=4\), PDM-16QAM (quadrature-amplitude modulation), with \(M=8\), and PDM-64QAM, with \(M=12\). However, the use of high-order modulation formats requires a higher optical signal-to-noise ratio (OSNR), which may result in a significantly reduced achievable transmission distance.

An alternative way of increasing the SE, and consequently the overall system capacity, involves reducing the frequency spacing \(\Delta{f}\) between the WDM sub-carriers.

In this tutorial, the focus is on the minimization of the ** normalized** frequency spacing \(\delta{f}\), defined as

\[\tag{4.2}\delta{f}=\frac{\Delta{f}}{R_s}\]

To achieve ultimate spectral efficiency, WDM channel spacings are reduced until the optical spectra of neighboring channels start to noticeably overlap. In this limit of ultra-dense WDM systems, linear crosstalk between adjacent WDM channels becomes a main source of degradation.

An efficient countermeasure to limit the crosstalk is based on an accurate spectral shaping of each subchannel of the WDM comb: this technique has been widely used in radio links for decades and has been lately proposed and demonstrated for optical links too.

Employing this technique, known in optical communications as “Nyquist-WDM,” the transmission of PDM-QPSK WDM signals with channel spacing equal to the symbol rate has been demonstrated over transpacific distances. The technique has also been successfully applied to the generation and transmission of higher-order modulation formats, such as PDM-8QAM, PDM-16QAM, PDM-32QAM, and PDM-64QAM, with frequency spacing values equal or very close to the symbol rate.

The ideal shape of the transmitted spectra, which allows to achieve a channel spacing equal to the symbol rate \(R_s\) with no crosstalk and no intersymbol interference (ISI), is rectangular with bandwidth equal to \(R_s\).

In such an ideal scenario, Nyquist-WDM can achieve the optimum matched filter performance in additive white Gaussian noise (AWGN) systems. In practice, penalties are to be expected when the ideal constraints on Nyquist WDM implementation are relaxed, for instance with the transmission of channels with not perfectly rectangular spectra.

In this tutorial, the fundamental Nyquist-WDM signaling theory is presented and various aspects regarding its implementation and application in coherent optical communication systems to improve the transport spectral efficiency are discussed.

Depending on the normalized frequency spacing \(\delta{f}\) among the WDM channels, three different categories of Nyquist-WDM signaling, which are described in detail in Section 2, can be identified:

- \(\delta{f}=1\) (i.e., \(\delta{f}=R_s\)): Ideal Nyquist-WDM (Section 2.1).
- \(\delta{f}\gt1\) (i.e., \(\delta{f}\gt{R_2}\)): Quasi-Nyquist-WDM (Section 2.2).
- \(\delta{f}\lt1\) (i.e., \(\delta{f}\lt{R_s}\)): Super-Nyquist-WDM (Section 2.3).

Figure 4.1 shows an example of the WDM spectrum in the three cases described here, as well as in a more standard WDM configuration.

- Section 3 discusses the receiver (Rx) structure used to detect Nyquist-WDM signals.
- Section 4 is devoted to a general discussion of practical implementations of the Nyquist-WDM transmitter (Tx), focusing on the different available options to perform spectral shaping in the optical and digital domains.
- In Section 5, a theoretical analysis of the trade-offs between capacity and reach is presented, followed by a review of the most relevant experimental demonstrations of Nyquist-WDM transmission over multispan optical links.

## 2. Nyquist Signaling Schemes

This section reports a review of the fundamental results of the digital communications theory, highlighting the properties of the transmitted signal spectrum that avoids both linear crosstalk between subcarriers (in the frequency domain) and ISI between adjacent pulses (in the time domain).

The section is organized in three subsections, for each of the categories of Nyquist signaling identified in Section 1.

### 2.1. Ideal Nyquist-WDM (𝚫f = R_{s})

Figure 4.2 shows the baseband model of a classical digital modulation system, which is a good approximation of the behavior of an optical system with coherent detection.

The model assumes propagation in the linear regime, as well as ideal modulation onto the optical carrier and perfect carrier recovery. Complex symbols \(\alpha_k\) are transmitted with a continuous-time pulse shape \(h_\text{Tx}(t)\), resulting in a transmit electrical field that can be written as

\[\tag{4.3}E_\text{Tx}(t)=\boldsymbol{\sum}_k\alpha_kh_\text{Tx}(t-kT)\]

where \(T\) is the signaling period, equal to the inverse of the symbol rate (\(T=1∕R_s\)).

This can alternatively be viewed as applying a filter with transfer function \(H_\text{Tx}(f)\) (equal to the Fourier transform of the impulse response \(h_\text{Tx}(t)\)) to a signal \(s(t)\), defined as

\[\tag{4.4}s(t)=\boldsymbol{\sum}_k\alpha_k\delta(t-kT)\]

The transfer function \(H_\text{Tx}(f)\) takes into account all filtering effects at the transmitter side. Under the assumption of zero-mean and uncorrelated modulation symbols \(\alpha_k\), the power spectral density (PSD) of the baseband transmitted electrical field of Equation 4.3 is proportional to \(|H_\text{Tx}(f)|^2\); that is, it depends exclusively on the transmit pulse shape.

The optical channel is modeled as an AWGN channel, with a flat frequency response. In an optical transmission system, this corresponds to assuming that the effects of chromatic dispersion (CD) and other propagation effects are perfectly compensated for. The signal at the input of the receiver can then be written as

\[\tag{4.5}E_\text{Tx}(t)=E_\text{Tx}(t)+n(t)=\boldsymbol{\sum}_k\alpha_kh_\text{Tx}(t-kT)+n(t)\]

where \(n(t)\) is a Gaussian random process with two-sided PSD equal to \(N_0/2\).

The coherent receiver (Rx) is modeled as a linear filter with transfer function \(H_\text{Rx}(f)\), followed by a sampler, corresponding to an analog-to-digital conversion device, operating at the symbol rate \(R_s\). The analog waveform at the decision point can be written as

\[\tag{4.6}r(t)=E_\text{Rx}(t)*h_\text{Rx}=\boldsymbol{\sum}_k\alpha_kh(t-kT)+v(t)\]

with \(h(t)=h_\text{Tx}(t)∗h_\text{Rx}(t)\) and \(v(t)=n(t)∗h_\text{Rx}(t)\). After sampling,

\[\tag{4.7}r_n=r(t_0+nT)=\alpha_nh(t_0)+\boldsymbol{\sum}_{k\ne{n}}\alpha_kh_{n-k}+v_k\]

where \(h_{n-k}=h(t_0+(n-k)T)\), \(v_k=v(t_0+nT)\) and \(t_0\) is the optimum sampling point. The condition for no ISI is

\[\tag{4.8}h_i=\begin{cases}1\quad(i=0)\\0\quad(i\ne0)\end{cases}\]

The Nyquist theorem states that the necessary and sufficient condition for \(h(t)\) to satisfy Equation 4.8 is that its Fourier transform \(H(f)\) satisfies

\[\tag{4.9}B(f)=\boldsymbol{\sum}_kH\left(f+\frac{k}{T}\right)=T\]

Three cases can be distinguished, depending on the spectral width \(B_H\) of the overall transfer function \(H(f)\):

- \(R_s\lt{B_H}\): the left term of Equation 4.9 consists of overlapping replicas of \(H(f)\), separated by \(R_s=1/T\), and there exist numerous choices for \(H(t)\) that satisfy Equation 4.9 . This case is discussed in Section 2.2.
- \(R_s\gt{B_H}\): since the left term of Equation 4.9 consists of nonoverlapping replicas of \(H(f)\), separated by \(R_s=1/T\), there is no choice of \(H(f)\) that satisfies Equation 4.9 ; that is, there is no way for a system to be designed without ISI. This case is discussed in Section 2.3.
- \(R_s=B_H\): this case corresponds to the so-called “Nyquist limit,” which is the subject of this section.

In the “Nyquist limit” case, there exists only one \(H(f)\) satisfying Equation 4.9, namely

\[\tag{4.10}H(f)=\begin{cases}T\quad(|f|\lt{B_H/2})\\0\quad(\text{otherwise})\end{cases}\]

which corresponds to the pulse in time

\[\tag{4.11}h(t)=\frac{\sin\left(\frac{\pi{t}}{T}\right)}{\frac{\pi{t}}{T}}=\text{sinc}\left(\frac{\pi{t}}{T}\right)\]

The difficulty with this choice of \(h(t)\) is that it is noncausal, and therefore nonrealizable. In order to make it realizable, usually a delayed version of it is used: \(h(t−T_d)\), with \(h(t−T_d)\approx0\) if \(t\lt0\), which also implies a shift in the sampling instant. Another problem with this pulse is that it decays to zero very slowly and, consequently, a small error in the sampling time results in a large ISI.

Figure 4.3 shows the time- and frequency-domain pulse shapes corresponding to the ideal Nyquist-WDM signal. From the optimum detection theory in AWGN channel, the receiver filter yielding the best performance is matched to the transmit pulse shape; that is, \(|H_\text{Rx}(f)|=|H_\text{Tx}(f)|=\sqrt{H(f)}\).

In the ideal Nyquist case, this would in theory require an infinite length digital filter at the receiver, which is not physically realizable, as discussed in Section 3.

One way of limiting this problem involves increasing the bandwidth of the spectrum (which corresponds to the case \(R_s\lt{B_H}\)). Since an increase of the bandwidth of the spectrum of the WDM channels would induce linear crosstalk between them, the frequency spacing needs to be increased, as well, yielding the “quasi-Nyquist-WDM” transmission described in the following section.

###
2.2. Quasi-Nyquist-WDM (𝚫f > R_{s})

A particular class of pulse spectra that satisfy Equation 4.9 is characterized by a raised-cosine (RC) shape, whose frequency characteristic is given by

\[\tag{4.12}H_\text{RC}(f)=\begin{cases}T\qquad\qquad\qquad\qquad\qquad\qquad\quad\left(0\le|f|\le\frac{1-\rho}{2T}\right)\\\frac{T}{2}\left\{1+\cos\left[\frac{\pi{T}}{\rho}\left(|f|-\frac{1-\rho}{2T}\right)\right]\right\}\quad\left(\frac{1-\rho}{2T}\le|f|\le\frac{1+\rho}{2T}\right)\\0\qquad\qquad\qquad\qquad\qquad\qquad\qquad\left(|f|\gt\frac{1+\rho}{2T}\right)\end{cases}\]

where \(\rho\) is called the roll-off factor and takes values in the range [0, 1]. In the time domain, the shape of the pulse having an RC spectrum is

\[\tag{4.13}h_\text{RC}(t)=\frac{\sin\left(\frac{\pi{t}}{T}\right)}{\frac{\pi{t}}{T}}\frac{\cos\left(\frac{\pi\rho{t}}{T}\right)}{1-\frac{4\rho^2t^2}{T^2}}\]

The pulse shape in the time and frequency domains for different values of \(\rho\) is shown in Figure 4.4.

The case \(\rho=0\) corresponds to the rectangular spectrum with bandwidth \(R_s\). Due to the smooth characteristics of the raised cosine spectrum, it is possible to design practical filters for both the transmitter and the receiver that approximate the overall desired frequency response.

In the case in which the channel has a flat frequency response, the optimum receiver filter is matched to the transmit one, thus both assume a square-root raised cosine (SRRC) shape. Note that an additional delay is required to ensure the physical realizability of the filters.

Figure 4.5 shows the eye diagrams of the signal after the receiver filter for both two- and four-level amplitude modulations, when SRRC filters are used both at the Tx and at the Rx side, for different values of roll-off. The lowest the roll-off, the more sensitive is the system to error in the sampling instant due to a nonperfect timing recovery.

When a WDM signal is generated by multiplexing subchannels with nonrectangular spectra, depending on the relationship between the frequency spacing and the symbol rate linear crosstalk among subchannels can be generated, potentially degrading performance.

The impact of crosstalk as a function of the normalized frequency spacing is shown in Figure 4.6(a), in terms of signal-to-noise ratio (SNR) needed to achieve a target bit error rate (BER) of \(10^{-3}\), for three different modulation formats (QPSK, 16QAM, and 64QAM).

The SNR is defined as

\[\tag{4.14}\text{SNR}=\frac{P_\text{Rx}}{pN_0R_s}\]

where \(P_\text{Rx}\) is the power of the useful signal at the input of the Rx filter, \(N_0\) is the one-sided PSD of the additive Gaussian noise (GN) and \(p\) is a parameter that is equal to 1 in the single-polarization case and is equal to 2 in the dual-polarization case.

SRRC filters with roll-off equal to 0.1 were used both at Tx and Rx sides. Ideally, no crosstalk is present when \(\delta{f}\gt(1+\rho)\). In fact, if \(\delta{f}\gt1.1\), the performance of all three formats converges to the theoretical value, whilst for lower spacings an SNR penalty is incurred, which is higher for higher cardinality modulation formats.

Figure 4.6(b) shows the impact of crosstalk at a fixed frequency spacing equal to \(1.1\cdot{R_s}\) as function of the roll-off. For values of \(\rho\) lesser than 0.1, no penalty is present, whilst for higher values the SNR penalty due to crosstalk increases, with a larger impact for higher-order modulation formats.

### 2.3. Super-Nyquist-WDM (𝚫f < R_{s})

As observed in Section 2.1, it is not possible to achieve zero-ISI if \(\delta{f}\lt{R_s}\). However, it is possible to achieve a channel spacing lower than the symbol rate by exploiting the so-called ** partial response signaling**.

The basic idea of partial response signaling involves introducing some amount of controlled ISI, which can be removed at the Rx side: this process allows to reduce the spectral width of the signal, and thus to increase the SE. The counterpart is some power penalty and/or additional complexity in the transponder.

Duobinary (DB) line-coding is an example of partial-response transmission format. It is based on the introduction of correlation among symbols through some amount of controlled ISI.

The transmission based on DB was first proposed in the 1960s by A. Lender, for radio frequency (RF) communications. In the 1990s, DB has re-emerged in the field of optical communications thanks to its high tolerance against fiber chromatic dispersion (CD).

Later, it was overcome by multilevel modulation schemes that could reach even higher spectral efficiencies, combined with coherent detection, which allows to compensate for huge amounts of CD in the digital domain.

Recently, the use of the DB concept has been proposed in order to further increase the SE in coherent optical transmission systems based on WDM multilevel modulation schemes: since the optical signals have a narrower spectrum bandwidth than conventional signals, a tighter frequency spacing can be used, thus potentially increasing the SE and the overall capacity.

The condition for zero ISI (see Eq. 4.8) is that \(h(iT)=0\) for \(i\ne0\). The DB coding allows one additional nonzero value in the samples \(h(iT)\):

\[\tag{4.15}h_i=\begin{cases}1\quad(i=0,1)\\0\quad(i\ne0,1)\end{cases}\]

The introduced ISI is deterministic, thus it can be taken into account and compensated for at the Rx.

The effect of partial response coding can be viewed as a filtering of the signal, that is, as the multiplication of the global transfer function \(H(f)\) by a periodic transfer function \(C(f)\) with period \(R_s\).

In the case of DB modulation scheme, \(C(f)\) is given by Proakis and Salehi

\[\tag{4.16}C(f)=1+\exp\{-j2\pi{f}T\}=2\cos(\pi{fT})\exp\{-j\pi{f}T\}\]

It corresponds in the time domain to the addition of a signal and a replica delayed by \(T\).

Figure 4.7 shows the shape of the duobinary pulse in frequency domain

\[\tag{4.17}H_\text{DB}(f)=\begin{cases}2\cos(\pi{f}T)\exp\{-j\pi{f}T\}\quad\quad|f|\lt\frac{1}{2T}\\0\qquad\qquad\qquad\qquad\qquad\qquad\text{otherwise}\end{cases}\]

and in time domain

\[\tag{4.18}h_\text{DB}(t)=\text{sinc}\left(\pi\frac{t}{T}\right)+\text{sinc}\left(\pi\frac{(t-T)}{T}\right)\]

where the sinc function is defined as in Equation 4.11 . Since the spectrum decays to zero smoothly, physically realizable filters that closely approximate this spectrum can be designed.

The effect of a DB coding on the scattering diagram of a QPSK modulation is shown in Figure 4.8: while the standard constellation (a) is composed of four points, the number of constellation points is increased to nine for the DB QPSK signal (b), due to the ISI introduced by the DB coding.

Two methods can be used to detect signals that contain a controlled amount of ISI. One is the symbol-by-symbol method, which is easy to implement, but gives an asymptotic SNR penalty in the order of 2 dB for DB line coding modulations. The other is based on multisymbol detection techniques, such as maximum-likelihood sequence estimation (MLSE) and maximum-a-posteriori probability (MAP) algorithms, which minimize the error probability, at the expenses of a higher computational complexity.

As an example, Figure 4.9 shows the back-to-back performance, in terms of Q-factor versus OSNR, for both the symbol-by-symbol detection scheme and the maximum-likelihood sequence detection (MLSD) algorithm. A Q-factor gain in the order of 1.7 dB was achieved thanks to the use of the more complex MLSD scheme.

## 3. Detection of a Nyquist-WDM Signal

In conventional WDM systems, subchannels are first demultiplexed in the optical domain and then separately detected. This approach cannot be applied to Nyquist-WDM signals, in which the subchannels are too closely spaced to be separated through optical filtering without incurring in any penalty. However, sharp filter shapes can be efficiently implemented at the Rx in the digital domain, enabling detection of single subchannels without the need of any tight optical filtering at the Rx.

The schematic of a coherent Rx, which detects a single PDM subchannel in a Nyquist-WDM comb, is shown in Figure 4.10. It is composed of a 90\(^\circ\)-hybrid, followed by four balanced photodetectors (BPDs), whose functionality is to map the optical field into four electrical signals, corresponding to the in-phase and quadrature field components for the two polarizations.

The single subchannel is selected by tuning the local oscillator (LO) to the center frequency of the subchannel. The electrical low-pass filters (LPFs) in Figure 4.10 represents the cascade of all band-limiting components in the Rx. The four analog electrical signals are sampled by four analog-to-digital converters (ADCs) with sampling speed \(f_\text{ADC}\) and the signal samples are elaborated by ** ad hoc** DSP algorithms, which perform polarization recovery and compensation of propagation linear (and possibly nonlinear) impairments.

Note that antialiasing electrical filters can be placed before the ADCs in order to reduce the bandwidth of the input signal, thus relaxing the requirements for the ADC sampling frequency and enabling more efficient DSP for polarization recovery and impairments compensation.

The Nyquist sampling theorem states that a sufficient and necessary condition to avoid the generation of aliasing replica of the analog signal is that the sampling frequency is greater than twice the bandwidth occupation of the signal.

In the case of ideal (rectangular) Nyquist shaping with ideal (rectangular) antialiasing filters, the sampling frequency needs to be greater than \(R_s\) in order to avoid aliasing.

In practical cases (quasi-Nyquist-WDM with realistic antialiasing filters), a higher sampling speed is needed. Typically, \(f_\text{ADC}=2\cdot{R_s}\), corresponding to a number of samples per symbol \(N_\text{SpS}\) equal to 2, but lower values can be used without incurring in substantial penalties where a value of \(N_\text{SpS}\) equal to 1.67 was used.

As an example, Figure 4.11(a) shows the results of a simulative analysis of the robustness of the system to the limited speed of the ADC (which translates into a limited number of samples per symbol) in terms of the back-to-back SNR penalty, at a reference BER of \(4\times10^{-3}\), with respect to the 2 SpS case.

A Nyquist-WDM comb was simulated, with an SRRC shape with \(\rho=0.1\). The Rx electrical low-pass characteristic was modeled as a fifth-order Bessel filter with bandwidth \(B_\text{Rx}\). The value of \(B_\text{Rx}\) was optimized for each value of \(N_\text{SpS}\), obtaining the optimum values shown in Figure 4.11(b) (normalized with respect to the symbol rate).

A low value of \(N_\text{SpS}\) corresponds to a low value of \(f_\text{ADC}/R_s\): to avoid aliasing, the Rx bandwidth has to be decreased accordingly. This, however, causes a distortion on the useful signal, which in turn induces an SNR penalty, which is higher for higher-order modulation formats.

A relevant characteristics of Nyquist-WDM signaling is its good performance even at very low values of the Rx bandwidth. In fact, thanks to the compact spectral shape of the Nyquist-WDM subchannels, the bandwidth of the electrical Rx filter can be kept low with respect to the symbol rate (close to \(R_s/2\)), without introducing filtering penalty on the subchannel and without breaking the orthogonality between subcarriers.

The matched filter design for SRRC spectrally shaped Nyquist-WDM systems has been addressed. Typically, two equalizers are implemented in the coherent Rx DSP: one CD equalizer and one adaptive butterfly blind equalizer. The CD equalizer is used to compensate for the large amount of accumulated CD in the fiber link and usually is a static frequency domain equalizer (FDE).

The adaptive equalizer is typically implemented using finite-impulse-response (FIR) digital filters, which perform polarization demultiplexing, polarization-mode dispersion (PMD) compensation, residual CD compensation, and ISI mitigation.

As the butterfly equalizer is dynamically adjusted, it is harder to implement and usually much smaller than the CD equalizer in terms of the FIR tap numbers or FDE overlap lengths. Optimum performance can be achieved if the adaptive equalizer can converge to a matched filter.

For a Nyquist signal with a smaller roll-off factor, a larger number of taps are required for an FIR filter-based adaptive equalizer, as shown in Figure 4.12, which reports the plots of the impulse response of SRRC filters for different values of roll-off. The length of the impulse response in time increases when the roll-off decreases.

It is shown that incorporating a matched filter in the bulk CD equalizer for an SRRC-shaped signal can significantly reduce the complexity of the blind equalizer, with no additional complexity added to the CD equalizer.

It is also shown that Nyquist-WDM systems with matched filtering are sensitive to the frequency offset between the Tx laser and the LO, and that the induced penalty decreases with increased SRRC roll-off factor.

## 4. Practical Nyquist-WDM Transmitter Implementations

As shown in Section 2.1, the ideal shape of the transmitted spectra, which allows to achieve a channel spacing equal to the symbol rate \(R_s\), is rectangular with bandwidth equal to \(R_s\). In such an ideal scenario, Nyquist-WDM can achieve the optimum matched filter performance in AWGN systems.

In practice, penalties are to be expected when the ideal constraints on Nyquist WDM implementation are relaxed, like for instance with the transmission of channels with not perfectly rectangular spectra.

The key-component in the generation of Nyquist-WDM signals is the “Nyquist filter”, which performs a tight spectral shaping on the generated signals at the Tx side in order to obtain an almost rectangular spectrum.

Figure 4.13 shows the shape of the Nyquist filter needed to transform an ideal nonreturn-to-zero (NRZ) pulse, characterized by a rectangular pulse in the time domain with length equal to \(T\), into a Nyquist spectrum with SRRC shape, for different values of roll-off.

If \(H_\text{NRZ}(f)\) is the Fourier transform of the NRZ pulse, the transfer function of the Nyquist filter is obtained as the product between the SRRC transfer function \(\sqrt{H_\text{RC}(f)}\) (see Eq. 4.12) and the inverse of \(H_\text{NRZ}(f)\):

\[\tag{4.19}H(f)=\sqrt{H_\text{RC}(f)}\frac{\pi{f}T}{\sin(\pi{f}T)}\]

The quasi-rectangular subchannel spectral shaping needed for Nyquist-WDM can be obtained either by band-limiting the signal coming out of each transmitter through an optical filter, as schematically shown in Figure 4.14(a), or by driving the electro-optical modulator with suitable electrical signals so that the optical modulated signal takes on the desired spectral shape, as shown in Figure 4.14(b).

The two techniques are known as optical Nyquist-WDM and digital Nyquist-WDM, respectively. The latter technique requires digital-to-analog-converters (DACs) to generate the electrical driving signals.

Early demonstrations of near-Nyquist channel spacing were performed by approximating the desired rectangular spectral shape through the use of optical filters (see Section 4.1). More recently, several experimental demonstrations appeared based on digital Nyquist-WDM generation and transmission (see Section 4.2).

In the following, the characteristics of Nyquist-WDM signals generated through spectral shaping in either optical or electrical/digital domain are analyzed and discussed, taking into account the implementation nonidealities of state-of-the-art components.

### 4.1. Optical Nyquist-WDM

The Tx setup for Nyquist-WDM signal generation using spectral shaping in the optical domain is shown in Figure 4.15. Each of the \(N\) transmitters generates a modulated optical signal with NRZ spectral shape. An array of optical filters is then used to transform the NRZ shape into a quasi-rectangular one. The \(N\) Nyquist signals are then wavelength multiplexed, generating the overall Nyquist-WDM spectrum.

Successful ultra-long-haul experiments exploiting Nyquist-WDM, based on BPSK, QPSK, and 8QAM modulation format (with polarization domain multiplexing) using tight optical filtering at the Tx side, have been performed (see Section 5.1).

In all experiments, the ideal optical filter shape was approximated using state-of-the art components, such as the Finisar Waveshaper filter. In addition to standard filtering, the Waveshaper allows a high-frequency pre-emphasis to be introduced in the optical spectrum of each channel in order to better approximate the Nyquist filter transfer function.

Figure 4.16 shows the transfer function of the optical shaping filter in four different cases: Finisar filter w/o pre-enhancement, super-Gaussian (SG) filter, which better approximates the Finisar filter transfer function, Finisar filter with pre-enhancement, and SRRC filter shape with \(\rho=0.1\).

Clearly, the main limitation of the Waveshaper component is the fact that its profile is not particularly steep (approximately second-order super-Gaussian) with respect to the ideal one. This introduces linear crosstalk between adjacent subchannels, inducing a penalty in the back-to-back transceiver performance.

The impact of such a nonideal filtering on system performance was investigated by simulations, showing that the constraints on the steep spectral shaping requested to satisfy the orthogonality condition and to minimize crosstalk between the subcarriers can be relaxed by increasing the channel spacing, at the expenses of a little loss in spectral efficiency.

As an example, in Figure 4.17 the SNR needed to achieve a target BER equal to \(4\times10^{-3}\) is shown as a function of the normalized frequency spacing. The modulation format is PDM-QPSK. Since the profile of the realistic filters is less steep than the ideal filter (see Figure 4.16), substantial crosstalk would occur at symbol rate spacing. Penalties can, however, be canceled by increasing the subchannel spacing.

A transmission of a 96 × 128 Gbit/s PDM-QPSK quasi-Nyquist WDM comb over 11,680 km was demonstrated, with a channel spacing equal to \(1.17\cdot{R_s}\). A programmable optical filter was used, engineering the top of the intensity response in order to follow a quadratic intensity profile of variable depth in decibels.

The narrow profile of the standard flat-top filter was found to be responsible for 0.5-dB penalty with respect to the single-channel case without optical filtering. However, engineering the spectral response according to a quadratic profile with variable depth, which enhances the power of the spectral components that are farther from the carrier frequency, yielded a significant performance improvement, with the Q-factor becoming larger than in the case of single channel without optical filter.

The optimum depth (6 dB) was found to be identical in the back-to-back configuration and after transmission. The improvement with respect to the single-channel case without filtering was enabled by the fact that, in addition to narrow spectral shaping, the programmable filtering device allows a high-frequency pre-emphasis to be introduced in the optical spectrum of each channel in order to precompensate for Tx and Rx electrical bandwidth limitations.

Using a similar technique, the transmission of a 10 × 120 Gbit/s PDM-QPSK Nyquist-WDM signal over 9000 km was demonstrated, with a channel spacing equal to the symbol rate.

Even though several record experiments have been performed in the past years using optical spectral shaping on PDM-QPSK signals, the difficulty in applying the technique to higher-order modulation formats was evident.

Quasi-Nyquist-WDM signals were generated using a PDM-8QAM modulation, but the normalized frequency spacing had to be kept as high as \(1.22\cdot{R_s}\) in order not to incur in substantial crosstalk penalty.

In practice, the main drawback in performing spectral shaping in the optical domain is the need of optical filters with very steep transfer functions, with the requirements on tight filtering becoming more stringent with the increase of the modulation cardinality.

A possible solution is to perform the spectral shaping in the electrical/digital domain by using DACs. In such a way, it is possible to accurately “design” through DSP the signal spectrum at the output of the DAC and ideally obtain a perfectly square spectrum, as shown in the following section.

### 4.2. Digital Nyquist-WDM

In the following, it is shown that very accurate spectral shaping can potentially be performed in the digital domain, with the main limitations being the sampling speed of the DAC and the availability of suitable analog antialiasing filters. The fundamental results from the signal theory are first reviewed and then applied to the case of optical transmission.

The “Nyquist sampling theorem” states that any analog signal \(x(t)\), band-limited in \([−W,W]\), can be perfectly reconstructed from its samples provided that the sampling frequency \(f_s\) is greater than \(2\cdot{W}\).

Figure 4.18 shows the schematics of an ideal digital-to-analog (D/A) conversion process. Ideally, to generate a perfectly rectangular Nyquist-spectrum with bandwidth equal to the symbol rate \(R_s\), a DAC is needed operating at a speed equal to \(R_s\) samples/s and with a perfectly rectangular transfer function with bandwidth \(0.5\cdot{R_s}\).

On the other hand, today commercial DACs are characterized by a transfer function which is far from rectangular, which makes it not possible to perform Nyquist spectral shaping to generate signals with a symbol rate equal to the DAC sampling speed.

The scheme of a realistic D/A conversion process used to generate each quadrature of a Nyquist-WDM signal is shown in Figure 4.19. The DAC can be modeled as a sample&hold (S&H) device, which generates a “step” function whose levels correspond to the samples of the ideal signal, followed by an interpolating filter, used to reconstruct the original signal.

The evolution of the signal spectrum is shown in Figure 4.20(a)–(d) for the case \(f_s=2\cdot{R_s}\) and in Figure 4.20(e) and (f) for the case \(f_s=1.5\cdot{R_s}\). The presence in the generated spectrum of spurious frequencies is due to non-perfectly-rectangular antialiasing filtering (the transfer function of the DAC is shown in plot (d) and (f) as dashed line).

A certain amount of ISI, introduced by both the interpolating filter and the S&H device, is also present, resulting in a non-flat spectrum in Figures 4.20(c)–(f). ISI can be partially mitigated by performing a pre-enhancement on the signal samples generated in DSP, while crosstalk can be canceled by adding a steep analog antialias filter at the output of the DAC.

A realistic DAC device is characterized by two main parameters:

- The sampling speed \(f_\text{DAC}\), which limits the achievable symbol rate \(R_s=f_\text{DAC}/N_\text{SpS}\), where \(N_\text{SpS}\) is the number of samples per symbol (also indicated as “oversampling factor”). State-of-the art DACs are characterized by a maximum sampling speed around 34 Gsamples/s.
- The number of resolution bits \(N_\text{DAC}\), which limits the cardinality of the modulation format. In fact, the higher is the order of the modulation format, the higher is the required value of \(N_\text{DAC}\).

Typically if \(f_\text{DAC}\) increases, \(N_\text{DAC}\) decreases. The achievable symbol rate can clearly be increased by decreasing the oversampling factor. In doing so, penalties could be incurred due to interference produced by spectral replica of the useful spectrum in the DAC process.

In one experiment, a 1.5 SpS DAC-supported Nyquist-WDM PDM-16QAM experiment was reported, using a DAC with \(f_\text{DAC}\) = 23.4 GHz and thus achieving a symbol rate \(R_s=15.6\) Gbaud. In another experiment, 1.33 SpS were employed in a 100-km PDM-64QAM single-channel transmission at 252 Gbit/s.

In one experiment, an oversampling factor as low as 1.15 was used, limiting the penalty due to spectrum replica thanks to the use of ad hoc antialiasing electrical filters. The modulation format was 10.4-Gbaud PDM-64QAM. The filtered eight-level signals driving the IQ modulator were generated by the DACs operating at 11.96 GS/s.

As a result, an alias signal replica was also generated, centered at 11.96 GHz, as shown in Figure 4.21(a). The replica was partially filtered out by the low-pass frequency response of the DAC, but to suppress it to the extent of making it negligible, a specifically designed antialiasing filter, with steep cutoff, was interposed between the DAC and the modulators. The antialiasing filter frequency response and its output are shown in Figure 4.21(a) and (b), respectively.

In the following, a set of simulation results is reported showing a comparison between digital and optical spectral shaping. Both PDM-QPSK and PDM-16QAM signaling are considered.

The use of a DAC working at 24 Gsamples/s with bandwidth equal to 9.6 GHz is assumed. An oversampling factor equal to 2 was used, yielding a value of symbol rate equal to 12 Gbaud. The DAC transfer function was modeled as a two-pole filter with dumping factor equal to \(\sqrt{2}/2\), which is a good approximation of the profile of state-of-the-art commercial devices.

The modulator driving voltages were optimized and a proper pre-enhancement was applied to the digital samples in order to compensate for both the interpolating filter and the S&H process. For the Nyquist filter generation, instead of an ideal square shape in [0, \(R_s/2\)], a more realistic raised-cosine shape with roll-off 0.15 was used.

In case of optical shaping, a fourth-order super-Gaussian shaping filter was assumed. The back-to-back performance is shown in Figure 4.22. The channel spacing, equal to Rs for PDM-QPSK, was increased to \(1.1\cdot{R_s}\) for PDM-16QAM in order to limit the crosstalk penalty, which for this format can heavily affect the performance.

The advantage of digital over optical spectral shaping highlighted by the results of Figure 4.22 is mainly due to the ability to precisely control the Tx spectrum to a degree far beyond what it possible with practical analog optical filters.

An additional advantage of a DAC-based Tx is the ability to compensate for the linear transfer function of the analog components of the Tx and Rx, thus relaxing the requirements for these components. Compensation is typically performed by first measuring the transfer function between the desired waveform and the optical output. This transfer function is then inverted and included in the digital filtering performed at the Tx. This is very attractive since the linear transfer function of the required equalization can be combined with the spectral shaping filter without increasing the ASIC resources needed to perform the filtering.

Nyquist pulse shaping in the digital domain is typically performed using FIR filters. The required length of the digital filter needed to generate SRRC pulses increases when the roll-off decreases (see Figure 4.12).

Complexity and performance of digital pulse shaping has been investigated, showing that FIR filters with 17 taps allow for a reduction in channel spacing to 1.1 the symbol rate within a 1-dB penalty. A higher number of FIR filter taps are needed for tighter channel spacing.

Using a 600-tap FIR filter to perform digital preshaping, the generation of 20 Gbit/s QPSK WDM signals with \(\Delta{f}=R_s=10\) GHz was demonstrated without back-to-back penalty. A twofold oversampling was used in the DAC.

A real-time demonstration of generation of Nyquist-like pulses with 14-Gbaud PDM-16QAM was reported, showing that the use of a 32-tap FIR filter to shape the signals would allow a channel spacing equal to \(1.06\cdot{R_s}\) without substantial crosstalk penalty.

## 5. Nyquist-WDM Transmission

The transmission performance of Nyquist-WDM systems has been extensively studied both experimentally (see Sections 5.1 and 5.2) and numerically.

Based on the GN-model for nonlinear propagation in uncompensated coherent optical systems, the relationship between the SE and total link length for Nyquist-WDM or quasi-Nyquist-WDM signals in arbitrary transmission scenarios can be derived.

In this section, this possibility is exemplified by analyzing PDM-QAM formats in two different multispan link scenarios with erbium-doped fiber amplifier (EDFA) amplification (noise figure \(F\) = 5 dB): a terrestrial link over standard single-mode fiber (SSMF) with 100-km span length and a submarine link over pure silica-core fiber (PSCF) with 60-km span length. The parameters of the fibers are shown in Table 4.1.

The WDM signal is assumed to occupy the entire C-band (\(B_\text{WDM}\) =5 THz) with a spacing among WDM subchannels equal to 1.1 \(R_s\), that is, the lowest value assuring the absence of any linear crosstalk for RC spectra with roll-off 0.1.

In order to analyze a realistic scenario, it is assumed to operate with a conservative 2-dB margin with respect to the ideal BER-versus-SNR performance and with a realistic soft FEC 1.5-dB penalty with respect to ideal soft FEC performance.

The dependence of the SE on total link length is plotted in Figure 4.23 for PDM-QAM modulation formats with cardinality ranging from 4 to 64.

The results of Figure 4.23 clearly highlight the trade-off between distance and SE, in relation to the different modulation formats: increasing the cardinality of the constellation, a higher SE can be achieved, but typically over a shorter transmission distance and/or at a higher required FEC overhead.

State-of-the-art soft FEC with 20% overhead can now operate at pre-FEC BER of \(2.7\times10^{-2}\): so, for all modulation formats, the points corresponding to BER=\(2.7\times10^{-2}\) are marked in the figures. The section of the SE lines to the left of the dots is, therefore, the “practicable” section, whereas moving to the right will be possible only if better FECs become available.

Considering the currently possible systems, in a terrestrial link with EDFA-only amplification, PDM-QPSK is the best choice for ultra-long-haul transmissions beyond 5000 km, while PDM-8QAM can be used to achieve an SE around 5–5.5 bit/s/Hz in 3000 km links.

PDM-16QAM allows to reach 2000 km with an SE of 6 bit/s/Hz. The reach of higher-order modulation formats, such as PDM-32QAM and PDM-64QAM, is very limited in this kind of systems, but can be significantly increased by using new generation fibers and shorter span length, similar to that in the analyzed submarine-like system over PSCF, where they reach 4000 and 2000 km, respectively.

The plot also shows that PDM-16QAM could reach ultra-long-haul distances over submarine-like links. Note that this reach can be further increased by using better-performing fibers, higher-performance FEC, and nonlinearity compensation techniques at the Rx, where 10,000 km could be achieved at an SE of 6 bit/(s Hz).

### 5.1. Optical Nyquist-WDM Transmission Experiments

In Table 4.2, experimental demonstrations of long-haul transmission based on super-Nyquist-WDM, Nyquist-WDM, and quasi-Nyquist-WDM using optical shaping at the Tx are listed. The most commonly used modulation format was PDM-QPSK, with a symbol rate in the order of 30 Gbaud. Only one demonstration is present with a higher-order modulation (PDM-8QAM), achieving the highest value of SE (4.1 bit/s/Hz), but with a limited reach (4000 km).

In some experiments, the transmission of 96×100 Gbit/s PDM-QPSK super-Nyquist-WDM signal was demonstrated over 4730 km with a sub-Nyquist channel spacing equal to \(0.89\cdot{R_s}\). The individual subchannels were bandwidth constrained by applying an aggressive prefiltering at the Tx: this created a significant ISI that produced a complex signal constellation (composed of a total of 36 points). Using a 5-tap MAP algorithm at the Rx side, ISI was efficiently removed and the constellation was recovered back to that of a typical QPSK signal.

A similar technique to mitigate ISI was used to reach a record length of 11,860 km. With the use of symbol-by-symbol detection, the maximum system length at the same BER was around 9000 km. The reduced performance was due to the fact that the adaptive digital equalizer, while compensating for the ISI introduced by strong filtering, simultaneously enhanced the high-frequency noise, thus inducing a significant SNR penalty. With a sub-Nyquist spacing equal to \(1.2\cdot{R_s}\), more than 15,000 km could be reached without the need of multisymbol detection.

### 5.2. Digital Nyquist-WDM Transmission Experiments

In Table 4.3, experimental demonstrations of long-haul transmission based on Nyquist-WDM and quasi-Nyquist-WDM using digital shaping at the Tx are listed. The modulation formats range from PDM-16QAM to PDM-64QAM.

Thanks to digital spectral shaping, values of frequency spacing very close to the symbol rate could be used in all experimental demonstrations. Typically, SRRC shapes were generated, with roll-off values ranging from 0.001 to 0.05.

In one setup, the Nyquist-WDM approach was used together with hybrid time-domain QAM modulation. A net SE of 8.25 bit/s/Hz was achieved interleaving PDM-32QAM and PDM-64QAM symbols in the time domain. Digital spectral shaping was used to achieve a frequency spacing as low as \(1.02\cdot{R_s}\).

In another setup, it is shown that time-domain hybrid QAM, together with digital spectral shaping, provides a new degree of design freedom to optimize the transmission performance by fine tuning the SE of the modulation format for a specific channel bandwidth and FEC redundancy requirement.

In another one, Nyquist-WDM based on time-domain hybrid QAM is indeed proposed as an enabling technology for future elastic optical networks, thanks to its improved ability to optimize system SE as a function of optical channel conditions compared with conventional individual format-based transceivers.

## 6. Conclusions

The principles and recent developments of Nyquist signaling in coherent optical transmission systems have been reviewed. The generation and detection of Nyquist-WDM signals have been discussed, together with the main experimental demonstrations of long-haul transmission. It was shown that, with either optical or electrical spectral shaping, WDM channel spacing equal or close to the symbol rate has been achieved, significantly outperforming unfiltered NRZ in terms of spectral efficiency.

The main drawback when performing spectral shaping in the optical domain is the need for analog filters with very steep profile, which are critical to design and build using state-of-the art technology. This limits the achievable spectral efficiency, especially when using higher-order modulation formats.

On the contrary, one of the main advantages in performing spectral shaping in the digital domain is the ability to precisely control the Tx spectrum to a degree far beyond what it possible with practical analog filters, which enables the generation of closely spaced Nyquist-WDM combs with channel spacing equal (or very close) to the symbol rate, even with high-cardinality constellations.

The main limitation of digital spectral shaping is the finite sampling speed of state-of-the-art DACs, which limits the achievable symbol rate. Also, an advantage of optical spectral shaping compared with digital pulse shaping is the fact that costly DACs are not required and power consumption can be significantly reduced.

Finally, a major advantage of digital spectral shaping through DSP and DACs over analog optical shaping is its high flexibility: in fact, keeping the symbol rate fixed, the same Tx hardware can be used to generate different modulation formats, and thus achieve different line rates. This will enable software-defined optical transmission, based on the optimization of the channel throughput depending on the link conditions.

In conclusion, recent experimental demonstrations confirmed that the Nyquist-WDM concept is a promising technology for ultrahigh spectral density long-haul links and that, exploiting state-of-the-art DAC and DSP technology for digital spectral shaping, it is a good candidate for next generation flexible optical networks.

The next tutorial gives a complete ** introduction to miniature and micro-optics**.