Self-interference cancelation in the presence of non-linear power amplifier and receiver IQ imbalance

In the in-band full-duplex(FD) systems, the self-interference (SI) power can be more than 100 dB higher than the power of the received data signal. In order to enable the FD transmission, several SI cancelation stages are needed in a FD transceiver. By combining the cancelation at the radio frequency (RF) with a specially designed antenna and cancelation circuitry and SI cancelation at the digital baseband, the required level of SI cancelation can be achieved even with a non-linear power amplifier. In this paper, a FD transceiver architecture is modeled with simulation tools that allow to use realistic antenna and analog transceiver models and at the same time enable algorithm studies. The analog SI cancelation at the RF is controlled by the baseband digital processing unit, and the tuning of the RF canceler is performed with an automatic gain control enhanced iterative algorithm. The combined cancelation performance of the antenna and RF canceler varies between 62 and 82 dB depending on the studied cases. The digital baseband SI cancelation is based on the Hammerstein model in order to take the power amplifier non-linearity into account. The coefficients of the Hammerstein model are estimated with a self-orthogonalizing adaptive algorithm. When realistic phase noise and IQ imbalance values are taken into account, the SI after all the cancelation stages can decrease the signal-to-interference-and-noise-ratio (SINR) by few decibels (dB). In order to further enhance the SI cancelation, the Hammerstein based SI canceler is extended to cancel also the effect of the receiver IQ imbalance. With the extended baseband canceler, the cancelation performance is mainly limited by the phase noise.