1,721,104 research outputs found
Saddle Point in the Minimax Converse for Channel Coding
Get PDFA minimax metaconverse has recently been proposed as a simultaneous generalization of a number of classical results and a tool for the nonasymptotic analysis. In this paper, it is shown that the order of optimizing the input and output distributions can be interchanged without affecting the bound. In the course of the proof, a number of auxiliary results of separate interest are obtained. In particular, it is shown that the optimization problem is convex and can be solved in many cases by the symmetry considerations. As a consequence, it is demonstrated that in the latter cases, the (multiletter) input distribution in information-spectrum (Verdú-Han) converse bound can be taken to be a (memoryless) product of single-letter ones. A tight converse for the binary erasure channel is rederived by computing the optimal (nonproduct) output distribution. For discrete memoryless channels, a conjecture of Poor and Verdú regarding the tightness of the information spectrum bound on the error exponents is resolved in the negative. Concept of the channel symmetry group is established and relations with the definitions of symmetry by Gallager and Dobrushin are investigated.National Science Foundation (U.S.) (Center for Science of Information, under Grant CCF-0939370
Asynchronous Communication: Exact Synchronization, Universality, and Dispersion
Get PDFRecently, Tchamkerten and coworkers proposed a novel variation of the problem of joint synchronization and error correction. This paper considers a strengthened formulation that requires the decoder to estimate both the message and the location of the codeword exactly. Such a scheme allows for transmitting data bits in the synchronization phase of the communication, thereby improving bandwidth and energy efficiencies. It is shown that the capacity region remains unchanged under the exact synchronization requirement. Furthermore, asynchronous capacity can be achieved by universal (channel independent) codes. Comparisons with earlier results on another (delay compensated) definition of rate are made. The finite blocklength regime is investigated and it is demonstrated that even for moderate blocklengths, it is possible to construct capacity-achieving codes that tolerate exponential level of asynchronism and experience only a rather small loss in rate compared to the perfectly synchronized setting; in particular, the channel dispersion does not suffer any degradation due to asynchronism. For the binary symmetric channel, a translation (coset) of a good linear code is shown to achieve the capacity-synchronization tradeoff.National Science Foundation (U.S.) (Center for Science of Information Grant CCF-0939370
Dissipation of Information in Channels With Input Constraints
No full textOne of the basic tenets in information theory, the data processing inequality states that the output divergence does not exceed the input divergence for any channel. For channels without input constraints, various estimates on the amount of such contraction are known, Dobrushin's coefficient for the total variation being perhaps the most well-known. This paper investigates channels with an average input cost constraint. It is found that, while the contraction coefficient typically equals one (no contraction), the information nevertheless dissipates. A certain nonlinear function, the Dobrushin curve of the channel, is proposed to quantify the amount of dissipation. Tools for evaluating the Dobrushin curve of additive-noise channels are developed based on coupling arguments. Some basic applications in stochastic control, uniqueness of Gibbs measures, and fundamental limits of noisy circuits are discussed. As an application, it is shown that, in the chain of n power-constrained relays and Gaussian channels, the end-to-end mutual information and maximal squared correlation decay as O(log log n/log n), which is in stark contrast with the exponential decay in chains of discrete channels. Similarly, the behavior of noisy circuits (composed of gates with bounded fan-in) and broadcasting of information on trees (of bounded degree) does not experience threshold behavior in the signal-to-noise ratio (SNR). Namely, unlike the case of discrete channels, the probability of bit error stays bounded away from 1/2 regardless of the SNR
Upper bound on list-decoding radius of binary codes
No full textConsider the problem of packing Hamming balls of a given relative radius subject to the constraint that they cover any point of the ambient Hamming space with multiplicity at most L. For odd L ≥ 3 an asymptotic upper bound on the rate of any such packing is proven. The resulting bound improves the best known bound (due to Blinovsky' 1986) for rates below a certain threshold. The method is a superposition of the linear- programming idea of Ashikhmin, Barg and Litsyn (that was used previously to improve the estimates of Blinovsky for L = 2) and a Ramsey-theoretic technique of Blinovsky. As an application it is shown that for all odd L the slope of the rate-radius tradeoff is zero at zero rate.National Science Foundation (U.S.) (Grant CCF-13-18620)National Science Foundation (U.S.). Science and Technology Center (Grant CCF-09-39370
Hypothesis testing via a comparator
Get PDFThis paper investigates the best achievable performance by a hypothesis test satisfying a structural constraint: two functions are computed at two different terminals and the detector consists of a simple comparator verifying whether the functions agree. Such tests arise as part of study of fundamental limits of channel coding, but are also useful in other contexts. A simple expression for the Stein exponent is found and applied to showing a strong converse in the problem of multi-terminal hypothesis testing with rate constraints. Connections to the Gács-Körner common information and to spectral properties of conditional expectation operator are identified. Further tightening of results hinges on finding λ-blocks of minimal weight. Application of Delsarte's linear programming method to this problem is described.Center for Science of Information (Grant Agreement CCF-09-39370
ℓp-norms of codewords from capacity- and dispersion-achieveing Gaussian codes
Get PDFIt is demonstrated that codewords of good codes for the additive white Gaussian noise (AWGN) channel become more and more isotropically distributed (in the sense of evaluating quadratic forms) and resemble white Gaussian noise (in the sense of ℓp norms) as the code approaches closer to the fundamental limits. In particular, it is shown that the optimal Gaussian code must necessarily have peak-to-average power ratio (PAPR) of order log n.National Science Foundation (U.S.) (Center for Science of Information Grant CCF-0939370
Peak-to-Average Power Ratio of Good Codes for Gaussian Channel
Get PDFConsider a problem of forward error-correction for the additive white Gaussian noise (AWGN) channel. For finite blocklength codes, the backoff from the channel capacity is inversely proportional to the square root of the blocklength. In this paper, it is shown that the codes achieving this tradeoff must necessarily have peak-to-average power ratio (PAPR) proportional to logarithm of the blocklength. This is extended to codes approaching capacity slower, and to PAPR measured at the output of an orthogonal frequency division multiplexing modulator. As a by-product, the convergence of (Smith's) amplitude-constrained AWGN capacity to Shannon's classical formula is characterized in the regime of large amplitudes. This converse-type result builds upon recent contributions in the study of empirical output distributions of good channel codes.National Science Foundation (U.S.). Center for Science of InformationNational Science Foundation (U.S.). Science and Technology Center (Grant CCF-0939370)National Science Foundation (U.S.) (CAREER Award CCF-12-53205
On asynchronous capacity and dispersion
Get PDFRecently Tchamkerten et al. proposed a mathematical formulation of the problem of joint synchronization and error-correction in noisy channels. A variation of their formulation in this paper considers a strengthened requirement that the decoder estimate both the message and the location of the codeword exactly. It is shown that the capacity region remains unchanged and that the strong converse holds. The finite blocklength regime is investigated and it is demonstrated that even for moderate blocklengths, it is possible to construct capacity-achieving codes that tolerate exponential level of asynchronism and experience only a rather small loss in rate compared to the perfectly synchronized setting; in particular, the channel dispersion does not suffer any degradation due to asynchronism
On dispersion of compound DMCs
Get PDFCode for a compound discrete memoryless channel (DMC) is required to have small probability of error regardless of which channel in the collection perturbs the codewords. Capacity of the compound DMC has been derived classically: it equals the maximum (over input distributions) of the minimal (over channels in the collection) mutual information. In this paper the expression for the channel dispersion of the compound DMC is derived under certain regularity assumptions on the channel. Interestingly, dispersion is found to depend on a subtle interaction between the channels encoded in the geometric arrangement of the gradients of their mutual informations. It is also shown that the third-order term need not be logarithmic (unlike single-state DMCs). By a natural equivalence with compound DMC, all results (dispersion and bounds) carry over verbatim to a common message broadcast channel.National Science Foundation (U.S.) (CAREER Award CCF-12-53205)National Science Foundation (U.S.). Center for Science of Information (Grant Agreement CCF-0939370
- …
