Browsing by Author "Sinanovic, Sinan"

Now showing 1 - 5 of 5
  • Thumbnail Image
    Item
    Asymptotic rates of the information transfer ratio
    (2002-05-20) Sinanovic, Sinan; Johnson, Don; Digital Signal Processing (http://dsp.rice.edu/)
    Information processing is performed when a system preserves aspects of the input related to what the input represents while it removes other aspects. To describe a system's information processing capability, input and output need to be compared in a way invariant to the way signals represent information. Kullback-Leibler distance, an information-theoretic measure which reflects the data processing theorem, is calculated on the input and output separately and compared to obtain information transfer ratio. We consider the special case where input serves several parallel systems and show that this configuration has the capability to represent the input information without loss. We also derive bounds for asymptotic rates at which the loss decreases as more parallel systems are added and show that the rate depends on the input distribution.
  • Thumbnail Image
    Item
    Limits of acoustic waveguide communication
    (2006) Sinanovic, Sinan; Johnson, Don H.
    A new method of wireless data telemetry in oil well services uses compressional acoustic waves to transmit data along the drill string. The compressional acoustic waves are produced as coded wave trains by an acoustic transducer, travel through the drill string and subsequently decoded to recover the data. Normal drilling operations produce in-band acoustic noise from multiple sources at intensities comparable to the transducer output. Based on a theoretical channel model, we predict that the drill string acoustic channel has a capacity of several hundreds bits per second in such noisy drilling conditions. In making these calculations, the surface noise is shown to be the limiting factor. We explore methods of improving channel capacity by exploiting upward and downward propagation modes. Simulations based on our acoustic model show that substantial (an order of magnitude or more) capacity increases can result if we can cancel the noise-laden downward mode. We show that our two-receiver cancellation algorithm can be derived with a nonparametric, training-based approach that assumes little about the acoustic model. Finally, we investigate the effects of noise attenuators on capacity.
  • Thumbnail Image
    Item
    Symmetrizing the Kullback-Leibler Distance
    (2001-03-20) Johnson, Don; Sinanovic, Sinan; Digital Signal Processing (http://dsp.rice.edu/)
    We define a new distance measure - the resistor-average distance - between two probability distributions that is closely related to the Kullback-Leibler distance. While the Kullback-Leibler distance is asymmetric in the two distributions, the resistor-average distance is not. It arises from geometric considerations similar to those used to derive the Chernoff distance. Determining its relation to well-known distance measures reveals a newway to depict how commonly used distance measures relate to each other.
  • Thumbnail Image
    Item
    Toward a theory of information processing
    (2000) Sinanovic, Sinan; Johnson, Don H.
    Information processing is performed when a system preserves some aspects of the information encoded by its input while it suppresses others. To describe a system's information processing capability, input and output need to be compared in a way invariant to the signal's form and how it represents information. We describe an approach to quantify information processing based on applying controlled changes to the input and observing the corresponding output. Information-theoretic distance measures---those that reflect the data processing theorem---are calculated on the input and output separately and compared. The best candidate for the distance measure to describe signal processing is the information theoretic distance measure called the Kullback-Leibler distance (also known as the relative entropy). Properties of the resulting information transfer ratio are used to derive fundamental information processing properties of systems and interconnected systems.
  • Thumbnail Image
    Item
    Toward a theory of information processing
    (2004-06-01) Sinanovic, Sinan; Johnson, Don; Digital Signal Processing (http://dsp.rice.edu/)
    Information processing theory endeavors to quantify how well signals encode information and how well systems, by acting on signals, process information. We use information-theoretic distance measures, the Kullback-Leibler distance in particular, to quantify how well signals represent information. The ratio of distances calculated between two informationally different signals at a system's output and input quantifies the system's information processing properties. Using this approach, we derive the fundamental processing capabilities of simple system architectures that apply universally: the systems and the kinds of signals they process and produce don't affect our general results. Applications in array signal processing and in neural signal analysis illustrate how to apply the theory.