Johnson, Don H.2009-06-042009-06-042000Sinanovic, Sinan. "Toward a theory of information processing." (2000) Master’s Thesis, Rice University. <a href="https://hdl.handle.net/1911/17380">https://hdl.handle.net/1911/17380</a>.https://hdl.handle.net/1911/17380Information processing is performed when a system preserves some aspects of the information encoded by its input while it suppresses others. To describe a system's information processing capability, input and output need to be compared in a way invariant to the signal's form and how it represents information. We describe an approach to quantify information processing based on applying controlled changes to the input and observing the corresponding output. Information-theoretic distance measures---those that reflect the data processing theorem---are calculated on the input and output separately and compared. The best candidate for the distance measure to describe signal processing is the information theoretic distance measure called the Kullback-Leibler distance (also known as the relative entropy). Properties of the resulting information transfer ratio are used to derive fundamental information processing properties of systems and interconnected systems.35 p.application/pdfengCopyright is held by the author, unless otherwise indicated. Permission to reuse, publish, or reproduce the work beyond the bounds of fair use or other exemptions to copyright law must be obtained from the copyright holder.ElectronicsElectrical engineeringThesisTHESIS E.E. 2000 SINANOVIC