Information Theory Books

Canon EOS 40D Guide to Digital Photography

David D. Busch

Amazon Price: $19.79 List Price: $29.99 Usually ships in 24 hours By: Course Technology PTR Amazon Marketplace: 32 new & used starting at $18.80

Buy at Amazon.com

Browse similar items by category:
Subjects -> Arts & Photography -> Photography -> Digital Photography
Subjects -> Computers & Internet -> Computer Science -> Information Theory
Subjects -> Computers & Internet -> Digital Photography & Video -> Camera Guides

Customer Reviews:
Total reviews: 73 Average rating: 4.5 of 5

Editorial Review:

As the new owner of Canon's most advanced intermediate digital SLR, you want to get started taking professional-looking photographs using all of the exciting features at your fingertips. "Canon EOS 40D Guide to Digital SLR Photography" is a concise introduction and guide to your camera's essential controls and functions, such as Live View, built-in dust reduction, and the blistering 6.5-frames-per-second continuous shooting mode that is an action photographer’s dream. The book provides detailed instructions showing you how, when, and why to make optimized settings with the Canon EOS 40D’s enhanced menus, which include a half-dozen versatile new custom functions. You'll learn about the camera's improved automatic focus, flash synchronization tricks, how to choose lenses that will provide the perspective and effects you want, and which exposure modes are ideal for each picture-taking opportunity. Packed with full-color images and examples that illustrate the recommended techniques and settings for your Canon EOS 40D, this book helps photographers of any skill level begin maximizing their equipment as soon as you open the cover!

The Big Switch: Rewiring the World, from Edison to Google

Nicholas Carr

Amazon Price: $17.13 List Price: $25.95 Usually ships in 24 hours By: W. W. Norton Amazon Marketplace: 54 new & used starting at $12.97

Buy at Amazon.com

Browse similar items by category:
Subjects -> Computers & Internet -> Business & Culture -> Culture
Subjects -> Computers & Internet -> Computer Science -> Information Theory
Subjects -> Computers & Internet -> General

Customer Reviews:
Total reviews: 33 Average rating: 3.5 of 5

Editorial Review:

An eye-opening look at the new computer revolution and the coming transformation of our economy, society, and culture.

A hundred years ago, companies stopped producing their own power with steam engines and generators and plugged into the newly built electric grid. The cheap power pumped out by electric utilities not only changed how businesses operated but also brought the modern world into existence. Today a similar revolution is under way. Companies are dismantling their private computer systems and tapping into rich services delivered over the Internet. This time it's computing that's turning into a utility. The shift is already remaking the computer industry, bringing new competitors like Google to the fore and threatening traditional stalwarts like Microsoft and Dell. But the effects will reach much further. Cheap computing will ultimately change society as profoundly as cheap electricity did. In this lucid and compelling book, Nicholas Carr weaves together history, economics, and technology to explain why computing is changing—and what it means for all of us.

Emergence: The Connected Lives of Ants, Brains, Cities, and Software

Steven Johnson

Amazon Price: $10.88 List Price: $16.00 Usually ships in 24 hours By: Scribner Amazon Marketplace: 49 new & used starting at $6.91

Buy at Amazon.com

Browse similar items by category:
Subjects -> Computers & Internet -> Computer Science -> Information Theory
Subjects -> Nonfiction -> Social Sciences -> Sociology -> Urban
Subjects -> Professional & Technical -> Engineering -> Civil -> Environmental -> General

Customer Reviews:
Total reviews: 83 Average rating: 3.5 of 5

Editorial Review:

An individual ant, like an individual neuron, is just about as dumb as can be. Connect enough of them together properly, though, and you get spontaneous intelligence. Web pundit Steven Johnson explains what we know about this phenomenon with a rare lucidity in Emergence: The Connected Lives of Ants, Brains, Cities, and Software. Starting with the weird behavior of the semi-colonial organisms we call slime molds, Johnson details the development of increasingly complex and familiar behavior among simple components: cells, insects, and software developers all find their place in greater schemes.

Most game players, alas, live on something close to day-trader time, at least when they're in the middle of a game--thinking more about their next move than their next meal, and usually blissfully oblivious to the ten- or twenty-year trajectory of software development. No one wants to play with a toy that's going to be fun after a few decades of tinkering--the toys have to be engaging now, or kids will find other toys.

Johnson has a knack for explaining complicated and counterintuitive ideas cleverly without stealing the scene. Though we're far from fully understanding how complex behavior manifests from simple units and rules, our awareness that such emergence is possible is guiding research across disciplines. Readers unfamiliar with the sciences of complexity will find Emergence an excellent starting point, while those who were chaotic before it was cool will appreciate its updates and wider scope. --Rob Lightner

Collective Intelligence in Action

Satnam Alag

Amazon Price: $29.69 List Price: $44.99 Usually ships in 24 hours By: Manning Publications Amazon Marketplace: 33 new & used starting at $25.06

Buy at Amazon.com

Browse similar items by category:
Subjects -> Computers & Internet -> Home Computing -> Internet -> General AAS
Subjects -> Computers & Internet -> Computer Science -> Artificial Intelligence -> Computer Mathematics
Subjects -> Computers & Internet -> Computer Science -> Artificial Intelligence -> Machine Learning

Customer Reviews:
Total reviews: 4 Average rating: 5.0 of 5

Editorial Review:

There's a great deal of wisdom in a crowd, but how do you listen to a thousand people talking at once? Identifying the wants, needs, and knowledge of internet users can be like listening to a mob.

In the Web 2.0 era, leveraging the collective power of user contributions, interactions, and feedback is the key to market dominance. A new category of powerful programming techniques lets you discover the patterns, inter-relationships, and individual profiles-the collective intelligence--locked in the data people leave behind as they surf websites, post blogs, and interact with other users.

Collective Intelligence in Action is a hands-on guidebook for implementing collective intelligence concepts using Java. It is the first Java-based book to emphasize the underlying algorithms and technical implementation of vital data gathering and mining techniques like analyzing trends, discovering relationships, and making predictions. It provides a pragmatic approach to personalization by combining content-based analysis with collaborative approaches.

This book is for Java developers implementing Collective Intelligence in real, high-use applications. Following a running example in which you harvest and use information from blogs, you learn to develop software that you can embed in your own applications. The code examples are immediately reusable and give the Java developer a working collective intelligence toolkit.

Along the way, you work with, a number of APIs and open-source toolkits including text analysis and search using Lucene, web-crawling using Nutch, and applying machine learning algorithms using WEKA and the Java Data Mining (JDM) standard.

An Introduction to Information Theory

John R. Pierce

Amazon Price: $10.36 List Price: $12.95 Usually ships in 24 hours By: Dover Publications Amazon Marketplace: 54 new & used starting at $4.98

Buy at Amazon.com

Browse similar items by category:
Subjects -> Computers & Internet -> Computer Science -> Artificial Intelligence -> General
Subjects -> Computers & Internet -> Computer Science -> Artificial Intelligence -> General AAS
Subjects -> Computers & Internet -> Computer Science -> Information Theory

Customer Reviews:
Total reviews: 16 Average rating: 4.5 of 5

Good intro but dated 3 out of 5 stars.
6 of 8 people found this review helpful.

The update of this book should have been updated. While it is understandable that at the time of the first print of this book in 1961 the author saw little or no practical use for Shannon's information theory (other than perhaps his channel capacity theorem) it was well known by the second printing in 1980 that it has profound implications in studying biology (and modern technology). For instance in an article published in Nature in 1967, A. L. MacKay showed how the genetic code is highly optimal using Huffman's algorithm. More recently Ardell and Sella (with summaries available on the net) have 'demonstrated that the code's present structure was also shaped by natural selection (though non-Darwinian, see below). In this process, the codons - the triplets of nucleotides that map a particular nucleic acid sequence into proteins - are arranged to minimize the negative effects of genetic error, and to optimize the process of 'readout' of genes during protein synthesis. By permuting all 20 amino acids across all possible codon sets, both groups found that the 'universal' genetic code - the one found in nearly every organism on earth...-falls in the best .0001% of all possible codes and perhaps even better, in terms of its capacity to be an error-correcting code...' By showing modifications are possible in one generation the evidence points away from Crick's thesis of the genetic code being a 'fozen accident' but instead possible Lamarckian beginnings with horizantal gene transfer leading to Carl Woese's early RNA World hypothesis before Darwinian vertical descent begins.

The author also tends to perpetuate the widespread misunderstanding (generally by physicists who tend to contort the meaning away from Shannon's into 'available' states or choices such as with Black Holes) that information is uncertainty; he confuses (readers potentially with) surprise versus information by not taking into account the other half of the necessary equation for information transmission, being noise. He says "The amount of information conveyed by the message increases as the amount of uncertainty as to what message actually will be produced becomes greater." [pg 23] While he clears this up in a later chapter on noise it becomes so technical that it appears most readers of Shannon's theory have been mislead. At this point the scientists (usually physicists who actually work with a different concept of 'available information') typically equate the uncertainty with Kolmogorov complexity and assume that maximum information and complexity is randomess.

For instance consider Philip Nelson's comment in his book Biological Physics that 'random messages carry the most information!' In one footnote of his nearly 600 page book he effectively dismisses all of Nobel Prize winner Shannon's information achievements.

Much of the trouble is with terminology. We think of noise as impure sound. Shannon tried to avoid this problem by introducing the term 'equivocation' but on the other hand this seems to have no intuitive meaning in this context. One really has to go to the math to sort it out. The critical equation to potentially eradicate the confusion does not appear in the book -
R = Hbefore - Hafter
H is an entropy-like formula without Boltzman's constant; however the concepts are very different. (Reportedly Von Neuman told Shannon in the 1940's to call his uncertainty 'entropy, as noone will know what you mean!' Apparently this is still working!) Entropy of the universe apparently increases under the 2nd law of thermodynamics (at least ignoring gravity and extensivity), information begins and ends with life (one needs a recognizer to measure it). A random message in fact carries no information as there is no resolution (reduction) of uncertainty. This is all explained at molecular biologist's Dr. Tom Schneider's website, I know of no other comprehensive source and certainly no book that gets it right. (As yet! 'Hope springs eternal!' A. Pope; 1688 - 1744)

Editorial Review:

Covers encoding and binary digits, entropy, language and meaning, efficient encoding and the noisy channel, and explores ways in which information theory relates to physics, cybernetics, psychology, and art. "Uncommonly good...the most satisfying discussion to be found." — Scientific American. 1980 edition.

Elements of Information Theory 2nd Edition (Wiley Series in Telecommunications and Signal Processing)

Thomas M. Cover, Joy A. Thomas

Amazon Price: $77.89 List Price: $89.95 Usually ships in 10 to 13 days By: Wiley-Interscience Amazon Marketplace: 43 new & used starting at $54.99

Buy at Amazon.com

Browse similar items by category:
Subjects -> Computers & Internet -> Computer Science -> Information Theory
Subjects -> Computers & Internet -> General
Subjects -> Computers & Internet -> General AAS

Customer Reviews:
Total reviews: 24 Average rating: 4.0 of 5

Editorial Review:

The latest edition of this classic is updated with new problem sets and material

The Second Edition of this fundamental textbook maintains the book's tradition of clear, thought-provoking instruction. Readers are provided once again with an instructive mix of mathematics, physics, statistics, and information theory.

All the essential topics in information theory are covered in detail, including entropy, data compression, channel capacity, rate distortion, network information theory, and hypothesis testing. The authors provide readers with a solid understanding of the underlying theory and applications. Problem sets and a telegraphic summary at the end of each chapter further assist readers. The historical notes that follow each chapter recap the main points.

The Second Edition features:
* Chapters reorganized to improve teaching
* 200 new problems
* New material on source coding, portfolio theory, and feedback capacity
* Updated references

Now current and enhanced, the Second Edition of Elements of Information Theory remains the ideal textbook for upper-level undergraduate and graduate courses in electrical engineering, statistics, and telecommunications.

An Instructor's Manual presenting detailed solutions to all the problems in the book is available from the Wiley editorial department.

Pragmatic Unit Testing in Java with JUnit

Andy Hunt, Dave Thomas

Amazon Price: $19.77 List Price: $29.95 Usually ships in 24 hours By: The Pragmatic Programmers Amazon Marketplace: 49 new & used starting at $11.08

Buy at Amazon.com

Browse similar items by category:
Subjects -> Computers & Internet -> Computer Science -> Information Theory
Subjects -> Computers & Internet -> Computer Science -> Software Engineering -> Information Systems
Subjects -> Computers & Internet -> Operating Systems -> General

Customer Reviews:
Total reviews: 12 Average rating: 4.0 of 5

Editorial Review:

Learn how to improve your Java coding skills using unit testing. Despite it's name, unit testing is really a coding technique, not a testing technique. Unit testing is done by programmers, for programmers. It's primarily for our benefit: we get improved confidence in our code, better ability to make deadlines, less time spent in the debugger, and less time beating on the code to make it work correctly.

This book shows how to write tests, but more importantly, it goes where other books fear to tread and gives you concrete advice and examples of what to test--the common things that go wrong in all of our programs. Discover the tricky hiding places where bugs breed, and how to catch them using the freely available JUnit framework. It's easy to learn how to think of all the things in your code that are likely to break. We'll show you how with helpful mnemonics, summarized in a handy tip sheet (also available from our website) to help you remember all this stuff.

With this book you will:

Write better code, and take less time to write it

Discover the tricky places where bugs breed

Learn how to think of all the things that could go wrong

Test individual pieces of code without having to include the whole project

Test effectively with the whole team

We'll also cover how to use Mock Objects for testing, how to write high quality test code, and how to use unit testing to improve your design skills. We'll show you frequent "gotchas"--along with the fixes--to save you time when problems come up. We'll show you how with helpful mnemonics, summarized in a handy tip sheet (also available from our website).

But the best part is that you don't need a sweeping mandate to change your whole team or your whole company. You don't need to adopt Extreme Programming or Test-Driven Development, or change your development process in order to reap the proven benefits of unit testing. You can start unit testing, the pragmatic way, right away.

Other Pragmatic Programmer books:

Pragmatic Project Automation: How to Build, Deploy, and Monitor Java Applications (Mike Clark)
Pragmatic Unit Testing in C# with NUnit (Andy Hunt and Dave Thomas)
Pragmatic Version Control using CVS (Dave Thomas and Andy Hunt)

Information Theory, Inference & Learning Algorithms

David J. C. MacKay

Amazon Price: $51.20 List Price: $64.00 Usually ships in 24 hours By: Cambridge University Press Amazon Marketplace: 40 new & used starting at $42.44

Buy at Amazon.com

Browse similar items by category:
Subjects -> Computers & Internet -> Computer Science -> Artificial Intelligence -> Neural Networks
Subjects -> Computers & Internet -> Computer Science -> Artificial Intelligence -> Theory of Computing
Subjects -> Computers & Internet -> Computer Science -> Artificial Intelligence -> General

Customer Reviews:
Total reviews: 9 Average rating: 4.5 of 5

Editorial Review:

Information theory and inference, often taught separately, are here united in one entertaining textbook. These topics lie at the heart of many exciting areas of contemporary science and engineering - communication, signal processing, data mining, machine learning, pattern recognition, computational neuroscience, bioinformatics, and cryptography. This textbook introduces theory in tandem with applications. Information theory is taught alongside practical communication systems, such as arithmetic coding for data compression and sparse-graph codes for error-correction. A toolbox of inference techniques, including message-passing algorithms, Monte Carlo methods, and variational approximations, are developed alongside applications of these tools to clustering, convolutional codes, independent component analysis, and neural networks. The final part of the book describes the state of the art in error-correcting codes, including low-density parity-check codes, turbo codes, and digital fountain codes -- the twenty-first century standards for satellite communications, disk drives, and data broadcast. Richly illustrated, filled with worked examples and over 400 exercises, some with detailed solutions, David MacKay's groundbreaking book is ideal for self-learning and for undergraduate or graduate courses. Interludes on crosswords, evolution, and sex provide entertainment along the way. In sum, this is a textbook on information, communication, and coding for a new generation of students, and an unparalleled entry point into these subjects for professionals in areas as diverse as computational biology, financial engineering, and machine learning.

Decoding the Universe: How the New Science of Information Is Explaining Everything in the Cosmos, from Our Brains to Black Holes

Charles Seife

Amazon Price: $10.20 List Price: $15.00 Usually ships in 24 hours By: Penguin (Non-Classics) Amazon Marketplace: 52 new & used starting at $7.59

Buy at Amazon.com

Browse similar items by category:
Subjects -> Computers & Internet -> Computer Science -> Artificial Intelligence -> Expert Systems
Subjects -> Computers & Internet -> Computer Science -> Information Theory
Subjects -> Computers & Internet -> General

Customer Reviews:
Total reviews: 29 Average rating: 4.0 of 5

Information theory, the third physics revolution of the XXth century 5 out of 5 stars.
2 of 2 people found this review helpful.

The author has a degree in probability theory and artificial intelligence, but he is a professor of journalism and has therefore written a book which is both very entertaining and not too difficult to understand. The subject is information, which Seife claims is the third XXth century revolution in physics started by Claude Shannon and which has relations with the other two: Relativity and Quantum Mechanics.

Of course, information is also related to thermodynamics and entropy, so the book contains a discussion of all these topics: thermodynamics, relativity and quantum mechanics. Famous conundrums such as Schroedinger's cat, entanglement, Maxwell's demon, etc. are analyzed from the point of view of information theory.

Here are some snippets of the book:

According to Seife, Einstein dictum "Nothing can travel faster than light" is really about information:" Information speed cannot exceed c". Another interesting fact is that what really causes computers to heat is the erasure of bits.

Seife describes recent achievements and experiments, proof that he is familiar with the latest results. One curious example is the solution of "the knight problem" in 2000 by using a DNA computer! Another one is that the entire human race has less genetic diversity than a few scores of chimps due to some kind of cataclysm about 500,000 years ago. A third one is the 1996 experiment demonstrating the existence of virtual particles (the so called Casimir effect).

In chapter 7, quantum computers are introduced and the possibility of the brain being one is briefly discussed. Unfortunately, it seems that Max Tegmark proved Roger Penrose wrong on this count. You begin to understand the power of quantum computation when the author describes Grover's algorithm to guess a number out of 16. Classically you need four yes/no answers to four questions. Grover manages the same task with two. Quantum computation reduces the complexity of some problems from n to square root of n.

I found also very interesting the reasons why the photoelectric effect cannot be explained by waves. On the other hand, interference cannot be explained by a corpuscular theory of light, so we are stuck with duality.

Towards the end, the author discusses black holes and the holographic principle: the quantity of information contained in a ball is not limited by its volume (surprisingly), but by its area. Since most cosmologists consider now the universe infinite (inflation seems to imply this) we are led, via the holographic bound, to the conclusion that the universe contains infinite copies of our own bubble universe. Seife admits that this is the most bizarre thing among the many ones described in his book.

Editorial Review:

The author of Zero explains the scientific revolution that is transforming the way we understand our world

Previously the domain of philosophers and linguists, information theory has now moved beyond the province of code breakers to become the crucial science of our time. In Decoding the Universe, Charles Seife draws on his gift for making cutting-edge science accessible to explain how this new tool is deciphering everything from the purpose of our DNA to the parallel universes of our Byzantine cosmos. The result is an exhilarating adventure that deftly combines cryptology, physics, biology, and mathematics to cast light on the new understanding of the laws that govern life and the universe.

Model Based Inference in the Life Sciences: A Primer on Evidence

David R. Anderson

Amazon Price: $34.83 List Price: $39.95 Usually ships in 24 hours By: Springer Amazon Marketplace: 46 new & used starting at $30.88

Buy at Amazon.com

Browse similar items by category:
Subjects -> Computers & Internet -> Computer Science -> Information Theory
Subjects -> Medicine -> Internal Medicine -> Infectious Disease -> Epidemiology
Subjects -> Medicine -> Research -> Biostatistics

Customer Reviews:
Total reviews: 1 Average rating: 5.0 of 5

Editorial Review:

The abstract concept of “information” can be quantified and this has led to many important advances in the analysis of data in the empirical sciences. This text focuses on a science philosophy based on “multiple working hypotheses” and statistical models to represent them. The fundamental science question relates to the empirical evidence for hypotheses in this set—a formal strength of evidence. Kullback-Leibler information is the information lost when a model is used to approximate full reality. Hirotugu Akaike found a link between K-L information (a cornerstone of information theory) and the maximized log-likelihood (a cornerstone of mathematical statistics). This combination has become the basis for a new paradigm in model based inference. The text advocates formal inference from all the hypotheses/models in the a priori set—multimodel inference.

This compelling approach allows a simple ranking of the science hypothesis and their models. Simple methods are introduced for computing the likelihood of model i, given the data; the probability of model i, given the data; and evidence ratios. These quantities represent a formal strength of evidence and are easy to compute and understand, given the estimated model parameters and associated quantities (e.g., residual sum of squares, maximized log-likelihood, and covariance matrices). Additional forms of multimodel inference include model averaging, unconditional variances, and ways to rank the relative importance of predictor variables.

This textbook is written for people new to the information-theoretic approaches to statistical inference, whether graduate students, post-docs, or professionals in various universities, agencies or institutes. Readers are expected to have a background in general statistical principles, regression analysis, and some exposure to likelihood methods. This is not an elementary text as it assumes reasonable competence in modeling and parameter estimation.