Statistics How To

Independent Component Analysis: Definition

Multivariate Analysis >

Independent component analysis is used in statistics and signal processing to express a multivariate function by its hidden factors or subcomponents. These component signals are independent non-Gaussian signals, and the intention is that these independent subcomponents accurately represent the composite signal.

The ‘cocktail party problem‘ is an often cited example of independent component analysis at work. Suppose you have a loud cocktail party, with many conversations going on in the same room. Now suppose you have a number of microphones located at various locations throughout the room, picking up the sound data. With no prior knowledge of the speakers, can you separate the signal from each microphone (in each case, a composite of all the noise in the room) into its component parts, i.e., each speakers voice?

You can do a surprisingly good job, assuming you have enough observation points. If there are N sources of noise (N guests, for instance) you will need N microphones in order to fully determine the original signals.

Assumptions in Independent Component Analysis.

Independent component analysis only works if the sources are non-Gaussian (i.e. they have non-normal distributions), and so that is one of the first assumptions you make if you use this analysis on a multivariate function. Since you are attempting to break the function down into independent components, you’re also making the assumption that the original sources were in fact independent.

Mixing Effects in Independent Component Analysis

There are three principles of mixing signals which make up the foundation for independent component analysis.

  1. Mixing signals involves going from a series of independent signals to a series of signals that is dependent. Although your source signals are all independently generated, the composite signals are all made from the same source signals, and so cannot be independent of each other.
  2. Although our source signals are non-Gaussian (by the assumption we made to begin with), the composite signals are in fact Gaussian (normally distributed). This is by the Central Limit Theorem, which tells us the the probability distribution function of the sum of independent variables with finite variance (like our signals) will tend toward the Gaussian distribution.
  3. The complexity of a signal mixture must always be greater than, or equal to, the complexity of its simplest source signal.

These three principles form the foundation for Independent Component Analysis.

References

  1. Hyvärinen, Aapo. What is Independent Component Analysis?
    Retrieved from https://www.cs.helsinki.fi/u/ahyvarin/whatisica.shtml on April 10, 2018
  2. Hyvärinen, Karhunen, & Oja. Independent Component Analysis. Chapter 1.
    Retrieved from http://research.ics.aalto.fi/ica/book/intro.pdf on April 10, 2018
  3. Ng, Andrew. CS 229 Lecture Notes: Independent Component Analysis.
    Retrieved from http://cs229.stanford.edu/notes/cs229-notes11.pdf on April 10, 2018
  4. Stone, James. Independent Component Analysis. Encyclopedia of Statistics in Behavioral Science, Volume 2, pp. 907–912
    Retrieved from https://pdfs.semanticscholar.org/6cdc/d22d69479c6c19f1583a281a95bc4029631e.pdf on April 10, 2018
------------------------------------------------------------------------------

Confused and have questions? Head over to Chegg and use code “CS5OFFBTS18” (exp. 11/30/2018) to get $5 off your first month of Chegg Study, so you can understand any concept by asking a subject expert and getting an in-depth explanation online 24/7.

Comments? Need to post a correction? Please post a comment on our Facebook page.

Check out our updated Privacy policy and Cookie Policy