ⓘ Factor graph. A factor graph is a bipartite graph representing the factorization of a function. In probability theory and its applications, factor graphs are us ..


ⓘ Factor graph

A factor graph is a bipartite graph representing the factorization of a function. In probability theory and its applications, factor graphs are used to represent factorization of a probability distribution function, enabling efficient computations, such as the computation of marginal distributions through the sum-product algorithm. One of the important success stories of factor graphs and the sum-product algorithm is the decoding of capacity-approaching error-correcting codes, such as LDPC and turbo codes.

Factor graphs generalize constraint graphs. A factor whose value is either 0 or 1 is called a constraint. A constraint graph is a factor graph where all factors are constraints. The max-product algorithm for factor graphs can be viewed as a generalization of the arc-consistency algorithm for constraint processing.


1. Definition

A factor graph is a bipartite graph representing the factorization of a function. Given a factorization of a function g {\displaystyle gX_{1},X_{2},\dots,X_{n}},

g = ∏ j = 1 m f j S j, {\displaystyle gX_{1},X_{2},\dots,X_{n}=\prod _{j=1}^{m}f_{j}S_{j},}

where S j ⊆ { X 1, X 2, …, X n } {\displaystyle S_{j}\subseteq \{X_{1},X_{2},\dots,X_{n}\}}, the corresponding factor graph G = X, F, E {\displaystyle G=X,F,E} consists of variable vertices X = { X 1, X 2, …, X n } {\displaystyle X=\{X_{1},X_{2},\dots,X_{n}\}}, factor vertices F = { f 1, f 2, …, f m } {\displaystyle F=\{f_{1},f_{2},\dots,f_{m}\}}, and edges E {\displaystyle E}. The edges depend on the factorization as follows: there is an undirected edge between factor vertex f j {\displaystyle f_{j}} and variable vertex X k {\displaystyle X_{k}} iff X k ∈ S j {\displaystyle X_{k}\in S_{j}}. The function is tacitly assumed to be real-valued: g ∈ R {\displaystyle gX_{1},X_{2},\dots,X_{n}\in \mathbb {R} }.

Factor graphs can be combined with message passing algorithms to efficiently compute certain characteristics of the function g {\displaystyle gX_{1},X_{2},\dots,X_{n}}, such as the marginal distributions.


2. Examples

Consider a function that factorizes as follows:

g = f 1 X 1 f 2 X 1, X 2 f 3 X 1, X 2 f 4 X 2, X 3 {\displaystyle gX_{1},X_{2},X_{3}=f_{1}X_{1}f_{2}X_{1},X_{2}f_{3}X_{1},X_{2}f_{4}X_{2},X_{3}},

with a corresponding factor graph shown on the right. Observe that the factor graph has a cycle. If we merge f 2 X 1, X 2 f 3 X 1, X 2 {\displaystyle f_{2}X_{1},X_{2}f_{3}X_{1},X_{2}} into a single factor, the resulting factor graph will be a tree. This is an important distinction, as message passing algorithms are usually exact for trees, but only approximate for graphs with cycles.


3. Message passing on factor graphs

A popular message passing algorithm on factor graphs is the sum-product algorithm, which efficiently computes all the marginals of the individual variables of the function. In particular, the marginal of variable X k {\displaystyle X_{k}} is defined as

g k X k = ∑ X k ¯ g {\displaystyle g_{k}X_{k}=\sum _{X_{\bar {k}}}gX_{1},X_{2},\dots,X_{n}}

where the notation X k ¯ {\displaystyle X_{\bar {k}}} means that the summation goes over all the variables, except X k {\displaystyle X_{k}}. The messages of the sum-product algorithm are conceptually computed in the vertices and passed along the edges. A message from or to a variable vertex is always a function of that particular variable. For instance, when a variable is binary, the messages over the edges incident to the corresponding vertex can be represented as vectors of length 2: the first entry is the message evaluated in 0, the second entry is the message evaluated in 1. When a variable belongs to the field of real numbers, messages can be arbitrary functions, and special care needs to be taken in their representation.

In practice, the sum-product algorithm is used for statistical inference, whereby g {\displaystyle gX_{1},X_{2},\dots,X_{n}} is a joint distribution or a joint likelihood function, and the factorization depends on the conditional independencies among the variables.

The Hammersley–Clifford theorem shows that other probabilistic models such as Bayesian networks and Markov networks can be represented as factor graphs; the latter representation is frequently used when performing inference over such networks using belief propagation. On the other hand, Bayesian networks are more naturally suited for generative models, as they can directly represent the causalities of the model.

  • graph theory, graph coloring is a special case of graph labeling it is an assignment of labels traditionally called colors to elements of a graph subject
  • generalized compressibility factor graphs based on as many as 25 or more different pure gases, such as the Nelson - Obert graphs Such graphs are said to have an
  • In the mathematical area of graph theory, a chordal graph is one in which all cycles of four or more vertices have a chord, which is an edge that is not
  • graph theory, the line graph of an undirected graph G is another graph L G that represents the adjacencies between edges of G. The name line graph comes
  • of graph theory, a cubic graph is a graph in which all vertices have degree three. In other words, a cubic graph is a 3 - regular graph Cubic graphs are
  • However, uniform graph partitioning or a balanced graph partition problem can be shown to be NP - complete to approximate within any finite factor Even for special
  • mathematical field of graph theory, the Petersen graph is an undirected graph with 10 vertices and 15 edges. It is a small graph that serves as a useful
  • Graph drawing is an area of mathematics and computer science combining methods from geometric graph theory and information visualization to derive two - dimensional
  • mathematical field of graph theory, the Ellingham Horton graphs are two 3 - regular graphs on 54 and 78 vertices: the Ellingham Horton 54 - graph and the Ellingham Horton

Users also searched:

probability propagation,