지시정보

지시 정보는 임의 $Y^{n}=(Y_{1},Y_{2},\dots ,Y_{n})$ $X^{n}=(X_{1},X_{2},\dots ,X_{n})$ n = ( $X^{n}=(X_{1},X_{2},\dots ,X_{n})$ $X^{n}=(X_{1},X_{2},\dots ,X_{n})$ $Y^{n}=(Y_{1},Y_{2},\dots ,Y_{n})$ $Y^{n}=(Y_{1},Y_{2},\dots ,Y_{n})$ $X^{n}=(X_{1},X_{2},\dots ,X_{n})$ $Y^{n}=(Y_{1},Y_{2},\dots ,Y_{n})$ $X^{n}=(X_{1},X_{2},\dots ,X_{n})$ {\ $displaystyle$ X $^{n$ }=( $Y^{n}=(Y_{1},Y_{2},\dots ,Y_{n})$ ${1},X_{2},\dots,$ $Y^{n}=(Y_{1},Y_{2},\dots ,Y_{n})$ ${n})$ ${\displaystyle$ Y $^{$ n $Y^{n}=(Y_{1},Y_{2},\dots ,Y_{n})$ = $Y_1$ {2},\ $Y^{n}=(Y_{1},Y_{2},\dots ,Y_{n})$ ,X_{ $n}) {\$ displaystyle Y^{n}=(Y_1 $Y_{2},\dots,Y_{n$ }}.지시 정보라는 용어는 James Massey에 의해 만들어졌고 다음과 같이 정의됩니다^[1].

I(X^{n}\toY^{n})\triangleq \sum _{i=1}^{n}I(X^{i};Y_{i} Y^{i-1})

$I(X^{i};Y_{i}|Y^{i-1})$ 서 I $I(X^{i};Y_{i}|Y^{i-1})$ ( $I(X^{i};Y_{i}|Y^{i-1})$ $I(X^{i};Y_{i}|Y^{i-1})$ $I(X^{i};Y_{i}|Y^{i-1})$ - $I(X^{i};Y_{i}|Y^{i-1})$ {\ $displaystyle$ I ( $X^{i});$ $Y_{i} Y^{i-1}}$ 는 $I(X^{i};Y_{i}|Y^{i-1})$ 조건부 상호 $I(X_{1},X_{2},...,X_{i};Y_{i}|Y_{1},Y_{2},...,Y_{i-1})$ I $I(X_{1},X_{2},...,X_{i};Y_{i}|Y_{1},Y_{2},...,Y_{i-1})$ $I(X_{1},X_{2},...,X_{i};Y_{i}|Y_{1},Y_{2},...,Y_{i-1})$ $I(X_{1},X_{2},...,X_{i};Y_{i}|Y_{1},Y_{2},...,Y_{i-1})$ $I(X_{1},X_{2},...,X_{i};Y_{i}|Y_{1},Y_{2},...,Y_{i-1})$ $I(X_{1},X_{2},...,X_{i};Y_{i}|Y_{1},Y_{2},...,Y_{i-1})$ $I(X_{1},X_{2},...,X_{i};Y_{i}|Y_{1},Y_{2},...,Y_{i-1})$ $I(X_{1},X_{2},...,X_{i};Y_{i}|Y_{1},Y_{2},...,Y_{i-1})$ $I(X_{1},X_{2},...,X_{i};Y_{i}|Y_{1},Y_{2},...,Y_{i-1})$ $I(X_{1},X_{2},...,X_{i};Y_{i}|Y_{1},Y_{2},...,Y_{i-1})$ {\ $displaystyle$ I $(X_{1}, X_{2},$ ..., $X_{i};$ $Y_{i} Y_{1},Y_{2},...,Y_{i-1$

지시 정보는 ^[1]^[2]^[3]^[4]피드백이 있는 채널의 용량, 이산 메모리리스 ^[5]네트워크의 용량, 인-블록 ^[6]메모리가 있는 네트워크의 용량, 인과적 측면 ^[7]정보가 있는 도박, 인과적 측면 ^[8]정보가 있는 압축, 실시간 제어 통신 ^[9]^[10]설정 등 인과성이 중요한 역할을 하는 문제에 적용됩니다.그리고 통계 물리학.^[11]

인과조건

지시 정보의 본질은 인과적 조건입니다. $x^{n}$ n ${\$ x $^{n}$ 에서 $y^n$ $x^{n}$ 인과적으로 조건화된 $y^{n}$ n {\ $displaystyle$ y $^{n}$ 의 확률은 다음과 같이 정의됩니다^[5].

P(x^{n}||y^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i})

(

P(x^{n}||y^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i})

P(x^{n}||y^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i})

y

P(x^{n}||y^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i})

)

P(x^{n}||y^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i})

P(x^{n}||y^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i})

=

P(x^{n}||y^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i})

n

P(x^{n}||y^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i})

(

P(x^{n}||y^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i})

P(x^{n}||y^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i})

x

P(x^{n}||y^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i})

- 1

P(x^{n}||y^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i})

P(x^{n}||y^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i})

)

P(x^{n}||y^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i})

{\

displaystyle

P

(x^{n}

y

^{n})\displaystyle

P

(x_

{n}

P(x^{n}||y^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i})

^{i=

1}^{n}P(x_{i} x^{i-1},y^{i

이는 모든 $y^{n}$ y $y^{n}$ ${\displaystyle$ y $^{n$ 가 아닌 "과거" 및 "현재" $y^{i}$ y $y^{i}$ $P(x^{n}|y^{n})=\prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{n})$ {\ $displaystyle$ y $P(x^{n}|y^{n})=\prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{n})$ }}에 대한 한 조건을 제외하고는 기존 $P(x^{n}|y^{n})=\prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{n})$ P ( $P(x^{n}|y^{n})=\prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{n})$ $P(x^{n}|y^{n})=\prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{n})$ y $P(x^{n}|y^{n})=\prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{n})$ ) $P(x^{n}|y^{n})=\prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{n})$ = $1$ n $P(x^{n}|y^{n})=\prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{n})$ ( $x ^{n}$ y $^{n}$ = \ $displaystyle$ y $^{n}}$ 에 대한 체인 규칙과 유사합니다. "과거" 기호만 포함하려면상수 기호를 붙여 지연을 도입할 수 있습니다.

P(x^{n}||(0,y^{n-1}))\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i-1})

(

P(x^{n}||(0,y^{n-1}))\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i-1})

n (

P(x^{n}||(0,y^{n-1}))\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i-1})

P(x^{n}||(0,y^{n-1}))\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i-1})

n -

P(x^{n}||(0,y^{n-1}))\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i-1})

P(x^{n}||(0,y^{n-1}))\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i-1})

P(x^{n}||(0,y^{n-1}))\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i-1})

=

P(x^{n}||(0,y^{n-1}))\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i-1})

P(x^{n}||(0,y^{n-1}))\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i-1})

(

P(x^{n}||(0,y^{n-1}))\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i-1})

x

P(x^{n}||(0,y^{n-1}))\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i-1})

-

P(x^{n}||(0,y^{n-1}))\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i-1})

P(x^{n}||(0,y^{n-1}))\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i-1})

i -

P(x^{n}||(0,y^{n-1}))\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i-1})

1 ) {\

displaystyle P(x^{n}

(

0,y

^{n-1

})\displaystyle P(x

^{

n})\displayq

P(x_{i} x

P(x^{n}||(0,y^{n-1}))\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i-1})

i-1},

y^{i-1})}.

형식적으로는 모든 문자열에 동일한 수의 기호가 있어야 하지만 이 식에 대해 P $P(x^{n}||y^{n-1})$ $P(x^{n}||y^{n-1})$ y $P(x^{n}||y^{n-1})$ - $P(x^{n}||y^{n-1})$ ) $P(x^{n}||y^{n-1})$ {\ $displaystyle$ P $(x^{n}$ y $^{n-1})}$ 를 $P(x^{n}||y^{n-1})$ (를) 써 표기법을 남용하는 것이 일반적입니다.

여러 문자열에 조건을 붙일 수도 있습니다. $P(x^{n}||y^{n},z^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i},z^{i})$ ( $P(x^{n}||y^{n},z^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i},z^{i})$ $P(x^{n}||y^{n},z^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i},z^{i})$ $P(x^{n}||y^{n},z^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i},z^{i})$ n $P(x^{n}||y^{n},z^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i},z^{i})$ $P(x^{n}||y^{n},z^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i},z^{i})$ ) ∏ $P(x^{n}||y^{n},z^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i},z^{i})$ = $P(x^{n}||y^{n},z^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i},z^{i})$ $P(x^{n}||y^{n},z^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i},z^{i})$ ( $P(x^{n}||y^{n},z^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i},z^{i})$ $P(x^{n}||y^{n},z^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i},z^{i})$ - 1 $P(x^{n}||y^{n},z^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i},z^{i})$ $P(x^{n}||y^{n},z^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i},z^{i})$ , $P(x^{n}||y^{n},z^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i},z^{i})$ $P(x^{n}||y^{n},z^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i},z^{i})$ {\ $displaystyle$ P $(x^{n}$ y $^{n}},z^{n}\twoq \two_{i$ = $1}^{n}P($ x_{ $i}$ x $^{$ i-1}, $y^{i$ }, $z^{$ i $P(x^{n}||y^{n},z^{n})\triangleq \prod _{i=1}^{n}P(x_{i}|x^{i-1},y^{i},z^{i})$

인과조건부 엔트로피

인과 조건화 엔트로피는 ^[2]다음과 같이 정의됩니다.

H(X^{n} Y^{n})=\mathbf {E} \left[-\log {P(X^{n} Y^{n}}\right]=\sum_{i=1}^{n}H(X_{i} X^{i-1},Y^{i}

마찬가지로 여러 문자열에 인과적으로 조건을 지정하고 H $H(X^{n}||Y^{n},Z^{n})=\mathbf {E} \left[-\log {P(X^{n}||Y^{n},Z^{n})}\right]$ ( $H(X^{n}||Y^{n},Z^{n})=\mathbf {E} \left[-\log {P(X^{n}||Y^{n},Z^{n})}\right]$ $H(X^{n}||Y^{n},Z^{n})=\mathbf {E} \left[-\log {P(X^{n}||Y^{n},Z^{n})}\right]$ $H(X^{n}||Y^{n},Z^{n})=\mathbf {E} \left[-\log {P(X^{n}||Y^{n},Z^{n})}\right]$ n $H(X^{n}||Y^{n},Z^{n})=\mathbf {E} \left[-\log {P(X^{n}||Y^{n},Z^{n})}\right]$ $H(X^{n}||Y^{n},Z^{n})=\mathbf {E} \left[-\log {P(X^{n}||Y^{n},Z^{n})}\right]$ ) = $H(X^{n}||Y^{n},Z^{n})=\mathbf {E} \left[-\log {P(X^{n}||Y^{n},Z^{n})}\right]$ [ - $H(X^{n}||Y^{n},Z^{n})=\mathbf {E} \left[-\log {P(X^{n}||Y^{n},Z^{n})}\right]$ ⁡ $H(X^{n}||Y^{n},Z^{n})=\mathbf {E} \left[-\log {P(X^{n}||Y^{n},Z^{n})}\right]$ ( $H(X^{n}||Y^{n},Z^{n})=\mathbf {E} \left[-\log {P(X^{n}||Y^{n},Z^{n})}\right]$ $H(X^{n}||Y^{n},Z^{n})=\mathbf {E} \left[-\log {P(X^{n}||Y^{n},Z^{n})}\right]$ Y $H(X^{n}||Y^{n},Z^{n})=\mathbf {E} \left[-\log {P(X^{n}||Y^{n},Z^{n})}\right]$ , $H(X^{n}||Y^{n},Z^{n})=\mathbf {E} \left[-\log {P(X^{n}||Y^{n},Z^{n})}\right]$ n ) $H(X^{n}||Y^{n},Z^{n})=\mathbf {E} \left[-\log {P(X^{n}||Y^{n},Z^{n})}\right]$ ] {\ $displaystyle H(X^{n}$ $Y^{n$ }, $Z$ ^{n $})$ = $\mathbf {E}$ \left $[-\log {P(X^{n}$ $Y^{n}},$ Z $^{n$ }\right $H(X^{n}||Y^{n},Z^{n})=\mathbf {E} \left[-\log {P(X^{n}||Y^{n},Z^{n})}\right]$ 를 쓸 수 있습니다.

특성.

인과^[1] 조건화를 위한 분해 규칙은

=

P(y^{n}

x

^{n

이 규칙은 P $P(x^{n}||y^{n-1}),P(y^{n}||x^{n})$ n $P(x^{n}||y^{n-1}),P(y^{n}||x^{n})$ $P(x^{n}||y^{n-1}),P(y^{n}||x^{n})$ - 1 $P(x^{n}||y^{n-1}),P(y^{n}||x^{n})$ ), $P(x^{n}||y^{n-1}),P(y^{n}||x^{n})$ n $P(x^{n}||y^{n-1}),P(y^{n}||x^{n})$ n ) $P(x^{n}||y^{n-1}),P(y^{n}||x^{n})$ {\ $displaystyle$ P $(x^{n}$ y $^{n},$ P $(y^{n}$ x $^{n})}$ 의 $P(x^{n}||y^{n-1}),P(y^{n}||x^{n})$ 모든 곱이 공동 $P(x^{n},y^{n})$ P $P(x^{n},y^{n})$ n $P(x^{n},y^{n})$ $P(x^{n},y^{n})$ n $P(x^{n},y^{n})$ {\ $displaystyle$ P $(x^{n},y^{n$ 를 제공함을 보여줍니다.

캐주얼 $P(y^{n}||x^{n})=\prod _{i=1}^{n}P(y_{i}|y^{i-1},x^{i})$ $P(y^{n}||x^{n})=\prod _{i=1}^{n}P(y_{i}|y^{i-1},x^{i})$ P ( $P(y^{n}||x^{n})=\prod _{i=1}^{n}P(y_{i}|y^{i-1},x^{i})$ n $P(y^{n}||x^{n})=\prod _{i=1}^{n}P(y_{i}|y^{i-1},x^{i})$ n ) $P(y^{n}||x^{n})=\prod _{i=1}^{n}P(y_{i}|y^{i-1},x^{i})$ = ∏ $P(y^{n}||x^{n})=\prod _{i=1}^{n}P(y_{i}|y^{i-1},x^{i})$ = $P(y^{n}||x^{n})=\prod _{i=1}^{n}P(y_{i}|y^{i-1},x^{i})$ $P(y^{n}||x^{n})=\prod _{i=1}^{n}P(y_{i}|y^{i-1},x^{i})$ $P(y^{n}||x^{n})=\prod _{i=1}^{n}P(y_{i}|y^{i-1},x^{i})$ ( $P(y^{n}||x^{n})=\prod _{i=1}^{n}P(y_{i}|y^{i-1},x^{i})$ $P(y^{n}||x^{n})=\prod _{i=1}^{n}P(y_{i}|y^{i-1},x^{i})$ $P(y^{n}||x^{n})=\prod _{i=1}^{n}P(y_{i}|y^{i-1},x^{i})$ - 1 $P(y^{n}||x^{n})=\prod _{i=1}^{n}P(y_{i}|y^{i-1},x^{i})$ i ) {\ $displaystyle$ P $(y^{n}$ x^{n})=\ $display_{i$ = $1}^{n}P(y_{i} y^{i-1}, x^{i}}}$ 는 확률 벡터, 즉,

P(y^{n}||x^{n})\geq 0\quad {\text{and}}\quad \sum _{y^{n}}P(y^{n}||x^{n})=1\quad {\text{for all }}(x^{n},y^{n})

( y

P(y^{n}||x^{n})\geq 0\quad {\text{and}}\quad \sum _{y^{n}}P(y^{n}||x^{n})=1\quad {\text{for all }}(x^{n},y^{n})

x

P(y^{n}||x^{n})\geq 0\quad {\text{and}}\quad \sum _{y^{n}}P(y^{n}||x^{n})=1\quad {\text{for all }}(x^{n},y^{n})

)

P(y^{n}||x^{n})\geq 0\quad {\text{and}}\quad \sum _{y^{n}}P(y^{n}||x^{n})=1\quad {\text{for all }}(x^{n},y^{n})

P(y^{n}||x^{n})\geq 0\quad {\text{and}}\quad \sum _{y^{n}}P(y^{n}||x^{n})=1\quad {\text{for all }}(x^{n},y^{n})

이고 ∑

P(y^{n}||x^{n})\geq 0\quad {\text{and}}\quad \sum _{y^{n}}P(y^{n}||x^{n})=1\quad {\text{for all }}(x^{n},y^{n})

n

P(y^{n}||x^{n})\geq 0\quad {\text{and}}\quad \sum _{y^{n}}P(y^{n}||x^{n})=1\quad {\text{for all }}(x^{n},y^{n})

(

P(y^{n}||x^{n})\geq 0\quad {\text{and}}\quad \sum _{y^{n}}P(y^{n}||x^{n})=1\quad {\text{for all }}(x^{n},y^{n})

P(y^{n}||x^{n})\geq 0\quad {\text{and}}\quad \sum _{y^{n}}P(y^{n}||x^{n})=1\quad {\text{for all }}(x^{n},y^{n})

)

=

P(y^{n}||x^{n})\geq 0\quad {\text{and}}\quad \sum _{y^{n}}P(y^{n}||x^{n})=1\quad {\text{for all }}(x^{n},y^{n})

에

P(y^{n}||x^{n})\geq 0\quad {\text{and}}\quad \sum _{y^{n}}P(y^{n}||x^{n})=1\quad {\text{for all }}(x^{n},y^{n})

1 입니다

.

{\

displaystyle

P

(y^{n}

x

^{n})\geq

0

\text{and

}\

text

{and

}}\

text \

sum _{y^{n

} P

(y^{n}

x

^

})=

1\text{모두에 대해

}(

x^{n},y^{n

지시 정보는 인과 조건의 ^[2]관점에서 작성될 수 있습니다.

=

H(Y^{n})-H(Y^{n}

X

^{n

관계는 세 개의 문자열로 일반화됩니다. $X^{n}$ {\ $displaystyle$ X $^{n}}$ 에서 $X^{n}$ $Z^{n}$ ${\$ Z $^{n}$ 에 $Z^{n}$ $Y^{n}$ 으로 $Y^{n}$ $Y^{n}$ 된 Yn {\displaystyle Y $^{n}$ 로 흐르는 지시 정보는

=

H(Y^{n}

Z

^{n})-H(Y^{n}

X

^{n},Z^{n

정보의 보존법칙

James Massey와 그의 ^[12]아들 Peter Massey에 의해 만들어진 이 법은 지시된 정보와 상호 정보를 연관시킴으로써 직관을 줍니다.법칙에 따르면 $X^{n},Y^{n}$ 의 $X^{n},Y^{n}$ , $X^{n},Y^{n}$ {\ $displaystyle X^{$ }, $Y^{n$ 다음 등식이 성립합니다.

I(X^{n});Y^{n})=I(X^{n}\toY^{n})+I(Y^{n-1}\to X^{n}).

이 법칙의^[2]^[13] 두가지 대안적 형태는

I(X^{n});Y^{n})=I(X^{n}\toY^{n})+I(Y^{n}\toX^{n})-I(X^{n}\좌측 오른쪽 화살표 Y^{n})

I(X^{n});Y^{n})=I(X^{n-1}\toY^{n})+I(Y^{n-1}\to X^{n})+I(X^{n}\좌측 오른쪽 화살표 Y^{n})

$I(X^{n}\leftrightarrow Y^{n})=\sum _{i=1}^{n}I(X_{i};Y_{i}|X^{i-1},Y^{i-1})$ 서 $I(X^{n}\leftrightarrow Y^{n})=\sum _{i=1}^{n}I(X_{i};Y_{i}|X^{i-1},Y^{i-1})$ I ( $I(X^{n}\leftrightarrow Y^{n})=\sum _{i=1}^{n}I(X_{i};Y_{i}|X^{i-1},Y^{i-1})$ n ↔ $I(X^{n}\leftrightarrow Y^{n})=\sum _{i=1}^{n}I(X_{i};Y_{i}|X^{i-1},Y^{i-1})$ ) = ∑ $I(X^{n}\leftrightarrow Y^{n})=\sum _{i=1}^{n}I(X_{i};Y_{i}|X^{i-1},Y^{i-1})$ = $I(X^{n}\leftrightarrow Y^{n})=\sum _{i=1}^{n}I(X_{i};Y_{i}|X^{i-1},Y^{i-1})$ $I(X^{n}\leftrightarrow Y^{n})=\sum _{i=1}^{n}I(X_{i};Y_{i}|X^{i-1},Y^{i-1})$ I ( $I(X^{n}\leftrightarrow Y^{n})=\sum _{i=1}^{n}I(X_{i};Y_{i}|X^{i-1},Y^{i-1})$ $I(X^{n}\leftrightarrow Y^{n})=\sum _{i=1}^{n}I(X_{i};Y_{i}|X^{i-1},Y^{i-1})$ $I(X^{n}\leftrightarrow Y^{n})=\sum _{i=1}^{n}I(X_{i};Y_{i}|X^{i-1},Y^{i-1})$ $I(X^{n}\leftrightarrow Y^{n})=\sum _{i=1}^{n}I(X_{i};Y_{i}|X^{i-1},Y^{i-1})$ - 1 $I(X^{n}\leftrightarrow Y^{n})=\sum _{i=1}^{n}I(X_{i};Y_{i}|X^{i-1},Y^{i-1})$ $I(X^{n}\leftrightarrow Y^{n})=\sum _{i=1}^{n}I(X_{i};Y_{i}|X^{i-1},Y^{i-1})$ $I(X^{n}\leftrightarrow Y^{n})=\sum _{i=1}^{n}I(X_{i};Y_{i}|X^{i-1},Y^{i-1})$ ) $I(X^{n}\leftrightarrow Y^{n})=\sum _{i=1}^{n}I(X_{i};Y_{i}|X^{i-1},Y^{i-1})$ {\ $displaystyle$ I( $X^{n}\좌우 화살표$ Y $^{n$ })=\ $sum$ _ ${i$ = $1}^{n}I(X_{i};$ $Y_{i} X^{i-1},Y^{i-1$

추정 및 최적화

n개의 ${\$ $displaystyle$ n $}$ 개의 $n$ 항이 $있고$ n개의{\displaystyle n $}$ 이 $n$ (가) 크기 때문에 지시된 정보를 추정하고 최적화하는 것이 어렵습니다.많은 경우, 제한 평균, 즉 $n개$ 의 {\ $displaystyle$ n $n$ }이(가 $)$ 무한대로 성장할 $n$ 를 다중 문자 표현식이라고 하는 최적화에 관심이 있습니다.

견적

방향 정보 식을 표본에 $\{P(x_{i},y_{i}|x^{i-1},y^{i-1})_{i=1}^{n}\}$ 하지 않고 합동 분포 {P $\{P(x_{i},y_{i}|x^{i-1},y^{i-1})_{i=1}^{n}\}$ ( $\{P(x_{i},y_{i}|x^{i-1},y^{i-1})_{i=1}^{n}\}$ $\{P(x_{i},y_{i}|x^{i-1},y^{i-1})_{i=1}^{n}\}$ $\{P(x_{i},y_{i}|x^{i-1},y^{i-1})_{i=1}^{n}\}$ - 1 $\{P(x_{i},y_{i}|x^{i-1},y^{i-1})_{i=1}^{n}\}$ $\{P(x_{i},y_{i}|x^{i-1},y^{i-1})_{i=1}^{n}\}$ $\{P(x_{i},y_{i}|x^{i-1},y^{i-1})_{i=1}^{n}\}$ ) $\{P(x_{i},y_{i}|x^{i-1},y^{i-1})_{i=1}^{n}\}$ = $\{P(x_{i},y_{i}|x^{i-1},y^{i-1})_{i=1}^{n}\}$ n $\{P(x_{i},y_{i}|x^{i-1},y^{i-1})_{i=1}^{n}\}$ ${\displaystyle$ \{ $P(x_{i$ }, $y_{i} x^{i-1},y^{i-1})_{i$ = $1}^{n}\}:$ 알 수 없는 샘플로부터 방향 정보를 추정하는 것은 어려운 문제입니다.상황 트리^[14] 가중치 및 경험적 모수 분포를^[15] 기반으로 하고 장기 단기 ^[16]메모리를 사용하는 여러 알고리즘이 있습니다.

최적화

지시 정보를 극대화하는 것은 정보 이론의 근본적인 문제입니다.예를 들어 채널 $\{P(y_{i}|x^{i},y^{i-1}\}_{i=1}^{n})$ 분포 { $\{P(y_{i}|x^{i},y^{i-1}\}_{i=1}^{n})$ ( $\{P(y_{i}|x^{i},y^{i-1}\}_{i=1}^{n})$ $\{P(y_{i}|x^{i},y^{i-1}\}_{i=1}^{n})$ $\{P(y_{i}|x^{i},y^{i-1}\}_{i=1}^{n})$ $\{P(y_{i}|x^{i},y^{i-1}\}_{i=1}^{n})$ i - $\{P(y_{i}|x^{i},y^{i-1}\}_{i=1}^{n})$ $\{P(y_{i}|x^{i},y^{i-1}\}_{i=1}^{n})$ $\{P(y_{i}|x^{i},y^{i-1}\}_{i=1}^{n})$ = $\{P(y_{i}|x^{i},y^{i-1}\}_{i=1}^{n})$ n ) $\{P(y_{i}|x^{i},y^{i-1}\}_{i=1}^{n})$ {\ $displaystyle$ \{ $P(y_{i} x^{i},y^{i-1}\}_{i$ = $I(X^{n}\to Y^{n})$ $}^{n$ 인 경우 채널 입력 $\{P(x_{i}|x^{i-1},y^{i-1}\}_{i=1}^{n})$ 분포 { $\{P(x_{i}|x^{i-1},y^{i-1}\}_{i=1}^{n})$ $\{P(x_{i}|x^{i-1},y^{i-1}\}_{i=1}^{n})$ $\{P(x_{i}|x^{i-1},y^{i-1}\}_{i=1}^{n})$ $\{P(x_{i}|x^{i-1},y^{i-1}\}_{i=1}^{n})$ $\{P(x_{i}|x^{i-1},y^{i-1}\}_{i=1}^{n})$ i - $\{P(x_{i}|x^{i-1},y^{i-1}\}_{i=1}^{n})$ $\{P(x_{i}|x^{i-1},y^{i-1}\}_{i=1}^{n})$ - $\{P(x_{i}|x^{i-1},y^{i-1}\}_{i=1}^{n})$ $\{P(x_{i}|x^{i-1},y^{i-1}\}_{i=1}^{n})$ $\{P(x_{i}|x^{i-1},y^{i-1}\}_{i=1}^{n})$ = $\{P(x_{i}|x^{i-1},y^{i-1}\}_{i=1}^{n})$ ) $\{P(x_{i}|x^{i-1},y^{i-1}\}_{i=1}^{n})$ ${\displaystyle$ I $(X^{n}\to$ Y $^{n})$ {\ $displaystyle \{P(x_{i$ } x $^{i-1},y^{i-1}\}_{i$ = $1}^{{$ i}^{{i→1}) $n$

Blahut-Arimoto,^[17] Markov 의사결정 프로세스,^[18]^[19]^[20]^[21] Recurrent neural ^[16]network, Reinforcement learning.^[22] 및 Graphical method(Q-graphs)^[23]^[24]를 기반으로 지시 정보를 최적화하는 알고리즘이 있습니다.Blahut-Arimoto ^[17]알고리즘의 주요 아이디어는 지시 정보 표현의 마지막 상호 정보에서 시작하여 뒤로 가는 것입니다.마코프 결정 과정의 경우,^[18]^[19]^[20]^[21] 주요 아이디어는 최적화를 무한 수평 평균 보상 마코프 결정 과정으로 변환하는 것입니다.순환 신경망의 ^[16]경우, 주요 아이디어는 순환 신경망을 사용하여 입력 분포를 모델링하고 그래디언트 강하를 사용하여 파라미터를 최적화하는 것입니다.강화 ^[22]학습의 주요 아이디어는 강화 학습 도구를 사용하여 용량의 마르코프 결정 프로세스 공식을 해결하는 것으로, 이를 통해 큰 알파벳이나 연속적인 알파벳을 다룰 수 있습니다.

마르코의 쌍방향 커뮤니케이션 이론

Massey의 지시 정보는 Marko의 초기 양방향 의사소통 ^[25]^[26]이론 개발(1966)에 의해 동기 부여되었습니다.Marko의 방향 전환 정보에 대한 정의는 {\ $displaystyle$ n $n$ 에서 $X^{n-1},Y^{n-1}$ $X^{n-1},Y^{n-1}$ - $X^{n-1},Y^{n-1}$ $X^{n-1},Y^{n-1}$ $-$ 1 {\ $displaystyle X^{n-1},$ Y^{ $n-1$ 에 대한 조건을 하나씩만 적용하고 한계를 갖는다는 점에서 Massey의 정의와 약간 다릅니다.

{\displaystyle T_{12}=\lim _{n\to \infty}\mathbf {E} \left[-\log {P(X_{n} X^{n-1}}}{P(X_{n} X^{n-1}}, Y^{n-1}}\right]\quad{\text{and}}\lim T_{21}=\lim _{n\to \infty}\mathbf {E} \left[-\log {P(Y_{n} Y^{n-1}}{P(Y_{n} Y^{n-1}}}\right}.

마르코는 다음과 같은 몇 가지 다른 양을 정의했습니다.

총 정보: $H_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1})\right]$ 1 $H_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1})\right]$ $H_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1})\right]$ n → $H_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1})\right]$ $H_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1})\right]$ [ - $H_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1})\right]$ ⁡ $H_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1})\right]$ ( $H_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1})\right]$ $H_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1})\right]$ $H_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1})\right]$ n - $H_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1})\right]$ ) ] $H_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1})\right]$ {\ $displaystyle$ _ ${n\to$ \ $infty}\mathbf {E}$ \ $left[-\log$ P( $H_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1})\right]$ ${n}$ X $^{n-1})\right]}$ $H_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1})\right]$ $H_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1})\right]$ $H_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1})\right]$ $H_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1})\right]$ $H_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1})\right]$ $H_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1})\right]$ $H_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1})\right]$ n → $H_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1})\right]$ $H_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1})\right]$ [ - $H_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1})\right]$ $H_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1})\right]$ ⁡ $H_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1})\right]$ ( $H_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1})\right]$ $H_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1})\right]$ $H_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1})\right]$ - $H_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1})\right]$ ) ] $H_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1})\right]$ {\ $displaystyle$ $H_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1})\right]$ _ ${2}=\lim$ _ ${n\to \infty}\mathbf {E}$ \ $left[-\log$ P( $Y_{n}$ Y $^{n-1})\right]}$
무료 정보: $F_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1},Y^{n-1})\right]$ 1 $F_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1},X^{n-1})\right]$ $F_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1},Y^{n-1})\right]$ n → $F_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1},X^{n-1})\right]$ $F_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1},Y^{n-1})\right]$ [ - $F_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1},Y^{n-1})\right]$ ⁡ $F_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1},Y^{n-1})\right]$ ( $F_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1},Y^{n-1})\right]$ $F_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1},Y^{n-1})\right]$ X $F_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1},Y^{n-1})\right]$ - $F_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1},Y^{n-1})\right]$ , $F_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1},Y^{n-1})\right]$ n - $F_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1},Y^{n-1})\right]$ ) ] ${\displaystyle$ _ ${n\to$ \ $infty}\mathbf {E}$ \left[ -\ $log$ P( $X_{n} X^{n-1$ }, Y^{ $n-1})\right]}$ $F_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1},Y^{n-1})\right]$ $F_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1},X^{n-1})\right]$ $F_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1},X^{n-1})\right]$ $F_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1},X^{n-1})\right]$ $F_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1},Y^{n-1})\right]$ $F_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1},X^{n-1})\right]$ $F_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1},Y^{n-1})\right]$ n → $F_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1},X^{n-1})\right]$ [ - $F_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1},Y^{n-1})\right]$ log $F_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1},X^{n-1})\right]$ $F_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1},X^{n-1})\right]$ ( $F_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1},X^{n-1})\right]$ Y $F_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1},X^{n-1})\right]$ - $F_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1},X^{n-1})\right]$ $F_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1},X^{n-1})\right]$ - $F_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1},X^{n-1})\right]$ ) ] $F_{2}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(Y_{n}|Y^{n-1},X^{n-1})\right]$ {\ $displaystyle$ $F_{1}=\lim _{n\to \infty }\mathbf {E} \left[-\log P(X_{n}|X^{n-1},Y^{n-1})\right]$ _ ${2}=\lim$ _ ${n\to \infty}\mathbf {E}$ \ $left[-\log$ P( $Y_{n} Y^{n-1}, X^{n-1})\right]$
일치: $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ = $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ n → ∞ $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ [ - $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ ⁡ $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ ( X $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ - $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ ) $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ ( $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ n $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ - $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ ) $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ ( $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ n , $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ - $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ ) $K=\lim _{n\to \infty }\mathbf {E} \left[-\log {\frac {P(X_{n}|X^{n-1})P(Y_{n}|Y^{n-1})}{P(X_{n},Y_{n}|X^{n-1},Y^{n-1})}}\right].$ ] $displaystyle$ K=\ $lim$ _ ${n\to \infty}\mathbf {E}$ \ $left[-\log$ {\ $frac {P(X_{n}$ X $^{n-1})P(Y_{n}$ Y $^{n-1}){P(X_{n},$ $Y_{n} X^{n-1},Y^{n-1}}\right]}.$

총 정보는 보통 엔트로피 속도라고 불립니다.Marko는 그가 관심을 가지고 있는 문제에 대해 다음과 같은 관계를 보여주었습니다.

$K=T_{12}+T_{21}$
$H_{1}=T_{12}+F_{1}$ 1 $H_{2}=T_{21}+F_{2}$ $H_{1}=T_{12}+F_{1}$ 12 $H_{1}=T_{12}+F_{1}$ + $H_{1}=T_{12}+F_{1}$ $H_{1}=T_{12}+F_{1}$ {\ $displaystyle H$ } = $T_{12}$ + $F_{1},$ $H_{2}=T_{21}+F_{2}$ 2 = $H_{2}=T_{21}+F_{2}$ $H_{2}=T_{21}+F_{2}$ $H_{2}=T_{21}+F_{2}$ $H_{2}=T_{21}+F_{2}$ 2 $H_{2}=T_{21}+F_{2}$ {\ $displaystyle H$ $H_{1}=T_{12}+F_{1}$ ${2$ } = $T_{21}$ + $F_{2}}$

또한 그는 잔류 엔트로피라고 부르는 양을 다음과 같이 정의했습니다.

$R_{1}=H_{1}-K=F_{1}-T_{21}$
$R_{2}=H_{2}-K=F_{2}-T_{12}$

그리고 $F_{1}+F_{2}=R_{1}+R_{2}+K=H_{1}+H_{2}-K$ $F_{1}+F_{2}=R_{1}+R_{2}+K=H_{1}+H_{2}-K$ 1 + $F_{1}+F_{2}=R_{1}+R_{2}+K=H_{1}+H_{2}-K$ 2 = $F_{1}+F_{2}=R_{1}+R_{2}+K=H_{1}+H_{2}-K$ + $F_{1}+F_{2}=R_{1}+R_{2}+K=H_{1}+H_{2}-K$ + $F_{1}+F_{2}=R_{1}+R_{2}+K=H_{1}+H_{2}-K$ = $F_{1}+F_{2}=R_{1}+R_{2}+K=H_{1}+H_{2}-K$ 1 + $F_{1}+F_{2}=R_{1}+R_{2}+K=H_{1}+H_{2}-K$ - K $F_{1}+F_{2}=R_{1}+R_{2}+K=H_{1}+H_{2}-K$ {\ $displaystyle$ F_ ${1}$ + $F_{2$ } = $R_{1}$ + $R_{2}$ + K = $H_{1}+H_{2}-K}$ 및 $F_{1}+F_{2}=R_{1}+R_{2}+K=H_{1}+H_{2}-K$ 여러 경계가 있습니다.

전이 엔트로피와의 관계

지시 정보는 Marko의 지시 전달 $T_{21}$ $T_{21}$ {\ $displaystyle$ T_ ${21$ 의 절단된 버전인 전달 엔트로피와 관련이 있습니다.

${\displaystyle$ i $}$ $d$ 에서 $i$ 메모리가 {\ $displaystyle$ d $}$ 인 $d$ 전송 엔트로피는

{\displaystyle T_{X\toY}=I(X_{i-1},\dots,X_{i-d};Y_{i} Y_{i-1},\dots,Y_{i-d}).

현재 $X_{i}$ $X_{i}$ {\ $displaystyle$ X_ ${i}$ 또는 $X_{i}$ 과거 $X^{i-d-1},Y^{i-d-1}$ $X^{i-d-1},Y^{i-d-1}$ - $X^{i-d-1},Y^{i-d-1}$ - $X^{i-d-1},Y^{i-d-1}$ $X^{i-d-1},Y^{i-d-1}$ - $X^{i-d-1},Y^{i-d-1}$ - $X^{i-d-1},Y^{i-d-1}$ {\ $displaystyle$ X $^{i-d-1}, Y^{i-displaystyle$ $i-d}$ 를 $X^{i-d-1},Y^{i-d-1}$ 포함하지 않는 경우 i $i-d$ - $i-d$ {\ $displaystyle$ i-d $i-d$

전송 엔트로피는 일반적으로 정지성을 가정합니다. $T_{X\to Y}$ , $T_{X\to Y}$ → $T_{X\to Y}$ $T_{X\toY$ 는 i ${\displaystyle$ i $i$ 시간에 의존하지 $i$ .

참고문헌

^ ^a ^b ^c Massey, James (1990). "Causality, Feedback And Directed Information". Proceedings 1990 International Symposium on Information Theory and its Applications, Waikiki, Hawaii, Nov. 27-30, 1990.
^ ^a ^b ^c ^d Kramer, Gerhard (1998). Directed information for channels with feedback (Doctoral). ETH Zurich. doi:10.3929/ethz-a-001988524. hdl:20.500.11850/143796.
^ Tatikonda, Sekhar Chandra (2000). Control under communication constraints (Doctoral). Massachusetts Institute of Technology. hdl:1721.1/16755.
^ Permuter, Haim Henry; Weissman, Tsachy; Goldsmith, Andrea J. (February 2009). "Finite State Channels With Time-Invariant Deterministic Feedback". IEEE Transactions on Information Theory. 55 (2): 644–662. arXiv:cs/0608070. doi:10.1109/TIT.2008.2009849. S2CID 13178.
^ ^a ^b Kramer, G. (January 2003). "Capacity results for the discrete memoryless network". IEEE Transactions on Information Theory. 49 (1): 4–21. doi:10.1109/TIT.2002.806135.
^ Kramer, Gerhard (April 2014). "Information Networks With In-Block Memory". IEEE Transactions on Information Theory. 60 (4): 2105–2120. arXiv:1206.5389. doi:10.1109/TIT.2014.2303120. S2CID 16382644.
^ Permuter, Haim H.; Kim, Young-Han; Weissman, Tsachy (June 2011). "Interpretations of Directed Information in Portfolio Theory, Data Compression, and Hypothesis Testing". IEEE Transactions on Information Theory. 57 (6): 3248–3259. arXiv:0912.4872. doi:10.1109/TIT.2011.2136270. S2CID 11722596.
^ Simeone, Osvaldo; Permuter, Haim Henri (June 2013). "Source Coding When the Side Information May Be Delayed". IEEE Transactions on Information Theory. 59 (6): 3607–3618. arXiv:1109.1293. doi:10.1109/TIT.2013.2248192. S2CID 3211485.
^ Charalambous, Charalambos D.; Stavrou, Photios A. (August 2016). "Directed Information on Abstract Spaces: Properties and Variational Equalities". IEEE Transactions on Information Theory. 62 (11): 6019–6052. arXiv:1302.3971. doi:10.1109/TIT.2016.2604846. S2CID 8107565.
^ Tanaka, Takashi; Esfahani, Peyman Mohajerin; Mitter, Sanjoy K. (January 2018). "LQG Control With Minimum Directed Information: Semidefinite Programming Approach". IEEE Transactions on Automatic Control. 63 (1): 37–52. arXiv:1510.04214. doi:10.1109/TAC.2017.2709618. S2CID 1401958.
^ Vinkler, Dror A; Permuter, Haim H; Merhav, Neri (20 April 2016). "Analogy between gambling and measurement-based work extraction". Journal of Statistical Mechanics: Theory and Experiment. 2016 (4): 043403. arXiv:1404.6788. Bibcode:2016JSMTE..04.3403V. doi:10.1088/1742-5468/2016/04/043403. S2CID 124719237.
^ Massey, J.L.; Massey, P.C. (September 2005). "Conservation of mutual and directed information". Proceedings. International Symposium on Information Theory, 2005. ISIT 2005. pp. 157–158. doi:10.1109/ISIT.2005.1523313. ISBN 0-7803-9151-9. S2CID 38053218.
^ Amblard, Pierre-Olivier; Michel, Olivier (28 December 2012). "The Relation between Granger Causality and Directed Information Theory: A Review". Entropy. 15 (1): 113–143. arXiv:1211.3169. Bibcode:2012Entrp..15..113A. doi:10.3390/e15010113.
^ Jiao, Jiantao; Permuter, Haim H.; Zhao, Lei; Kim, Young-Han; Weissman, Tsachy (October 2013). "Universal Estimation of Directed Information". IEEE Trans. Inf. Theory. 59 (10): 6220–6242. arXiv:1201.2334. doi:10.1109/TIT.2013.2267934. S2CID 10855063.
^ Quinn, Christopher J.; Kiyavash, Negar; Coleman, Todd P. (December 2015). "Directed Information Graphs". IEEE Trans. Inf. Theory. 61 (12): 6887–6909. arXiv:1204.2003. doi:10.1109/TIT.2015.2478440. S2CID 3121664.
^ ^a ^b ^c Aharoni, Ziv; Tsur, Dor; Goldfeld, Ziv; Permuter, Haim Henry (June 2020). "Capacity of Continuous Channels with Memory via Directed Information Neural Estimator". 2020 IEEE International Symposium on Information Theory (ISIT). pp. 2014–2019. arXiv:2003.04179. doi:10.1109/ISIT44484.2020.9174109. ISBN 978-1-7281-6432-8. S2CID 212634151. {{cite book}}: journal=무시됨(도움말)
^ ^a ^b Naiss, Iddo; Permuter, Haim H. (January 2013). "Extension of the Blahut–Arimoto Algorithm for Maximizing Directed Information". IEEE Trans. Inf. Theory. 59 (1): 204–222. arXiv:1012.5071. doi:10.1109/TIT.2012.2214202. S2CID 3115749.
^ ^a ^b Permuter, Haim; Cuff, Paul; Van Roy, Benjamin; Weissman, Tsachy (July 2008). "Capacity of the Trapdoor Channel With Feedback". IEEE Trans. Inf. Theory. 54 (7): 3150–3165. arXiv:cs/0610047. doi:10.1109/TIT.2008.924681. S2CID 1265.
^ ^a ^b Elishco, Ohad; Permuter, Haim (September 2014). "Capacity and Coding for the Ising Channel With Feedback". IEEE Trans. Inf. Theory. 60 (9): 5138–5149. arXiv:1205.4674. doi:10.1109/TIT.2014.2331951. S2CID 9761759.
^ ^a ^b Sabag, Oron; Permuter, Haim H.; Kashyap, Navin (January 2016). "The Feedback Capacity of the Binary Erasure Channel With a No-Consecutive-Ones Input Constraint". IEEE Trans. Inf. Theory. 62 (1): 8–22. doi:10.1109/TIT.2015.2495239. S2CID 476381.
^ ^a ^b Peled, Ori; Sabag, Oron; Permuter, Haim H. (July 2019). "Feedback Capacity and Coding for the $(0,k)$ -RLL Input-Constrained BEC". IEEE Trans. Inf. Theory. 65 (7): 4097–4114. arXiv:1712.02690. doi:10.1109/TIT.2019.2903252. S2CID 86582654.
^ ^a ^b Aharoni, Ziv; Sabag, Oron; Permuter, Haim Henri (18 August 2020). "Reinforcement Learning Evaluation and Solution for the Feedback Capacity of the Ising Channel with Large Alphabet". arXiv:2008.07983 [cs.IT].
^ Sabag, Oron; Permuter, Haim Henry; Pfister, Henry (March 2017). "A Single-Letter Upper Bound on the Feedback Capacity of Unifilar Finite-State Channels". IEEE Trans. Inf. Theory. 63 (3): 1392–1409. arXiv:1604.01878. doi:10.1109/TIT.2016.2636851. S2CID 3259603.
^ Sabag, Oron; Huleihel, Bashar; Permuter, Haim Henry (2020). "Graph-Based Encoders and their Performance for Finite-State Channels with Feedback". IEEE Trans. Commun. 68 (4): 2106–2117. arXiv:1907.08063. doi:10.1109/TCOMM.2020.2965454. S2CID 197544824.
^ Marko, Hans (1 September 1966). "Die Theorie der bidirektionalen Kommunikation und ihre Anwendung auf die Nachrichtenübermittlung zwischen Menschen (Subjektive Information)". Kybernetik (in German). 3 (3): 128–136. doi:10.1007/BF00288922. ISSN 1432-0770. PMID 5920460. S2CID 33275199.
^ Marko, H. (December 1973). "The Bidirectional Communication Theory--A Generalization of Information Theory". IEEE Transactions on Communications. 21 (12): 1345–1351. doi:10.1109/TCOM.1973.1091610. S2CID 51664185.

[Massey_1990-1] Massey, James (1990). "Causality, Feedback And Directed Information". Proceedings 1990 International Symposium on Information Theory and its Applications, Waikiki, Hawaii, Nov. 27-30, 1990.

[Kramer_1998-2] Kramer, Gerhard (1998). Directed information for channels with feedback (Doctoral). ETH Zurich. doi:10.3929/ethz-a-001988524. hdl:20.500.11850/143796.

[3] Tatikonda, Sekhar Chandra (2000). Control under communication constraints (Doctoral). Massachusetts Institute of Technology. hdl:1721.1/16755.

[2008.2009849-4] Permuter, Haim Henry; Weissman, Tsachy; Goldsmith, Andrea J. (February 2009). "Finite State Channels With Time-Invariant Deterministic Feedback". IEEE Transactions on Information Theory. 55 (2): 644–662. arXiv:cs/0608070. doi:10.1109/TIT.2008.2009849. S2CID 13178.

[Kramer_2003-5] Kramer, G. (January 2003). "Capacity results for the discrete memoryless network". IEEE Transactions on Information Theory. 49 (1): 4–21. doi:10.1109/TIT.2002.806135.

[6] Kramer, Gerhard (April 2014). "Information Networks With In-Block Memory". IEEE Transactions on Information Theory. 60 (4): 2105–2120. arXiv:1206.5389. doi:10.1109/TIT.2014.2303120. S2CID 16382644.

[7] Permuter, Haim H.; Kim, Young-Han; Weissman, Tsachy (June 2011). "Interpretations of Directed Information in Portfolio Theory, Data Compression, and Hypothesis Testing". IEEE Transactions on Information Theory. 57 (6): 3248–3259. arXiv:0912.4872. doi:10.1109/TIT.2011.2136270. S2CID 11722596.

[8] Simeone, Osvaldo; Permuter, Haim Henri (June 2013). "Source Coding When the Side Information May Be Delayed". IEEE Transactions on Information Theory. 59 (6): 3607–3618. arXiv:1109.1293. doi:10.1109/TIT.2013.2248192. S2CID 3211485.

[9] Charalambous, Charalambos D.; Stavrou, Photios A. (August 2016). "Directed Information on Abstract Spaces: Properties and Variational Equalities". IEEE Transactions on Information Theory. 62 (11): 6019–6052. arXiv:1302.3971. doi:10.1109/TIT.2016.2604846. S2CID 8107565.

[10] Tanaka, Takashi; Esfahani, Peyman Mohajerin; Mitter, Sanjoy K. (January 2018). "LQG Control With Minimum Directed Information: Semidefinite Programming Approach". IEEE Transactions on Automatic Control. 63 (1): 37–52. arXiv:1510.04214. doi:10.1109/TAC.2017.2709618. S2CID 1401958.

[11] Vinkler, Dror A; Permuter, Haim H; Merhav, Neri (20 April 2016). "Analogy between gambling and measurement-based work extraction". Journal of Statistical Mechanics: Theory and Experiment. 2016 (4): 043403. arXiv:1404.6788. Bibcode:2016JSMTE..04.3403V. doi:10.1088/1742-5468/2016/04/043403. S2CID 124719237.

[12] Massey, J.L.; Massey, P.C. (September 2005). "Conservation of mutual and directed information". Proceedings. International Symposium on Information Theory, 2005. ISIT 2005. pp. 157–158. doi:10.1109/ISIT.2005.1523313. ISBN 0-7803-9151-9. S2CID 38053218.

[13] Amblard, Pierre-Olivier; Michel, Olivier (28 December 2012). "The Relation between Granger Causality and Directed Information Theory: A Review". Entropy. 15 (1): 113–143. arXiv:1211.3169. Bibcode:2012Entrp..15..113A. doi:10.3390/e15010113.

[14] Jiao, Jiantao; Permuter, Haim H.; Zhao, Lei; Kim, Young-Han; Weissman, Tsachy (October 2013). "Universal Estimation of Directed Information". IEEE Trans. Inf. Theory. 59 (10): 6220–6242. arXiv:1201.2334. doi:10.1109/TIT.2013.2267934. S2CID 10855063.

[15] Quinn, Christopher J.; Kiyavash, Negar; Coleman, Todd P. (December 2015). "Directed Information Graphs". IEEE Trans. Inf. Theory. 61 (12): 6887–6909. arXiv:1204.2003. doi:10.1109/TIT.2015.2478440. S2CID 3121664.

[2003.04179-16] Aharoni, Ziv; Tsur, Dor; Goldfeld, Ziv; Permuter, Haim Henry (June 2020). "Capacity of Continuous Channels with Memory via Directed Information Neural Estimator". 2020 IEEE International Symposium on Information Theory (ISIT). pp. 2014–2019. arXiv:2003.04179. doi:10.1109/ISIT44484.2020.9174109. ISBN 978-1-7281-6432-8. S2CID 212634151. {{cite book}}: journal=무시됨(도움말)

[1012.5071-17] Naiss, Iddo; Permuter, Haim H. (January 2013). "Extension of the Blahut–Arimoto Algorithm for Maximizing Directed Information". IEEE Trans. Inf. Theory. 59 (1): 204–222. arXiv:1012.5071. doi:10.1109/TIT.2012.2214202. S2CID 3115749.

[2008.924681-18] Permuter, Haim; Cuff, Paul; Van Roy, Benjamin; Weissman, Tsachy (July 2008). "Capacity of the Trapdoor Channel With Feedback". IEEE Trans. Inf. Theory. 54 (7): 3150–3165. arXiv:cs/0610047. doi:10.1109/TIT.2008.924681. S2CID 1265.

[1205.4674-19] Elishco, Ohad; Permuter, Haim (September 2014). "Capacity and Coding for the Ising Channel With Feedback". IEEE Trans. Inf. Theory. 60 (9): 5138–5149. arXiv:1205.4674. doi:10.1109/TIT.2014.2331951. S2CID 9761759.

[2015.2495239-20] Sabag, Oron; Permuter, Haim H.; Kashyap, Navin (January 2016). "The Feedback Capacity of the Binary Erasure Channel With a No-Consecutive-Ones Input Constraint". IEEE Trans. Inf. Theory. 62 (1): 8–22. doi:10.1109/TIT.2015.2495239. S2CID 476381.

[1712.02690-21] Peled, Ori; Sabag, Oron; Permuter, Haim H. (July 2019). "Feedback Capacity and Coding for the $(0,k)$ -RLL Input-Constrained BEC". IEEE Trans. Inf. Theory. 65 (7): 4097–4114. arXiv:1712.02690. doi:10.1109/TIT.2019.2903252. S2CID 86582654.

[2008.07983-22] Aharoni, Ziv; Sabag, Oron; Permuter, Haim Henri (18 August 2020). "Reinforcement Learning Evaluation and Solution for the Feedback Capacity of the Ising Channel with Large Alphabet". arXiv:2008.07983 [cs.IT].

[23] Sabag, Oron; Permuter, Haim Henry; Pfister, Henry (March 2017). "A Single-Letter Upper Bound on the Feedback Capacity of Unifilar Finite-State Channels". IEEE Trans. Inf. Theory. 63 (3): 1392–1409. arXiv:1604.01878. doi:10.1109/TIT.2016.2636851. S2CID 3259603.

[24] Sabag, Oron; Huleihel, Bashar; Permuter, Haim Henry (2020). "Graph-Based Encoders and their Performance for Finite-State Channels with Feedback". IEEE Trans. Commun. 68 (4): 2106–2117. arXiv:1907.08063. doi:10.1109/TCOMM.2020.2965454. S2CID 197544824.

[25] Marko, Hans (1 September 1966). "Die Theorie der bidirektionalen Kommunikation und ihre Anwendung auf die Nachrichtenübermittlung zwischen Menschen (Subjektive Information)". Kybernetik (in German). 3 (3): 128–136. doi:10.1007/BF00288922. ISSN 1432-0770. PMID 5920460. S2CID 33275199.

[26] Marko, H. (December 1973). "The Bidirectional Communication Theory--A Generalization of Information Theory". IEEE Transactions on Communications. 21 (12): 1345–1351. doi:10.1109/TCOM.1973.1091610. S2CID 51664185.

[1]

[2]

[3]

[4]

[5]

[6]

[7]

[8]

[9]

[10]

[11]

[12]

[13]

[14]

[15]

[16]

[17]

[18]

[19]

[20]

[21]

[22]

[23]

[24]

[25]

[26]

Search