首页 > 电脑常识 > 人工智能

鏈€澶ф湡鏈涚畻娉?Expectation-Maximization algorithm锛孍M)

admin 人工智能 2021-05-25 09:19:48
后台-系统设置-扩展变量-手机广告位-内容正文底部

鏈€澶ф湡鏈涚畻娉?Expectation-Maximization algorithm锛孍M)

  • 鏈€澶ф湡鏈涚畻娉?Expectation-Maximization algorithm锛孍M)
    • 涓€銆丒M绠楁硶鐨勫箍涔夋楠?/a>
    • 浜屻€佸厛鍐欏嚭EM鐨勫叕寮?/a>
    • 涓夈€佸叾鏀舵暃鎬х殑璇佹槑
    • 鍥涖€佸叕寮忔帹瀵兼柟娉?
      • 4.1 E-M姝ラ鍏紡
      • 4.2 鎺ㄥ杩囩▼
    • 浜斻€佸叕寮忔帹瀵兼柟娉?锛堟秹鍙奐ensen涓嶇瓑寮忥級
      • 5.1 Jensen涓嶇瓑寮?/a>
      • 5.2 鍏充簬E-M绠楁硶鐨勭悊瑙?/a>
      • 5.3 鎺ㄥ杩囩▼
    • 鍏€佸箍涔塃M绠楁硶
      • 骞夸箟EM姝ラ锛?/a>
    • 涓冦€丒M绠楁硶鐨勬敼杩?/a>

鈥冣€冩渶澶ф湡鏈涚畻娉曪紝涔熻璇戜綔鏈€澶у寲绠楁硶锛圡inorize-Maxization,MM锛?鏄湪姒傜巼妯″瀷涓鎵惧弬鏁版渶澶т技鐒朵及璁℃垨鑰呮渶澶у悗楠屼及璁$殑绠楁硶锛屽叾涓鐜囨ā鍨嬩緷璧栦簬鏃犳硶瑙傛祴鐨勯殣鎬у彉閲忋€?br> 鈥冣€冩渶澶ф湡鏈涚畻娉曞氨鏄疎-step鍜孧-step浜ゆ浛杩涜璁$畻锛岀洿鑷虫弧瓒虫敹鏁涙潯浠躲€傛墍浠ュ畠鏄竴绉嶈凯浠g畻娉曘€?br> 鈥冣€僂M绠楁硶閫傜敤鍦烘櫙锛氬綋鏁版嵁鏈夌己澶卞€兼椂锛屽嵆鏁版嵁涓嶅畬鏁存椂銆傝繕鏈夊緢澶氭満鍣ㄥ涔犳ā鍨嬬殑姹傝В缁忓父鐢ㄥ埌Em锛屾瘮濡侴MM锛堥珮鏂贩鍚堟ā鍨嬶級銆丠MM锛堥殣椹皵绉戝か妯″瀷锛夌瓑绛夈€?/p>

浜屻€佸厛鍐欏嚭EM鐨勫叕寮?/h2>

鈥冣€冭繖閲屼互鏈€澶т技鐒朵及璁′綔涓哄噯鍒欙細

\[ {\hat \theta _{MLE}} = \arg \max \log P(X|\theta ) \]

鈥冣€僂M鍏紡

\[{\theta ^{(t + 1)}} = \arg \mathop {\max }\limits_\theta \int_Z {\log } P(X,Z|\theta )P(Z|X,{\theta ^{(t)}})dZ \]

鈥冣€?span class="math inline">\(\int_Z {\log } P(X,Z|\theta )P(Z|X,{\theta ^{(t)}})dZ\)涔熷彲浠ュ啓浣?span class="math inline">\({E_{Z|X,{\theta ^{(t)}}}}[\log P(X,Z|\theta )]\)鎴栬€?span class="math inline">\(\sum\limits_Z {\log P(X,Z|\theta )P(Z|X,{\theta ^{(t)}})}\)

涓夈€佸叾鏀舵暃鎬х殑璇佹槑

鈥冣€冩澶勭殑璇佹槑骞朵笉鏄潪甯镐弗鏍肩殑璇佹槑銆傝璇佹槑鍏舵敹鏁涳紝灏辨槸瑕佽瘉鏄庡綋\({\theta ^{(t)}} \to {\theta ^{(t + 1)}}\)锛?span class="math inline">\(P(X|{\theta ^{(t)}}) \to P(X|{\theta ^{(t + 1)}})\)鏃?span class="math inline">\(P(X|{\theta ^{(t)}}) \le P(X|{\theta ^{(t + 1)}})\)銆傝瘉鏄庤繃绋嬪涓嬶細

鈥冣€冭瘉鏄庯細

\[\log P(X|\theta ) = \log P(X,Z|\theta ) - \log P(Z|X,\theta ) \]

鈥冣€冪瓑寮忎袱杈瑰叧浜?span class="math inline">\({Z|X,{\theta ^{(t)}}}\)鍒嗗竷鍚屾椂姹傛湡鏈涳細

鈥冣€冨乏杈?

\[\begin{array}{l} {E_{Z|X,{\theta ^{(t)}}}}[\log P(X|\theta )]\\ = \int_Z {\log } P(X|\theta )P(Z|X,{\theta ^{(t)}})dZ\\ = \log P(X|\theta )\int_Z {P(Z|X,{\theta ^{(t)}})} dZ\\ = \log P(X|\theta ) \end{array} \]

鈥冣€冨彸杈癸細

\[\begin{array}{l} {E_{Z|X,{\theta ^{(t)}}}}\left[ {\log P(X,Z|\theta ) - \log P(Z|X,\theta )} \right]\\ = \int_Z {\log P(X,Z|\theta )P(Z|X,{\theta ^{(t)}})dZ - } \int_Z {\log P(Z|X,\theta )P(Z|X,{\theta ^{(t)}})dZ} \end{array} \]

鈥冣€冧护\(Q锛圽theta ,{\theta ^{(t)}}) = \int_Z {\log P(X,Z|\theta )P(Z|X,{\theta ^{(t)}})dZ}\)锛?span class="math inline">\(H(\theta ,{\theta ^{(t)}}) = \int_Z {\log P(Z|X,\theta )P(Z|X,{\theta ^{(t)}})dZ}\)銆?/p>

鈥冣€冨垯

\[{E_{Z|X,{\theta ^{(t)}}}}\left[ {\log P(X,Z|\theta ) - \log P(Z|X,\theta )} \right] = Q(\theta ,{\theta ^{(t)}}) - H(\theta ,{\theta ^{(t)}}) \]

鈥冣€冪敱EM鐨勭畻娉曞叕寮忓彲寰楋細锛堝洜涓哄氨鏄姹傛弧瓒宠姹傜殑鏈€澶?span class="math inline">\(\theta\)浣滀负\(\theta ^{(t+1)}\)锛?/p>

\[Q({\theta ^{(t + 1)}},{\theta ^{(t)}}) \ge Q(\theta ,{\theta ^{(t)}}) \]

鈥冣€冧篃鍗筹細锛?span class="math inline">\(\theta\)鍙?span class="math inline">\(\theta ^{(t)}\)鏃讹級

\[Q({\theta ^{(t + 1)}},{\theta ^{(t)}}) \ge Q({\theta ^{(t)}},{\theta ^{(t)}}) \]

鈥冣€冨浜?span class="math inline">\(H(\theta ,{\theta ^{(t)}})\)锛?/p>

\[\begin{array}{l} H({\theta ^{(t + 1)}},{\theta ^{(t)}}) - H({\theta ^{(t)}},{\theta ^{(t)}})\\ = \int_Z {\log P(Z|X,{\theta ^{(t + 1)}})} P(Z|X,{\theta ^{(t)}})dZ - \int_Z {\log P(Z|X,{\theta ^t})} P(Z|X,{\theta ^{(t)}})dZ\\ = \int_Z {P(Z|X,{\theta ^{(t)}})} \log \frac{{P(Z|X,{\theta ^{(t + 1)}})}}{{P(Z|X,{\theta ^t})}}dZ\\ = - KL(P(Z|X,{\theta ^{(t)}})||P(Z|X,{\theta ^{(t + 1)}}))\\ \le 0 \end{array} \]

鈥冣€冧篃鍗?span class="math inline">\(H({\theta ^{(t + 1)}},{\theta ^{(t)}}) \le H({\theta ^{(t)}},{\theta ^{(t)}})\)
鈥冣€冪患涓婏紝\(P(X|{\theta ^{(t)}}) \le P(X|{\theta ^{(t + 1)}})\)锛岃瘉姣曘€?/p>

鍥涖€佸叕寮忔帹瀵兼柟娉?

璇存槑涓嬫暟鎹細
鈥冣€僗: observed data 鈥冣€?span class="math inline">\(X = \{ {x_1},{x_2}, \cdots {x_N}\}\)
鈥冣€僙: unovserved data(latent data) 鈥冣€?\(Z = \{ {z_i}\} _{i = 1}^K\)
鈥冣€?X,Z): complete data
鈥冣€?span class="math inline">\(\theta\): parameter

4.1 E-M姝ラ鍏紡

鈥冣€僂-step:

\[P(Z|X,{\theta ^{(t)}}) \to {E_{Z|X,{\theta ^{(t)}}}}[\log P(X,Z|\theta )] \]

鈥冣€僊-step:

\[{\theta ^{(t + 1)}} = \arg \mathop {\max }\limits_\theta {E_{Z|X,{\theta ^{(t)}}}}[\log P(X,Z|\theta )] \]

4.2 鎺ㄥ杩囩▼

\[\log P(X|\theta ) = \log (X,Z|\theta ) - \log (Z|X,\theta ) \]

鈥冣€冪瓑浠蜂唬鎹紝寮曞叆鍒嗗竷\(q(Z)\):

\[\log P(X|\theta ) = \log \frac{{P(X,Z|\theta )}}{{q(z)}} - \log \frac{{P(Z|X,\theta )}}{{q(z)}} , q(Z) \ne 0 \]

鈥冣€冧袱杈瑰悓鏃跺叧浜庡垎甯?span class="math inline">\(q(Z)\)姹傛湡鏈?/p>

鈥冣€冨浜庡乏杈癸細

\[\begin{array}{l} {E_{q(Z)}}[\log P(X|\theta )] = \int_Z {\log } P(X|\theta )P(Z|X,{\theta ^{(t)}})dZ\\ = \log P(X|\theta )\int_Z {P(Z|X,{\theta ^{(t)}})} dZ\\ = \log P(X|\theta ) \end{array} \]

鈥冣€冨浜庡彸杈癸細

\[\begin{array}{l} {E_{Z|X,{\theta ^{(t)}}}}\left[ {\log \frac{{P(X,Z|\theta )}}{{q(z)}} - \log \frac{{P(Z|X,\theta )}}{{q(z)}}} \right]\\ = \int_Z {\log \frac{{P(X,Z|\theta )}}{{q(z)}}} q(z)dZ - \int_Z {\log \frac{{P(Z|X,\theta )}}{{q(z)}}} q(z)dZ\\ = ELBO + KL\left( {q(Z)||P(Z|X,\theta )} \right) \end{array} \]

鈥冣€冨叾涓?span class="math inline">\({P(Z|X,\theta )}\)涓哄悗楠屾鐜囥€侲LBO涓篹vidence lower bound

\[\begin{array}{l} \therefore \log P(X|\theta ) = ELBO + KL\left( {q||P} \right)\\ \because KL\left( {q||P} \right) \ge 0\\ \therefore \log P(X|\theta ) \ge ELBO \end{array} \]

鈥冣€冨垯鍙栨渶澶у€硷紝灏辩瓑浠蜂簬ELBO鍙栨渶澶у€硷紝姝ゆ椂锛?/p>

\[{{\hat \theta }^{(t + 1)}} = \arg \mathop {\max }\limits_\theta ELBO = \arg \mathop {\max }\limits_\theta \int_Z {\log \frac{{P(X,Z|\theta )}}{{q(z)}}} q(z)dZ \]

鈥冣€?span class="math inline">\(\because\)褰?span class="math inline">\(q=P\)鏃讹紝\(KL=0\)锛屽嵆\(\log P(X|\theta ) = ELBO\)鐨勭瓑鍙锋垚绔嬨€?/p>

鈥冣€冨垯鍙?span class="math inline">\(q(z) = P(Z|X,{\theta ^{(t)}})\)锛屽嵆涓婁竴鏃跺埢鐨勫悗楠屻€?br> 鈥冣€?span class="math inline">\(\therefore\)

\[\begin{array}{l} {{\hat \theta }^{(t + 1)}} = \arg \mathop {\max }\limits_\theta \int_Z {\log \frac{{P(X,Z|\theta )}}{{P(Z|X,{\theta ^{(t)}})}}} P(Z|X,{\theta ^{(t)}})dZ\\ = \arg \mathop {\max }\limits_\theta \int_Z {\left[ {\log P(X,Z|\theta ) - \log P(Z|X,{\theta ^{(t)}})} \right]} P(Z|X,{\theta ^{(t)}})dZ \end{array} \]

鈥冣€冩鏃?span class="math inline">\(\theta ^{(t)}\)涓轰笂涓€鏃跺埢鐨勫弬鏁帮紝鏁呭湪姝ゅ彲瑙嗕綔甯告暟锛屽垯鍑忓彿鍚庨潰鐨勫弬鏁伴兘瑙嗕綔甯告暟锛屼笌鐩爣\(\theta\)鏃犲叧锛屽湪姹傝В鏄棤鐢紝鏁呭彲浠ョ渷鍘汇€?/p>

鈥冣€冨垯姝ゆ椂锛?/p>

\[{{\hat \theta }^{(t + 1)}} = \arg \mathop {\max }\limits_\theta \int_Z {\log P(X,Z|\theta )} P(Z|X,{\theta ^{(t)}})dZ \]

鈥冣€冭瘉姣?/p>

浜斻€佸叕寮忔帹瀵兼柟娉?锛堟秹鍙奐ensen涓嶇瓑寮忥級

\(t \in [0,1]\)锛屽垯鏈?span class="math inline">\(f\left( {t \cdot a + (1 - t) \cdot b} \right) \ge tf\left( a \right) + (1 - t)f\left( b \right)\)銆傜壒鍒殑锛屽綋\(a = b = \frac{1}{2}\)鏃讹紝\(f\left( {\frac{{a + b}}{2}} \right) \ge \frac{{f\left( a \right) + f\left( b \right)}}{2}\)銆備粠鏈熸湜鐨勮搴︽潵鐪嬪氨鏄?span class="math inline">\(f\left( E \right) \ge E\left( f \right)\)锛屽厛鏈熸湜鍚庡嚱鏁板ぇ浜庣瓑浜庡厛鍑芥暟鍚庢湡鏈涖€?/p>

\({\log P(X,Z|\theta )}\)鐨勬湡鏈涳紝鍗冲啓鍑轰竴涓叧浜?span class="math inline">\(\theta\)鐨勫嚱鏁般€?br> 鈥冣€冨浜?strong>M-step锛?閽堝E-step鍐欏嚭鐨勬湡鏈涘嚱鏁帮紝姹備娇鍙傛暟\(\theta\)婊¤冻鍏跺彇鏈€澶у€肩殑鍙傛暟浣滀负褰撳墠鏃跺埢鐨勫弬鏁扮洰鏍?span class="math inline">\(\theta ^{(t+1)}\).

5.3 鎺ㄥ杩囩▼

\[\log P(X|\theta ) = \log \int_Z {P(X,Z|\theta )} dZ \]

鑱斿悎鍒嗗竷鐨勮竟缂樼Н鍒?杈圭紭姒傜巼鍒嗗竷

鈥冣€冨紩鍏ュ垎甯?span class="math inline">\(q(Z)\):

\[\begin{array}{l}鍘熷紡= \log \int_Z {\frac{{P(X,Z|\theta )}}{{q(Z)}}} \cdot q(Z)dZ\\ = \log \left[ {{E_{Z|X,{\theta ^{(t)}}}}\left( {\frac{{P(X,Z|\theta )}}{{q(Z)}}} \right)} \right] \end{array} \]

鈥冣€冪敱Jensen涓嶇瓑寮?

\[鍘熷紡\ge {E_{Z|X,{\theta ^{(t)}}}}\left[ {\log \left( {\frac{{P(X,Z|\theta )}}{{q(Z)}}} \right)} \right] = ELBO \]

鈥冣€冨綋\({\frac{{P(X,Z|\theta )}}{{q(Z)}}} = c\),\(c\)涓哄父鏁版椂锛岀瓑鍙锋垚绔嬨€?/p>

鈥冣€冨垯\(q(Z) = \frac{1}{c}P(X,Z|\theta )\)锛屼袱杈瑰悓鏃跺\(Z\)姹傜Н鍒嗭細

\[\begin{array}{c} \int_Z {q(Z)dZ = \int_Z {\frac{1}{c}P(X,Z|\theta )dZ} } \\ \\ 1 = \frac{1}{c}P(X|\theta ) \end{array} \]

鈥冣€?span class="math inline">\(\therefore\)\(q(Z) = P(X|\theta ) \cdot P(X,Z|\theta ) = P(Z|X,\theta )\)

鈥冣€冨悗缁楠ゅ悓鏂规硶涓€銆?/p>

鍏€佸箍涔塃M绠楁硶

鈥冣€冣憼鐙箟EM绠楁硶鏄箍涔塃M绠楁硶鐨勪竴绉嶇壒渚嬶紱
鈥冣€冣憽鐢熸垚妯″瀷涓鏋?span class="math inline">\(Z\)鐨勫鏉傚害澶珮锛屽垯鍚庨獙姒傜巼\(P(Z|X,\theta)\)寰堥毦姹傚嚭锛坕ntractable锛夈€備絾鏄儚GMM鍜孒NN鐨?span class="math inline">\(Z\)鏄粨鏋勫寲鐨勶紝鐩稿绠€鍗曪紝鎵€浠ュ彲浠ョ敤鐙箟EM绠楁硶杩涜浼樺寲銆?/p>

\[\begin{array}{l} \log P(X|\theta ) = ELBO + KL(q||P)\\ = {E_{q(Z)}}\left[ {\log \frac{{P(X,Z|\theta )}}{{q(Z)}}} \right] - {E_{q(Z)}}\left[ {\log \frac{{P(Z|X,\theta )}}{{q(Z)}}} \right] \end{array} \]

\( \left\{ \begin{array}{l} 1.鍥哄畾\theta :\hat q = \arg \mathop {\min }\limits_q KL(q||P) = \arg \mathop {\max }\limits_q ELBO(\hat q,\theta)\\ 2.鍥哄畾\hat q:\theta = \arg \mathop {\max }\limits_\theta ELBO(\hat q,\theta) \end{array} \right. \)

鈥冣€冨搴旂殑锛?/p>

鈥冣€?span class="math inline">\( \left\{ \begin{array}{l} E-step:{q^{(t + 1)}} = \arg \mathop {\max }\limits_q ELBO(q,{\theta ^{(t)}})\\ M-step:{\theta ^{(t + 1)}} = \arg \mathop {\max }\limits_\theta ELBO({q^{(t + 1)}},{\theta ^{(t)}}) \end{array} \right. \)


\[ELBO(q,\theta ) = {E_{q(Z)}} \log P(X,Z|\theta ) - {E_{q(Z)}}\log q(Z) \]

鈥冣€冨叾涓?span class="math inline">\(- {E_{q(Z)}}\log q(Z)\)鏄喌\(H[q(z)]\)锛屽垯

\[ELBO(q,\theta ) = {E_{q(Z)}} \log P(X,Z|\theta ) + H[q(z)] \]

鈥冣€冨箍涔塃M鏄厛鍥哄畾涓€涓弬鏁板湪璁$畻鍙︿竴涓弬鏁帮紝鏁呭彲浠ヤ粠鍧愭爣涓婂崌娉曠殑瑙掑害鍘荤湅銆?/p>

后台-系统设置-扩展变量-手机广告位-内容正文底部
版权声明

本文仅代表作者观点,不代表本站立场。
本文系作者授权发表,未经许可,不得转载。
本文地址:https://www.jcdi.cn/rengongzhineng/30564.html

留言与评论(共有 0 条评论)
   
验证码:
后台-系统设置-扩展变量-手机广告位-评论底部广告位

教程弟

https://www.jcdi.cn/

统计代码 | 京ICP1234567-2号

Powered By 教程弟 教程弟

使用手机软件扫描微信二维码