36.Python实现马尔科夫链

36.Python实现马尔科夫链
学习参考网址：MCMC(二)马尔科夫链 - 刘建平Pinard - 博客园

目录

1.问题描述

2.马尔科夫链模型状态转移矩阵的性质

3.代码实例(一)

3.1运行结果

4.代码实例(2)

4.1运行结果

5.小结

1.问题描述

上图马尔科夫链表示股市模型，有三种状态，每一种状态都以一定的概率转化到下一个状态，这个状态概率转化图也可以用矩阵的形式表示，定义矩阵P某一位置的值为，即从状态i转化到状态j的概率，并定义牛市为状态0，熊市为状态1，横盘为状态2。这样我们得到了马尔科夫链模型的状态转移矩阵为(由下图)：

$P=\begin{pmatrix} 0.9 &0.075 &0.025 \\ 0.15 &0.8 &0.05 \\ 0.25& 0.25 &0.5 \end{pmatrix}$

2.马尔科夫链模型状态转移矩阵的性质

仍然以上面的这个状态转移矩阵为例。假设我们当前股市的概率分布为：[0.3,0.4,0.3]，即30%概率的牛市，40%概率的熊盘与30%的横盘。然后这个状态作为序列概率分布的初始状态 $t_{0}$ ，将其带入这个状态转移矩阵计算 $t_{1},t_{2},t_{3}\cdot\cdot \cdot$ 的状态。

3.代码实例(一)
```
import numpy as np
matrix =  np.matrix([[0.9,0.075,0.025],[0.15,0.8,0.05],[0.25,0.25,0.5]], dtype=float)
vector1 = np.matrix([[0.3,0.4,0.3]], dtype = float)
 
for i in range(100):
    vector1 = vector1*matrix
    print ("Current round:", i+1)
    print (vector1)
```
3.1运行结果

Current round: 1
[[0.405 0.4175 0.1775]]
Current round: 2
[[0.4715 0.40875 0.11975]]
Current round: 3
[[0.5156 0.3923 0.0921]]
Current round: 4
[[0.54591 0.375535 0.078555]]
Current round: 5
[[0.567288 0.36101 0.071702]]
Current round: 6
[[0.5826362 0.3492801 0.0680837]]
Current round: 7
[[0.59378552 0.34014272 0.06607176]]
Current round: 8
[[0.60194632 0.33316603 0.06488765]]
Current round: 9
[[0.6079485 0.32790071 0.06415079]]
Current round: 10
[[0.61237646 0.3239544 0.06366914]]
Current round: 11
[[0.61564926 0.32100904 0.0633417 ]]
Current round: 12
[[0.61807111 0.31881635 0.06311253]]
Current round: 13
[[0.61986459 0.31718655 0.06294886]]
Current round: 14
[[0.62119333 0.3159763 0.06283037]]
Current round: 15
[[0.62217803 0.31507813 0.06274383]]
Current round: 16
[[0.62290791 0.31441182 0.06268027]]
Current round: 17
[[0.62344896 0.31391762 0.06263343]]
Current round: 18
[[0.62385006 0.31355112 0.06259882]]
Current round: 19
[[0.62414743 0.31327936 0.06257322]]
Current round: 20
[[0.62436789 0.31307785 0.06255426]]
Current round: 21
[[0.62453135 0.31292843 0.06254022]]
Current round: 22
[[0.62465253 0.31281765 0.06252982]]
Current round: 23
[[0.62474238 0.31273552 0.0625221 ]]
Current round: 24
[[0.624809 0.31267462 0.06251639]]
Current round: 25
[[0.62485839 0.31262947 0.06251215]]
Current round: 26
[[0.624895 0.31259599 0.06250901]]
Current round: 27
[[0.62492215 0.31257117 0.06250668]]
Current round: 28
[[0.62494228 0.31255277 0.06250495]]
Current round: 29
[[0.62495721 0.31253912 0.06250367]]
Current round: 30
[[0.62496827 0.31252901 0.06250272]]
Current round: 31
[[0.62497648 0.31252151 0.06250202]]
Current round: 32
[[0.62498256 0.31251594 0.0625015 ]]
Current round: 33
[[0.62498707 0.31251182 0.06250111]]
Current round: 34
[[0.62499041 0.31250876 0.06250082]]
Current round: 35
[[0.62499289 0.3125065 0.06250061]]
Current round: 36
[[0.62499473 0.31250482 0.06250045]]
Current round: 37
[[0.62499609 0.31250357 0.06250034]]
Current round: 38
[[0.6249971 0.31250265 0.06250025]]
Current round: 39
[[0.62499785 0.31250196 0.06250018]]
Current round: 40
[[0.62499841 0.31250146 0.06250014]]
Current round: 41
[[0.62499882 0.31250108 0.0625001 ]]
Current round: 42
[[0.62499912 0.3125008 0.06250008]]
Current round: 43
[[0.62499935 0.31250059 0.06250006]]
Current round: 44
[[0.62499952 0.31250044 0.06250004]]
Current round: 45
[[0.62499964 0.31250033 0.06250003]]
Current round: 46
[[0.62499974 0.31250024 0.06250002]]
Current round: 47
[[0.6249998 0.31250018 0.06250002]]
Current round: 48
[[0.62499985 0.31250013 0.06250001]]
Current round: 49
[[0.62499989 0.3125001 0.06250001]]
Current round: 50
[[0.62499992 0.31250007 0.06250001]]
Current round: 51
[[0.62499994 0.31250005 0.06250001]]
Current round: 52
[[0.62499996 0.31250004 0.0625 ]]
Current round: 53
[[0.62499997 0.31250003 0.0625 ]]
Current round: 54
[[0.62499998 0.31250002 0.0625 ]]
Current round: 55
[[0.62499998 0.31250002 0.0625 ]]
Current round: 56
[[0.62499999 0.31250001 0.0625 ]]
Current round: 57
[[0.62499999 0.31250001 0.0625 ]]
Current round: 58
[[0.62499999 0.31250001 0.0625 ]]
Current round: 59
[[0.62499999 0.3125 0.0625 ]]
Current round: 60
[[0.625 0.3125 0.0625]]
Current round: 61
[[0.625 0.3125 0.0625]]
Current round: 62
[[0.625 0.3125 0.0625]]
Current round: 63
[[0.625 0.3125 0.0625]]
Current round: 64
[[0.625 0.3125 0.0625]]
Current round: 65
[[0.625 0.3125 0.0625]]
Current round: 66
[[0.625 0.3125 0.0625]]
Current round: 67
[[0.625 0.3125 0.0625]]
Current round: 68
[[0.625 0.3125 0.0625]]
Current round: 69
[[0.625 0.3125 0.0625]]
Current round: 70
[[0.625 0.3125 0.0625]]
Current round: 71
[[0.625 0.3125 0.0625]]
Current round: 72
[[0.625 0.3125 0.0625]]
Current round: 73
[[0.625 0.3125 0.0625]]
Current round: 74
[[0.625 0.3125 0.0625]]
Current round: 75
[[0.625 0.3125 0.0625]]
Current round: 76
[[0.625 0.3125 0.0625]]
Current round: 77
[[0.625 0.3125 0.0625]]
Current round: 78
[[0.625 0.3125 0.0625]]
Current round: 79
[[0.625 0.3125 0.0625]]
Current round: 80
[[0.625 0.3125 0.0625]]
Current round: 81
[[0.625 0.3125 0.0625]]
Current round: 82
[[0.625 0.3125 0.0625]]
Current round: 83
[[0.625 0.3125 0.0625]]
Current round: 84
[[0.625 0.3125 0.0625]]
Current round: 85
[[0.625 0.3125 0.0625]]
Current round: 86
[[0.625 0.3125 0.0625]]
Current round: 87
[[0.625 0.3125 0.0625]]
Current round: 88
[[0.625 0.3125 0.0625]]
Current round: 89
[[0.625 0.3125 0.0625]]
Current round: 90
[[0.625 0.3125 0.0625]]
Current round: 91
[[0.625 0.3125 0.0625]]
Current round: 92
[[0.625 0.3125 0.0625]]
Current round: 93
[[0.625 0.3125 0.0625]]
Current round: 94
[[0.625 0.3125 0.0625]]
Current round: 95
[[0.625 0.3125 0.0625]]
Current round: 96
[[0.625 0.3125 0.0625]]
Current round: 97
[[0.625 0.3125 0.0625]]
Current round: 98
[[0.625 0.3125 0.0625]]
Current round: 99
[[0.625 0.3125 0.0625]]
Current round: 100
[[0.625 0.3125 0.0625]]

可以发现，从第60轮开始，我们的状态概率分布就不变了，一直保持在[0.625 0.3125 0.0625]，即62.5%的牛市，31.25%的熊市与6.25%的横盘。我们现在换一个初始概率分布试一试，现在我们用[0.7,0.1,0.2]作为初始概率分布。

4.代码实例(2)
```
import numpy as np
matrix =  np.matrix([[0.9,0.075,0.025],[0.15,0.8,0.05],[0.25,0.25,0.5]], dtype=float)
vector1 = np.matrix([[0.7,0.1,0.2]], dtype = float)
 
for i in range(100):
    vector1 = vector1*matrix
    print ("Current round:", i+1)
    print (vector1)
```
4.1运行结果

Current round: 1
[[0.695 0.1825 0.1225]]
Current round: 2
[[0.6835 0.22875 0.08775]]
Current round: 3
[[0.6714 0.2562 0.0724]]
Current round: 4
[[0.66079 0.273415 0.065795]]
Current round: 5
[[0.652172 0.28474 0.063088]]
Current round: 6
[[0.6454378 0.2924769 0.0620853]]
Current round: 7
[[0.64028688 0.29791068 0.06180244]]
Current round: 8
[[0.6363954 0.30180067 0.06180393]]
Current round: 9
[[0.63347695 0.30462117 0.06190188]]
Current round: 10
[[0.6312979 0.30668318 0.06201892]]
Current round: 11
[[0.62967532 0.30819862 0.06212607]]
Current round: 12
[[0.62846909 0.30931606 0.06221485]]
Current round: 13
[[0.6275733 0.31014174 0.06228495]]
Current round: 14
[[0.62690847 0.31075263 0.0623389 ]]
Current round: 15
[[0.62641525 0.31120496 0.06237979]]
Current round: 16
[[0.62604941 0.31154006 0.06241053]]
Current round: 17
[[0.62577811 0.31178839 0.0624335 ]]
Current round: 18
[[0.62557693 0.31197244 0.06245062]]
Current round: 19
[[0.62542776 0.31210888 0.06246336]]
Current round: 20
[[0.62531716 0.31221003 0.06247282]]
Current round: 21
[[0.62523515 0.31228501 0.06247984]]
Current round: 22
[[0.62517435 0.31234061 0.06248505]]
Current round: 23
[[0.62512926 0.31238182 0.06248891]]
Current round: 24
[[0.62509584 0.31241238 0.06249178]]
Current round: 25
[[0.62507106 0.31243504 0.0624939 ]]
Current round: 26
[[0.62505268 0.31245184 0.06249548]]
Current round: 27
[[0.62503906 0.31246429 0.06249665]]
Current round: 28
[[0.62502896 0.31247352 0.06249752]]
Current round: 29
[[0.62502147 0.31248037 0.06249816]]
Current round: 30
[[0.62501592 0.31248545 0.06249863]]
Current round: 31
[[0.6250118 0.31248921 0.06249899]]
Current round: 32
[[0.62500875 0.312492 0.06249925]]
Current round: 33
[[0.62500649 0.31249407 0.06249944]]
Current round: 34
[[0.62500481 0.3124956 0.06249959]]
Current round: 35
[[0.62500357 0.31249674 0.06249969]]
Current round: 36
[[0.62500264 0.31249758 0.06249977]]
Current round: 37
[[0.62500196 0.31249821 0.06249983]]
Current round: 38
[[0.62500145 0.31249867 0.06249988]]
Current round: 39
[[0.62500108 0.31249901 0.06249991]]
Current round: 40
[[0.6250008 0.31249927 0.06249993]]
Current round: 41
[[0.62500059 0.31249946 0.06249995]]
Current round: 42
[[0.62500044 0.3124996 0.06249996]]
Current round: 43
[[0.62500033 0.3124997 0.06249997]]
Current round: 44
[[0.62500024 0.31249978 0.06249998]]
Current round: 45
[[0.62500018 0.31249984 0.06249998]]
Current round: 46
[[0.62500013 0.31249988 0.06249999]]
Current round: 47
[[0.6250001 0.31249991 0.06249999]]
Current round: 48
[[0.62500007 0.31249993 0.06249999]]
Current round: 49
[[0.62500005 0.31249995 0.0625 ]]
Current round: 50
[[0.62500004 0.31249996 0.0625 ]]
Current round: 51
[[0.62500003 0.31249997 0.0625 ]]
Current round: 52
[[0.62500002 0.31249998 0.0625 ]]
Current round: 53
[[0.62500002 0.31249999 0.0625 ]]
Current round: 54
[[0.62500001 0.31249999 0.0625 ]]
Current round: 55
[[0.62500001 0.31249999 0.0625 ]]
Current round: 56
[[0.62500001 0.31249999 0.0625 ]]
Current round: 57
[[0.625 0.3125 0.0625]]
Current round: 58
[[0.625 0.3125 0.0625]]
Current round: 59
[[0.625 0.3125 0.0625]]
Current round: 60
[[0.625 0.3125 0.0625]]
Current round: 61
[[0.625 0.3125 0.0625]]
Current round: 62
[[0.625 0.3125 0.0625]]
Current round: 63
[[0.625 0.3125 0.0625]]
Current round: 64
[[0.625 0.3125 0.0625]]
Current round: 65
[[0.625 0.3125 0.0625]]
Current round: 66
[[0.625 0.3125 0.0625]]
Current round: 67
[[0.625 0.3125 0.0625]]
Current round: 68
[[0.625 0.3125 0.0625]]
Current round: 69
[[0.625 0.3125 0.0625]]
Current round: 70
[[0.625 0.3125 0.0625]]
Current round: 71
[[0.625 0.3125 0.0625]]
Current round: 72
[[0.625 0.3125 0.0625]]
Current round: 73
[[0.625 0.3125 0.0625]]
Current round: 74
[[0.625 0.3125 0.0625]]
Current round: 75
[[0.625 0.3125 0.0625]]
Current round: 76
[[0.625 0.3125 0.0625]]
Current round: 77
[[0.625 0.3125 0.0625]]
Current round: 78
[[0.625 0.3125 0.0625]]
Current round: 79
[[0.625 0.3125 0.0625]]
Current round: 80
[[0.625 0.3125 0.0625]]
Current round: 81
[[0.625 0.3125 0.0625]]
Current round: 82
[[0.625 0.3125 0.0625]]
Current round: 83
[[0.625 0.3125 0.0625]]
Current round: 84
[[0.625 0.3125 0.0625]]
Current round: 85
[[0.625 0.3125 0.0625]]
Current round: 86
[[0.625 0.3125 0.0625]]
Current round: 87
[[0.625 0.3125 0.0625]]
Current round: 88
[[0.625 0.3125 0.0625]]
Current round: 89
[[0.625 0.3125 0.0625]]
Current round: 90
[[0.625 0.3125 0.0625]]
Current round: 91
[[0.625 0.3125 0.0625]]
Current round: 92
[[0.625 0.3125 0.0625]]
Current round: 93
[[0.625 0.3125 0.0625]]
Current round: 94
[[0.625 0.3125 0.0625]]
Current round: 95
[[0.625 0.3125 0.0625]]
Current round: 96
[[0.625 0.3125 0.0625]]
Current round: 97
[[0.625 0.3125 0.0625]]
Current round: 98
[[0.625 0.3125 0.0625]]
Current round: 99
[[0.625 0.3125 0.0625]]
Current round: 100
[[0.625 0.3125 0.0625]]

5.小结

可以看出，尽管这次我们采用了不同初始概率分布，最终状态的概率分布趋于同一个稳定的概率分布[0.625 0.3125 0.0625]，也就是说我们的马尔科夫链模型的状态转移矩阵收敛到的稳定概率分布与我们的初始状态概率分布无关。如果我们得到了这个稳定概率分布对应的马尔科夫链模型的状态转移矩阵，则我们可以用任意的概率分布样本开始，代入马尔科夫链模型的状态转移矩阵，这样经过一些序列的转换，最终就可以得到符合对应稳定概率分布的样本。

这个性质不光对我们上面的状态转移矩阵有效，对于绝大多数的其他的马尔科夫链模型的状态转移矩阵也有效。同时不仅是离散状态下成立，连续状态时也成立。
相关阅读:
自动化项目实战 [个人博客系统]
day41
python获取已安装程序列表
 地图资源下载工具数据在线、离线查询及数据激活功能
 回归预测 | MATLAB实现PCA-GRU主成分门控循环单元多输入单输出回归预测
 Tensorflow、Pytorch和Ray（张量，计算图）
MySQL 面试题
 查找两个总和为特定值的索引
 学习算法和数据结构的第15天
 打工人都觉得香的 Java 程序员 306 道面试秘笈
原文地址：https://blog.csdn.net/m0_57656758/article/details/127847398

1.问题描述

2.马尔科夫链模型状态转移矩阵的性质

3.代码实例(一)

3.1运行结果

4.代码实例(2)

4.1运行结果

5.小结