<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>学而知之</title>
  
  
  <link href="http://www.wangyulong.cn/atom.xml" rel="self"/>
  
  <link href="http://www.wangyulong.cn/"/>
  <updated>2026-02-19T07:45:16.917Z</updated>
  <id>http://www.wangyulong.cn/</id>
  
  <author>
    <name>wangyulong</name>
    
  </author>
  
  <generator uri="https://hexo.io/">Hexo</generator>
  
  <entry>
    <title>Linux SSH 封禁多次登录失败 IP</title>
    <link href="http://www.wangyulong.cn/1761284676/"/>
    <id>http://www.wangyulong.cn/1761284676/</id>
    <published>2025-10-24T13:46:30.000Z</published>
    <updated>2026-02-19T07:45:16.917Z</updated>
    
    <content type="html"><![CDATA[<h3 id="背景">背景</h3><p>服务器只要对外暴露端口，就会引来各种扫描。今天登录服务器，查看/var/log/auth.log 日志，发现有几乎一直有人尝试用 ssh登录服务器。虽然没有成功的，但是让人一直暴力破解也有些安全隐患。本文也记录一下本次操作的方法，便于以后再有新的服务器，也能快速做自动封禁。</p><h3 id="封禁-ip">封禁 IP</h3><p>创建一个脚本 <code>/root/scripts/ban_auth_failed_ip.sh</code>，读取日志 /var/log/auth.log，提取登录失败次数过多的IP。有的系统没有这个文件，可以通过 <code>systemctl -u ssh -n 500</code>或者 <code>systemctl -u sshd -n 500</code> 获取最近的 500 条日志。这里首先提前登录失败的 IP，并统计次数，只有被封禁的 IP 不在/etc/hosts.deny 的时候才新增封禁。 脚本里把阈值设定为 6次了，如果有需要可以自行修改。</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#! /usr/bin/bash</span></span><br><span class="line"></span><br><span class="line">BLACKS=`<span class="built_in">tail</span> -500 /var/log/auth.log | grep Failed | sed  <span class="string">&quot;s/^.*from \(.*\) port.*$/\1/g&quot;</span> | <span class="built_in">sort</span> | <span class="built_in">uniq</span> -c | awk <span class="string">&#x27;&#123;if ($1 &gt;= 6) &#123;printf &quot;sshd:%s:deny\n&quot;,$2&#125;&#125;&#x27;</span>`</span><br><span class="line"><span class="keyword">for</span> line <span class="keyword">in</span> <span class="variable">$BLACKS</span></span><br><span class="line"><span class="keyword">do</span></span><br><span class="line">    cnt=`fgrep <span class="variable">$line</span> /etc/hosts.deny | <span class="built_in">wc</span> -l`</span><br><span class="line">    <span class="keyword">if</span> [ <span class="variable">$cnt</span> -ne 0 ]</span><br><span class="line">    <span class="keyword">then</span></span><br><span class="line">        <span class="built_in">continue</span></span><br><span class="line">    <span class="keyword">fi</span></span><br><span class="line">    <span class="built_in">echo</span> <span class="variable">$line</span> &gt;&gt; /etc/hosts.deny</span><br><span class="line"><span class="keyword">done</span></span><br></pre></td></tr></table></figure><h3 id="解封-ip">解封 IP</h3><p>防止封禁的 IP 列表过长，创建一个脚本<code>/root/scripts/recover_host_deny.sh</code>，自动清空/etc/hosts.deny</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#! /usr/bin/bash</span></span><br><span class="line">sed -ni <span class="string">&#x27;/^#.*$/p; /^$/p&#x27;</span> /etc/hosts.deny</span><br></pre></td></tr></table></figure><p>同时为了防止把自己的 IP 封禁了，可以修改 /etc/hosts.allow，把常用的IP 添加进去。hosts.allow 的优先级是高于 hosts.deny 的。</p><figure class="highlight apache"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># /etc/hosts.allow: list of hosts that are allowed to access the system.</span></span><br><span class="line"><span class="comment">#                   See the manual pages hosts_access(5) and hosts_options(5).</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># Example:    ALL: LOCAL @some_netgroup</span></span><br><span class="line"><span class="comment">#             ALL: .foobar.edu EXCEPT terminalserver.foobar.edu</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"><span class="comment"># If you&#x27;re going to protect the portmapper use the name &quot;rpcbind&quot; for the</span></span><br><span class="line"><span class="comment"># daemon name. See rpcbind(8) and rpc.mountd(8) for further information.</span></span><br><span class="line"><span class="comment">#</span></span><br><span class="line"></span><br><span class="line"><span class="attribute">sshd</span>: <span class="number">192.168.10.0</span>/<span class="number">24</span></span><br><span class="line"><span class="attribute">sshd</span>: <span class="number">10.28.13.12</span></span><br><span class="line"><span class="attribute">sshd</span>: <span class="number">10.28.12.15</span></span><br></pre></td></tr></table></figure><h3 id="定时任务">定时任务</h3><p>为了实现自动封禁和解封，可以使用 crontab设置定时任务，自动执行脚本。由于脚本需要有权限修改 /etc/hosts.deny文件，可以设置 root 的 crontab，执行 <code>crontab -e</code></p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">*/2 * * * * /root/scripts/ban_auth_failed_ip.sh &gt; /dev/null</span><br><span class="line">1 23 * * * /root/scripts/recover_host_deny.sh &gt; /dev/null</span><br></pre></td></tr></table></figure><p>设置每隔 2 分钟自动执行<code>/root/scripts/ban_auth_failed_ip.sh</code> 封禁 IP。</p><p>每天的 23:01 分，自动执行<code>/root/scripts/recover_host_deny.sh</code> 解封全部 IP。</p>]]></content>
    
    
      
      
    <summary type="html">&lt;h3 id=&quot;背景&quot;&gt;背景&lt;/h3&gt;
&lt;p&gt;服务器只要对外暴露端口，就会引来各种扫描。今天登录服务器，查看
/var/log/auth.log 日志，发现有几乎一直有人尝试用 ssh
登录服务器。虽然没有成功的，但是让人一直暴力破解也有些安全隐患。
本文也记录一下本次操作的方法</summary>
      
    
    
    
    <category term="Linux" scheme="http://www.wangyulong.cn/categories/Linux/"/>
    
    
  </entry>
  
  <entry>
    <title>从头编写 Skip-Gram Word2Vec</title>
    <link href="http://www.wangyulong.cn/1734076558/"/>
    <id>http://www.wangyulong.cn/1734076558/</id>
    <published>2024-12-13T17:06:46.000Z</published>
    <updated>2026-02-19T07:45:16.917Z</updated>
    
    <content type="html"><![CDATA[<h3 id="摘要">摘要</h3><p>本文介绍了 Word2Vec 的相关背景知识，以及背后的数学原理，并通过 Python源码实现 Skip-Gram 以及负采样方法。 本文并不会涉及 Word2Vec的所有方面，而是从 Skip-Gram 入手，关注 Word2Vec最核心的思想，从头开始实现一个精简版的 Skip-Gram Word2Vec。对于算法的优化本文只会略微提及，感兴趣的读者可以自行阅读相关文献，或者原版的Word2Vec 源码。</p><h3 id="背景">背景</h3><p>Word2Vec 是谷歌出品，自发布以引起了广泛的关注，论文引用次数已经达到了4 万多次。 第一篇论文 <em>Efficient Estimation of word Representationsin vector Space</em> 对比了多种模型，并且引入了 CBOW 和 Skip-Gram 2 种Word2Vec 模型。 Word2Vec模型和神经网络模型相比，最大的优势加速了计算，据论文描述，单台机器每天可以训练1000 亿个单次。 而且通过对比发现 Skip-Gram 模型在语义理解方面表现要比CBOW 更好。 第二篇论文 <em>Distributed Representations of Words andPhrases and their Compositionality</em> 重点介绍的也是 Skip-Gram模型，提出了层次 SoftMax 以及负采样和 SubSample 优化方法。 本文将只关注Skip-Gram 和负采样，并用 Python 源码实现该功能。</p><h3 id="单词的表示">单词的表示</h3><p>首先分析一下，不使用Word2Vec，直接使用神经网络训练词向量的方法以及问题。通常单词使用字符串表示，但神经网络只能接收浮点型向量，不能接收字符串，所以得换一种表示方式。一种简单的方式是使用 One-Hot 编码，这种方式可以把单词表示为向量。具体的做法是，先构建一个词表，词表的大小就是向量的长度，用将单词在词表的下标位置设置为1，其他位置设置为 0。 比如词表大小为 6，构建的词表为<code>[the, dog, is, walking, in, park]</code> 对于第一个单词<code>dog</code> 的 One-Hot 编码为 <span class="math inline">\([0, 1, 0,0, 0, 0]\)</span> 。 第二个单词 <code>is</code> 的 One-Hot 编码为 <spanclass="math inline">\([0, 0, 1, 0, 0, 0]\)</span> 。</p><p><img src='../images/1734076558.one-hot.png'></p><h3 id="网络结构">网络结构</h3><p>这里以最简单的例子说明：只考虑连续的 2 个单词，输入为第一个单词的One-Hot 表示，输出为第二个单词的 One-Hot 概率分布，构建一个 3层的神经网络。 仍以上面的词表为例，那么输入和输出的 One-Hot 向量长度就是<span class="math inline">\(6\)</span>, 隐藏层的个数为 <spanclass="math inline">\(2\)</span>，同时隐藏层不使用激活函数，只在最终的输出层后面使用SoftMax 激活，生成输出词的 One-Hot 概率分布。损失函数使用神经网络的输出和真实词 One-Hot 编码的 CrossEntropy。</p><p><img src='../images/1734076558.fwd.png'></p><p>可能有人要产生疑问了，既然隐藏层没有激活函数，实际上第三层的 <spanclass="math inline">\(\mathbf{y}\)</span> 直接就相当于对输入层 <spanclass="math inline">\({\mathbf{x}}\)</span>做线性变换，隐藏层还有什么存在的必要呢？</p><p><span class="math display">\[\mathbf{y=b \cdot W = (x \cdot V) \cdot W = x \cdot (V \cdot W)}\]</span></p><p>实际上 Word2Vec 就是要训练隐藏层，把隐藏层的输出 <spanclass="math inline">\(\mathbf{b}\)</span>当作单词的向量表示，可以理解为其实就是在做矩阵分解。</p><h3 id="时间复杂度分析">时间复杂度分析</h3><p>假设词表大小为 <span class="math inline">\(N\)</span>，隐藏层大小为<span class="math inline">\(B\)</span>。那么输入词向量长度为 <spanclass="math inline">\(N\)</span>；矩阵 <spanclass="math inline">\(\mathbf{V}\)</span> 的大小为 <spanclass="math inline">\(N \times B\)</span>，即 <spanclass="math inline">\(N\)</span> 行 <spanclass="math inline">\(B\)</span> 列；矩阵 <spanclass="math inline">\(\mathbf{W}\)</span> 的大小为 <spanclass="math inline">\(B \times N\)</span>, 即 <spanclass="math inline">\(B\)</span> 行, <spanclass="math inline">\(N\)</span> 列。</p><p>第一层是输入，无需计算。</p><p>第二层的计算 <span class="math inline">\(\mathbf{b = x \cdotV}\)</span> ，对于常规的矩阵计算，时间复杂度为 <spanclass="math inline">\(N \times B\)</span>，由于输入向量是 One-Hot编码，相当于直接从 <span class="math inline">\(\mathbf{V}\)</span>中取出第 <span class="math inline">\(k\)</span> 行，时间复杂度为 <spanclass="math inline">\(O(1)\)</span>。</p><p>第三层的计算 <span class="math inline">\(\mathbf{y = b \cdotW}\)</span>，由于 <span class="math inline">\(\mathbf{b}\)</span> 和<span class="math inline">\(\mathbf{W}\)</span>都是稠密向量，无法优化，计算的时间复杂度为 <span class="math inline">\(B\times N\)</span>。</p><p>可以发现第一层和第二层的计算开销可忽略不计，最关键的是第三层的计算开销。如果词表大小达到百万量级，隐藏层达到 <spanclass="math inline">\(1000\)</span>维的话，单是训练一个单词，计算量就达到了 <spanclass="math inline">\(10^3 \times 10^6 =10^9\)</span>，而语料库中的单词可能达到数十亿以上。这种时间复杂度对于训练整个语料库几乎是不可行的，需要进行优化。</p><p>在 Word2Vec 中，提到了 2 种优化的方法：层次化 SoftMax 和负采样。由于层次化 SoftMax实现起来更加复杂，而且优化效果也不如负采样，本文重点儿阐述负采样方法。</p><h3 id="层次化-softmax">层次化 SoftMax</h3><p>层次化 SoftMax方法构建了一颗哈夫曼树，在训练的过程中使用贪心方式沿着二叉树路径从上到下做二分类，最终遍历到叶子节点，也就是概率最大的词。该方法可以将时间复杂度降低为 <span class="math inline">\(B \timeslog(N)\)</span>。</p><h3 id="负采样">负采样</h3><p>使用负采样方法时，神经网络的输入为一个词的 One-Hot编码，输出层不再使用 SoftMax，而是使用 Sigmoid 做二分类。仍然以上面的词表为例，假设有这样一条样本：第一个词为<code>dog</code>，第二个词为<code>is</code>，把在原始训练中的样本称为正样本。 由于 <code>dog</code>在词表中的下标为 <spanclass="math inline">\(1\)</span>，那么神经网络的输入就是向量 <spanclass="math inline">\([0,1,0,0,0,0]\)</span>；<code>is</code>在词表中的下标为 <spanclass="math inline">\(2\)</span>，在训练的时候，在输出层只计算 <spanclass="math inline">\(z_2 = Sigmoid(y_2)\)</span>，把 <spanclass="math inline">\(z_2\)</span> 的当成是输出为 <code>is</code>的概率。 我们希望 <span class="math inline">\(z_2\)</span> 尽可能的接近<span class="math inline">\(1\)</span>，损失函数为 <spanclass="math inline">\(z_2\)</span> 和 <spanclass="math inline">\(1\)</span> 的 CrossEntropy。</p><p><img src='../images/1734076558.neg.png'></p><p>如此一来，训练的开销会大大降低，但是也丢失了一些信息，需要加入一些负样本。负样本的生成方法是随机生成，原始的 Word2Vec方法是按照词频的概率生成负样本，本文为了简单起见，使用的是均匀分布生成负样本。比如对于第一个词 <code>dog</code>，随机生成的下一个词是<code>park</code>，把这条训练数据当作负样本。 由于 <code>park</code>在词表的下标为 <spanclass="math inline">\(5\)</span>，神经网络在输出层只计算 <spanclass="math inline">\(z_5=Sigmoid(y_5)\)</span>，把 <spanclass="math inline">\(z_5\)</span> 的当成是输出为 <code>park</code>的概率。 由于是负样本，我们希望 <span class="math inline">\(z_5\)</span>尽可能的接近 <span class="math inline">\(0\)</span>，损失函数为 <spanclass="math inline">\(z_5\)</span> 和 <spanclass="math inline">\(0\)</span> 的 CrossEntropy。</p><p>对于每条正样本，同时构造生成 <span class="math inline">\(K\)</span>条负样本，对于每条训练样本，第三层的时间复杂度为 <spanclass="math inline">\(O(K \times B)\)</span>，这里 <spanclass="math inline">\(K\)</span> 是超参数。 <spanclass="math inline">\(K\)</span> 可以取 <spanclass="math inline">\(10\)</span> 左右，而 <spanclass="math inline">\(B\)</span> 取到 <spanclass="math inline">\(1000\)</span>也足够了，训练一个单词的开销比原始的方法下降了好几个数量级，更具备可操作性。谷歌的论文提到单台机器每天可训练千亿个词。</p><h3 id="skip-gram">Skip-Gram</h3><p>前面的介绍只是为了便于理解，简化后的场景。 实际上 Skip-Gram的做法是：给定一个单词，需要预测其周围 <span class="math inline">\(2\times c\)</span> 个单词的概率，即这个单词之前的 <spanclass="math inline">\(c\)</span> 个词以及之后的 <spanclass="math inline">\(c\)</span>个词，而且不考虑单词之间的顺序，以及单词和中心词的距离。假设语料库中有一句话是<code>the dog is walking in the park</code>，当前中心词是<code>is</code>，<span class="math inline">\(c\)</span> 等于2，那么就有三条正样本：</p><figure class="highlight hy"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">(<span class="name"><span class="built_in">is</span></span>, the)</span><br><span class="line">(<span class="name"><span class="built_in">is</span></span>, dog)</span><br><span class="line">(<span class="name"><span class="built_in">is</span></span>, walking)</span><br><span class="line">(<span class="name"><span class="built_in">is</span></span>, in)</span><br></pre></td></tr></table></figure><p>以下开始实战部分，通过 Python 代码以及 numpy 库实现一个简化版的Word2Vec。</p><h3 id="数据准备">数据准备</h3><p>这里的数据集使用原版的 Word2Vec 用到的数据集。 <figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">wget http://mattmahoney.net/dc/text8.zip -O text8.gz</span><br><span class="line">gzip -d text8.gz -f</span><br></pre></td></tr></table></figure></p><h3 id="语料库解析">语料库解析</h3><p>语料库中就是一个接一个的单词，但是这个语料库中没有句子的分隔符。以下代码把它们当成一句话来处理了。为了便于测试，可以在解析时支持指定最大的单词个数，本例中指定单词个数为10000。 在解析时需要将单词转换为整数，这个整数就是单词在词表中的下标。比如语料库为：<code>the dog is walking in the park</code>，词表为：<code>[the, dog, is, walking, in, park]</code>，输出的Tokens 为 <code>[0, 1, 2, 3, 4, 0, 5]</code></p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">#!/usr/bin/env python</span></span><br><span class="line"><span class="comment">#-*- coding:utf-8 -*-</span></span><br><span class="line"></span><br><span class="line">max_words_size = <span class="number">10000</span></span><br><span class="line">data_map = &#123;&#125;</span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">read_tokens</span>():</span><br><span class="line">    tokens = []</span><br><span class="line">    <span class="keyword">with</span> <span class="built_in">open</span>(<span class="string">&#x27;text8&#x27;</span>) <span class="keyword">as</span> f:</span><br><span class="line">        content = f.read()</span><br><span class="line">        word = []</span><br><span class="line">        <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(<span class="number">0</span>, <span class="built_in">len</span>(content)):</span><br><span class="line">            <span class="keyword">if</span> content[i].isspace():</span><br><span class="line">                <span class="keyword">if</span> <span class="built_in">len</span>(word) &gt; <span class="number">0</span>:</span><br><span class="line">                    k = <span class="string">&#x27;&#x27;</span>.join(word)</span><br><span class="line">                    <span class="keyword">if</span> k <span class="keyword">in</span> data_map:</span><br><span class="line">                        v = data_map[k]</span><br><span class="line">                    <span class="keyword">else</span>:</span><br><span class="line">                        v = <span class="built_in">len</span>(data_map)</span><br><span class="line">                        data_map[k] = v</span><br><span class="line">                    tokens.append(v) <span class="comment"># 将单词作为 Token</span></span><br><span class="line">                    word = []</span><br><span class="line">            <span class="keyword">else</span>:</span><br><span class="line">                word.append(content[i])</span><br><span class="line">            <span class="comment"># 最大读取单词个数为 max_words_size</span></span><br><span class="line">            <span class="keyword">if</span> <span class="built_in">len</span>(tokens) &gt;= max_words_size:</span><br><span class="line">                <span class="keyword">break</span></span><br><span class="line">    <span class="keyword">return</span> tokens</span><br><span class="line"></span><br></pre></td></tr></table></figure><h3 id="构建训练正样本">构建训练正样本</h3><p>首先枚举每一个单词作为中心词，然后再枚举周围词，作为正样本。</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">get_skip_gram_pairs</span>(<span class="params">tokens, window</span>):</span><br><span class="line">    pairs = []</span><br><span class="line">    <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(<span class="built_in">len</span>(tokens)): <span class="comment"># 枚举中心词</span></span><br><span class="line">        <span class="keyword">for</span> j <span class="keyword">in</span> <span class="built_in">range</span>(<span class="built_in">max</span>(<span class="number">0</span>, i - window), <span class="built_in">min</span>(<span class="built_in">len</span>(tokens), i + window + <span class="number">1</span>)): <span class="comment"># 枚举周围词</span></span><br><span class="line">            <span class="keyword">if</span> j == i:</span><br><span class="line">                <span class="keyword">continue</span></span><br><span class="line">            <span class="comment"># 添加元组（中心词_id，周围词_id）</span></span><br><span class="line">            pairs.append((tokens[i], tokens[j]))</span><br><span class="line">    <span class="keyword">return</span> pairs</span><br></pre></td></tr></table></figure><h3 id="梯度更新">梯度更新</h3><p>这里没有使用机器学习框架，也没有自动微分功能，需要推导梯度更新公式。损失函数为 <span class="math inline">\(\sigma(y)\)</span> 和真实值之间的CrossEntropy。 <span class="math display">\[  L = −[label*log(\sigma(y))+(1−label)log(1−\sigma(y))]\]</span></p><p>其中 <span class="math inline">\(\displaystyle \sigma(y) =\frac{1}{1+e^{-y}}\)</span>，<span class="math inline">\(\displaystyle\frac{\mathrm{d}}{\mathrm{d} y} log(\sigma(y)) =1-\sigma(y)\)</span>，<span class="math inline">\(label\)</span>的取值只能是 <span class="math inline">\(0\)</span> 或者 <spanclass="math inline">\(1\)</span>。</p><p>当 <span class="math inline">\(label\)</span> 取值为 <spanclass="math inline">\(1\)</span> 时： <span class="math display">\[\begin{align}L_+ &amp;= -\log(\sigma(y)) \\\frac{\partial L_+}{\partial y} &amp;= \sigma(y) - 1 \\\end{align}\]</span> 当 <span class="math inline">\(label\)</span> 取值为 <spanclass="math inline">\(0\)</span> 时： <span class="math display">\[\begin{align}L_- &amp;= -\log[1 - \sigma(y)] \\\frac{\partial L_-}{\partial y} &amp;= \sigma(y) - 0 \\\end{align}\]</span> 综合起来，可以得到输出层的梯度公式： <spanclass="math display">\[\begin{align}\frac{\partial L}{\partial y} &amp;= \sigma(y) - label \\\end{align}\]</span> 而之前层的网络参数，可以通过反向传播算法更新梯度计算。</p><h3 id="模型训练">模型训练</h3><p>这里使用随机梯度下降以及反向传播训练模型。 需要定义 <spanclass="math inline">\(2\)</span> 个变量 <spanclass="math inline">\(embed\_in\)</span> 和 <spanclass="math inline">\(embed\_out\)</span>来存储网络模型参数，分别对应之前介绍的矩阵 <spanclass="math inline">\(V\)</span> 和矩阵 <spanclass="math inline">\(W\)</span>。 输入为中心词的下标假设为 <spanclass="math inline">\(s\)</span>，当前预测的周围词下标为 <spanclass="math inline">\(t\)</span>。 第二层的输出直接就是 <spanclass="math inline">\(embed\_in[s]\)</span>，第三层的输出为 <spanclass="math inline">\(y = embed\_in[s] \cdot embed\_out[t]\)</span></p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</span><br><span class="line"><span class="keyword">import</span> math</span><br><span class="line"><span class="keyword">import</span> random</span><br><span class="line"></span><br><span class="line">hidden_dim = <span class="number">4</span></span><br><span class="line">vocab_size = <span class="number">0</span></span><br><span class="line">embed_in = <span class="literal">None</span></span><br><span class="line">embed_out = <span class="literal">None</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">soft_max</span>(<span class="params">x</span>):</span><br><span class="line">    <span class="keyword">return</span> <span class="number">1.0</span> / (<span class="number">1</span> + math.exp(-x))</span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">train</span>(<span class="params">pairs</span>):</span><br><span class="line">    learning_rate = <span class="number">1e-4</span></span><br><span class="line">    <span class="keyword">for</span> epoch <span class="keyword">in</span> <span class="built_in">range</span>(<span class="number">3</span>):</span><br><span class="line">        <span class="keyword">for</span> p <span class="keyword">in</span> pairs:</span><br><span class="line">            <span class="comment"># 第一个为正样本，之后的是构造的负样本</span></span><br><span class="line">            examples = [(<span class="number">1</span>, p[<span class="number">1</span>])]</span><br><span class="line">            <span class="comment"># 这里写死构造 8 个负样本</span></span><br><span class="line">            <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(<span class="number">8</span>):</span><br><span class="line">                j = random.randrange(<span class="number">0</span>, vocab_size) <span class="comment"># 随机选一个词当负样本</span></span><br><span class="line">                <span class="keyword">while</span> j == p[<span class="number">1</span>]: <span class="comment"># 由于 p[1] 是正样本，随机选其他词</span></span><br><span class="line">                    j = random.randrange(<span class="number">0</span>, vocab_size)</span><br><span class="line">                examples.append((<span class="number">0</span>, j))</span><br><span class="line">            <span class="comment"># 遍历样本，使用梯度下降更新参数值</span></span><br><span class="line">            <span class="keyword">for</span> examp <span class="keyword">in</span> examples:</span><br><span class="line">                label = examp[<span class="number">0</span>] <span class="comment"># label 为 1 表示当前为正样本，label 为 0 表示当前为负样本</span></span><br><span class="line">                <span class="comment"># 这里 x0 和 x1 就是词的 Embedding，通过梯度下降算法自动更新求解</span></span><br><span class="line">                x0 = embed_in[p[<span class="number">0</span>]] <span class="comment"># p[0] 是中心词_id</span></span><br><span class="line">                x1 = embed_out[examp[<span class="number">1</span>]] <span class="comment"># examp[1] 是周围词_id</span></span><br><span class="line">                y = np.dot(x0, x1) <span class="comment"># 计算 Embedding 的内积</span></span><br><span class="line">                g = (soft_max(y) - label) * learning_rate <span class="comment"># 计算梯度，learning_rate 是超参数学习率</span></span><br><span class="line">                tmp0 = x0.copy() <span class="comment"># 保留原来的 x0</span></span><br><span class="line">                x0 -= g * x1 <span class="comment"># 反向传播，梯度下降，更新 x0</span></span><br><span class="line">                x1 -= g * tmp0 <span class="comment"># 反向传播，梯度下降，更新 x1</span></span><br><span class="line">    nc = <span class="number">0</span></span><br><span class="line">    <span class="keyword">for</span> k,v <span class="keyword">in</span> data_map.items():</span><br><span class="line">        <span class="built_in">print</span>(k, embed_in[v])</span><br><span class="line">        nc += <span class="number">1</span></span><br><span class="line">        <span class="keyword">if</span> nc &gt; <span class="number">5</span>:</span><br><span class="line">            <span class="keyword">break</span></span><br></pre></td></tr></table></figure><h3 id="主流程">主流程</h3><p>将之前的代码组合起来，形成完成代码。</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">main</span>():</span><br><span class="line">    <span class="keyword">global</span> vocab_size, embed_in, embed_out</span><br><span class="line">    <span class="comment"># 1. 读入语料库</span></span><br><span class="line">    tokens = read_tokens()</span><br><span class="line">    <span class="comment"># 2. 构建训练样本</span></span><br><span class="line">    pairs = get_skip_gram_pairs(tokens, <span class="number">3</span>)</span><br><span class="line">    <span class="comment"># 3. 初始化模型参数</span></span><br><span class="line">    vocab_size = <span class="built_in">len</span>(data_map)</span><br><span class="line">    embed_in = np.random.rand(vocab_size, hidden_dim)</span><br><span class="line">    embed_out = np.random.rand(vocab_size, hidden_dim)</span><br><span class="line">    <span class="comment"># 4. 训练模型</span></span><br><span class="line">    train(pairs)</span><br></pre></td></tr></table></figure>]]></content>
    
    
      
      
    <summary type="html">&lt;h3 id=&quot;摘要&quot;&gt;摘要&lt;/h3&gt;
&lt;p&gt;本文介绍了 Word2Vec 的相关背景知识，以及背后的数学原理，并通过 Python
源码实现 Skip-Gram 以及负采样方法。 本文并不会涉及 Word2Vec
的所有方面，而是从 Skip-Gram 入手，关注 Word2V</summary>
      
    
    
    
    <category term="人工智能" scheme="http://www.wangyulong.cn/categories/%E4%BA%BA%E5%B7%A5%E6%99%BA%E8%83%BD/"/>
    
    
    <category term="Word2Vec" scheme="http://www.wangyulong.cn/tags/Word2Vec/"/>
    
  </entry>
  
  <entry>
    <title>最长回文子串</title>
    <link href="http://www.wangyulong.cn/1688195692/"/>
    <id>http://www.wangyulong.cn/1688195692/</id>
    <published>2023-07-01T15:13:46.000Z</published>
    <updated>2026-02-19T07:45:16.917Z</updated>
    
    <content type="html"><![CDATA[<p>原题为 POJ 4974 最长回文子串。</p><h3 id="题目大意">题目大意：</h3><p>输入多个字符串，对于每个字符串输出一个整数 <spanclass="math inline">\(N\)</span>，表示该字符串的最长回文子串。</p><h3 id="输入">输入</h3><p>最多 <span class="math inline">\(30\)</span>个字符串，每个字符串长度不超过 <spanclass="math inline">\(1000000\)</span>，最后一个字符串为<code>END</code>(该字符串不需要处理)。</p><h3 id="输出">输出</h3><p>对于每组测试用例，输出以"Case N: "作为开头，N是测试用例的编号,然后是对应字符串的最长回文子串。</p><h3 id="样例输入">样例输入</h3><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">abcbabcbabcba</span><br><span class="line">abacacbaaaab</span><br><span class="line">END</span><br></pre></td></tr></table></figure><h3 id="样例输出">样例输出</h3><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">Case 1: 13</span><br><span class="line">Case 2: 6</span><br></pre></td></tr></table></figure><p>求解最长回文子串的经典算法为 Manacher 算法。</p><p><strong>定理1</strong>：如果某个字符串本身是回文串，对于其内部的任何一个回文子串<span class="math inline">\(s\)</span>，它的 <spanclass="math inline">\(s\)</span> 的镜像串 <spanclass="math inline">\(s&#39;\)</span>也是回文子串，并且 <spanclass="math inline">\(s = s&#39;\)</span>。</p><p>这是由回文串本身的对称性决定的，对 <spanclass="math inline">\(s\)</span> 做中心对称变化，然后再做平移，就会得到<spanclass="math inline">\(s&#39;\)</span>。由于对回文串做对称变换保持不变，所以<span class="math inline">\(s\)</span> 跟 <spanclass="math inline">\(s&#39;\)</span> 相等。</p><p>当回文串的长度为偶数时，回文串的中心并不是某一个字符，原版的 Manacher算法为了解决这个问题，通过对原字符串添加特殊字符的方法，使得每一个字串长度都是奇数。实际上这一步是不必要的，因为找回文中心的本质原因是为了找回文串的镜像变换，只需要知道镜像变换公式即可。</p><p>对于一个回文子串，其中心对称的两个下标之和是定值 <spanclass="math inline">\(C\)</span>，而 <spanclass="math inline">\(\displaystyle \frac C 2\)</span>就是回文子串的中心，这里把 <span class="math inline">\(C\)</span>称为回文串的二倍中心。 对于回文子串的任意一个下标 <spanclass="math inline">\(j\)</span>，其对称的下标变换公式为 <spanclass="math inline">\(i = C - j\)</span>。假设回文子串的右边界为 <spanclass="math inline">\(r\)</span>，那么回文子串左边界 <spanclass="math inline">\(l = C - r\)</span>，回文串的长度为 <spanclass="math inline">\(len = r - l + 1 = 2*r -C + 1\)</span> 。</p><p><img src='../images/1688195692.1.png'></p><p>对于每一个中心都有一个最长的回文子串，而每个回文子串也都有一个唯一的中心，也就有一个唯一的变换公式，所以我们可以用<span class="math inline">\(C\)</span> 来唯一标识这个子串，至于 <spanclass="math inline">\(\displaystyle \frac C 2\)</span>是什么并不重要。由于有以上的算术关系，只需要知道 <spanclass="math inline">\(C\)</span> 和 <spanclass="math inline">\(r\)</span>，就能知道当前回文子串的全部信息，所以在实现的时候也是只需要维护<span class="math inline">\(C\)</span> 和 <spanclass="math inline">\(r\)</span> 即可。</p><h2 id="最长回文子串算法">最长回文子串算法</h2><p>该算法是动态规划的，定义 <span class="math inline">\(rd[c]\)</span>表示二倍中心为 <span class="math inline">\(c\)</span>的最长回文子串的长度，用 <span class="math inline">\(r\)</span> 表示以<span class="math inline">\(c\)</span>为二倍中心的回文串向右最远扩展的位置加一，可以把这个状态称为<strong>循环不变状态</strong>，并用下图表示。那么<span class="math inline">\(l = c - r\)</span>，从 <spanclass="math inline">\(l+1\)</span> 至 <spanclass="math inline">\(r-1\)</span>为回文子串，回文子串的长度 <spanclass="math inline">\(rd[c] = r - (l+1) = r - (c - r) - 1 =2r-c-1\)</span>。</p><p><img src='../images/1688195692.2.png'></p><p>让 <span class="math inline">\(c\)</span> 从零开始依次递增并计算<span class="math inline">\(rd[c]\)</span>，当 <spanclass="math inline">\(rd[c]\)</span> 被计算出来之后，二倍中心位于区间<span class="math inline">\([c+1, 2r)\)</span>的最长回文子串，是可以被快速计算的。</p><p><img src='../images/1688195692.3.png'></p><p>最简单的就是上图中的情况，即回文子串的右边界无法触及 <spanclass="math inline">\(r-1\)</span>，这种可以利用定理1直接计算。不妨设中心位置为<span class="math inline">\(\displaystyle \fracn2\)</span>，根据定理1可知，关于 <spanclass="math inline">\(\displaystyle \fracc2\)</span>对称处存在一个完全一样的回文串，只要对称串的左边界无法触及<span class="math inline">\(l+1\)</span>，那么以 <spanclass="math inline">\(\displaystyle \frac n2\)</span>为中心的最大回文串右边界就无法触及 <spanclass="math inline">\(r-1\)</span>，最大回文串长度也就确定了 <spanclass="math inline">\(rd[n] = rd[2c-n]\)</span>。</p><p>处理完全部这种情况之后，会遇到第一个右边界达到 <spanclass="math inline">\(r-1\)</span> 的回文子串，变成下图情况。</p><p><img src='../images/1688195692.4.png'></p><p>由于此时对于 <span class="math inline">\(s[l] = s[r] ?\)</span>我们无法确定，所以就不能确定 <span class="math inline">\(r-1-l\)</span>是否为最长回文子串长度。如果二者不想等 <spanclass="math inline">\(r-1-l\)</span>就是最大回文字串长度，而且可以发现当前就是循环不变状态。而如果 <spanclass="math inline">\(s[l] =s[r]\)</span>，就要一直向右扩展，直到二者不相等。扩展完成之后就会发现，也回到了循环不变状态，可以开启下一轮的计算了。</p><h2 id="c-代码">C++ 代码</h2><figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string">&lt;iostream&gt;</span></span></span><br><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string">&lt;string&gt;</span></span></span><br><span class="line"></span><br><span class="line"><span class="keyword">using</span> <span class="keyword">namespace</span> std;</span><br><span class="line">std::string str;</span><br><span class="line"><span class="type">int</span> rd[<span class="number">2000008</span>];</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="type">int</span> <span class="title">main</span><span class="params">()</span> </span>&#123;</span><br><span class="line">  ios_base::<span class="built_in">sync_with_stdio</span>(<span class="literal">false</span>);</span><br><span class="line">  std::cin &gt;&gt; str;</span><br><span class="line">  <span class="type">int</span> T = <span class="number">0</span>;</span><br><span class="line">  rd[<span class="number">0</span>] = <span class="number">1</span>;</span><br><span class="line">  <span class="keyword">while</span> (str != <span class="string">&quot;END&quot;</span>) &#123;</span><br><span class="line">    <span class="type">int</span> r = <span class="number">1</span>, n = <span class="number">0</span>, res = <span class="number">1</span>;</span><br><span class="line">    <span class="keyword">while</span> (r &lt; str.<span class="built_in">size</span>()) &#123;</span><br><span class="line">      <span class="type">int</span> c = n;</span><br><span class="line">      <span class="keyword">while</span> (++n &lt;= <span class="number">2</span>*(r - <span class="number">1</span>) &amp;&amp; rd[<span class="number">2</span>*c - n] &lt; r - (n - r) - <span class="number">1</span>) &#123;</span><br><span class="line">          rd[n] = rd[<span class="number">2</span>*c - n];</span><br><span class="line">      &#125;</span><br><span class="line">      <span class="keyword">while</span> (<span class="number">0</span> &lt;= n - r &amp;&amp; str[n-r] == str[r]) &#123;</span><br><span class="line">          r++;</span><br><span class="line">      &#125;</span><br><span class="line">      rd[n] = (r - <span class="number">1</span>) - (n - (r - <span class="number">1</span>)) + <span class="number">1</span>;</span><br><span class="line">      <span class="keyword">if</span> (res &lt; rd[n]) &#123;</span><br><span class="line">          res = rd[n];</span><br><span class="line">      &#125;</span><br><span class="line">    &#125;</span><br><span class="line">    std::cout &lt;&lt; <span class="string">&quot;Case &quot;</span> &lt;&lt; ++T &lt;&lt; <span class="string">&quot;: &quot;</span> &lt;&lt; res &lt;&lt; std::endl;</span><br><span class="line">    std::cin &gt;&gt; str;</span><br><span class="line">  &#125;</span><br><span class="line">  <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;原题为 POJ 4974 最长回文子串。&lt;/p&gt;
&lt;h3 id=&quot;题目大意&quot;&gt;题目大意：&lt;/h3&gt;
&lt;p&gt;输入多个字符串，对于每个字符串输出一个整数 &lt;span
class=&quot;math inline&quot;&gt;&#92;(N&#92;)&lt;/span&gt;，表示该字符串的最长回文子串。&lt;/p&gt;
&lt;h3 </summary>
      
    
    
    
    <category term="算法" scheme="http://www.wangyulong.cn/categories/%E7%AE%97%E6%B3%95/"/>
    
    
    <category term="算法" scheme="http://www.wangyulong.cn/tags/%E7%AE%97%E6%B3%95/"/>
    
  </entry>
  
  <entry>
    <title>线性规划单纯形法</title>
    <link href="http://www.wangyulong.cn/1659146474/"/>
    <id>http://www.wangyulong.cn/1659146474/</id>
    <published>2022-07-30T10:53:46.000Z</published>
    <updated>2026-02-19T07:45:16.917Z</updated>
    
    <content type="html"><![CDATA[<p>线性规划是最优化问题中的一个重要领域。许多实际问题都可以归结为线性规划问题，例如：网络流、多商品流量等问题。我们在高中已经学习过只有<span class="math inline">\(2\)</span>个变量的场景，可以使用图解法来求最优解。该方法虽然直观，易于理解，但是难以推广到<span class="math inline">\(n\)</span>个变量的场景。面对更加复杂的线性规划问题，单纯形法是则是更加常用的方法。本文尝试从线性代数基础知识出发，一步一步的推导出单纯形法，并提出一种避免退化基变换的迭代方法，从而避免了单纯形陷入无限循环的情况。</p><h2 id="线性规划的标准形式">线性规划的标准形式</h2><p>线性规划问题的标准形式如下</p><p>目标：<span class="math inline">\(\max : c_1 \cdot x_1 + c_2 \cdotx_2 + ... + c_n \cdot x_n\)</span> ，<spanclass="math inline">\(c_i\)</span> 是给定的常量。<spanclass="math inline">\(x_i\)</span> 是变量，需要满足如下约束：</p><p><span class="math display">\[\begin{aligned}a_{11} \cdot x_1 + a_{12} \cdot x_2 + ... + a_{1n} \cdot x_n &amp;= b_1\\a_{21} \cdot x_1 + a_{22} \cdot x_2 + ... + a_{2n} \cdot x_n &amp;= b_2\\... \\a_{m1} \cdot x_1 + a_{m2} \cdot x_2 + ... + a_{mn} \cdot x_n &amp;= b_m\end{aligned}\]</span> 而且对于每一个 <span class="math inline">\(x_i\)</span>需要满足约束条件： <span class="math inline">\(x_i \geq 0\)</span>。在标准形式中，要求 <span class="math inline">\(b_i \geq0\)</span>，但是如果其中某个 <span class="math inline">\(b_i &lt;0\)</span>，可以将该方程两边同时乘以 <spanclass="math inline">\(-1\)</span> 转化为标准形式。</p><p>将标准形式写成矩阵形式：</p><p>目标：<span class="math inline">\(\max : \mathbf{c^T \cdotx}\)</span></p><p>约束：<span class="math inline">\(\mathbf{Ax} = \mathbf{b}, \mathbf{x\geq 0}\)</span></p><p>在标准形式下，<span class="math inline">\(\mathbf{b}\)</span>也需要满足 <span class="math inline">\(\mathbf{b \geq 0}\)</span>，即<span class="math inline">\(\mathbf{b}\)</span> 的每一个分量都大于等于0。</p><blockquote><p>有时候目标是求最小值，基本方法是一样的，也可以通过将目标函数乘以<span class="math inline">\(-1\)</span>，转化为最大值问题求解。</p></blockquote><h2 id="基础知识">基础知识</h2><h3 id="线性代数">线性代数</h3><p>回顾一下线性代数基础知识，对于线性方程组 <spanclass="math inline">\(\mathbf{Ax} = \mathbf{b}\)</span>，其中 <spanclass="math inline">\(\mathbf{A}\)</span> 是一个 <spanclass="math inline">\(m \times n\)</span> 的矩阵（<spanclass="math inline">\(m\)</span> 个方程，<spanclass="math inline">\(n\)</span> 个未知量）。用 <spanclass="math inline">\(rank(\mathbf{A})\)</span> 代表矩阵 <spanclass="math inline">\(\mathbf{A}\)</span> 的秩，方程组的增广矩阵为 <spanclass="math inline">\([\mathbf{A|b}]\)</span>，只有当 <spanclass="math inline">\(rank(\mathbf{A}) = rank([\mathbf{A|b}])\)</span>时，该方程组才可能有解，在此情况下，还可以分为以下 3 种情况：</p><ol type="1"><li>如果 <span class="math inline">\(rank(\mathbf{A}) &lt; n\)</span>，那么该线性方程组有无穷多组解</li><li>如果 <span class="math inline">\(rank(\mathbf{A}) = n\)</span>，那么该线性方程组有唯一解</li><li>如果 <span class="math inline">\(rank(\mathbf{A}) &gt; n\)</span>，那么该线性方程组无解</li></ol><p>对于情况 2 和情况3，对于使用线性代数方法很容易解决，这里不在赘述。</p><p>情况 <span class="math inline">\(1\)</span>是我们研究的对象，在这种情况下，设矩阵 <spanclass="math inline">\(\mathbf{A}\)</span> 的秩是 <spanclass="math inline">\(K\)</span>，由于 <span class="math inline">\(K&lt; n\)</span>，那么矩阵 <spanclass="math inline">\(\mathbf{A}\)</span>可以通过高斯消元法（加减消元法）消除冗余的方程，最终只剩下由 <spanclass="math inline">\(K\)</span>个线性无关的方程（通过手动计算，或者计算机程序都很容易做到）。<strong>在以下的讨论中，均假设<span class="math inline">\(\mathbf{A}\)</span>已经是经过高斯消元之后的方程组</strong>。</p><p><strong>定义一</strong> ：如果对于某个 <spanclass="math inline">\(\mathbf{x_0}\)</span>，满足约束条件：<spanclass="math inline">\(\mathbf{Ax_0} = \mathbf{b}, \mathbf{x_0 \geq0}\)</span>，那么称 <span class="math inline">\(\mathbf{x_0}\)</span>是该线性规划的一个<strong>可行解</strong>。由所有可行解组成的集合为<strong>可行域</strong>。</p><p><strong>定义二</strong> ：已知线性规划问题中，矩阵 <spanclass="math inline">\(\mathbf{A}\)</span> 的秩是 <spanclass="math inline">\(m\)</span>，则矩阵 <spanclass="math inline">\(\mathbf{A}\)</span> 中存在 <spanclass="math inline">\(m\)</span> 个线性无关的列向量。只将这 <spanclass="math inline">\(m\)</span> 个列向量对应的 <spanclass="math inline">\(x\)</span> 视作未知量，将其他列向量对应的 <spanclass="math inline">\(x\)</span> 取值为 <spanclass="math inline">\(0\)</span>，则方程 <spanclass="math inline">\(\mathbf{Ax} = \mathbf{b}\)</span> 有唯一解 <spanclass="math inline">\(\mathbf{x_0}\)</span>，将这个解称之为<strong>基解</strong>。这<span class="math inline">\(m\)</span>个列向量被称为<strong>基向量</strong>。基向量对应的 <spanclass="math inline">\(x\)</span>分量称为<strong>基变量</strong>，基变量的个数也是 <spanclass="math inline">\(m\)</span>。基变量中也有可能取值为 <spanclass="math inline">\(0\)</span>，这种情况称为退化。如果所有基变量都大于等于<span class="math inline">\(0\)</span>，那么称 <spanclass="math inline">\(\mathbf{x_0}\)</span>是一个<strong>基可行解</strong>。</p><blockquote><p>备注：基解、可行解、基可行解只和约束条件有关，和目标函数无关</p></blockquote><p><strong>例一</strong>：对于如下线性方程组，求它的一个基解。 <spanclass="math display">\[\begin{aligned}x_1 + x_2 + 2x_3 + x_4 + x_5 &amp;= 9 \\2x_1 + x_2 + x_3 + 3x_4 + x_5 &amp;= 12\end{aligned}\]</span> 该方程组中有 <span class="math inline">\(5\)</span>个未知量，<span class="math inline">\(2\)</span>个线性无关的方程，所以矩阵 <spanclass="math inline">\(\mathbf{A}\)</span> 的秩是 <spanclass="math inline">\(2\)</span>。前 <spanclass="math inline">\(2\)</span> 个列向量为： <spanclass="math display">\[\left (\begin{array}{c|c}1 &amp; 1 \\2 &amp; 1 \\\end{array}\right )\]</span> 这 <span class="math inline">\(2\)</span>个列向量是线性无关的，对应的 <spanclass="math inline">\(\mathbf{x}\)</span> 的分量分别为 <spanclass="math inline">\(x_1, x_2\)</span>，将其他分量 <spanclass="math inline">\(x_3,x_4,x_5\)</span> 取值为 <spanclass="math inline">\(0\)</span>，方程组可简化为： <spanclass="math display">\[\begin{array}{rrrrr}x_1 &amp;+&amp; x_2 &amp;+&amp; 2 \cdot 0 &amp;+&amp; 1 \cdot 0&amp;+&amp; 1 \cdot 0 &amp;=&amp; 9 \\2x_1 &amp;+&amp; x_2 &amp;+&amp; 1\cdot 0 &amp;+&amp; 3 \cdot 0&amp;+&amp; 1 \cdot 0 &amp;=&amp; 12\end{array}\]</span> 该方程有唯一解，可以解得： <spanclass="math inline">\(\mathbf{x_0} = (3,6,0,0,0)^T\)</span>，这里的<span class="math inline">\(\mathbf{x_0}\)</span> 就是一个基解，对应的<span class="math inline">\(x_1\)</span> 和 <spanclass="math inline">\(x_2\)</span>是基变量。由于每一个非零分量都是正数，所以也是基可行解。</p><p>对于有 <span class="math inline">\(n\)</span>个未知量的线性规划问题，由于基变量有 <spanclass="math inline">\(m\)</span> 个，所以基解的个数最多为 <spanclass="math inline">\(\displaystyle \binom{n}{m} =\frac{n!}{m!(n-m)!}\)</span></p><p><strong>定理一</strong>：已知 <span class="math inline">\(m\)</span>个向量组 <spanclass="math inline">\(\{\mathbf{a_1},\mathbf{a_2},...,\mathbf{a_m}\}\)</span>线性无关，向量 <span class="math inline">\(\mathbf{b}\)</span>可以唯一表示为 <span class="math inline">\(\mathbf{b} = \lambda_1\mathbf{a_1} + \lambda_2  \mathbf{a_2} + ... + \lambda_m\mathbf{a_m}\)</span>，而且其中存在一个 <spanclass="math inline">\(\lambda_i \neq 0\)</span>，那么向量组 <spanclass="math inline">\(\{\mathbf{a_1}, ..., \mathbf{a_{i-1}}, \mathbf{b},\mathbf{a_{i+1}}, ..., \mathbf{a_{m}}\}\)</span> 线性无关。</p><p>证明：由于 <span class="math inline">\(\lambda_i \neq 0\)</span>，对<span class="math inline">\(\mathbf{b} = \lambda_1 \mathbf{a_1} +\lambda_2  \mathbf{a_2} + ... + \lambda_m \mathbf{a_m}\)</span>做等价变换可得： <span class="math display">\[\frac{-\lambda_1}{\lambda_i}\mathbf{a_1} +... +\frac{-\lambda_{i-1}}{\lambda_i}\mathbf{a_{i-1}} +\frac{1}{\lambda_i}\mathbf{b} +\frac{-\lambda_{i+1}}{\lambda_i}\mathbf{a_{i+1}} +... +\frac{-\lambda_{m}}{\lambda_i}\mathbf{a_{m}}= \mathbf{a_i}\]</span> 由于以上的表示法是唯一的，所以线性方程组 <spanclass="math inline">\(\mathbf{[{a_1}, ..., a_{i-1}, b, a_{i+1}, ...,a_m]x = a_i}\)</span> 有唯一解，矩阵 <spanclass="math inline">\(\mathbf{[{a_1}, ..., a_{i-1}, b, a_{i+1}, ...,a_m]}\)</span> 的秩为 <spanclass="math inline">\(m\)</span>。由于向量的个数也是 <spanclass="math inline">\(m\)</span>，所以向量组 <spanclass="math inline">\(\{\mathbf{a_1}, ..., \mathbf{a_{i-1}}, \mathbf{b},\mathbf{a_{i+1}}, ..., \mathbf{a_{m}}\}\)</span> 线性无关。</p><h3 id="凸集">凸集</h3><p><strong>定义三</strong>：对于集合 <spanclass="math inline">\(\mathbf{C}\)</span> 中的任意 <spanclass="math inline">\(2\)</span> 个点 <spanclass="math inline">\(\mathbf{x_1,x_2}\)</span>，如果其连线上的任意一个点也都属于集合<span class="math inline">\(\mathbf{C}\)</span>，那么称集合 <spanclass="math inline">\(\mathbf{C}\)</span> 为<strong>凸集</strong>。其中<span class="math inline">\(\mathbf{x_1,x_2}\)</span>连线上的点可以形式化表示为 <span class="math inline">\(a \cdot\mathbf{x_1} + (1-a) \cdot \mathbf{x_2}\)</span>，其中 <spanclass="math inline">\(0 \leq a \leq 1\)</span> 。</p><p><strong>定理二</strong>：如果线性规划存在可行解，那么所有可行解组成的集合是一个凸集。</p><p><strong>证明</strong>：假设 <spanclass="math inline">\(\mathbf{x_1,x_2}\)</span> 是线性规划 <spanclass="math inline">\(\mathbf{Ax} = \mathbf{b}, \mathbf{x \geq0}\)</span> 的可行解。那么对于任意 <span class="math inline">\(0 \leq a\leq 1\)</span>，都有： <span class="math display">\[\begin{aligned}\mathbf{x_1 \geq 0} &amp;\Rightarrow a \cdot \mathbf{x_1} \geq\mathbf{0} \\\mathbf{x_2 \geq 0} &amp;\Rightarrow (1 - a) \cdot \mathbf{x_2} \geq\mathbf{0} \\\end{aligned}\]</span> 将以上两式相加可得： <span class="math display">\[a \cdot \mathbf{x_1} + (1-a) \cdot \mathbf{x_2} \geq \mathbf{0}\]</span> 类似的： <span class="math display">\[\begin{aligned}\mathbf{Ax_1} = \mathbf{b} &amp;\Rightarrow \mathbf{A} \cdot (a\mathbf{x_1}) = a \cdot \mathbf{b} \\\mathbf{Ax_2} = \mathbf{b} &amp;\Rightarrow \mathbf{A} \cdot ((1-a)\mathbf{x_1}) = (1-a) \cdot \mathbf{b}\end{aligned}\]</span> 将以上两式相加可得： <span class="math display">\[\mathbf{A}(a \cdot \mathbf{x_1} + (1-a) \cdot \mathbf{x_2}) = \mathbf{b}\]</span> 所以 <span class="math inline">\(a \cdot \mathbf{x_1} + (1-a)\cdot \mathbf{x_2}\)</span> 也是一个可行解。证毕！</p><p><strong>定义三</strong>：在凸集 <spanclass="math inline">\(\mathbf{C}\)</span> 中，如果点 <spanclass="math inline">\(\mathbf{x}\)</span>不在任何两个不同的点的连线上，那么称 <spanclass="math inline">\(\mathbf{x}\)</span> 为 <spanclass="math inline">\(\mathbf{C}\)</span>的<strong>顶点</strong>。换而言之，如果 <spanclass="math inline">\(\mathbf{x}\)</span> 是顶点，且 <spanclass="math inline">\(\mathbf{x} = a \cdot \mathbf{x_1} + (1-a) \cdot\mathbf{x_2}\)</span>，其中 <span class="math inline">\(0 \leq a \leq1\)</span>，那么必然有 <span class="math inline">\(\mathbf{x_1} =\mathbf{x_2}\)</span> 。</p><p><strong>定理二</strong>：如果凸集 <spanclass="math inline">\(\mathbf{C}\)</span> 中顶点的个数是有限的，那么<span class="math inline">\(\mathbf{C}\)</span>中的任意一个点，都可以写成顶点的凸组合。即 <spanclass="math inline">\(\mathbf{C}\)</span> 中有 <spanclass="math inline">\(n\)</span> 个顶点 <spanclass="math inline">\(\mathbf{P_1,P_2,...P_n}\)</span>，对于 <spanclass="math inline">\(\mathbf{C}\)</span> 中任意一个点 <spanclass="math inline">\(\mathbf{P}\)</span>，可以写为 <spanclass="math inline">\(\mathbf{P} =\sum\limits_{i=1}^{n}\lambda_i\mathbf{P_i}\)</span>，其中 <spanclass="math inline">\(0 \leq \lambda_i \leq 1\)</span>，并且 <spanclass="math inline">\(\sum\limits_{i=1}^{n}\lambda_i = 1\)</span>。</p><p><strong>证明</strong>：使用数学归纳法证明。首先考虑最简单的情况，<spanclass="math inline">\(\mathbf{C}\)</span> 中只有 <spanclass="math inline">\(2\)</span> 个顶点 <spanclass="math inline">\(\mathbf{P_1,P_2}\)</span>，那么根据凸集的定义，其内的点<span class="math inline">\(\mathbf{P}\)</span> 满足关系： <spanclass="math inline">\(\mathbf{P} = \lambda_1\mathbf{P_1} +\lambda_2\mathbf{P_2}\)</span>，其中 <span class="math inline">\(0 \leq\lambda_1,\lambda_2 \leq 1\)</span> ，且 <spanclass="math inline">\(\lambda_1 + \lambda_2 = 1\)</span>。</p><p>归纳假设对于任何一个顶点个数为 <span class="math inline">\(n\)</span>的凸集 <spanclass="math inline">\(\mathbf{C_n}\)</span>，该结论是成立的，那么再新增一个顶点<span class="math inline">\(\mathbf{P_{n+1}}\)</span>，使得 <spanclass="math inline">\(\mathbf{C_n}\)</span> 中的部分区域和 <spanclass="math inline">\(\mathbf{P_{n+1}}\)</span>围成一个新的区域，再加上原有的 <spanclass="math inline">\(\mathbf{C_n}\)</span> 会形成一个顶点个数为 <spanclass="math inline">\(n+1\)</span> 的凸集 <spanclass="math inline">\(\mathbf{C_{n+1}}\)</span>。对于老区域中的点，由归纳假设可直接得证。而从新区域的构成可知，新区域中的任何一个点<span class="math inline">\(\mathbf{Q}\)</span> 都是老区域中的点 <spanclass="math inline">\(\mathbf{P}\)</span> 和 <spanclass="math inline">\(\mathbf{P_{n+1}}\)</span> 连接线上的点。所以 <spanclass="math inline">\(\mathbf{Q} = \lambda\mathbf{P} +\lambda_{n+1}\mathbf{P_{n+1}}\)</span>，其中 <spanclass="math inline">\(0 \leq \lambda,\lambda_{n+1} \leq 1\)</span> ，且<span class="math inline">\(\lambda + \lambda_{n+1} = 1\)</span>。</p><p>由归纳假设可知：<span class="math inline">\(\mathbf{P} =\sum\limits_{i=1}^{n}\lambda_i\mathbf{P_i}\)</span>，其中 <spanclass="math inline">\(0 \leq \lambda_i \leq 1\)</span>，并且 <spanclass="math inline">\(\sum\limits_{i=1}^{n}\lambda_i = 1\)</span>。</p>所以： <span class="math display">\[\begin{aligned}\mathbf{Q} &amp;= \lambda\mathbf{P} + \lambda_{n+1}\mathbf{P_{n+1}} \\           &amp;= \lambda\sum\limits_{i=1}^{n}\lambda_i\mathbf{P_i} +\lambda_{n+1}\mathbf{P_{n+1}} \\           &amp;= \sum\limits_{i=1}^{n}\lambda\lambda_i\mathbf{P_i} +\lambda_{n+1}\mathbf{P_{n+1}}\end{aligned}\]</span> 由于 <span class="math inline">\(\lambda 和 \lambda_i\)</span>均为非负，所以 <span class="math inline">\(\mathbf{P_i}\)</span> 的系数<span class="math inline">\(\lambda\lambda_i\)</span> 和 <spanclass="math inline">\(\lambda_{n+1}\)</span> 均为非负，而且满足关系： $$<span class="math display">\[\begin{aligned}\sum\limits_{i=1}^{n}\lambda\lambda_i + \lambda_{n+1}  &amp;= \lambda\sum\limits_{i=1}^{n}\lambda_i + \lambda_{n+1} \\  &amp;= \lambda + \lambda_{n+1} \\  &amp;= 1      \end{aligned}\]</span><p>$$ 由此可知，如果凸集 <span class="math inline">\(\mathbf{C}\)</span>中顶点的个数是有限的，那么 <spanclass="math inline">\(\mathbf{C}\)</span>中的任意一个点，都可以写成顶点的凸组合。</p><p><strong>定理三</strong>：如果线性规划问题的可行域是有界的，那么在可行域中一定存在某个顶点是最优解。</p><p><strong>证明</strong>：线性规划的目标为 <spanclass="math inline">\(\max : \mathbf{c^T \cdot x}\)</span>，假设最优解为 <spanclass="math inline">\(\mathbf{x_0}\)</span>。由定理二可知， <spanclass="math inline">\(\mathbf{x_0}\)</span>可以写为顶点的凸组合。不妨设可行域有 <spanclass="math inline">\(n\)</span> 个顶点：<spanclass="math inline">\(\mathbf{x_1,x_2,...x_n}\)</span> 。那么 <spanclass="math inline">\(\mathbf{x_0} =\sum\limits_{i=1}^{n}\mathbf{x_i}\)</span>，其中 <spanclass="math inline">\(0 \leq \lambda_i \leq 1\)</span>，并且 <spanclass="math inline">\(\sum\limits_{i=1}^{n}\lambda_i =1\)</span>。假设在集合 <span class="math inline">\(\{\mathbf{c^T x_1},\mathbf{c^T x_2}, ..., \mathbf{c^T x_n}, \}\)</span> 中，最大的元素为<span class="math inline">\(\mathbf{c^Tx_k}\)</span>。那么可以得到以下关系： <span class="math display">\[\begin{aligned}\mathbf{c^T x_0} &amp;=\mathbf{c^T}\sum\limits_{i=1}^{n}\lambda_i\mathbf{x_i} \\             &amp;= \sum\limits_{i=1}^{n}\lambda_i\mathbf{c^T x_i} \\             &amp;\leq \sum\limits_{i=1}^{n}\lambda_i\mathbf{c^T x_k} \\             &amp;= \mathbf{c^T x_k}\sum\limits_{i=1}^{n}\lambda_i \\             &amp;= \mathbf{c^T x_k}\end{aligned}\]</span> 由于 <span class="math inline">\(\mathbf{x_0}\)</span>是最优解（不管是否为顶点），而 <span class="math inline">\(\mathbf{c^Tx_0} \leq \mathbf{c^T x_k}\)</span>，所以 <spanclass="math inline">\(\mathbf{x_k}\)</span> 也是最优解。证毕！</p><blockquote><p>注：如何可行域是无界的，如果存在最优解的话，也一定可以在顶点处取得，这里将证明略去。</p></blockquote><p><strong>定理四</strong>：线性规划中可行域的每一个顶点都是基可行解；同样每个基可行解也都是可行域的顶点。</p><p><strong>证明</strong>：<strong>首先证明线性规划中可行域的每一个顶点都是基可行解</strong>。</p><p>假设线性规划约束条件中，矩阵 <spanclass="math inline">\(\mathbf{A}\)</span> 的秩为 <spanclass="math inline">\(m\)</span>，当前顶点为 <spanclass="math inline">\(\mathbf{x}\)</span>，且 <spanclass="math inline">\(\mathbf{x}\)</span> 的分量中，非 <spanclass="math inline">\(0\)</span> 分量的个数为 <spanclass="math inline">\(k\)</span>。分为以下 <spanclass="math inline">\(2\)</span> 种情况讨论：</p><ol type="1"><li><span class="math inline">\(\mathbf{x}\)</span> 的非零分量对应 <spanclass="math inline">\(\mathbf{A}\)</span>中的列向量线性无关，有基可行解的定义可知， <spanclass="math inline">\(\mathbf{x}\)</span> 是一个基可行解。</li><li><span class="math inline">\(\mathbf{x}\)</span> 的非零分量对应 <spanclass="math inline">\(\mathbf{A}\)</span>中的列向量线性相关。以下使用反正法证明该情况不存在。首先假设这种情况存在，为使描述更加方便，这里假设<span class="math inline">\(\mathbf{x}\)</span> 的前 <spanclass="math inline">\(k\)</span> 个分量不为 <spanclass="math inline">\(0\)</span>，其余分量均为 <spanclass="math inline">\(0\)</span>，所以 <spanclass="math inline">\(\mathbf{x} =(x_1,x_2,...,x_k,0,0,..,0)^T\)</span>。由于 <spanclass="math inline">\(\mathbf{x}\)</span>在可行域中，所以非零分量都只能是正数。将矩阵 <spanclass="math inline">\(\mathbf{A}\)</span> 写成列向量的形式：<spanclass="math inline">\(\mathbf{A} =[\mathbf{a_1},\mathbf{a_2},...,\mathbf{a_n}]\)</span>。线性方程组的形式为</li></ol><p><span class="math display">\[x_1 \mathbf{a_1} + x_2 \mathbf{a_2} + ... + x_n \mathbf{a_n} =\mathbf{b}\]</span></p><p>由于 <span class="math inline">\(\mathbf{x}\)</span> 中，只有前 <spanclass="math inline">\(k\)</span> 个分量非零，其余分量为 <spanclass="math inline">\(0\)</span>，化简如下： <spanclass="math display">\[\begin{equation}x_1 \mathbf{a_1} + x_2 \mathbf{a_2} + ... + x_k \mathbf{a_k} =\mathbf{b} \tag{1}\end{equation}\]</span> 另外由于向量 <spanclass="math inline">\(\mathbf{a_1},\mathbf{a_2},...,\mathbf{a_k}\)</span>线性相关，所以方程组：<spanclass="math inline">\([\mathbf{a_1},\mathbf{a_2},...,\mathbf{a_k}] \cdot\mathbf{x} = \mathbf{0}\)</span> 有无穷多组解。设其中一组非零解为 <spanclass="math inline">\(\mathbf{y}\)</span>，即： <spanclass="math display">\[y_1 \mathbf{a_1} + y_2 \mathbf{a_2} + ... + y_k \mathbf{a_k} =\mathbf{0}\]</span> 将以上方程两边同时乘以极小的正实数 <spanclass="math inline">\(\varepsilon\)</span> 可得： <spanclass="math display">\[\varepsilon y_1 \mathbf{a_1} + \varepsilon y_2 \mathbf{a_2} + ... +\varepsilon y_k \mathbf{a_k} = \mathbf{0} \tag{2}\]</span> <span class="math inline">\((1)\)</span> 式减 <spanclass="math inline">\((2)\)</span>式可得： <span class="math display">\[(x_1 - \varepsilon y_1) \mathbf{a_1} +(x_2 - \varepsilon y_2) \mathbf{a_2} +... +(x_k -\varepsilon y_k) \mathbf{a_k} = \mathbf{b} \tag{3}\]</span> 由于 <span class="math inline">\(\varepsilon\)</span>是一个任意小的正实数，可以保证方程 <spanclass="math inline">\((3)\)</span> 中的每一项 <spanclass="math inline">\(x_i - \varepsilon y_i\)</span>都是正数，所以得到了一个新的可行解 <spanclass="math inline">\(\mathbf{x} - \varepsilon \mathbf{y}\)</span>。同理，将 <span class="math inline">\((1)\)</span> 式加 <spanclass="math inline">\((2)\)</span> 式也可得的一个新的可行解 <spanclass="math inline">\(\mathbf{x} + \varepsilon \mathbf{y}\)</span>。由于<span class="math inline">\(\varepsilon \neq 0\)</span> ，所以 <spanclass="math inline">\(\mathbf{x} - \varepsilon \mathbf{y} \neq\mathbf{x} + \varepsilon \mathbf{y}\)</span>。同时 <spanclass="math inline">\(\mathbf{x} = \frac{1}{2}(\mathbf{x} - \varepsilon\mathbf{y}) + \frac{1}{2} (\mathbf{x} + \varepsilon\mathbf{y})\)</span>，所以 <spanclass="math inline">\(\mathbf{x}\)</span>不是可行域的顶点，与前提条件矛盾，所以该情况不存在。</p><p><strong>以下为证明线性规划的每个基可行解都是可行域的顶点</strong>。</p><p>设当前基可行解为 <spanclass="math inline">\(\mathbf{x}\)</span>，矩阵 <spanclass="math inline">\(\mathbf{A}\)</span> 的秩为 <spanclass="math inline">\(m\)</span> ，那么 <spanclass="math inline">\(\mathbf{x}\)</span> 中就有 <spanclass="math inline">\(m\)</span> 个基变量，同时其余的非基变量均为 <spanclass="math inline">\(0\)</span>。为了描述更加简便，不妨设 <spanclass="math inline">\(\mathbf{x}\)</span> 的前 <spanclass="math inline">\(m\)</span> 个分量为基变量，则 <spanclass="math inline">\(\mathbf{x} = (x_1, x_2, ..., x_m,0,0,...,0)^T\)</span>，其中当 <span class="math inline">\(1 \leq i \leqm\)</span>时，<span class="math inline">\(x_i \ge0\)</span>。由于可行解是一个凸集，<spanclass="math inline">\(\mathbf{x}\)</span> 可以写为形式： <spanclass="math inline">\(\mathbf{x} = \lambda \cdot \mathbf{y} +(1-\lambda) \cdot \mathbf{z}\)</span>，其中 <spanclass="math inline">\(\mathbf{y,z}\)</span> 也是可行解，<spanclass="math inline">\(0 \leq \lambda \leq 1\)</span>。为了证明 <spanclass="math inline">\(\mathbf{x}\)</span> 是顶点，只需要证明 <spanclass="math inline">\(\mathbf{y} = \mathbf{z}\)</span> 即可。</p><p>由于 <span class="math inline">\(\mathbf{y,z}\)</span>是可行解，所以它们的每一个分量都大于等于 <spanclass="math inline">\(0\)</span>，而 <spanclass="math inline">\(\lambda\)</span> 和 <span class="math inline">\(1- \lambda\)</span> 也都大于等于 <spanclass="math inline">\(0\)</span>，对于 <spanclass="math inline">\(\mathbf{x}\)</span> 取 <spanclass="math inline">\(0\)</span> 的分量， <spanclass="math inline">\(\mathbf{y,z}\)</span> 也只能取 0。<spanclass="math inline">\(\mathbf{y,z}\)</span> 的形式如下:</p><p><span class="math display">\[\mathbf{x} = (x_1, x_2, ... x_m, 0, 0, ..., 0)^T \\\mathbf{y} = (y_1, y_2, ... y_m, 0, 0, ..., 0)^T \\\mathbf{z} = (z_1, z_2, ... z_m, 0, 0, ..., 0)^T \\\]</span> 另一方面，由于 <spanclass="math inline">\(\mathbf{y,z}\)</span>是可行解，它们也满足关系：<spanclass="math inline">\(\mathbf{Ay=b},\mathbf{Az=b}\)</span></p><p>对 <span class="math inline">\(\mathbf{x,y,z}\)</span> 只截取前 <spanclass="math inline">\(m\)</span> 个分量，分别命名为 <spanclass="math inline">\(\mathbf{\tilde x, \tilde y,\tilde z}\)</span>。<span class="math display">\[\mathbf{\tilde x} = (x_1, x_2, ... x_m)^T \\\mathbf{\tilde y} = (y_1, y_2, ... y_m)^T \\\mathbf{\tilde z} = (z_1, z_2, ... z_m)^T \\\]</span> 取矩阵 <span class="math inline">\(\mathbf{A}\)</span>中对应的 <span class="math inline">\(m\)</span> 个列向量，组成一个 <spanclass="math inline">\(m \times m\)</span> 的方阵 <spanclass="math inline">\(\mathbf{\tilde A}\)</span>，对应 <spanclass="math inline">\(\mathbf{b}\)</span> 中的 <spanclass="math inline">\(m\)</span> 个分量组成向量 <spanclass="math inline">\(\mathbf{\tilde b}\)</span>。那么有以下关系： <spanclass="math display">\[\begin{align*}\mathbf{\tilde A \tilde y} &amp;= \mathbf{\tilde b}     &amp; (1) \\\mathbf{\tilde A \tilde z} &amp;= \mathbf{\tilde b}     &amp; (2) \\\mathbf{\tilde A (\tilde y - \tilde z)} &amp;= \mathbf{0}     &amp;(1)-(2) \\\end{align*}\]</span> 由于 <span class="math inline">\(\mathbf{\tilde A}\)</span>的列向量线性无关，方程 <span class="math inline">\(\mathbf{\tilde A(\tilde y - \tilde z)} = \mathbf{0}\)</span> 只有零解，所以 <spanclass="math inline">\(\mathbf{\tilde y - \tilde z} =\mathbf{0}\)</span>。即 <span class="math inline">\(\mathbf{\tilde y=\tilde z}\)</span>，再拼接上剩余的 <spanclass="math inline">\(0\)</span> 分量，可得 <spanclass="math inline">\(\mathbf{y = z}\)</span>。所以 <spanclass="math inline">\(\mathbf{x}\)</span> 是可行域的顶点。证毕！</p><h2 id="单纯形法">单纯形法</h2><p>单纯形法的基本思路是基于迭代的思想，并非直接计算目标，而是通过迭代计算，一步一步的靠近目标，从而最终达到目标值。通过前面的介绍，我们知道线性规划如果能取到最大值，一定可以在某个基可行解取到最大值，所以迭代的方向是从一个基可行解变换到另一个相邻的基可行解，同时使得结果更加接近最大值。</p><p><strong>定义四</strong>：如果两个基可行解仅有一个基变量互换，那么称这<span class="math inline">\(2\)</span> 个基可行解是相邻的。</p><p><strong>例二</strong>：基可行解 <spanclass="math inline">\((x_1,x_2,x_3,0,0)^T\)</span> 和 <spanclass="math inline">\((x_1,x_2,0,0,x_5)^T\)</span>就是相邻的，它们可以通过互换 <span class="math inline">\(x_3\leftrightarrow x_5\)</span> 获得。</p><h3 id="最简形式">最简形式</h3><p>如果我们已经确定了基变量，那么通过初等行变换，必然能使基变量对应的矩阵形成一个单位矩阵，这里这种形式的矩阵称为<strong>最简形式</strong>。单位矩阵的<span class="math inline">\(m\)</span>个列向量线性无关，被作为基向量，相应的 <spanclass="math inline">\(x_i\)</span>为基变量。如果忽略列的下标，进行初等列变换之后，可以将单位矩阵放到最左边。<span class="math display">\[\begin{pmatrix}1 &amp; 0 &amp; \dots &amp; 0 &amp; a_{1,m+1} &amp; a_{1,m+2}, &amp;\dots &amp; a_{1,n} \\0 &amp; 1 &amp; \dots &amp; 0 &amp; a_{2,m+1} &amp; a_{2,m+2}, &amp;\dots &amp; a_{2,n} \\\vdots &amp; \vdots &amp; \ddots &amp; \vdots \\0 &amp; 0 &amp; \dots &amp; 1 &amp; a_{m,m+1} &amp; a_{m,m+2}, &amp;\dots &amp; a_{m,n}\end{pmatrix}\cdot\begin{pmatrix}x_1 \\x_2 \\\vdots \\x_n\end{pmatrix}=\begin{pmatrix}b_1 \\b_2 \\\vdots \\b_m \\\end{pmatrix}\]</span>矩阵这样变换之后，使得后面的分析和处理都更加方便。比如可以直接把对应的基解写出来：<span class="math display">\[\mathbf{x} = (b_1, b_2, ... b_m, 0, 0, ..., 0)^T\]</span> 对于矩阵 <span class="math inline">\(\mathbf{A} =[\mathbf{a_1},\mathbf{a_2},...,\mathbf{a_n}]\)</span> 中的列向量 <spanclass="math inline">\(\mathbf{a_j}\)</span> ，其中 <spanclass="math inline">\(m+1 \leq j \leqn\)</span>，可以写为基向量的线性组合： <span class="math display">\[\begin{equation}\mathbf{a_j} = a_{1j} \cdot \mathbf{a_1} + a_{2j} \cdot \mathbf{a_2} +... + a_{mj} \cdot \mathbf{a_m} \tag{4}\end{equation}\]</span></p><p><strong>例三</strong>：考虑如下线性方程组： <spanclass="math display">\[\begin{pmatrix}{\color{red} 1}  &amp;  {\color{red} 0} &amp; {\color{red} 0}  &amp;-2  &amp; 0 \\{\color{red} 0}  &amp;  {\color{red} 1} &amp; {\color{red} 0}  &amp;5  &amp; 1 \\{\color{red} 0}  &amp;  {\color{red} 0} &amp; {\color{red} 1}  &amp;1  &amp; 3\end{pmatrix}\cdot\begin{pmatrix}x_1 \\x_2 \\x_3 \\x_4 \\x_5\end{pmatrix}=\begin{pmatrix}2 \\4 \\1 \\\end{pmatrix}\]</span> 由于矩阵 <span class="math inline">\(\mathbf{A}\)</span> 中前3 个列向量组成单位矩阵，所以是最简形式。可以把 <spanclass="math inline">\(x_1,x_2,x_3\)</span> 为基变量，直接写出基解：<spanclass="math inline">\(\mathbf{x} = (2, 4, 1, 0, 0)^T\)</span>。把前<span class="math inline">\(3\)</span> 个列向量作为基向量，矩阵的第<span class="math inline">\(4\)</span>个列向量可以写为基向量的线性组合，如下形式： <spanclass="math display">\[\begin{pmatrix}-2 \\5 \\1 \\\end{pmatrix}=-2 \cdot\begin{pmatrix}1 \\0 \\0 \\\end{pmatrix}+5 \cdot\begin{pmatrix}0 \\1 \\0 \\\end{pmatrix}+1 \cdot\begin{pmatrix}0 \\0 \\1 \\\end{pmatrix}\]</span></p><h2 id="基变换">基变换</h2><p>如果我们已经把线性规划化的矩阵 <spanclass="math inline">\(\mathbf{A}\)</span>转化为了最简形式，并找到一个基可行解 <spanclass="math inline">\(\mathbf{x}\)</span>。再给定一个非基变量 <spanclass="math inline">\(x_j\)</span> ，如果想要让 <spanclass="math inline">\(x_j\)</span> 成为基变量，应该对矩阵 <spanclass="math inline">\(\mathbf{A}\)</span> 做何种变换呢？</p><p>设矩阵 <span class="math inline">\(\mathbf{A}\)</span> 的秩是 <spanclass="math inline">\(m\)</span>，由于矩阵 <spanclass="math inline">\(\mathbf{A}\)</span> 是最简形式，其基可行解的形式：<span class="math inline">\(\mathbf{x} =(b_1,b_2,...b_m,0,0,...,0)\)</span>，即： <span class="math display">\[\begin{equation}x_i =\left\{\begin{aligned}b_{i} &amp; , &amp; 1 \leq i \leq m, \quad &amp; x_i 是基变量 \\0 &amp; , &amp; m+1 \leq i \leq n, \quad &amp; x_i 是非基变量 \\\end{aligned}\right.\end{equation}\]</span> 将矩阵 <span class="math inline">\(\mathbf{A}\)</span>写成列向量的形式：<span class="math inline">\(\mathbf{A} =[\mathbf{a_1},\mathbf{a_2},...,\mathbf{a_n}]\)</span>。将基可行解带入线性方程组<span class="math inline">\(\mathbf{Ax=b}\)</span> 可得： <spanclass="math display">\[x_1 \mathbf{a_1} + x_2 \mathbf{a_2} + ... + x_n \mathbf{a_n} =\mathbf{b}\]</span> 由于当<span class="math inline">\(1 \leq i \leqm\)</span>时，<span class="math inline">\(x_i=b_i\)</span>； 当 <spanclass="math inline">\(i &gt; m\)</span> 时，<spanclass="math inline">\(x_i = 0\)</span>，所以方程简化为： <spanclass="math display">\[\begin{equation}b_1 \mathbf{a_1} + b_2 \mathbf{a_2} + ... + b_m \mathbf{a_m} =\mathbf{b} \tag{5}\end{equation}\]</span> 回顾基解的定义，当 <span class="math inline">\(j &gt;m\)</span> 时，<spanclass="math inline">\(\{\mathbf{a_1,a_2,...,a_m,a_j}\}\)</span>线性相关，<span class="math inline">\(\mathbf{a_j}\)</span>可以写成其他向量的线性组合，参考方程 <spanclass="math inline">\((4)\)</span> 即： <span class="math display">\[-a_{1j} \mathbf{a_1} - a_{2j} \mathbf{a_2} - ... - a_{mj} \mathbf{a_m} +\mathbf{a_j} = \mathbf{0}\]</span> 将上式两端同时乘上一个极小的正数 <spanclass="math inline">\(\varepsilon\)</span> 得： <spanclass="math display">\[\begin{equation}-\varepsilon a_{1j} \mathbf{a_1} - \varepsilon a_{2j} \mathbf{a_2} - ...- \varepsilon a_{mj} \mathbf{a_m} + \varepsilon \mathbf{a_j}= \mathbf{0}\tag{6}\end{equation}\]</span> <span class="math inline">\((5) + (6)\)</span> 可得： <spanclass="math display">\[\begin{equation}(b_1-\varepsilon a_{1j}) \mathbf{a_1} + (b_2-\varepsilon a_{2j})\mathbf{a_2} + ... + (b_m - \varepsilon a_{mj}) \mathbf{a_m} +\varepsilon \mathbf{a_j}= \mathbf{b} \tag{7}\end{equation}\]</span> 对于该方程，分为以下 <span class="math inline">\(2\)</span>种情况：</p><ol type="1"><li><p>基变量全都是非退化的，即当<span class="math inline">\(1 \leq i\leq m\)</span> 时 <span class="math inline">\(x_i &gt;0\)</span>，那么当 <span class="math inline">\(\varepsilon\)</span>从一个极小的正数开始逐渐增大时，<spanclass="math inline">\(b_i-\varepsilon a_{ij}\)</span>可能会增加，也可以会降低，也可能保持不变。只要有任何一个降低（当 <spanclass="math inline">\(a_{ij} &gt; 0\)</span> 时），当 <spanclass="math inline">\(\varepsilon\)</span>增加的足够大时，可以找到第一个降低为零的 <spanclass="math inline">\(b_i-\varepsilon a_{ij}\)</span>，这时其他的系数，仍然大于等于零，这样就又找到了另外一个解：</p><p><span class="math display">\[\mathbf{x} = (b_1-\varepsilon a_{1j},b_2-\varepsilon a_{2j}, ..., b_{i-1}-\varepsilona_{i-1,j},b_{i+1}-\varepsilon a_{i+1,j},...,b_m-\varepsilona_{mj},0,...,\varepsilon,0,...,0)\]</span></p><p>由于将 <span class="math inline">\(\mathbf{a_j}\)</span> 写成了 <spanclass="math inline">\(\mathbf{a_1} \sim \mathbf{a_m}\)</span>的线性组合时，线性组合中 <spanclass="math inline">\(\mathbf{a_i}\)</span> 的系数 <spanclass="math inline">\(a_{ij}\)</span> 不为零，由定理一可知 <spanclass="math inline">\(\mathbf{a_j}\)</span>和其他剩余的列向量必然线性无关，这样就让 <span class="math inline">\(x_j= \varepsilon\)</span> 成为了新的基变量，同时 <spanclass="math inline">\(x_i\)</span> 退出基变量。但是当 <spanclass="math inline">\(1 \leq i \leq m\)</span> 时，哪一个 <spanclass="math inline">\(b_i - \varepsilon a_{ij}\)</span> 最先变为 <spanclass="math inline">\(0\)</span> 呢？把 <spanclass="math inline">\(\varepsilon\)</span> 视作变量，如果 <spanclass="math inline">\(a_{ij} &gt; 0\)</span> ，同时 <spanclass="math inline">\(\displaystyle \frac{b_i}{a_{ij}}\)</span>最小，就说明分量 <span class="math inline">\(b_i - \varepsilona_{ij}\)</span> 最先变为 <span class="math inline">\(0\)</span>。</p><p>给定一个特定的 <span class="math inline">\(j\)</span> 之后，如果增加<span class="math inline">\(\varepsilon\)</span> 时，没有 <spanclass="math inline">\(b_i-\varepsilon a_{ij}\)</span> 降低，即所有的<span class="math inline">\(a_{ij}\)</span> （第 <spanclass="math inline">\(j\)</span> 列）都小于等于 <spanclass="math inline">\(0\)</span>，说明当前基解不可能把 <spanclass="math inline">\(x_j\)</span> 换入为基变量。</p></li><li><p>某个基变量是退化的，即存在某个 <span class="math inline">\(1 \leqi \leq m\)</span> ，使得 <span class="math inline">\(x_i =0\)</span>，也就是 <span class="math inline">\(b_i =0\)</span>。由于此时 <span class="math inline">\(x_j =0\)</span>，如果满足 <span class="math inline">\(a_{ij} \neq0\)</span>，在剔除 <span class="math inline">\(\mathbf{a_i}\)</span>之后，<span class="math inline">\(\mathbf{a_j}\)</span>和其他的基向量线性无关，只需要将 <spanclass="math inline">\(x_i\)</span> 换出，就能让 <spanclass="math inline">\(x_j\)</span>成为新的基变量。但是经过变换之后，新的基可行解仍然是退化的。而且经过变换之后，新的基可行解和变换前的基可行解完全相同，只是基变量不同而已。这种基变换并不会让线性规划朝着目标前进，所以是没有意义的，应该避免。能不能避免退化呢？还像处理情况<span class="math inline">\(1\)</span> 一样，考虑让 <spanclass="math inline">\(\varepsilon\)</span> 从零开始增大，如果 <spanclass="math inline">\(a_{ij} &gt; 0\)</span>，由于 <spanclass="math inline">\(b_i=0\)</span>，所以 <spanclass="math inline">\(b_i - \varepsilon a_{ij}\)</span>的值会降低，那么只能变为负数，负数不可能成为可行解。所以在 <spanclass="math inline">\(\varepsilon\)</span>从零开始增大时，只有当退化的基变量系数保持不变或者增加（<spanclass="math inline">\(a_{ij} \leq0\)</span>），某个非退化的基变量系数降低，才能使 <spanclass="math inline">\(x_j\)</span>换入为基变量，得到非退化的基可行解。<strong>在基变量退化的情况下，如果忽略了<spanclass="math inline">\(a_{ij}\)</span> 的符号，仍然按情况 <spanclass="math inline">\(1\)</span> 处理，可能出现某个 <spanclass="math inline">\(x_i\)</span> 为负数的情况，将 <spanclass="math inline">\(x_j\)</span>换入后，得到的不是可行解</strong>。</p></li></ol><p>如果可以使 <span class="math inline">\(x_j\)</span>换入为基变量，应该如何变换呢？通过对上面的分析可以知道：<strong>如果有退化的基变量（<spanclass="math inline">\(b_i = 0\)</span>），首先确保所有退化基对应的系数<span class="math inline">\(a_{ij} \leq0\)</span>，否则就只能变换到同一个退化的基可行解，这种变换没有意义；同时对于所有非退化基（<spanclass="math inline">\(b_i &gt; 0\)</span>），同时满足 <spanclass="math inline">\(a_{ij} &gt; 0\)</span> 的，取 <spanclass="math inline">\(\displaystyle \frac{b_i}{a_{ij}}\)</span>最小的行，将该行对应的 <span class="math inline">\(x_i\)</span>从基变量换出</strong>。如果上述情况不满足，说明无法把 <spanclass="math inline">\(x_j\)</span> 换入为基变量。</p><blockquote><p>有的做法是允许退化，但是可能出现死循环的情况，需要特殊的策略（比如Bland 法则）来解决这个问题。</p></blockquote><p><strong>例四</strong>：已知如下线性方程组，<spanclass="math inline">\(x_1,x_2,x_3\)</span> 为基变量，现在要把 <spanclass="math inline">\(x_4\)</span> 换入为基变量，应该如何操作？ <spanclass="math display">\[\begin{pmatrix}{\color{red} 1}  &amp;  {\color{red} 0} &amp; {\color{red} 0}  &amp;-2  &amp; 0 \\{\color{red} 0}  &amp;  {\color{red} 1} &amp; {\color{red} 0}  &amp;1  &amp; 1 \\{\color{red} 0}  &amp;  {\color{red} 0} &amp; {\color{red} 1}  &amp;2  &amp; 1\end{pmatrix}\cdot\begin{pmatrix}x_1 \\x_2 \\x_3 \\x_4 \\x_5\end{pmatrix}=\begin{pmatrix}1 \\3 \\4 \\\end{pmatrix}\]</span> 首先基解为：<span class="math inline">\(\mathbf{x} =(1,3,4,0,0)^T\)</span>，可以看到，基变量都是正数，所以没有退化的情况。找到第<span class="math inline">\(4\)</span> 列为正数的项：<spanclass="math inline">\(a_{2,4} = 1，a_{3,4} =2\)</span>，然后分别计算：<spanclass="math inline">\(\displaystyle\frac{b_2}{a_{2,4}} = 3\)</span> 和<span class="math inline">\(\displaystyle\frac{b_3}{a_{3,4}} =2\)</span>，可以看到 <spanclass="math inline">\(\displaystyle\frac{b_2}{a_{2,4}} &gt;\frac{b_3}{a_{3,4}}\)</span>。在第 <spanclass="math inline">\(4\)</span> 列中，因为 <spanclass="math inline">\(\displaystyle\frac{b_3}{a_{3,4}}\)</span>为正数，且最小，所以可以把 <span class="math inline">\(x_3\)</span>换出基变量，可以保证 <span class="math inline">\(\mathbf{x} =(x_1,x_2,0,x_4,0)^T\)</span>为基可行解。如何求出新的基可行解的值呢？由于新的基向量组成的矩阵不是单位矩阵，无法直接把新的基可行解写出来。我们希望把它转换为最简形式，可以这样操作，因为要把<span class="math inline">\(x_3\)</span> 换出，<spanclass="math inline">\(x_4\)</span> 换入，所以要以第 <spanclass="math inline">\(3\)</span> 行作为基准，通过高斯消元法把第 <spanclass="math inline">\(4\)</span> 列其他的元素变为 <spanclass="math inline">\(0\)</span>。</p><p>第一步：将矩阵转换为增广矩阵 <spanclass="math inline">\([\mathbf{A|b}]\)</span>。 <spanclass="math display">\[\left (\begin{array}{ccccc|c}1 &amp; 0 &amp; 0 &amp; -2 &amp; 0 &amp; 1 \\0 &amp; 1 &amp; 0 &amp; 1 &amp; 1 &amp; 3 \\0 &amp; 0 &amp; 1 &amp; 2 &amp; 1 &amp; 4 \\\end{array}\right )\]</span> 第二步：把第 <span class="math inline">\(3\)</span> 行除以<span class="math inline">\(2\)</span>，以把 <spanclass="math inline">\(a_{3,4}\)</span> 变为 <spanclass="math inline">\(1\)</span>，可得： <span class="math display">\[\left (\begin{array}{ccccc|c}1 &amp; 0 &amp; 0 &amp; -2 &amp; 0 &amp; 1 \\0 &amp; 1 &amp; 0 &amp; 1 &amp; 1 &amp; 3 \\0 &amp; 0 &amp; \frac{1}{2} &amp; 1 &amp; \frac{1}{2} &amp; 2 \\\end{array}\right )\]</span> 第三步：<span class="math inline">\((1) + 2 \times(3)\)</span> 同时 <span class="math inline">\((2) - (3)\)</span>，可得： <span class="math display">\[\left (\begin{array}{ccccc|c}{\color{red} 1} &amp; {\color{red} 0} &amp; 1 &amp; {\color{red} 0}&amp; 1 &amp; 5 \\{\color{red} 0} &amp; {\color{red} 1} &amp; -\frac{1}{2} &amp;{\color{red} 0} &amp; \frac{1}{2} &amp; 1 \\{\color{red} 0} &amp; {\color{red} 0} &amp; \frac{1}{2} &amp;{\color{red} 1} &amp; \frac{1}{2} &amp; 2 \\\end{array}\right )\]</span> 这样第 <span class="math inline">\(1,2,4\)</span>列就形成了一个新的单位矩阵，新的基可行解为：<spanclass="math inline">\(\mathbf{x} = (5,1,0,2,0)^T\)</span></p><p>在例四的操作，也称为 Pivot操作，翻译为中文有的为旋转，有的为转轴，其实就是高斯消元法，目的是为了形成一个新的单位矩阵，让下一步基变量的替换操作可以继续进行下去。</p><p><strong>例五</strong>：考虑如下线性规划方程组， <spanclass="math inline">\(x_1,x_2,x_3\)</span>为基变量，问能否把 <spanclass="math inline">\(x_4\)</span> 替换为基变量，获取一个新的基可行解？<span class="math display">\[\begin{pmatrix}{\color{red} 1}  &amp;  {\color{red} 0} &amp; {\color{red} 0}  &amp; 1\\{\color{red} 0}  &amp;  {\color{red} 1} &amp; {\color{red} 0}  &amp; 1\\{\color{red} 0}  &amp;  {\color{red} 0} &amp; {\color{red} 1}  &amp; 2\end{pmatrix}\cdot\begin{pmatrix}x_1 \\x_2 \\x_3 \\x_4\end{pmatrix}=\begin{pmatrix}2 \\1 \\0 \\\end{pmatrix}\]</span>注意到该方程最左边的三个列向量形成了一个单位矩阵，所以已经是最简形式了，令<span class="math inline">\(x_4=0\)</span>， 可以求得对应基解：<spanclass="math inline">\(\mathbf{x} = (2,1,0,0)^T\)</span>。由于 <spanclass="math inline">\(x_3\)</span> 也为 <spanclass="math inline">\(0\)</span>，所以该基解是退化的。由于 <spanclass="math inline">\(a_{3,4} \neq 0\)</span>，所以可以直接让 <spanclass="math inline">\(x_3\)</span> 退出基变量，同时让 <spanclass="math inline">\(x_4\)</span>加入基变量，对应的基可行解仍然是：<span class="math inline">\(\mathbf{x}= (2,1,0,0)^T\)</span> 。这只是换了基变量，并没有获得新的基可行解。</p><p>另一方面由于 <span class="math inline">\(a_{3,4} &gt;0\)</span>，通过上述对情况 <span class="math inline">\(2\)</span>的讨论知道，新的基可行解，是不可能把 <spanclass="math inline">\(x_4\)</span> 换入作为基变量的。如果强行把 <spanclass="math inline">\(x_4\)</span> 换入呢？由于 <spanclass="math inline">\(\displaystyle \frac{b_1}{a_{1,4}} =2\)</span>，<span class="math inline">\(\displaystyle\frac{b_2}{a_{2,4}} = 1\)</span>，所以 <spanclass="math inline">\(\displaystyle \frac{b_1}{a_{1,4}} &gt;\frac{b_2}{a_{2,4}}\)</span>，所以 <spanclass="math inline">\(\displaystyle\frac{b_2}{a_{2,4}}\)</span>最小，要把 <span class="math inline">\(x_2\)</span> 从基变量中换出。把第<span class="math inline">\(2\)</span> 行作为基准，对其他行做消元，把第<span class="math inline">\(4\)</span> 列其他元素变为 <spanclass="math inline">\(0\)</span>。</p><p>第一步：将矩阵转换为增广矩阵 <spanclass="math inline">\([\mathbf{A|b}]\)</span> <spanclass="math display">\[\left (\begin{array}{cccc|c}1 &amp; 0 &amp; 0 &amp; 1 &amp; 2 \\0 &amp; 1 &amp; 0 &amp; 1 &amp; 1 \\0 &amp; 0 &amp; 1 &amp; 2 &amp; 0 \\\end{array}\right )\]</span> 第一步：<span class="math inline">\((1)-(2)\)</span>，同时<span class="math inline">\((3)-2\times(2)\)</span> 可得 <spanclass="math display">\[\left (\begin{array}{cccc|c}1 &amp; -1 &amp; 0 &amp; 0 &amp; 1 \\0 &amp; 1 &amp; 0 &amp; 1 &amp; 1 \\0 &amp; -2 &amp; 1 &amp; 0 &amp; -2 \\\end{array}\right )\]</span> 令 <span class="math inline">\(x_2\)</span>为零，求的基解：<span class="math inline">\(\mathbf{x} =(1,0,-2,1)^T\)</span>，由于 <span class="math inline">\(x_3 &lt;0\)</span> ，所以不是基可行解。</p><p>由此可见，在处理线性规划问题时，如果遇到了退化的基可行解时，如果仍然盲目套用旋转操作，可能转换到一个非可行解，从而得到一个错误的结果。</p><h2 id="迭代算法">迭代算法</h2><p>在前面的介绍中，基变换、转轴操作都可以和目标函数无关，如果有了目标函数，能不能让基变换朝着离目标值更近的方向迭代，经过有限次的基变换，达到目标值？基变换需要假设我们已经获取到了一个基可行解，然后才能做新的变换。仍然有<span class="math inline">\(2\)</span> 个问题需要解决：</p><ol type="1"><li>当前基可行解是否已经达到了最优？</li><li>如果当前基可行解不是最优的，下一步应该那一个非基变量换入？</li></ol><h3 id="最优判定">最优判定</h3><p>首先来看，如何判断当前可行解是否为最优解。</p><p>这里给定目标：<span class="math inline">\(\max : c_1 \cdot x_1 + c_2\cdot x_2 + ... + c_n \cdot x_n\)</span>，将单纯形法约束化为最简形式：<span class="math display">\[\begin{pmatrix}1 &amp; 0 &amp; \dots &amp; 0 &amp; a_{1,m+1} &amp; a_{1,m+2}, &amp;\dots &amp; a_{1,n} \\0 &amp; 1 &amp; \dots &amp; 0 &amp; a_{2,m+1} &amp; a_{2,m+2}, &amp;\dots &amp; a_{2,n} \\\vdots &amp; \vdots &amp; \ddots &amp; \vdots \\0 &amp; 0 &amp; \dots &amp; 1 &amp; a_{m,m+1} &amp; a_{m,m+2}, &amp;\dots &amp; a_{m,n}\end{pmatrix}\cdot\begin{pmatrix}x_1 \\x_2 \\\vdots \\x_n\end{pmatrix}=\begin{pmatrix}b_1 \\b_2 \\\vdots \\b_m \\\end{pmatrix}\]</span> 写成线性方程组的形式，并移项可得： <spanclass="math display">\[\begin{aligned}x_1 &amp;= b_1 - (a_{1,m+1} \cdot x_{m+1} + a_{1,m+2} \cdot x_{m+2} +... + a_{1,n} \cdot x_{n}) \\x_2 &amp;= b_2 - (a_{2,m+1} \cdot x_{m+1} + a_{2,m+2} \cdot x_{m+2} +... + a_{2,n} \cdot x_{n}) \\... \\x_m &amp;= b_m - (a_{m,m+1} \cdot x_{m+1} + a_{m,m+2} \cdot x_{m+2} +... + a_{m,n} \cdot x_{n}) \\\end{aligned}\]</span>这样就把所有的基变量用非基变量表示出来了，因此目标函数也可以只用非基变量表示。</p><p><strong>例六</strong>：考察线性规划问题，目标 <spanclass="math inline">\(\max：z = x_1 + 3x_2\)</span>，约束为: <spanclass="math display">\[\begin{pmatrix}1  &amp;  0 &amp; 0  &amp; -2  &amp; 0 \\0  &amp;  1 &amp; 0  &amp; 1  &amp; 1 \\0  &amp;  0 &amp; 1  &amp; 2  &amp; 1\end{pmatrix}\cdot\begin{pmatrix}x_1 \\x_2 \\x_3 \\x_4 \\x_5\end{pmatrix}=\begin{pmatrix}1 \\3 \\4 \\\end{pmatrix}\]</span> 并且：<span class="math inline">\(x_i \geq 0\)</span>，其中<span class="math inline">\(1 \leq i \leq 5\)</span></p><p>由于已经是线性规划最简形式了，可以直接写出基可行解 <spanclass="math inline">\(\mathbf{x} =(1,3,4,0,0)^T\)</span>，同时对线性方程组移项可得： <spanclass="math display">\[\begin{aligned}x_1 &amp;= 1 - (-2 \cdot x_4 + 0 \cdot x_5) \\x_2 &amp;= 3 - (1 \cdot x_4 + 1 \cdot x_5) \\x_3 &amp;= 4 - (2 \cdot x_4 + 1 \cdot x_5)\end{aligned}\]</span> 将 <span class="math inline">\(x_1，x_2\)</span>带入目标可得：<span class="math inline">\(z=1 - (-2 \cdot x_4 + 0 \cdotx_5) + 3(3 - (1 \cdot x_4 + 1 \cdot x_5)) = 10 - x_4 -3x_5\)</span>。由于 <span class="math inline">\(x_4,x_5\)</span>是是非基变量，在基可行解中，取值为 <spanclass="math inline">\(0\)</span>，可以得到当前目标值 <spanclass="math inline">\(z_0=10\)</span>。于此同时，在任意一个可行解中，<spanclass="math inline">\(x_4 \geq 0, x_5 \geq 0\)</span>，都有 <spanclass="math inline">\(z=10 - x_4 - 3x_5 \leq 10\)</span>。所以目标值<span class="math inline">\(z\)</span> 最大取值为 <spanclass="math inline">\(10\)</span>，取值条件为当前的基解：<spanclass="math inline">\(\mathbf{x} = (1,3,4,0,0)^T\)</span>。</p><p>上述例子给了我们判定当前基解是否为最优解的思路：首先要求当前已经是最简形式了，同时有一个基可行解。然后对于目标函数，把基变量都用非基变量表示。如果目标函数中，所有非基变量的系数都是负数，说明当前基解就是优解；否则需要选择一个新的基，继续进行基变换。</p><p>具体的证明和例六是一样的，只不过需要多一些形式化的描述，这里就不展开了。</p><h3 id="选取基">选取基</h3><p>如果最优判定失败了，说明在目标公式中一定存在某个非基变量，系数为正，那么就可以把该变量换入为基变量。</p><p><strong>定理五</strong>：已知线性规划中，目标公式 <spanclass="math inline">\(z=C + c_{m+1}\cdot x_{m+1} + ... + c_{n} \cdotx_{n}\)</span> ，其中 <span class="math inline">\(C\)</span>为常数项，<span class="math inline">\(x_{m+1} \sim x_{n}\)</span>都是非基变量。如果存在某个非基变量 <spanclass="math inline">\(x_{j}\)</span> 的系数系数 <spanclass="math inline">\(c_j &gt; 0\)</span> ，而且可以把 <spanclass="math inline">\(x_{j}\)</span> 换入为基变量。那么把 <spanclass="math inline">\(x_{j}\)</span>换入为基变量之后，目标公式中的常数项 <spanclass="math inline">\(C\)</span>要么不变，要么增加，取决于换出的基变量是否退化。</p><p><strong>证明</strong>：这里仍然只考虑线性方程组已经被转化为最简形式，<spanclass="math inline">\(x_1 \sim x_m\)</span> 是基变量，<spanclass="math inline">\(x_{m+1} \sim x_n\)</span> 是非基变量。 <spanclass="math display">\[\begin{pmatrix}1 &amp; 0 &amp; \dots &amp; 0 &amp; a_{1,m+1} &amp; a_{1,m+2}, &amp;\dots &amp; a_{1,n} \\0 &amp; 1 &amp; \dots &amp; 0 &amp; a_{2,m+1} &amp; a_{2,m+2}, &amp;\dots &amp; a_{2,n} \\\vdots &amp; \vdots &amp; \ddots &amp; \vdots \\0 &amp; 0 &amp; \dots &amp; 1 &amp; a_{m,m+1} &amp; a_{m,m+2}, &amp;\dots &amp; a_{m,n}\end{pmatrix}\cdot\begin{pmatrix}x_1 \\x_2 \\\vdots \\x_n\end{pmatrix}=\begin{pmatrix}b_1 \\b_2 \\\vdots \\b_m \\\end{pmatrix}\]</span></p><p>假设将 <span class="math inline">\(x_{j}\)</span>换入为基变量的同时，<span class="math inline">\(x_i\)</span>被换出为非基变量，在目标函数中，<span class="math inline">\(x_j\)</span>也需要被 <span class="math inline">\(x_i\)</span>替代。由于目标函数的形式为：<span class="math inline">\(z=C+c_j \cdotx_{j} + P\)</span>，其中 <spanclass="math inline">\(P=\sum\limits_{k=1}^{j-1}c_k\cdot x_{k} +\sum\limits_{k=j+1}^{n}c_k\cdot x_{k}\)</span>。由于在非基变量中，只有<span class="math inline">\(x_j\)</span>变成了基变量，所以基变换之后，目标函数中的 <spanclass="math inline">\(P\)</span> 不会改变，只有 <spanclass="math inline">\(c_j \cdot x_{j}\)</span> 会被 <spanclass="math inline">\(x_i\)</span> 的一次项多项式替代。考察第 <spanclass="math inline">\(i\)</span> 个约束方程： <spanclass="math display">\[1 \cdot x_i + a_{i,m+1} \cdot x_{m+1} + ... + a_{i,j} \cdot x_{j} + ...+ a_{i,n} \cdot x_{n} = b_i\]</span> 等式两边同时除以 <span class="math inline">\(a_{ij}\)</span>得： <span class="math display">\[\frac{1}{a_{ij}} \cdot x_i + \frac{a_{i,m+1}}{a_{ij}} \cdot x_{m+1} +... + 1 \cdot x_{j} + ... + \frac{a_{i,n}}{a_{ij}} \cdot x_{n} = b_i\]</span> 移项可得： <span class="math display">\[\begin{aligned}x_{j} &amp;= b_i - (\frac{1}{a_{ij}} \cdot x_i +\frac{a_{i,m+1}}{a_{ij}} \cdot x_{m+1} + ...+\frac{a_{i,j-1}}{a_{i,j-1}} \cdot x_{j-1} + \frac{a_{i,j+1}}{a_{i,j+1}}\cdot x_{j+1} + ... + \frac{a_{i,n}}{a_{ij}} \cdot x_{n}) \\      &amp;= b_i - Q\end{aligned}\]</span> 上述式子中 <span class="math inline">\(Q\)</span> 仍然是 <spanclass="math inline">\(x_1 \sim x_n\)</span> 的线性组合，而常数项只有<span class="math inline">\(b_i\)</span>。把 <spanclass="math inline">\(x_j\)</span> 带入目标函数可得： <spanclass="math display">\[\begin{aligned}z &amp;= C + c_j \cdot x_j + P \\  &amp;= C + c_j \cdot (b_i - Q) + P \\  &amp;= (C + c_j \cdot b_i) - c_j \cdot Q + P\end{aligned}\]</span> 在该式子中 <span class="math inline">\(-c_j \cdot Q +P\)</span> 为 <span class="math inline">\(x_1 \sim x_n\)</span>的线性组合，常数项为 <span class="math inline">\(C+c_j \cdotb_i\)</span>，由于 <span class="math inline">\(c_j &gt; 0\)</span>，对<span class="math inline">\(b_i\)</span> 分两种情况讨论：</p><ol type="1"><li><span class="math inline">\(b_i = 0\)</span>，说明 <spanclass="math inline">\(x_i\)</span> 是退化的基变量，这时 <spanclass="math inline">\(C + c_j \cdot b_i = C\)</span>，即把 <spanclass="math inline">\(x_j\)</span>换入为基变量之后，目标函数中的常数项保持不变。前面基变换的小节已经讨论过这种情况，应该避免把退化的基变量换出，因为不会得到新的可行解。</li><li><span class="math inline">\(b_i &gt; 0\)</span>，说明 <spanclass="math inline">\(x_i\)</span> 是没有退化的基变量，这时 <spanclass="math inline">\(C + c_j \cdot b_i &gt; C\)</span>，把 <spanclass="math inline">\(x_j\)</span>换入为基变量之后，目标函数中的常数项增加 <span class="math inline">\(c_j\cdot b_i\)</span>。</li></ol><p>证毕！</p><p>通过上述定理可以知道，在非退化场景下，如果目标函数中 <spanclass="math inline">\(x_j\)</span> 的系数为正，把 <spanclass="math inline">\(x_j\)</span>换入作为基变量之后，得到的新基可行解一定比原基可行解更加接近目标值。如果所有系数为正的非基变量，都不能换入为基变量，说明可行域中已经没有顶点使得目标值更大，该目标的解是无界的。为什么不是无解呢？因为单纯形法的初始条件是从一个基可行解出发，所以一定有解。</p><blockquote><p>通过以上定理可以知道，通过一次单纯形法的基变换，目标函数的增量为：<spanclass="math inline">\(c_j \cdotb_i\)</span>。如果遍历所有非基变量，找到所有满足条件的 <spanclass="math inline">\(c_j \cdotb_i\)</span>，然后选取一个最大的值，把对应的 <spanclass="math inline">\(x_j\)</span>换入为基变量，能够最大幅度的提升单纯形法迭代速度。但是会增加代码复杂度，为了简化起见，本文不会使用这个策略，如果感兴趣，可由读者自行实现。</p></blockquote><p><strong>例七</strong>：求下列线性规划的最优可行解以及对应的最优值。</p><p>目标：<span class="math inline">\(\max：z = x_1 + 3x_2 -x_3\)</span></p><p>约束： <span class="math display">\[\begin{pmatrix}1 &amp; 0 &amp; 0 &amp; 1 &amp; 2 \\0 &amp; 1 &amp; 0 &amp; 2 &amp; -1 \\0 &amp; 0 &amp; 1 &amp; 1 &amp; 3 \\\end{pmatrix}\cdot\begin{pmatrix}x_1 \\x_2 \\x_3 \\x_4 \\x_5 \\\end{pmatrix}=\begin{pmatrix}2 \\2 \\6 \\\end{pmatrix}\]</span> 其中：<span class="math inline">\(x_i \geq 0\)</span>，<spanclass="math inline">\(1 \leq i \leq 5\)</span></p><p><strong>解法一</strong>：带入法求解，该方法易于理解，但是手动运算比较多。首先可以看到，约束方程中，已经出现了一个单位矩阵，在该形式下可以直接写出一个基解：<spanclass="math inline">\(\mathbf{x} =(2,2,6,0,0)^T\)</span>。与此同时，我们可以对约束方程做变形，把所有的基变量<span class="math inline">\(x_1,x_2,x_3\)</span> 都用非基变量 <spanclass="math inline">\(x_4\)</span> 和 <spanclass="math inline">\(x_5\)</span> 表示： <span class="math display">\[\begin{aligned}x_1 &amp;= 2 -x_4 -2x_5  \\x_2 &amp;= 2 -2x_4 +x_5  \\x_3 &amp;= 6 -x_4 -3x_5\end{aligned}\]</span> 带入到目标函数 <span class="math inline">\(z\)</span> 可得：<span class="math display">\[\begin{aligned}z &amp;= x_1 + 3x_2 - x_3 \\  &amp;= (2-x_4-2x_5) + 3 \cdot (2-2x_4+x_5) - (6 - x_4 -3x_5) \\  &amp;= 2 - 6x_4 + 4x_5\end{aligned}\]</span> 可以看到，在当前基可行解 <spanclass="math inline">\(\mathbf{x} = (2,2,6,0,0)^T\)</span>的情况下，目标函数 <span class="math inline">\(z\)</span> 取值为 <spanclass="math inline">\(2\)</span>。由于在当前基可行解中 <spanclass="math inline">\(x_5\)</span> 为 <spanclass="math inline">\(0\)</span>，而目标函数中，<spanclass="math inline">\(x_5\)</span> 的系数为正数，如果 <spanclass="math inline">\(x_5\)</span>增大的话，可以提高函数的取值，所以可以让 <spanclass="math inline">\(x_5\)</span>成为新的基变量（只有基变量才不为零）。查看线性方程组的系数矩阵第 <spanclass="math inline">\(5\)</span> 列，只有 <spanclass="math inline">\(a_{1,5}\)</span> 和 <spanclass="math inline">\(a_{3,5}\)</span> 为正数，并且 <spanclass="math inline">\(\displaystyle \frac{b_1}{a_{1,5}} &lt;\frac{b_3}{a_{3,5}}\)</span>，所以应该把 <spanclass="math inline">\(x_1\)</span>从基变量中换出。首先写出线性方程组的增广矩阵 <spanclass="math inline">\(\mathbf{[A|b]}\)</span>： <spanclass="math display">\[\left (\begin{array}{ccccc|c}1 &amp; 0 &amp; 0 &amp; 1 &amp; 2 &amp; 2 \\0 &amp; 1 &amp; 0 &amp; 2 &amp; -1 &amp; 2 \\0 &amp; 0 &amp; 1 &amp; 1 &amp; 3 &amp; 6 \\\end{array}\right )\]</span> 现在要用第 <span class="math inline">\(1\)</span> 行，第 <spanclass="math inline">\(5\)</span> 列，把第 <spanclass="math inline">\(5\)</span> 列的其他元素都消为 <spanclass="math inline">\(0\)</span>。首先应该把 <spanclass="math inline">\(a_{1,5}\)</span> 变为 <spanclass="math inline">\(1\)</span>，操作为 <spanclass="math inline">\(\displaystyle (1) \times \frac{1}{2}\)</span><span class="math display">\[\left (\begin{array}{ccccc|c}\frac{1}{2} &amp; 0 &amp; 0 &amp; \frac{1}{2} &amp; 1 &amp; 1 \\0 &amp; 1 &amp; 0 &amp; 2 &amp; -1 &amp; 2 \\0 &amp; 0 &amp; 1 &amp; 1 &amp; 3 &amp; 6 \\\end{array}\right )\]</span> 然后 <span class="math inline">\((2) + (1)\)</span> 和 <spanclass="math inline">\((3) - 3 \times (1)\)</span>: <spanclass="math display">\[\left (\begin{array}{ccccc|c}\frac{1}{2} &amp; 0 &amp; 0 &amp; \frac{1}{2} &amp; 1 &amp; 1 \\\frac{1}{2} &amp; 1 &amp; 0 &amp; \frac{5}{2} &amp; 0 &amp; 3 \\-\frac{3}{2} &amp; 0 &amp; 1 &amp; -\frac{1}{2} &amp; 0 &amp; 3 \\\end{array}\right )\]</span> 矩阵的第 <span class="math inline">\(2,3,5\)</span>列可以组成新的单位矩阵，对应基解为：<spanclass="math inline">\(\mathbf{x} =(0,3,3,0,1)^T\)</span>。写出矩阵的第一行对应的方程： <spanclass="math display">\[\frac{1}{2} \cdot x_1 + 0\cdot x_2 + 0\cdot x_3 + \frac{1}{2} \cdot x_4+ 1 \cdot x_5 = 1\]</span> 移项可得： <span class="math display">\[x_5 = 1 - \frac{1}{2} \cdot x_1 - \frac{1}{2} \cdot x_4\]</span> 带入目标函数中： <span class="math display">\[\begin{aligned}z &amp;= 2 - 6x_4 + 4x_5 \\  &amp;= 2 - 4x_4 + 4 \cdot (1 - \frac{1}{2} \cdot x_1 - \frac{1}{2}\cdot x_4) \\  &amp;= 6 - 2x_1 - 6x_4\end{aligned}\]</span> 即把基解 <span class="math inline">\(\mathbf{x} =(0,3,3,0,1)^T\)</span> 带入目标函数可取得目标值 <spanclass="math inline">\(6\)</span>，由于 <span class="math inline">\(z = 6- 2x_1 - 6x_4 \leq 6\)</span>，所以目标函数的最大值为 <spanclass="math inline">\(6\)</span>。</p><p><strong>解法二</strong>：高斯消元法求解，该方法以矩阵操作为主，便于用程序化实现，其本质和方法一没有区别。从方法一对线性方程组使用高斯消元，但是对目标函数使用的是带入法。实际上可以把目标函数也加入到线性方程组中，这样使用高斯消元法的时候能把目标函数做消元。<span class="math display">\[\begin{pmatrix}1 &amp; 0 &amp; 0 &amp; 1 &amp; 2 \\0 &amp; 1 &amp; 0 &amp; 2 &amp; -1 \\0 &amp; 0 &amp; 1 &amp; 1 &amp; 3 \\1 &amp; 3 &amp; -1 &amp; 0 &amp; 0 \\\end{pmatrix}\cdot\begin{pmatrix}x_1 \\x_2 \\x_3 \\x_4 \\x_5 \\\end{pmatrix}=\begin{pmatrix}2 \\2 \\6 \\z \\\end{pmatrix}\]</span></p><p>把原有的线性方程组和目标函数拼接为一个大矩阵 <spanclass="math inline">\(\mathbf{S}\)</span>： <spanclass="math display">\[\left (\begin{array}{ccccc|c}1 &amp; 0 &amp; 0 &amp; 1 &amp; 2 &amp; 2 \\0 &amp; 1 &amp; 0 &amp; 2 &amp; -1 &amp; 2 \\0 &amp; 0 &amp; 1 &amp; 1 &amp; 3 &amp; 6 \\\hline1 &amp; 3 &amp; -1 &amp; 0 &amp; 0 &amp; z\end{array}\right )\]</span> 仍然像解法一一样，找到 <span class="math inline">\(3\)</span>个基向量，确定初始可行解：<span class="math inline">\(\mathbf{x} =(2,2,6,0,0)^T\)</span>。然后用高斯消元法，把矩阵 <spanclass="math inline">\(\mathbf{S}\)</span>的最后一行中，基向量对应的分量消除，依次执行操作：<spanclass="math inline">\((4)-(1),(4)-3\times(2),(4)+(3)\)</span>，矩阵<span class="math inline">\(\mathbf{S}\)</span> 可转化为： <spanclass="math display">\[\left (\begin{array}{ccccc|c}1 &amp; 0 &amp; 0 &amp; 1 &amp; 2 &amp; 2 \\0 &amp; 1 &amp; 0 &amp; 2 &amp; -1 &amp; 2 \\0 &amp; 0 &amp; 1 &amp; 1 &amp; 3 &amp; 6 \\\hline0 &amp; 0 &amp; 0 &amp; -6 &amp; 4 &amp; z-2\end{array}\right )\]</span> 观察矩阵 <span class="math inline">\(\mathbf{S}\)</span>的最后一行，第 <span class="math inline">\(5\)</span>列为正数，所以可以把 <span class="math inline">\(x_5\)</span>换入为基变量；然后观察第 <span class="math inline">\(5\)</span>列的正数项所在行，可得 <span class="math inline">\(\displaystyle\frac{b_1}{a_{1,5}} &lt; \frac{b_3}{a_{3,5}}\)</span>。所以应该把 <spanclass="math inline">\(x_1\)</span> 从基变量中换出。为了继续把矩阵 <spanclass="math inline">\(\mathbf{S}\)</span> 化简，需要用第 <spanclass="math inline">\(1\)</span> 行，把其他行的第 <spanclass="math inline">\(5\)</span> 列都消为 <spanclass="math inline">\(0\)</span>。变换的方法和解法一类似，只不过矩阵的最后一行也参与变换，最终结果为：<span class="math display">\[\left (\begin{array}{ccccc|c}\frac{1}{2} &amp; 0 &amp; 0 &amp; \frac{1}{2} &amp; 1 &amp; 1 \\\frac{1}{2} &amp; 1 &amp; 0 &amp; \frac{5}{2} &amp; 0 &amp; 3 \\-\frac{3}{2} &amp; 0 &amp; 1 &amp; -\frac{1}{2} &amp; 0 &amp; 3 \\\hline-2 &amp; 0 &amp; 0 &amp; -6 &amp; 0 &amp; z-6\end{array}\right )\]</span> 矩阵最后一行对应的方程为<span class="math inline">\(- 2x_1 -6x_4 = z - 6\)</span>，移项得：<span class="math inline">\(z = 6 - 2x_1- 6x_4\)</span>。对应的基可行解：<span class="math inline">\(\mathbf{x}= (0,3,3,0,1)^T\)</span>，取得的目标值：<spanclass="math inline">\(6\)</span></p><h3 id="避免退化基变换单纯形算法">避免退化基变换单纯形算法</h3><p>在基变换过程中，如果允许从一个退化的基可行解变换为另一个退化的基可行解，在单纯形法的实施过程中，有可能出现环的情况，永远无法退出。通过前文的介绍，如果单纯形法中出现了一个退化的可行解，要么当前解就是最优解；要么可以通过基变换，变换到另外一个基可行解，使得目标函数进一步靠近目标值。通过避免退化的方法，可以使每一个基变换的迭代，目标函数都更加靠近目标值。所以通过有限步的迭代，可以达到目标值，永远不会出现死循环的情况。算法步骤如下：</p><ol type="1"><li><p>初始化矩阵：初始化单纯形矩阵 <spanclass="math inline">\(\mathbf{S}\)</span> 为如下形式： <spanclass="math display">\[\left [\begin{array}{c|c}\mathbf{A} &amp; \mathbf{b} \\\mathbf{c^T} &amp; 0\end{array}\right]\]</span></p></li><li><p>初始化基可行解：找到一个初始的基可行解 <spanclass="math inline">\(\mathbf{x}\)</span></p></li><li><p>最优判定：判定矩阵 <spanclass="math inline">\(\mathbf{S}\)</span>的最后一行元素，是否除了最后一个之外，其他元素都小于等于零。如果是说明当前基可行解可使目标函数达到最优，最优值为矩阵<span class="math inline">\(\mathbf{S}\)</span>最后一行最后一列的相反数，算法结束；否则进入下一步</p></li><li><p>换入基选择：对于矩阵 <spanclass="math inline">\(\mathbf{S}\)</span> 的最后一行，找到一个 <spanclass="math inline">\(c_j\)</span> 为正的值，判定 <spanclass="math inline">\(x_j\)</span>是否能成为换入的基变量。判定方法为，首先判断是否有 <spanclass="math inline">\(b_i = 0\)</span>的情况，如果没有这种情况，进入下一步。否则说明有退化的基变量，判定对应的<span class="math inline">\(a_{ij} &lt; 0\)</span>是否成立，如果成立进入下一步。否则说明把 <spanclass="math inline">\(x_j\)</span>换入不会得到新的基可行解。需要寻找另一个换入的基变量，在矩阵 <spanclass="math inline">\(\mathbf{S}\)</span> 的最后一行，找到另一个 <spanclass="math inline">\(c_j &gt; 0\)</span>。如果找到了，重复执行第 4步；如果没找到，说明目标函数是无界的，算法终止。</p></li><li><p>换出基选择：计算 <span class="math inline">\(i=  \mathop{argmin}\limits_{k}\displaystyle\frac{b_k}{a_{kj}}\)</span>，且 <spanclass="math inline">\(\displaystyle\frac{b_k}{a_{kj}} &gt;0\)</span>。如果能找到符合条件的 <spanclass="math inline">\(i\)</span>，说明可以把 <spanclass="math inline">\(x_i\)</span>从基变量中换出，进入下一步。否则说明目标函数是无界的，算法终止。</p></li><li><p>矩阵最简化：第 <span class="math inline">\(i\)</span> 行除以<span class="math inline">\(a_{ij}\)</span>，即把 <spanclass="math inline">\(a_{ij}\)</span> 变为 <spanclass="math inline">\(1\)</span>。然后通过高斯消元法把第 <spanclass="math inline">\(j\)</span>列中，其他行（包括最后一行）的元素都消为零。</p></li><li><p>跳转执行第 <span class="math inline">\(3\)</span> 步</p></li></ol><h2 id="初始基可行解">初始基可行解</h2><p>在前面的讨论中，我们均是从一个初始基可行解出发，经过一系列的基变换，最终达到目标值。但是初始的基可行解应该如何确定呢？</p><h3 id="特殊线性规划">特殊线性规划</h3><p>对于某些特殊的问题，初始基可行解可以直接确定。先看一类特殊的线性规划问题。</p><p>目标：<span class="math inline">\(\max：\mathbf{c^T \cdotx}\)</span></p><p>约束：<span class="math inline">\(\mathbf{Ax \leq b}\)</span>，<spanclass="math inline">\(\mathbf{x} \geq \mathbf{0}\)</span></p><p>前提条件：<span class="math inline">\(\mathbf{b} \geq\mathbf{0}\)</span></p><p>这类问题虽然看起来好像要比标准的线性规划问题要难，实际上经过转换之后，却是最简单的场景。对于不等式： <span class="math display">\[a_1 \cdot x_1 + a_2 \cdot x_2 + ... + a_n \cdot x_n \leq b\]</span> 可以添加一个松弛变量，使得上述不等式变为等式：</p><p><span class="math display">\[a_1 \cdot x_1 + a_2 \cdot x_2 + ... + a_n \cdot x_n + x_{n+1}= b\]</span> 其中：<span class="math inline">\(x_{n+1} \geq0\)</span>，<span class="math inline">\(x_{n+1}\)</span>叫做松弛变量。</p><p>由于不等式组中有 <span class="math inline">\(m\)</span>个不等式，可以添加 <span class="math inline">\(m\)</span>个松弛变量，变成 <span class="math inline">\(m\)</span> 个方程，而添加的<span class="math inline">\(m\)</span>个松弛变量也可以作为基变量。这样就直接找到了一个基可行解。</p><p>举个例子来说明：</p><p><strong>例八</strong>：线性规划问题描述如下，添加松弛变量，将其转化为线性规划的标准形式。</p><p>目标：<span class="math inline">\(\max：2x_1 + 3x_2 -x_3\)</span></p><p>约束： <span class="math display">\[\begin{aligned}x_1 + x_3 &amp;\leq 6 \\x_2 + x_3 &amp;\leq 4 \\2x_1 + x_2 &amp;\leq 6 \\\end{aligned}\]</span> 其中：<span class="math inline">\(x_1 \geq 0,x_2 \geq 0,x_3\geq 0\)</span></p><p><strong>解</strong>：首先通过添加松弛变量，使不等式变为等式： <spanclass="math display">\[\begin{aligned}x_1 + x_3 + &amp;x_4 &amp; &amp; &amp;= 6 \\x_2 + x_3 + &amp; &amp;x_5 &amp; &amp;= 4 \\2x_1 + x_2 + &amp; &amp; &amp;x_6 &amp;= 6 \\\end{aligned}\]</span> 其中：<span class="math inline">\(x_i \geq 0, 1 \leq i \leq6\)</span>。写成矩阵形式： <span class="math display">\[\begin{pmatrix}1 &amp; 0 &amp; 1 &amp; 1 &amp; 0 &amp; 0 \\0 &amp; 1 &amp; 1 &amp; 0 &amp; 1 &amp; 0 \\2 &amp; 1 &amp; 0 &amp; 0 &amp; 0 &amp; 1 \\\end{pmatrix}\cdot\begin{pmatrix}x_1 \\x_2 \\x_3 \\x_4 \\x_5 \\x_6 \\\end{pmatrix}=\begin{pmatrix}6 \\4 \\6 \\\end{pmatrix}\]</span> 可以看到，方程组的系数矩阵中的最右侧的 <spanclass="math inline">\(3\)</span> 列已经形成了一个单位矩阵，所以可以把<span class="math inline">\(x_4,x_5,x_6\)</span>作为基变量，得到一个基可行解：<span class="math inline">\(\mathbf{x} =(0,0,0,6,4,6)^T\)</span>。然后应用单纯形算法，就可以求出最优解。</p><h3 id="一般线性规划">一般线性规划</h3><p>对于一般线性规划问题，也可以通过类似的处理，一般有 <spanclass="math inline">\(2\)</span> 种方法，可以求得初始基可行解。一种是大<span class="math inline">\(M\)</span>法，另一种是两阶段法。以下面的例子作为说明。</p><p><strong>例九</strong>：已知一般线性规划问题描述如下，求初始基可行解</p><p>目标：<span class="math inline">\(\max：2x_1 + 3x_2 -x_3\)</span></p><p>约束： <span class="math display">\[\begin{aligned}x_1 + x_2 + x_3 &amp;= 6 \\x_1 - x_2 + 2x_3 &amp;= 4 \\\end{aligned}\]</span> 其中：<span class="math inline">\(x_1 \geq 0,x_2 \geq 0,x_3\geq 0\)</span></p><p><strong>解法一</strong>： 使用大 <spanclass="math inline">\(M\)</span> 法</p><p>所谓大 <span class="math inline">\(M\)</span>，是一个非常大的正数，让它作为人工变量的系数，如果要让目标取值为最大，人工变量只能取值为<span class="math inline">\(0\)</span>。将原问题转换为以下问题：</p><p>目标：<span class="math inline">\(\max：2x_1 + 3x_2 - x_3 - M \cdot(x_4 + x_5)\)</span></p><p>约束： <span class="math display">\[\begin{aligned}&amp;x_1 + x_2 + x_3 &amp;+x_4 &amp;&amp;= 6 \\&amp;x_1 - x_2 + 2x_3 &amp; &amp;+x_5 &amp;= 4 \\\end{aligned}\]</span> 其中：<span class="math inline">\(x_1 \geq 0,x_2 \geq 0,x_3\geq 0,x_4 \geq 0,x_5 \geq 0\)</span>。</p><p><span class="math inline">\(x_4\)</span> 和 <spanclass="math inline">\(x_5\)</span> 在原问题中不存在，是人工变量。由于<span class="math inline">\(M\)</span> 很大，要让目标值取得最大，<spanclass="math inline">\(x_4,x_5\)</span> 只能取值为 <spanclass="math inline">\(0\)</span>，所以原问题和新问题的目标取值解是一样的。在转换后的问题中，可以直接将<span class="math inline">\(x_4,x_5\)</span>作为基变量，求得初始基可行解：<span class="math inline">\(\mathbf{x} =(0,0,0,6,4)^T\)</span>，然后可以直接利用单纯形法求解。</p><p><strong>解法二</strong>： 使用两阶段法。 在大 <spanclass="math inline">\(M\)</span>法中，将问题转换之后，新问题和原问题是同解的，但是多出来的大 <spanclass="math inline">\(M\)</span>，会让强迫症感觉不爽。而所谓两阶段法是首先改造原问题，对改造后的问题使用单纯形法求最优解对应的基可行解；然后把该基可行解带回到原问题，再一次使用单纯形法求解。</p><p>第一阶段：改造问题</p><p>目标：<span class="math inline">\(\max：-(x_4 + x_5)\)</span></p><p>约束： <span class="math display">\[\begin{aligned}&amp;x_1 + x_2 + x_3 &amp;+x_4 &amp;&amp;= 6 \\&amp;x_1 - x_2 + 2x_3 &amp; &amp;+x_5 &amp;= 4 \\\end{aligned}\]</span> 其中：<span class="math inline">\(x_1 \geq 0,x_2 \geq 0,x_3\geq 0,x_4 \geq 0,x_5 \geq 0\)</span>。</p><p>对于该问题，如果要使得目标函数最大，显然 <spanclass="math inline">\(x_4\)</span> 和 <spanclass="math inline">\(x_5\)</span> 只能取值为 <spanclass="math inline">\(0\)</span>。所以我们要使用单纯形法求解该情况下基可行解。一阶段的初始基可行解：<span class="math inline">\(\mathbf{x} =(0,0,0,6,4)^T\)</span>，通过单纯形法求解（步骤略）该问题可得在最优解的情况下，对应的基可行解为：<spanclass="math inline">\(\mathbf{x} = (5,1,0,0,0)^T\)</span>，即 <spanclass="math inline">\(x_1=5,x_2=1\)</span>。所以原问题的基可行解为：<spanclass="math inline">\(\mathbf{x}=(5,1,0)^T\)</span>。</p><p>第二阶段：将 <spanclass="math inline">\(\mathbf{x}=(5,1,0)^T\)</span>作为初始基可行解，使用单纯形法求解原问题（步骤略）。</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;线性规划是最优化问题中的一个重要领域。许多实际问题都可以归结为线性规划问题，例如：网络流、多商品流量等问题。我们在高中已经学习过只有
&lt;span class=&quot;math inline&quot;&gt;&#92;(2&#92;)&lt;/span&gt;
个变量的场景，可以使用图解法来求最优解。该方法虽然直观，易于理</summary>
      
    
    
    
    <category term="数学" scheme="http://www.wangyulong.cn/categories/%E6%95%B0%E5%AD%A6/"/>
    
    
    <category term="数学" scheme="http://www.wangyulong.cn/tags/%E6%95%B0%E5%AD%A6/"/>
    
  </entry>
  
  <entry>
    <title>多项式插值</title>
    <link href="http://www.wangyulong.cn/1527838601/"/>
    <id>http://www.wangyulong.cn/1527838601/</id>
    <published>2022-07-16T07:48:06.000Z</published>
    <updated>2026-02-19T07:45:16.917Z</updated>
    
    <content type="html"><![CDATA[<p>插值是一种通过已知的数据点，求新数据点的过程或方法。比如我们已经通过采样、或者实验等方式，获取到了<span class="math inline">\(n+1\)</span> 个不同的采样点 <spanclass="math inline">\((x_i, y_i)\)</span>，其中 <spanclass="math inline">\(x_i\)</span> 为输入，<spanclass="math inline">\(y_i\)</span> 为输出。现在遇到了一个新的输入 <spanclass="math inline">\(x_t\)</span>，如何预测对应的输出 <spanclass="math inline">\(y_t\)</span>呢？插值就是一种能解决此问题的方法。而多项式由于形式简单，便于计算，是最常用的插值函数。使用多项式作为插值函数，就是多项式插值。</p><p>本文将介绍多项式插值的基本原理，以及常用的插值方法，其中包括了拉格朗日插值法和牛顿插值法。对于拉格朗日插值和牛顿插值，本文不仅介绍了原理，还给出了Python 代码的实现。虽然代码基于 numpy，但是并没有使用 numpy的高级特性。如果读者想替换为标准的 Python list，也是比较容易的。</p><h2 id="多项式求值">1. 多项式求值</h2><p>使用多项式插值，必然涉及到多项式求值问题：已知一个 <spanclass="math inline">\(n\)</span> 次多项式的表达式 <spanclass="math inline">\(f(x) = a_0 + a_1 \cdot x + a_2 \cdot x^2 + ... +a_n \cdot x^n\)</span>，再给一个特定的点 <spanclass="math inline">\(x\)</span>，如何计算对应的 <spanclass="math inline">\(f(x)\)</span>呢？当然可以直接按多项式的定义去求值，但是代码比较繁琐。这里使用霍纳法则，转换一下多项式的形式：</p><p><span class="math inline">\(f(x) = (((a_n \cdot x + a_{n-1})x + ... +a_2)x + a_1)x + a_0\)</span>在使用霍纳法则迭代计算多项式时，代码会比较简洁，时间复杂度为 <spanclass="math inline">\(O(n)\)</span>。</p><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment"># 霍纳法则计算多项式，a 是多项式的系数，x 是给定的点</span></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">Polynomial</span>(<span class="params">a, x</span>):</span><br><span class="line">    r = <span class="number">0</span></span><br><span class="line">    <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(<span class="built_in">len</span>(a)-<span class="number">1</span>, -<span class="number">1</span>, -<span class="number">1</span>):</span><br><span class="line">        r = x * r + a[i]</span><br><span class="line">    <span class="keyword">return</span> r</span><br></pre></td></tr></table></figure><h2 id="多项式插值基本思路">2. 多项式插值基本思路</h2><p>回到本文刚开始的问题：已知 <span class="math inline">\(n+1\)</span>个不同的采样点 <span class="math inline">\((x_i, y_i)\)</span>，其中<span class="math inline">\(x_i\)</span> 为输入，<spanclass="math inline">\(y_i\)</span> 为输出。现在遇到了一个新的输入 <spanclass="math inline">\(x_t\)</span>，如何预测对应的输出 <spanclass="math inline">\(y_t\)</span> 呢？</p><p>首先寻找一个多项式函数 <spanclass="math inline">\(f(x)\)</span>，使得 <spanclass="math inline">\(f(x)\)</span> 恰好经过这 <spanclass="math inline">\(n+1\)</span> 个点。然后假定对于任意的输入 <spanclass="math inline">\(x\)</span>，输出均为 <spanclass="math inline">\(f(x)\)</span>。最后对于给定的输入 <spanclass="math inline">\(x_t\)</span>，按照假定条件，输出 <spanclass="math inline">\(y_t = f(x_t)\)</span>。</p><h2 id="多项式插值原理">3. 多项式插值原理</h2><p><strong>定理1：</strong> 对于 <spanclass="math inline">\(n+1\)</span> 个不同的坐标<spanclass="math inline">\((x_0, y_0)\)</span>，<spanclass="math inline">\((x_1, y_1)\)</span>...，<spanclass="math inline">\((x_n, y_n)\)</span>。存在一个唯一的不超过 <spanclass="math inline">\(n\)</span> 次的多项式，恰好经过这 <spanclass="math inline">\(n+1\)</span> 个点。</p><p><strong>证明：</strong> 使用待定系数法，设多项式为： <spanclass="math inline">\(f(x) = a_0 + a_1 \cdot x + a_2 \cdot x^2 + ... +a_n \cdot x^n\)</span>，由于 <span class="math inline">\(f(x)\)</span>经过这 <span class="math inline">\(n+1\)</span> 个点。将 <spanclass="math inline">\((x_i, y_i)\)</span> 坐标带入 <spanclass="math inline">\(f(x)\)</span> 的表达式，可构建如下方程组 <spanclass="math display">\[f(x_0) = a_0 + a_1 \cdot x_0 + a_2 \cdot x_0^2 + ... + a_n \cdot x_0^n =y_0 \\f(x_1) = a_0 + a_1 \cdot x_1 + a_2 \cdot x_1^2 + ... + a_n \cdot x_1^n =y_1 \\... \\f(x_n) = a_0 + a_1 \cdot x_n + a_2 \cdot x_n^2 + ... + a_n \cdot x_n^n =y_n\]</span> 写成矩阵形式 <span class="math inline">\(\mathbf{X a} =\mathbf{y}\)</span>，其中 <span class="math inline">\(\mathbf{a} = (a_0,a_1, a_2, ..., a_n)^\mathrm{T}\)</span>，<spanclass="math inline">\(\mathbf{y} = (y_0, y_1, y_2, ...,y_n)^\mathrm{T}\)</span>，<spanclass="math inline">\(\mathbf{X}\)</span> 是一个 <spanclass="math inline">\((n+1) \times (n+1)\)</span> 的矩阵。</p><p><span class="math display">\[\begin{pmatrix}1  &amp;  x_0      &amp; x_0^2  &amp; \cdot  &amp; x_0^n \\1  &amp;  x_1      &amp; x_1^2  &amp; \cdot  &amp; x_1^n \\1  &amp;  x_2      &amp; x_2^2  &amp; \cdot  &amp; x_2^n \\\cdot  &amp;\cdot  &amp; \cdot  &amp; \cdot  &amp; \cdot \\1  &amp;  x_n      &amp; x_n^2  &amp; \cdot  &amp; x_n^n\end{pmatrix}\cdot\begin{pmatrix}a_0 \\a_1 \\a_2 \\\cdot \\a_n\end{pmatrix}=\begin{pmatrix}y_0 \\y_1 \\y_2 \\\cdot \\y_n\end{pmatrix}\]</span></p><p>可以看出，<span class="math inline">\(\mathbf{X}\)</span>是一个范德蒙矩阵，行列式不为 <spanclass="math inline">\(0\)</span>，所以该方程组有唯一解：<spanclass="math inline">\(\mathbf{a} = \mathbf{X}^{-1}\mathbf{y}\)</span>。所以存在一个唯一的多项式，经过这 <spanclass="math inline">\(n+1\)</span> 个点。</p><p>虽然可以使用高斯消元法对范德蒙矩阵矩阵求逆，进而求出多项式插值函数，但是该方法计算的复杂度比较高，时间复杂度为<spanclass="math inline">\(O(n^3)\)</span>，而且精度不高，实际很少使用。更常用的方法是拉格朗日插值和牛顿插值。由定理1可知，不管使用什么方法，最终的插值多项式都是一样的，只是计算的复杂程度不同而已。</p><h2 id="拉格朗日插值">4. 拉格朗日插值</h2><p>拉格朗日使用构造法来构造多项式插值函数。基本思想是寻找 <spanclass="math inline">\(n+1\)</span> 个多项式基函数，其中第 <spanclass="math inline">\(i\)</span> 个基函数在 <spanclass="math inline">\(x_i\)</span> 处取值为 <spanclass="math inline">\(1\)</span>，在其他 <spanclass="math inline">\(x_j\)</span> 处 (<span class="math inline">\(j\neq i\)</span>) 取值为 <span class="math inline">\(0\)</span>。即第<span class="math inline">\(i (0 \leq i \leq n)\)</span> 个多项式基函数<span class="math inline">\(\omega_i(x)\)</span> 满足如下特性：</p><p><span class="math display">\[\omega_i(x_k) = \left\{  \begin{aligned}  &amp;1 , i = k \\  &amp;0 , i \neq k  \end{aligned}\right.\]</span></p><p>构造多项式函数 <span class="math inline">\(f(x) =\sum\limits_{i=1}^{n}y_i \cdot {\omega_i(x)}\)</span>，那么显然函数<span class="math inline">\(f(x)\)</span> 满足插值点 <spanclass="math inline">\(f(x_i) = y_i\)</span>。由于每个基函数 <spanclass="math inline">\(\omega_i(x)\)</span> 是一个关于 <spanclass="math inline">\(x\)</span> 的 <spanclass="math inline">\(n\)</span> 次多项式，所以 <spanclass="math inline">\(f(x)\)</span> 也一个关于 <spanclass="math inline">\(x\)</span> 的 <spanclass="math inline">\(n\)</span>次多项式。关键的问题是如何寻找基函数。</p><p>对于 <span class="math inline">\(i (0 \leq i \leqn)\)</span>，拉格朗日选取的第 <span class="math inline">\(i\)</span>个基函数： <span class="math inline">\(\displaystyle \omega_i(x) =\prod\limits_{j=0, j \neq i}^{j=n}\frac{(x-x_j)}{(x_i-x_j)}\)</span>。</p><p>可以看到，基函数 <span class="math inline">\(\omega_i(x)\)</span>是一个关于 <span class="math inline">\(x\)</span> 的 <spanclass="math inline">\(n\)</span>次多项式。下面验证一下是否满足基函数特性。将 <spanclass="math inline">\(x_i\)</span> 带入第 <spanclass="math inline">\(i\)</span> 个基函数可得：</p><p><span class="math inline">\(\displaystyle \omega_i(x_i) =\prod\limits_{j=0,j \neq i}^{j=n}\frac{(x_i-x_j)}{(x_i-x_j)}\)</span>，约分后，可得 <spanclass="math inline">\(\omega_i(x_i) = 1\)</span></p><p>对于<span class="math inline">\(k \neq i\)</span> 且 <spanclass="math inline">\(0 \leq k \leq n\)</span>，将 <spanclass="math inline">\(x_k\)</span> 带入第 <spanclass="math inline">\(i\)</span> 个基函数可得： <spanclass="math inline">\(\displaystyle \omega_i(x_k) = \prod\limits_{j=0,j\neq i}^{j=n} \frac{(x_k-x_j)}{(x_i-x_j)}\)</span>。</p><p>在该式中，分子中必然存在一个 <span class="math inline">\(j\)</span>使得 <span class="math inline">\(k = j\)</span>，即<spanclass="math inline">\(x_k - x_j = 0\)</span>，那么分子的乘积也为 <spanclass="math inline">\(0\)</span>，即 <spanclass="math inline">\(\omega_i(x_k) =0\)</span>，所以满足基函数特性。</p><p>拉格朗日多项式插值函数表达式为：<span class="math inline">\(f(x) =\sum\limits_{i=1}^{n}y_i \cdot {\omega_i(x)}\)</span></p><h3 id="举例">举例</h3><p>以上的介绍有些抽象，看一个例子就会清晰一些。假设有 <spanclass="math inline">\(3\)</span> 个点 <spanclass="math inline">\((1,2)\)</span>，<spanclass="math inline">\((3,12)\)</span>，<spanclass="math inline">\((4,23)\)</span>，求一个二次函数 <spanclass="math inline">\(y = a_0 + a_1 \cdot x + a_2 \cdot x^2\)</span>经过这 <span class="math inline">\(3\)</span> 个点。首先找到 3个拉格朗日基：</p><p><span class="math inline">\(\displaystyle \omega_0(x) =\frac{x-x_1}{x_0-x_1} \cdot \frac{x-x_2}{x_0-x_2} = \frac{x-3}{1-3}\cdot \frac{x-4}{1-4} = \frac{(x-3)(x-4)}{6}\)</span></p><p><span class="math inline">\(\displaystyle \omega_1(x) =\frac{x-x_0}{x_1-x_0} \cdot \frac{x-x_2}{x_1-x_2} = \frac{x-1}{3-1}\cdot \frac{x-4}{3-4} = -\frac{(x-1)(x-4)}{2}\)</span></p><p><span class="math inline">\(\displaystyle \omega_2(x) =\frac{x-x_0}{x_2-x_0} \cdot \frac{x-x_1}{x_2-x_1} = \frac{x-1}{4-1}\cdot \frac{x-3}{4-3} = \frac{(x-1)(x-3)}{3}\)</span></p><p>所以最终的表达式：</p><p><span class="math display">\[\begin{aligned}f(x) &amp;= y_0 \cdot \omega_0(x) + y_1 \cdot \omega_1(x) + y_2 \cdot\omega_2(x) \\     &amp;= 2 \cdot \frac{(x-3)(x-4)}{6} + 12 \cdot-\frac{(x-1)(x-4)}{2} + 23 \cdot \frac{(x-1)(x-3)}{3}\end{aligned}\]</span></p><p>容易验证，对于以上表达式：<span class="math inline">\(f(1) =2\)</span>，<span class="math inline">\(f(3) = 12\)</span>，<spanclass="math inline">\(f(4) = 23\)</span></p><h3 id="python-代码">Python 代码</h3><p>直接使用拉格朗日基函数的定义，来求插值函数 <figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">Lagrange</span>(<span class="params">X, Y</span>):</span><br><span class="line">    <span class="keyword">def</span> <span class="title function_">f</span>(<span class="params">x</span>):</span><br><span class="line">        r = <span class="number">0</span></span><br><span class="line">        <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(<span class="built_in">len</span>(Y)):</span><br><span class="line">            w = Y[i]</span><br><span class="line">            <span class="keyword">for</span> j <span class="keyword">in</span> <span class="built_in">range</span>(<span class="built_in">len</span>(X)):</span><br><span class="line">                <span class="keyword">if</span> i == j:</span><br><span class="line">                    <span class="keyword">continue</span></span><br><span class="line">                w *= (x - X[j]) / (X[i] - X[j])</span><br><span class="line">            r += w</span><br><span class="line">        <span class="keyword">return</span> r</span><br><span class="line">    <span class="keyword">return</span> f <span class="comment"># 返回插值函数</span></span><br></pre></td></tr></table></figure></p><p>测试：给定一个 <span class="math inline">\(5\)</span> 次多项式，以及<span class="math inline">\(6\)</span>个插值点，可以得到拉格朗日插值函数。再给定一个新的点，测试通过原始的多项式和插值函数计算的结果是否一样。<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">test_lagrange</span>():</span><br><span class="line">    <span class="comment"># f(x) 是一个 5 次多项式，表达式如下：</span></span><br><span class="line">    <span class="comment"># f(x) = 1 + 5 * x + 2 * x^2 + 4 * x^3 + 6 * x^4 + 3 * x^5</span></span><br><span class="line">    a = np.array([<span class="number">1</span>, <span class="number">5</span>, <span class="number">2</span>, <span class="number">4</span>, <span class="number">6</span>, <span class="number">3</span>])</span><br><span class="line">    y = Polynomial(a, <span class="number">3.5</span>)  <span class="comment"># 直接使用多项式计算 f(3.5)</span></span><br><span class="line">    <span class="built_in">print</span>(y)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># 给定 6 个插值点</span></span><br><span class="line">    X = np.array([<span class="number">1</span>, <span class="number">2</span>, <span class="number">5</span>, <span class="number">7</span>, <span class="number">9</span>, <span class="number">10</span>])</span><br><span class="line">    Y = np.zeros_like(X)</span><br><span class="line">    <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(<span class="built_in">len</span>(X)):</span><br><span class="line">        Y[i] = Polynomial(a, X[i])</span><br><span class="line"></span><br><span class="line">    f = Lagrange(X, Y) <span class="comment"># 返回拉格朗日插值函数</span></span><br><span class="line">    y = f(<span class="number">3.5</span>) <span class="comment"># 使用插值函数计算</span></span><br><span class="line">    <span class="built_in">print</span>(y)</span><br><span class="line"></span><br><span class="line">test_lagrange()</span><br></pre></td></tr></table></figure>拉格朗日插值虽然形式简单，便于理解，但是不具备增量特特性，即在已经计算出<span class="math inline">\(n\)</span>个点的插值函数后，如果再增加一个点， 所有基函数都要重建。</p><p>如果不对插值多项式做化简，每次计算一个新的点时，时间复杂度为 <spanclass="math inline">\(O(n^2)\)</span>。</p><h2 id="牛顿插值">5. 牛顿插值</h2><p>牛顿插值法可以解决拉格朗日不具备增量计算的问题，解决方法是换了一组基函数。对于<span class="math inline">\(n+1\)</span> 个不同的点 <spanclass="math inline">\((x_0, y_0)\)</span>，<spanclass="math inline">\((x_1, y_1)\)</span>...，<spanclass="math inline">\((x_n, y_n)\)</span> ，牛顿法也是构造 <spanclass="math inline">\(n+1\)</span> 个基函数。对于上述每个点 <spanclass="math inline">\((x_i, y_i)\)</span> ，基函数需要满足如下特性：</p><p><span class="math display">\[  \begin{aligned}  \sum\limits_{j=0}^{i}{a_j\phi_j(x_i)} &amp;= y_i \\  \sum\limits_{j=i+1}^{n}{a_j\phi_j(x_i)} &amp;= 0  \end{aligned}\]</span></p><p>如果找到了这样一组基函数，那么最终插值多项式的表达式：<spanclass="math inline">\(\displaystyle f(x) =\sum\limits_{i=0}^{n}a_i\phi_i(x)\)</span>，根据基函数特性可得：<spanclass="math inline">\(f(x_i) = y_i\)</span>。</p><p>如何寻找基函数呢，仍然是构造法。牛顿构造的第 <spanclass="math inline">\(i\)</span> 个基函数表达式如下：</p><p><span class="math display">\[\phi_i(x) = \left\{  \begin{aligned}  &amp;1 , i = 0 \\  &amp;\prod\limits_{j=0}^{i-1}(x-x_j) , 1 \leq i \leq n  \end{aligned}\right.\]</span></p><p>牛顿插值多项式中的 <span class="math inline">\(a_i\)</span>是待定系数，需要满足牛顿基函数特性，并不能通过观测直接得到。为了求解<span class="math inline">\(a_i\)</span>，首先将牛顿插值多项式展开：<span class="math display">\[\begin{aligned}f(x) &amp;= \phi_0(x) + \phi_1(x) + \phi_2(x) + ... + \phi_n(x) \\     &amp;= a_0 + a_1(x-x_0) + a_2(x-x_0)(x-x_1) + ... +a_n(x-x_0)(x-x_1)...(x-x_{n-1})\end{aligned}\]</span></p><p>将 <span class="math inline">\(n+1\)</span> 个不同的点的坐标 <spanclass="math inline">\((x_0, y_0)\)</span>，<spanclass="math inline">\((x_1, y_1)\)</span>...，<spanclass="math inline">\((x_n, y_n)\)</span> 带入牛顿插值表达式 <spanclass="math inline">\(f(x)\)</span> 可得到以下方程组：</p><p><span class="math display">\[\begin{array}{llll}  f(x_0) &amp;= a_0 &amp;= y_0 \\  f(x_1) &amp;= a_0 + a_1(x_1-x_0) &amp;= y_1 \\  f(x_2) &amp;= a_0 + a_1(x_2-x_0) + a_2(x_2-x_0)(x_2-x_1) &amp;= y_2 \\  ... \\  f(x_n) &amp;= a_0 + a_1(x_n-x_0) + a_2(x_n-x_0)(x_n-x_1) + ... +a_n(x_n-x_0)...(x_n-x_{n-1}) &amp;= y_n\end{array}\]</span></p><p>上述方程组有 <span class="math inline">\(n+1\)</span> 个方程，<spanclass="math inline">\(n+1\)</span> 个未知量，由于这 <spanclass="math inline">\(n+1\)</span>个点的坐标不同，而且各个方程之间线性无关，该方程组有唯一解。虽然可以使用高斯消元法来求解，但是使用带入法更加快捷。可以先通过第一个方程求解出<span class="math inline">\(a_0\)</span>，然后带入到第二个方程求解 <spanclass="math inline">\(a_1\)</span>，等等以此类推直到求解出 <spanclass="math inline">\(a_n\)</span>。</p><h3 id="举例-1">举例</h3><p>和拉格朗日插值一样，假设有 <span class="math inline">\(3\)</span>个点 <span class="math inline">\((1,2)\)</span>，<spanclass="math inline">\((3,12)\)</span>，<spanclass="math inline">\((4,23)\)</span>，求一个经过这 <spanclass="math inline">\(3\)</span> 个点的二次函数。</p><p>牛顿插值的二次函数的形式为：<span class="math inline">\(f(x) = a_0 +a_1(x-1) + a_2(x-1)(x-3)\)</span>。带入坐标之后为：</p><p><span class="math inline">\(f(1) = a_0 = 2\)</span></p><p><span class="math inline">\(f(3) = a_0 + a_1(3-1) = 2 + a_1(3-1) = 12=&gt; a_1 = 5\)</span></p><p><span class="math inline">\(f(4) = a_0 + a_1(4-1) + a_2(4-1)(4-3) = 2+ 5 \times 3 + a_2 \times 3  = 23 =&gt; a_2 = 2\)</span></p><p>这样就把全部系数都求出来了，所以插值函数：<spanclass="math inline">\(f(x) = 2 + 5(x-1) + 2(x-1)(x-3)\)</span></p><h3 id="编程实现">编程实现</h3><p>考察一下关于 <span class="math inline">\(a_i\)</span> 的方程： <spanclass="math display">\[\begin{aligned}f(x_i) &amp;= a_0 \\       &amp;+ a_1(x_i-x_0) \\       &amp;+ a_2(x_i-x_0)(x_i-x_1) \\       &amp;+ ... \\       &amp;+ a_{i-1}(x_i-x_0)(x_i-x_1)...(x_i-x_{i-2}) \\       &amp;+ a_i(x_i-x_0)(x_i-x_1)...(x_i-x_{i-1})\end{aligned}\]</span></p><p>变形可得： <span class="math display">\[\begin{aligned}a_i &amp;= \frac{f(x_i)-a_0}{(x_i-x_0)(x_i-x_1)...(x_i-x_{i-1})} \\       &amp;- \frac{a_1}{(x_i-x_1)...(x_i-x_{i-1})} \\       &amp;- ... \\       &amp;- \frac{a_2}{(x_i-x_2)...(x_i-x_{i-1})} \\       &amp;- \frac{a_{i-1}}{x_i-x_{i-1}}\end{aligned}\]</span></p><p>仿照霍纳法则，再对上述公式变形：</p><p><span class="math inline">\(a_i =\displaystyle(((\frac{f(x_i)-a_0}{x_i-x_0} - a_1) \frac{1}{x_i-x_1} -a_2) \frac{1}{x_i-x_2} - ... - a_{i-1})\frac{1}{x_i-x_{i-1}}\)</span></p><p>这样就把系数计算出来了。对于一个未知的点 <spanclass="math inline">\(x\)</span>，如何计算 <spanclass="math inline">\(f(x)\)</span> 呢？仍然是仿霍纳法则</p><p><span class="math display">\[\begin{aligned}f(x) &amp;= a_0 + a_1(x-x_0) + a_2(x-x_0)(x-x_1) + ... +a_n(x-x_0)(x-x_1)...(x-x_{n-1}) \\ &amp;      = ((a_n(x-x_{n-1}) + a_{n-1})(x-x_{n-2}) + ... + a_1)(x-x_0) + a_0\end{aligned}\]</span></p><h3 id="python-代码-1">Python 代码</h3><figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">import</span> numpy <span class="keyword">as</span> np</span><br><span class="line"></span><br><span class="line"><span class="keyword">def</span> <span class="title function_">Newton</span>(<span class="params">X, Y</span>):</span><br><span class="line">    a = np.zeros_like(X)</span><br><span class="line">    a = Y.copy() <span class="comment"># 初始化 a[i] = Y[i]</span></span><br><span class="line">    <span class="comment"># 计算系数 a[i]</span></span><br><span class="line">    <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(<span class="number">1</span>, <span class="built_in">len</span>(a)):</span><br><span class="line">        <span class="keyword">for</span> j <span class="keyword">in</span> <span class="built_in">range</span>(<span class="number">0</span>, i):</span><br><span class="line">            a[i] = (a[i] - a[j]) / (X[i] - X[j])</span><br><span class="line"></span><br><span class="line">    <span class="comment"># 已知系数 a[i]， 返回插值多项式 f(x)</span></span><br><span class="line">    <span class="keyword">def</span> <span class="title function_">f</span>(<span class="params">x</span>):</span><br><span class="line">        r = a[<span class="built_in">len</span>(a) - <span class="number">1</span>]</span><br><span class="line">        <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(<span class="built_in">len</span>(a) - <span class="number">2</span>, -<span class="number">1</span>, -<span class="number">1</span>):</span><br><span class="line">            r = r * (x - X[i]) + a[i]</span><br><span class="line">        <span class="keyword">return</span> r</span><br><span class="line">    <span class="keyword">return</span> f</span><br></pre></td></tr></table></figure><p>测试：给定一个 <span class="math inline">\(5\)</span> 次多项式，以及<span class="math inline">\(6\)</span>个插值点，可以得到牛顿插值函数。再给定一个新的点，测试通过原始的多项式和插值函数计算的结果是否一样。<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">test_newton</span>():</span><br><span class="line">    <span class="comment"># f(x) 是一个 5 次多项式，表达式如下：</span></span><br><span class="line">    <span class="comment"># f(x) = 1 + 5 * x + 2 * x^2 + 4 * x^3 + 6 * x^4 + 3 * x^5</span></span><br><span class="line">    a = np.array([<span class="number">1</span>, <span class="number">5</span>, <span class="number">2</span>, <span class="number">4</span>, <span class="number">6</span>, <span class="number">3</span>])</span><br><span class="line">    y = Polynomial(a, <span class="number">3.5</span>)  <span class="comment"># 直接使用多项式计算 f(3.5)</span></span><br><span class="line">    <span class="built_in">print</span>(y)</span><br><span class="line"></span><br><span class="line">    <span class="comment"># 给定 6 个插值点</span></span><br><span class="line">    X = np.array([<span class="number">1</span>, <span class="number">2</span>, <span class="number">5</span>, <span class="number">7</span>, <span class="number">9</span>, <span class="number">10</span>])</span><br><span class="line">    Y = np.zeros_like(X)</span><br><span class="line">    <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(<span class="built_in">len</span>(X)):</span><br><span class="line">        Y[i] = Polynomial(a, X[i])</span><br><span class="line"></span><br><span class="line">    f = Newton(X, Y) <span class="comment"># 返回牛顿插值函数</span></span><br><span class="line">    y = f(<span class="number">3.5</span>) <span class="comment"># 使用插值函数计算</span></span><br><span class="line">    <span class="built_in">print</span>(y)</span><br><span class="line"></span><br><span class="line">test_newton()</span><br></pre></td></tr></table></figure></p><p>牛顿插值法有一个初始化阶段，时间复杂度为 <spanclass="math inline">\(O(n^2)\)</span>，初始化完成之后，在计算新的点对应的值时，时间复杂度为<spanclass="math inline">\(O(n)\)</span>。而且牛顿基函数具备增量计算特性：如果已知前<span class="math inline">\(n\)</span> 个基函数，再增加 1 个插值点时，前<span class="math inline">\(n\)</span>个基函数都不变，只需增加一个新的基函数即可。</p><h3 id="差商">差商</h3><p>虽然通过以上算法可以计算出牛顿插值基的系数，但是对于系数的物理意义并没有解释，实际上第<span class="math inline">\(i\)</span> 个插值基函数的系数，是第 <spanclass="math inline">\(i\)</span>阶差商。差商的定义是递归的，其定义如下：</p><p>零阶差商：<span class="math inline">\(F[x_i] = f(x_i)\)</span></p><p>一阶差商：<span class="math inline">\(\displaystyle F[x_i,x_j] =\frac{F[x_i] - F[x_j]}{x_i-x_j}\)</span></p><p>二阶差商：<span class="math inline">\(\displaystyle F[x_i,x_k,x_j] =\frac{F[x_i,x_k] - F[x_k,x_j]}{x_i-x_j}\)</span></p><p>...</p><p><span class="math inline">\(k\)</span> 阶差商：<spanclass="math inline">\(\displaystyle F[x_i,x_{i+1},...,x_{i+k}] =\frac{F[x_i,x_{i+1}, ..., x_{k-1}] - F[x_{i+1},x_{i+2}, ...,x_{i+k}]}{x_i-x_{i+k}}\)</span></p><p>为了得到插值函数 <span class="math inline">\(f(x)\)</span>和差商的关系，可以按阶从低到高考察差商函数。</p><p>考察一阶差商函数：<span class="math inline">\(\displaystyle F[x, x_0]= \frac{F[x] - F[x_0]}{x-x_0}\)</span>， 变形可得： <spanclass="math display">\[F[x] = F[x_0] + (F[x] - F[x_0])(x-x_0)\]</span></p><p>由于零阶差商： <span class="math inline">\(F[x] =f(x)\)</span>，替换之后可得： <span class="math display">\[f(x) = F[x_0] + (F[x] - F[x_0])(x-x_0)\]</span></p><p>再考察二阶差商函数：</p><p><span class="math display">\[\begin{aligned}F[x, x_0, x_1] &amp;= \frac{F[x,x_0] - F[x_0,x_1]}{x-x_1}               &amp;= \frac{\displaystyle\frac{F[x]-F[x_0]}{x-x_0} -F[x_0,x_1]}{x - x_1}\end{aligned}\]</span></p><p>等式左右两边同时乘以 <spanclass="math inline">\((x-x_0)(x-x_1)\)</span> 之后可得：</p><p><span class="math display">\[F[x, x_0, x_1](x-x_0)(x-x_1) = F[x]-F[x_0] - F[x_0,x_1](x-x_0)\]</span> 移项并用 <span class="math inline">\(f(x)\)</span> 替换 <spanclass="math inline">\(F[x]\)</span> 得：</p><p><span class="math display">\[f(x) = F[x_0] + F[x_0,x_1](x-x_0) + F[x, x_0, x_1](x-x_0)(x-x_1)\]</span></p><p>...</p><p>一步一步替换之后，最终可得： <span class="math display">\[\begin{aligned}f(x) &amp;= F[x_0] \\     &amp;+ F[x_0,x_1](x-x_0) \\     &amp;+ F[x_0, x_1, x_2](x-x_0)(x-x_1) \\     &amp;+ ... \\     &amp;+ F[x_0, x_1, ..., x_{n}](x-x_0)(x-x_1)...(x-x_{n-1}) \\     &amp;+ F[x, x_1, ..., x_{n-1}, x_{n}](x-x_1)...(x-x_{n-1})(x-x_n)\end{aligned}\]</span> 以上是推导过程，更严格的证明可以使用数学归纳法。</p><p>把 <span class="math inline">\(x_0\)</span> 带入以上表达式可得</p><p><span class="math display">\[f(x_0) = F[x_0] + F[x, x_1, ...,x_{n-1}, x_{n}](x_0-x_1)...(x_0-x_{n-1})(x_0-x_n)\]</span></p><p>由于零阶差商：<span class="math inline">\(F[x_0] =f(x_0)\)</span>，所以 <span class="math inline">\(F[x, x_1, ...,x_{n-1}, x_{n}] = 0\)</span>。最终 <spanclass="math inline">\(f(x)\)</span> 的表达式如下：</p><p><span class="math display">\[\begin{aligned}f(x) &amp;= F[x_0] \\     &amp;+ F[x_0,x_1](x-x_0) \\     &amp;+ F[x_0, x_1, x_2](x-x_0)(x-x_1) \\     &amp;+ ... \\     &amp;+ F[x_0, x_1, ..., x_{n}](x-x_0)(x-x_1)...(x-x_{n-1})\end{aligned}\]</span></p><p>通过以上推导可以发现，牛顿插值法的系数 <spanclass="math inline">\(a_k\)</span>，其实就是 <spanclass="math inline">\(k\)</span> 阶差商 <spanclass="math inline">\(F[x_0, x_1, ..., x_k]\)</span>。</p><h3 id="python-代码-2">Python 代码</h3><p>首先计算 <span class="math inline">\(0\)</span>阶差商，然后通过递推公式，依次计算 <spanclass="math inline">\(1\)</span> 阶，<spanclass="math inline">\(2\)</span> 阶，<spanclass="math inline">\(3\)</span> 阶， ...， <spanclass="math inline">\(n\)</span>阶差商。差商计算出来之后，仍然使用霍纳法则计算多项式的值。<figure class="highlight python"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">def</span> <span class="title function_">NewtonDivDiff</span>(<span class="params">X, Y</span>):</span><br><span class="line">    d = np.zeros([<span class="built_in">len</span>(Y), <span class="built_in">len</span>(Y)])</span><br><span class="line">    <span class="comment"># 计算 0 阶差商，这里表示为 d[i][i]</span></span><br><span class="line">    <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(<span class="number">0</span>, <span class="built_in">len</span>(Y)):</span><br><span class="line">        d[i][i] = Y[i]</span><br><span class="line"></span><br><span class="line">    <span class="comment"># k 为当前差商的阶数，先计算 1 阶，然后 2 阶，依次类推</span></span><br><span class="line">    <span class="keyword">for</span> k <span class="keyword">in</span> <span class="built_in">range</span>(<span class="number">1</span>, <span class="built_in">len</span>(Y)):</span><br><span class="line">        <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(<span class="number">0</span>, <span class="built_in">len</span>(Y) - k):</span><br><span class="line">            d[i][i+k] = (d[i][i+k-<span class="number">1</span>] - d[i+<span class="number">1</span>][i+k]) / (X[i] - X[i+k])</span><br><span class="line"></span><br><span class="line">    <span class="comment"># 已知系数 a[i]， 计算多项式 f(x)</span></span><br><span class="line">    <span class="keyword">def</span> <span class="title function_">f</span>(<span class="params">x</span>):</span><br><span class="line">        r = d[<span class="number">0</span>][<span class="built_in">len</span>(Y) - <span class="number">1</span>]</span><br><span class="line">        <span class="keyword">for</span> i <span class="keyword">in</span> <span class="built_in">range</span>(<span class="built_in">len</span>(Y) - <span class="number">2</span>, -<span class="number">1</span>, -<span class="number">1</span>):</span><br><span class="line">            r = r * (x - X[i]) + d[<span class="number">0</span>][i]</span><br><span class="line">        <span class="keyword">return</span> r</span><br><span class="line">    <span class="keyword">return</span> f</span><br></pre></td></tr></table></figure></p><p>可以看到，直接使用差商的定义和前面求解的方式代码是相似的，二者的时间复杂度是一样的，都是<spanclass="math inline">\(O(n)\)</span>。直接求解时，代码更加简洁，空间复杂度为<spanclass="math inline">\(O(n)\)</span>，而使用差商定义时，需要用一个二维的数组，空间复杂度为<spanclass="math inline">\(O(n^2)\)</span>，要高一些。但是使用差商的定义，物理意义更加明确一些。</p><h2 id="总结">总结</h2><p>本文介绍了一般的多项式插值、拉格朗日插值以及牛顿插值，不仅介绍了原理，还给出了Python 代码的实现，可以供读者研究使用。虽然通过定理 1可以知道，多项式插值最终的表达式是一样的，但是使用不同的插值方法其算法复杂程度是不一样的。一般多项式插值的算法复杂度最高，拉格朗日插值算法复杂度稍低，牛顿插值算法复杂度最低。$$</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;插值是一种通过已知的数据点，求新数据点的过程或方法。比如我们已经通过采样、或者实验等方式，获取到了
&lt;span class=&quot;math inline&quot;&gt;&#92;(n+1&#92;)&lt;/span&gt; 个不同的采样点 &lt;span
class=&quot;math inline&quot;&gt;&#92;((x_i, y_i)</summary>
      
    
    
    
    <category term="数学" scheme="http://www.wangyulong.cn/categories/%E6%95%B0%E5%AD%A6/"/>
    
    
    <category term="数学" scheme="http://www.wangyulong.cn/tags/%E6%95%B0%E5%AD%A6/"/>
    
  </entry>
  
  <entry>
    <title>Poj 2663 Tri Tiling</title>
    <link href="http://www.wangyulong.cn/1587210716/"/>
    <id>http://www.wangyulong.cn/1587210716/</id>
    <published>2020-04-18T19:51:56.000Z</published>
    <updated>2026-02-19T07:45:16.916Z</updated>
    
    <content type="html"><![CDATA[<h3 id="题目大意">题目大意：</h3><p>问用 <span class="math inline">\(1*2\)</span> 的多米诺骨牌覆盖 <spanclass="math inline">\(3*n\)</span>的矩形区域，总共有多少种不同的覆盖方式？ 下图是矩形大小为 <spanclass="math inline">\(3*12\)</span> 的一个有效覆盖。<img src="../images/2663_1.jpg" /></p><h3 id="输入">输入</h3><p>有若干组输入，每组输入为一个整数 <span class="math inline">\(n,(0&lt;= n &lt;= 30)\)</span> 。最后一个输入为 -1 ，表示输入结束。</p><h3 id="输出">输出</h3><p>对于每组输入，输出一个整数，表示总共可能的覆盖个数。</p><h3 id="样例输入">样例输入</h3><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">2</span><br><span class="line">8</span><br><span class="line">12</span><br><span class="line">-1</span><br></pre></td></tr></table></figure><h3 id="样例输出">样例输出</h3><figure class="highlight plaintext"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">3</span><br><span class="line">153</span><br><span class="line">2131</span><br></pre></td></tr></table></figure><h3 id="题目分析">题目分析</h3><p>如果矩形区域为 <spanclass="math inline">\(2*n\)</span>，很容易找到递推公式 <spanclass="math display">\[F[n] =\begin{cases}1,  &amp; \text{if n = 0 or n = 1} \\[2ex]F[n-1] + F[n-2], &amp; \text{if n &gt; 1}\end{cases}\]</span> 现在矩形区域为 <spanclass="math inline">\(3*n\)</span>，递归关系似乎不太容易推导，但是只要坚持下去，还是可以找到递推关系的。假设总的覆盖情况有 <spanclass="math inline">\(A[n]\)</span>种，我们先找第一层底递推关系。<img src="../images/20200418081819.png" /> 这里出现了一种新的情况 <spanclass="math inline">\(B[n]\)</span>，我们还无法表示，需要继续往下推导，直到找到递归式为止。</p><p><img src="../images/20200418082828.png" />这样我们就找到了2组递推关系： <span class="math display">\[\begin{cases}A[n] = 2*B[n-1] + A[n-2]  \\[2ex]B[n] = A[n-1] + B[n-2]\end{cases}\]</span> 然后，只要找到初始情况 <spanclass="math inline">\(A[n]\)</span> 和 <spanclass="math inline">\(B[n]\)</span> 的值，就可以递推求解了。 代码如下：<figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#<span class="keyword">include</span> <span class="string">&lt;stdio.h&gt;</span></span></span><br><span class="line"></span><br><span class="line"><span class="meta">#<span class="keyword">define</span> MAX_SIZE 32</span></span><br><span class="line"></span><br><span class="line"><span class="function"><span class="type">int</span> <span class="title">main</span><span class="params">()</span> </span>&#123;</span><br><span class="line">    <span class="type">int</span> n = <span class="number">0</span>;</span><br><span class="line">    <span class="type">int</span> a[MAX_SIZE] = &#123;<span class="number">1</span>, <span class="number">0</span>, <span class="number">3</span>&#125;;</span><br><span class="line">    <span class="type">int</span> b[MAX_SIZE] = &#123;<span class="number">0</span>, <span class="number">1</span>, <span class="number">0</span>&#125;;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">for</span> (<span class="type">int</span> i = <span class="number">3</span>; i &lt; MAX_SIZE; i++) &#123;</span><br><span class="line">        a[i] = <span class="number">2</span>*b[i<span class="number">-1</span>] + a[i<span class="number">-2</span>];</span><br><span class="line">        b[i] = a[i<span class="number">-1</span>] + b[i<span class="number">-2</span>];</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">while</span> (<span class="built_in">scanf</span>(<span class="string">&quot;%d&quot;</span>, &amp;n) &gt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="keyword">if</span> (n &lt; <span class="number">0</span>) &#123;</span><br><span class="line">            <span class="keyword">break</span>;</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="built_in">printf</span>(<span class="string">&quot;%d\n&quot;</span>, a[n]);</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure></p><p>题目网址：<ahref="http://poj.org/problem?id=2663">http://poj.org/problem?id=2663</a></p>]]></content>
    
    
      
      
    <summary type="html">&lt;h3 id=&quot;题目大意&quot;&gt;题目大意：&lt;/h3&gt;
&lt;p&gt;问用 &lt;span class=&quot;math inline&quot;&gt;&#92;(1*2&#92;)&lt;/span&gt; 的多米诺骨牌覆盖 &lt;span
class=&quot;math inline&quot;&gt;&#92;(3*n&#92;)&lt;/span&gt;
的矩形区域，总共有多少种不同的覆盖方</summary>
      
    
    
    
    <category term="算法" scheme="http://www.wangyulong.cn/categories/%E7%AE%97%E6%B3%95/"/>
    
    
    <category term="算法" scheme="http://www.wangyulong.cn/tags/%E7%AE%97%E6%B3%95/"/>
    
  </entry>
  
  <entry>
    <title>编写 MySQL 插件</title>
    <link href="http://www.wangyulong.cn/1534732562/"/>
    <id>http://www.wangyulong.cn/1534732562/</id>
    <published>2018-08-20T10:36:02.000Z</published>
    <updated>2026-02-19T07:45:16.917Z</updated>
    
    <content type="html"><![CDATA[<p>众所周知，MySQL是支持插件式存储引擎的，意思是MySQL源码中开放了存储引擎相关API，只要插件实现相关的API，就能安装到MySQL中，并作为存储引擎开始工作了。其实MySQL支持多种类型的插件，比如UDF，Daemon，认证，半同步，存储引擎。其中UDF和Daemon插件都非常简单，UDF只是实现了一个在MySQL的SQL接口里可以调用的函数；而Daemon插件只要按照MySQLPlugin的声明方式，声明plugin的描述以及入口函数，从这个plugin入口函数开始，你可以编写任意的代码，比如创建一个后台线程，甚至直接调用MySQLserver的源码，访问数据表的数据。</p><p>本文主要描述在MySQL中编写Daemonplugin的方法，本文创建一个为monitor的后台plugin，后台每隔若干秒将thread_count等打印到日志中。具体代码如下：</p><p>声明MySQL插件的描述以及入口出口函数 <figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">struct</span> <span class="title class_">st_mysql_daemon</span> monitor_info = &#123; MYSQL_DAEMON_INTERFACE_VERSION&#125;;</span><br><span class="line"> </span><br><span class="line"><span class="built_in">mysql_declare_plugin</span>(monitor_plugin)</span><br><span class="line">&#123;</span><br><span class="line">    MYSQL_DAEMON_PLUGIN,</span><br><span class="line">    &amp;monitor_info,</span><br><span class="line">    <span class="string">&quot;monitoring&quot;</span>,</span><br><span class="line">    <span class="string">&quot;wylazy&quot;</span>,</span><br><span class="line">    <span class="string">&quot;monitoring mysql thread&quot;</span>,</span><br><span class="line">    PLUGIN_LICENSE_BSD,</span><br><span class="line">    monitoring_plugin_init,</span><br><span class="line">    monitoring_plugin_deinit,</span><br><span class="line">    <span class="number">0x0100</span>, <span class="comment">//1.0</span></span><br><span class="line">    <span class="literal">NULL</span>,</span><br><span class="line">    vars_system_var,</span><br><span class="line">    <span class="literal">NULL</span></span><br><span class="line">&#125;</span><br><span class="line">mysql_declare_plugin_end;</span><br><span class="line"> </span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment"> * 具体的含义如下：</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> * struct st_mysql_plugin &#123;</span></span><br><span class="line"><span class="comment"> *   int type;                  // 插件类型, 比如MYSQL_DAEMON_PLUGIN,MYSQL_STORAGE_ENGINE_PLUGIN</span></span><br><span class="line"><span class="comment"> *   void * info;</span></span><br><span class="line"><span class="comment"> *   const char * name;         // INSTALL PLUGIN用到的插件名字</span></span><br><span class="line"><span class="comment"> *   const char * author;       // 插件的作者</span></span><br><span class="line"><span class="comment"> *   const char * descr;        // 插件的描述</span></span><br><span class="line"><span class="comment"> *   int license;               // PLUGIN_LICENSE_GPL, PLUGIN_LICENSE_BSD</span></span><br><span class="line"><span class="comment"> *   int (* init)(void *);      // 插件的入口函数</span></span><br><span class="line"><span class="comment"> *   int (* deinit)(void *);    // 插件的退出函数</span></span><br><span class="line"><span class="comment"> *   unsigned int version;      // 低8个bit存minor version，其他的为major version</span></span><br><span class="line"><span class="comment"> *   struct st_mysql_show_var * status_vars; // 通过SHOW STATUS显示的状态信息</span></span><br><span class="line"><span class="comment"> *   struct st_mysql_sys_var ** system_vars; // 通过SHOW VARIABLES显示的变量信息</span></span><br><span class="line"><span class="comment"> *   void * __reserved;         // 在MySQL 5.1里未用到</span></span><br><span class="line"><span class="comment"> * &#125;</span></span><br><span class="line"><span class="comment"> *</span></span><br><span class="line"><span class="comment"> */</span></span><br></pre></td></tr></table></figure>其中在结构体st_mysql_plugin中，成员name是插件的名称，(* init)(void<em>)是插件的入口函数，(</em> deinit)(void *)是插件的退出函数。<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">install plugin monitoring so name xxxxx.so</span><br></pre></td></tr></table></figure>当通过mysql命令行执行如上命令的时候，MySQL的执行过程大概为：通过插件的名称'monitoring'找到相应的插件，然后执行对应的init()函数。在uninstallplugin的时候执行deinit()函数。</p><p>因为我们的plugin想在后台执行，所以需要在init中创建一个后台执行的线程。具体代码如下：<figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br></pre></td><td class="code"><pre><span class="line"><span class="comment">//后台线程</span></span><br><span class="line"><span class="function"><span class="type">pthread_handler_t</span> <span class="title">monitoring</span><span class="params">(<span class="type">void</span> * p)</span> </span>&#123;</span><br><span class="line">    <span class="type">char</span> buffer[MONITOR_BUFFER];</span><br><span class="line">    <span class="type">char</span> time_str[<span class="number">20</span>];</span><br><span class="line"> </span><br><span class="line">    <span class="keyword">while</span> (<span class="number">1</span>) &#123;</span><br><span class="line">        <span class="keyword">struct</span> <span class="title class_">timeval</span> tv;</span><br><span class="line">        tv.tv_sec = waiting_seconds;</span><br><span class="line">        tv.tv_usec = <span class="number">0</span>;</span><br><span class="line">         </span><br><span class="line">        <span class="comment">//每隔若干秒，打印一次日志</span></span><br><span class="line">        <span class="keyword">if</span> (<span class="built_in">select</span>(<span class="number">1</span>, <span class="literal">NULL</span>, <span class="literal">NULL</span>, <span class="literal">NULL</span>, &amp;tv) == <span class="number">0</span>) &#123;</span><br><span class="line">            <span class="built_in">get_date</span>(time_str, GETDATE_DATE_TIME, <span class="number">0</span>);</span><br><span class="line">            <span class="built_in">sprintf</span>(buffer, <span class="string">&quot;%s: %u of %lu clients connected, %lu connections made\n&quot;</span>, </span><br><span class="line">                    time_str, thread_count, max_connections, thread_id);</span><br><span class="line">            <span class="built_in">write</span>(monitoring_file, buffer, <span class="built_in">strlen</span>(buffer));</span><br><span class="line">        &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">            <span class="built_in">fprintf</span>(stderr, <span class="string">&quot;select return error&quot;</span>);</span><br><span class="line">            <span class="keyword">break</span>;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"> </span><br><span class="line"><span class="comment">/**</span></span><br><span class="line"><span class="comment">* plugin的入口</span></span><br><span class="line"><span class="comment">* 返回0：安装插件成功</span></span><br><span class="line"><span class="comment">* 返回非0：安装插件失败。mysqld内部会调用deinit()方法做清理</span></span><br><span class="line"><span class="comment">*/</span></span><br><span class="line"><span class="function"><span class="type">static</span> <span class="type">int</span> <span class="title">monitoring_plugin_init</span><span class="params">(<span class="type">void</span> *p)</span> </span>&#123;</span><br><span class="line">    <span class="type">pthread_attr_t</span> attr;</span><br><span class="line">    <span class="type">char</span> monitoring_filename[FN_REFLEN];</span><br><span class="line">    <span class="type">char</span> buffer[MONITOR_BUFFER];</span><br><span class="line">    <span class="type">char</span> time_str[<span class="number">20</span>];</span><br><span class="line"> </span><br><span class="line">    <span class="comment">//打开日志文件</span></span><br><span class="line">    <span class="built_in">fn_format</span>(monitoring_filename, <span class="string">&quot;monitor&quot;</span>, <span class="string">&quot;&quot;</span>, <span class="string">&quot;.log&quot;</span>, MY_REPLACE_EXT | MY_UNPACK_FILENAME);</span><br><span class="line">    <span class="built_in">unlink</span>(monitoring_filename);</span><br><span class="line">    <span class="keyword">if</span> ((monitoring_file = <span class="built_in">open</span>(monitoring_filename, O_CREAT|O_RDWR, <span class="number">0644</span>)) &lt; <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="built_in">fprintf</span>(stderr, <span class="string">&quot;plugin &#x27;monitoring&#x27; could not create file %s&quot;</span>, monitoring_filename);</span><br><span class="line">        <span class="keyword">return</span> <span class="number">1</span>;</span><br><span class="line">    &#125;</span><br><span class="line"> </span><br><span class="line">    <span class="built_in">get_date</span>(time_str, GETDATE_DATE_TIME, <span class="number">0</span>);</span><br><span class="line">    <span class="built_in">sprintf</span>(buffer, <span class="string">&quot;Monitoring started at %s\n&quot;</span>, time_str);</span><br><span class="line">    <span class="built_in">write</span>(monitoring_file, buffer, <span class="built_in">strlen</span>(buffer));</span><br><span class="line"> </span><br><span class="line">    <span class="comment">//创建后台线程</span></span><br><span class="line">    <span class="keyword">if</span> (<span class="built_in">pthread_create</span>(&amp;monitoring_thread, <span class="literal">NULL</span>, monitoring, <span class="literal">NULL</span>) != <span class="number">0</span>) &#123;</span><br><span class="line">        <span class="built_in">fprintf</span>(stderr, <span class="string">&quot;Plugin monitoring could not create monitoring thread\n&quot;</span>);</span><br><span class="line">        <span class="keyword">return</span> <span class="number">1</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">&#125;</span><br><span class="line"> </span><br><span class="line"><span class="comment">//plugin的出口</span></span><br><span class="line"><span class="function"><span class="type">static</span> <span class="type">int</span> <span class="title">monitoring_plugin_deinit</span><span class="params">(<span class="type">void</span> * p)</span> </span>&#123;</span><br><span class="line">    <span class="type">char</span> buffer[MONITOR_BUFFER], time_str[<span class="number">20</span>];</span><br><span class="line">     </span><br><span class="line">    <span class="comment">//结束后台线程</span></span><br><span class="line">    <span class="built_in">pthread_cancel</span>(monitoring_thread);</span><br><span class="line">    <span class="built_in">pthread_join</span>(monitoring_thread, <span class="literal">NULL</span>);</span><br><span class="line"> </span><br><span class="line">    <span class="built_in">get_date</span>(time_str, GETDATE_DATE_TIME, <span class="number">0</span>);</span><br><span class="line">    <span class="built_in">sprintf</span>(buffer, <span class="string">&quot;Monitoring stopped at %s\n&quot;</span>, time_str);</span><br><span class="line">    <span class="built_in">write</span>(monitoring_file, buffer, <span class="built_in">strlen</span>(buffer));</span><br><span class="line"> </span><br><span class="line">    <span class="comment">//关闭日志文件</span></span><br><span class="line">    <span class="built_in">close</span>(monitoring_file);</span><br><span class="line">    <span class="keyword">return</span> <span class="number">0</span>;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>在plugin启动的时候，我们在mysql的数据目录下创建了一个日志文件monitor.log，同时创建了一个后台线程打印日志，在卸载plugin的时候，停止后台线程，并关闭日志文件。</p><p>为了可以动态的控制打印日志的时间间隔，我们还在plugin中添加了一个动态的变量waiting_seconds，控制后台线程打印日志的时间间隔。<figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br></pre></td><td class="code"><pre><span class="line"><span class="type">int</span> waiting_seconds = <span class="number">0</span>;</span><br><span class="line"> </span><br><span class="line"><span class="comment">//将变量与waiting_seconds 关联</span></span><br><span class="line"><span class="function"><span class="type">static</span> <span class="title">MYSQL_SYSVAR_INT</span><span class="params">(mo_waiting_seconds, waiting_seconds, PLUGIN_VAR_RQCMDARG, </span></span></span><br><span class="line"><span class="params"><span class="function">                       <span class="string">&quot;waiting time before print log&quot;</span>, <span class="literal">NULL</span>, <span class="literal">NULL</span></span></span></span><br><span class="line"><span class="params"><span class="function">                       , <span class="number">5</span>  <span class="comment">//默认值</span></span></span></span><br><span class="line"><span class="params"><span class="function">                       , <span class="number">0</span>  <span class="comment">//最小值</span></span></span></span><br><span class="line"><span class="params"><span class="function">                       , <span class="number">600</span> <span class="comment">//最大值</span></span></span></span><br><span class="line"><span class="params"><span class="function">                       , <span class="number">0</span>)</span></span>;</span><br><span class="line">                        </span><br><span class="line"><span class="comment">//将mo_waiting_seconds 添加到系统中                      </span></span><br><span class="line"><span class="keyword">struct</span> <span class="title class_">st_mysql_sys_var</span> * vars_system_var[] =</span><br><span class="line">&#123;</span><br><span class="line">    <span class="built_in">MYSQL_SYSVAR</span>(mo_waiting_seconds)</span><br><span class="line">    , <span class="literal">NULL</span></span><br><span class="line">&#125;;</span><br></pre></td></tr></table></figure>按照如上的方式，即可在mysql客户端中通过如下命令动态改变打印日志的时间间隔：<figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">set</span> <span class="keyword">global</span> monitoring_mo_waiting_seconds <span class="operator">=</span> <span class="number">8</span>;</span><br></pre></td></tr></table></figure>编译plugin的时候还需要依赖MySQL，既可以通过依赖MySQL源代码的方式编译，又可以通过依赖mysql库的方式编译。通过这两种方式编译所支持的功能是不同的，如果直接依赖mysql源码，就可以包含mysql在sql目录下的头文件，使用THD等服务器的数据类型，甚至是访问mysqld的全局变量，调用mysqld的函数等。而依赖mysql库编译的时候，功能就相对弱一些。这里为了方便起见，选择通过依赖mysql库的方式编译，Makefile的内容如下：<figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line">CC = gcc</span><br><span class="line">MYSQL_PATH = $(HOME)/programs/mysql</span><br><span class="line">MYSQL_CONFIG = $(MYSQL_PATH)/bin/mysql_config</span><br><span class="line"> </span><br><span class="line">INCLUDES = <span class="variable">$&#123;shell $(MYSQL_CONFIG) --include&#125;</span></span><br><span class="line"><span class="comment"># LIBS = $&#123;shell $(MYSQL_CONFIG) --libs&#125;</span></span><br><span class="line"> </span><br><span class="line">CPPFLAGS := -g $(CPPFLAGS) $(INCLUDES) -fPIC -DMYSQL_DYNAMIC_PLUGIN</span><br><span class="line"> </span><br><span class="line">SO_BASE=monitor</span><br><span class="line"> </span><br><span class="line">all: $(SO_BASE).so</span><br><span class="line"> </span><br><span class="line">$(SO_BASE).so: $(SO_BASE).o</span><br><span class="line">        $(CC) -o <span class="variable">$@</span> -shared $&lt;</span><br><span class="line"> </span><br><span class="line">install: all</span><br><span class="line">        <span class="built_in">cp</span> $(SO_BASE).so $(MYSQL_PATH)/lib/mysql/plugin/</span><br><span class="line"> </span><br><span class="line">clean:</span><br><span class="line">        <span class="built_in">rm</span> -f *.o *.gch *.so</span><br></pre></td></tr></table></figure></p><p>相关说明：</p><ol type="1"><li>同一个名称的plugin只能被install一次，如果install多次，只有第一次可以install成功，后面的都会失败，而且连plugin的init方法都不会调用</li><li>在installplugin以后，如果没有卸载plugin。如果将mysqld重启，plugin会在重启的时候自动install</li></ol>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;众所周知，MySQL是支持插件式存储引擎的，意思是MySQL源码中开放了存储引擎相关API，只要插件实现相关的API，就能安装到MySQL中，并作为存储引擎开始工作了。其实MySQL支持多种类型的插件，比如UDF，Daemon，认证，半同步，存储引擎。其中UDF和Daemo</summary>
      
    
    
    
    <category term="MySQL" scheme="http://www.wangyulong.cn/categories/MySQL/"/>
    
    
  </entry>
  
  <entry>
    <title>MySQL 复制中对 Load Data的处理（译）</title>
    <link href="http://www.wangyulong.cn/1534732376/"/>
    <id>http://www.wangyulong.cn/1534732376/</id>
    <published>2018-08-20T10:32:56.000Z</published>
    <updated>2026-02-19T07:45:16.916Z</updated>
    
    <content type="html"><![CDATA[<p>当MySQL执行LOAD DATAINFILE语句的时候，写binlog和其他语句有很大不同。一个LOAD DATAINFILE语句在binlog中可能变成了一个或者，若干个Event。这些Event记录了LoadData语句的附加信息，以及如何处理数据文件。</p><p>由于历史原因，一个LOAD DATA语句，可能对应4组不同的Event</p><p>1）在MySQL 3.23中，只有一个Event：Load_log_event（type codeLOAD_EVENT=6）,Load_log_event只记录了文件名，没有记录文件本身。当Slave遇到Load_log_event时，Slave会再和Master建立一个连接，让Master把文件发送过来。这有一个缺点，就是binlog不是自包含的。如果在Master上文件已经被删除了，或者Slave连不上Master，文件就会传输失败。</p><p>2）在MySQL 4.0.0中，文件本身也会记录到binlog中。一个LOAD DATAINFILE语句会对应多个Event，Create_file_log_event (type codeCREATE_FILE_EVENT = 8), Append_block_log_event (type codeAPPEND_BLOCK_EVENT = 9), Execute_load_log_event (type codeEXEC_LOAD_EVENT = 10), and Delete_file_log_event (type codeDELETE_FILE_EVENT = 11)，Event序列如下：</p><p>Create_file_log_event：传输1次</p><p>Append_block_log_event：传输0次，或者多次</p><p>Execute_load_log_event：传输1次（成功时）</p><p>Delete_file_log_event：传输1次（失败时）</p><p>Create_file_log_event也包含了LOAD DATAINFILE选项，这其实是一个设计上的缺陷。因为只有当遇到Execute_load_log_event时才能真正执行LOADDATA语句。所以当Slave收到Create_file_log_event时，会把它写入临时文件，只有当遇到Execute_load_log_event时才从这个临时文件构造完整的LOADDATA INFILE语句。</p><p>LOADDATA语句在LOAD一个大文件的时候，会把一个大文件分块传，每个块一个Append_block_log_event。块大小不超过2^17= 131072Bytes。</p><p>Create_file_log_event告诉Slave创建一个临时文件，并把文件的第一个块也写入临时文件（Create_file_log_event携带第一个文件块）。接下来会有若干Append_block_log_event，告诉Slave把它们追加写到这个临时文件。Execute_load_log_event告诉Slave把临时文件加载到数据表里面，或则是Delete_file_log_event告诉Slave不要加载临时文件，并把临时文件删除。当LOADDATA语句在Master上执行失败时，Master会记录一条Delete_file_log_event。</p><p>3）MySQL 4.0.0新引入了NEW_LOAD_EVENT类型，typecode = 12。</p><p>NEW_LOAD_EVENT和以前的LOAD_EVENT差不多，不过支持了更长的分隔符。原始的LOAD_DATA_EVENT只用了一个字节表示分隔符（FIELDSTERMINATEDBY）。后来在某个版本里，binlog格式支持了多字节作为分隔符，所以EVENT_TYPE也加入了支持。</p><p>4）MySQL 5.0.3里面又新添加了两个EVENT_TYPE。</p><p>Begin_load_query_log_event（typecode = 17）</p><p>Execute_load_query_log_event （typecode = 18）</p><p>一个LOAD DATA语句的Event序列可能如下：</p><p>Begin_load_query_log_event：传输1次</p><p>Append_block_log_event：传输0次，或者多次</p><p>Execute_load_log_event：传输1次（成功时）</p><p>Delete_file_log_event：传输1次（失败时）</p><p>在这个新序列中，Begin_load_query_log_event和Append_block_log_event几乎是相同的，Execute_load_log_event包含了LOADDATA语句的文本。（而在4.0里面，LOADDATA语句被记录在Create_file_log_event里面）。</p><p>这样就不在需要一个临时文件存放LOAD DATAINFILE的参数了，但是还是要有一个临时文件存放要被LOAD的数据。</p><p>示例部分请参照原文</p><p>原文地址：http://dev.mysql.com/doc/internals/en/load-data-infile-events.html</p>]]></content>
    
    
      
      
    <summary type="html">&lt;p&gt;当MySQL执行LOAD DATA
INFILE语句的时候，写binlog和其他语句有很大不同。一个LOAD DATA
INFILE语句在binlog中可能变成了一个或者，若干个Event。这些Event记录了Load
Data语句的附加信息，以及如何处理数据文件。&lt;/p&gt;</summary>
      
    
    
    
    <category term="MySQL" scheme="http://www.wangyulong.cn/categories/MySQL/"/>
    
    
  </entry>
  
  <entry>
    <title>MySQL 复制协议</title>
    <link href="http://www.wangyulong.cn/1534731278/"/>
    <id>http://www.wangyulong.cn/1534731278/</id>
    <published>2018-08-20T10:14:38.000Z</published>
    <updated>2026-02-19T07:45:16.916Z</updated>
    
    <content type="html"><![CDATA[<h3 id="概述">概述</h3><p>这里只介绍MySQL 的异步复制协议（MySQL5.5增加了半同步复制功能，感兴趣的同学可以自己研究）。</p><p>MySQL的主从复制工作模式大致为，主库将执行的语句写入Binlog，由Dump线程将Binlog发送到从库的IO线程，IO线程将日志保存为Relay-Log，再由从库的SQL线程重放执行。本文主要研究主库Dump线程将Binlog发送到从库IO线程所使用的协议。</p><h3 id="通信流程">通信流程</h3><p>首先从库向主库发起TCP连接，当连接建立完成后，主库向从库发送第一条数据包（InitialHandshakePacket），包含协议版本，服务器的版本，flag，以及认证相关信息。然后从库将用户名和加密的密码等认证信息发送给服务器。这一步的认证过程和普通MySQL客户端登录到MySQL服务器没什么区别。</p><p>然后从库向主库发送若干SELECT语句，获取服务器的时间戳，以及服务器版本等信息。</p><p>接着从库向主库发送一条COM_BINLOG_DUMP命令，开始复制过程。</p><p>当主库收到DUMP命令后，将Binlog中的Event一个接一个的发送到从库。</p><p><img src='../images/1397025016.21.mysql_conn_protocol.png' /></p><h3 id="mysql-一般数据包">MySQL 一般数据包</h3><p>每个MySQL数据包都是由的Header和Payload组成（包括Initial HandshakePacket在内的所有数据包）。Header由4个字节组成，3个字节的长度标识（FixedLengthInteger<ahref="http://dev.mysql.com/doc/internals/en/integer.html#packet-Protocol::FixedLengthInteger"class="uri">http://dev.mysql.com/doc/internals/en/integer.html#packet-Protocol::FixedLengthInteger</a>），1个字节的序号。Playoad长度由Header部分指定。<img src='../images/1397025142.11.mysql_packet.png' /></p><h3 id="握手包">握手包</h3><p>当从库与主库建立TCP连接后，主库向从库发送第一个数据包，包含了服务器的版本，以及服务器的能力，格式如下：<img src='../images/1397629175.59.init.png' /></p><h3 id="dump命令">DUMP命令</h3><p>当从库连接到主库以后，从库向主库发送一条DUMP命令，开始复制过程。DUMP命令的结构如下图所示，第一字节的命令标识为DUMP命令的标识12。<img src='../images/1397025217.19.dump_command.png' />随后，主库会将Binlog的Event一个接一个的发送给从库，第一个为RotateEvent，这个Event包含了下一个Event对应的binlog的文件名称。第二个Event为FormatDescriptionEvent，这个Event包含了所有Event的描述信息。MySQL协议里，在每个Event之前，都会有一个字节的00字段，这个字节在MySQL协议里叫做OK-Byte。参考：<ahref="http://dev.mysql.com/doc/internals/en/com-binlog-dump.html"class="uri">http://dev.mysql.com/doc/internals/en/com-binlog-dump.html</a></p><h3 id="event头字段">Event头字段</h3><p>主库向从库发送的Event数据包也由Event头和Event消息体组成。从MySQL4.0开始，Event头的长度固定为19字节。Event头的结构如下图所示：<img src='../images/1419938843.52.event_header.png' /></p><h3 id="rotate-event消息">Rotate Event消息</h3><p>Rotate Event比较简单，只包含一个Post Header和下一个Binlog的文件名称<img src='../images/1397025336.27.rotate.png' /></p><h3 id="format-description-event消息">Format Description Event消息</h3><p>FDE消息包含2个字节的Binlog版本，在MySQL5.0以后，Binlog版本是4；50字节的MySQL服务器版本，如果版本长度不足50，则后面补零；MySQL的Binlog支持26中Event，每种Event还可能有字节的Header，在FDE消息的最后，包含了各种Event对应的EventHeader长度，每种Event Header长度对应一个字节。<img src='../images/1397025391.31.fde.png' /></p><h3 id="query-event消息">Query Event消息</h3><p>QueryEvent消息如上图所示，其中从“slave_proxy_id”到“2字节的状态变量长度”之间为QueryEvent的Header，共占13字节（对应FDE消息中的描述）。剩余部分为Payload，其中状态变量和Schema的长度由Header所指定。然后是一个字节的“00”，最后是SQL查询语句。<img src='../images/1397025470.45.query.png' /></p><h3 id="其他event">其他Event</h3><p>请参考：<ahref="http://dev.mysql.com/doc/internals/en/binlog-event.html"class="uri">http://dev.mysql.com/doc/internals/en/binlog-event.html</a></p><h3 id="举例分析">举例分析</h3><p>场景为，主库在数据库db上的InnoDB数据表t上执行SQL 语句insertintot(val)values(‘a’)，其中在表t上带有自增主键。然后从库连接主库，开始复制过程。<b>DUMP请求</b> 从库向主库发起DUMP命令<img src='../images/1397025645.82.dump.png' /><b>前4个字节是MySQL数据包的头：</b></p><p>第1~3个字节：1b 00 00 为本数据包payload部分的长度27</p><p>第4个字节：00为包的序号</p><p><b>剩下的27个字节（从第5字节到第31字节）是DUMP命令部分：</b></p><p>第5个字节：12为COM_BINLOG_DUMP命令的标识</p><p>第6~9个字节：6a 00 00 00 为Binlog开始的位置</p><p>第10~11个字节：00 00 为两个字节的Flags</p><p>第12~15个字节：02 00 00 00 为从库的Server-id</p><p>第16~31个字节：Binlog的文件名</p><p><b>主库回复Binlog</b></p><p>主库收到DUMP命令后，向从库发送Event信息：<img src='../images/1397025752.58.binlog_res.png' />如上图，MySQL主库回复了6个部分的数据包，分别是6个Event：</p><p>Rotate Event</p><p>Format Description Event</p><p>Query Event: BEGIN</p><p>Intvar Event: #因为表t上带有自增主键，所以通过额外的IntvalEvent来保证主从在自增主键上的一致性</p><p>Query Event: insert into t(val) values(‘a’)</p><p>Xid Event: COMMIT</p><h3 id="event标识对照表">Event标识对照表</h3><p>参考：<ahref="http://dev.mysql.com/doc/internals/en/binlog-event-type.html"class="uri">http://dev.mysql.com/doc/internals/en/binlog-event-type.html</a></p><table><thead><tr><th>Hex</th><th>Event Name</th></tr></thead><tbody><tr><td>0x00</td><td>UNKNOWN_EVENT</td></tr><tr><td>0x01</td><td>START_EVENT_V3</td></tr><tr><td>0x02</td><td>QUERY_EVENT</td></tr><tr><td>0x03</td><td>STOP_EVENT</td></tr><tr><td>0x04</td><td>ROTATE_EVENT</td></tr><tr><td>0x05</td><td>INTVAR_EVENT</td></tr><tr><td>0x06</td><td>LOAD_EVENT</td></tr><tr><td>0x07</td><td>SLAVE_EVENT</td></tr><tr><td>0x08</td><td>CREATE_FILE_EVENT</td></tr><tr><td>0x09</td><td>APPEND_BLOCK_EVENT</td></tr><tr><td>0x0a</td><td>EXEC_LOAD_EVENT</td></tr><tr><td>0x0b</td><td>DELETE_FILE_EVENT</td></tr><tr><td>0x0c</td><td>NEW_LOAD_EVENT</td></tr><tr><td>0x0d</td><td>RAND_EVENT</td></tr><tr><td>0x0e</td><td>USER_VAR_EVENT</td></tr><tr><td>0x0f</td><td>FORMAT_DESCRIPTION_EVENT</td></tr><tr><td>0x10</td><td>XID_EVENT</td></tr><tr><td>0x11</td><td>BEGIN_LOAD_QUERY_EVENT</td></tr><tr><td>0x12</td><td>EXECUTE_LOAD_QUERY_EVENT</td></tr><tr><td>0x13</td><td>TABLE_MAP_EVENT</td></tr><tr><td>0x14</td><td>WRITE_ROWS_EVENTv0</td></tr><tr><td>0x15</td><td>UPDATE_ROWS_EVENTv0</td></tr><tr><td>0x16</td><td>DELETE_ROWS_EVENTv0</td></tr><tr><td>0x17</td><td>WRITE_ROWS_EVENTv1</td></tr><tr><td>0x18</td><td>UPDATE_ROWS_EVENTv1</td></tr><tr><td>0x19</td><td>DELETE_ROWS_EVENTv1</td></tr><tr><td>0x1a</td><td>INCIDENT_EVENT</td></tr><tr><td>0x1b</td><td>HEARTBEAT_EVENT</td></tr><tr><td>0x1c</td><td>IGNORABLE_EVENT</td></tr><tr><td>0x1d</td><td>ROWS_QUERY_EVENT</td></tr><tr><td>0x1e</td><td>WRITE_ROWS_EVENTv2</td></tr><tr><td>0x1f</td><td>UPDATE_ROWS_EVENTv2</td></tr><tr><td>0x20</td><td>DELETE_ROWS_EVENTv2</td></tr><tr><td>0x21</td><td>GTID_EVENT</td></tr><tr><td>0x22</td><td>ANONYMOUS_GTID_EVENT</td></tr><tr><td>0x23</td><td>PREVIOUS_GTIDS_EVENT</td></tr></tbody></table>]]></content>
    
    
      
      
    <summary type="html">&lt;h3 id=&quot;概述&quot;&gt;概述&lt;/h3&gt;
&lt;p&gt;这里只介绍MySQL 的异步复制协议（MySQL
5.5增加了半同步复制功能，感兴趣的同学可以自己研究）。&lt;/p&gt;
&lt;p&gt;MySQL的主从复制工作模式大致为，主库将执行的语句写入Binlog，由Dump线程将Binlog发送到从库的I</summary>
      
    
    
    
    <category term="MySQL" scheme="http://www.wangyulong.cn/categories/MySQL/"/>
    
    
  </entry>
  
  <entry>
    <title>MySQL 5.6 中Binlog Group Commit 实现</title>
    <link href="http://www.wangyulong.cn/1534730601/"/>
    <id>http://www.wangyulong.cn/1534730601/</id>
    <published>2018-08-20T10:03:21.000Z</published>
    <updated>2026-02-19T07:45:16.916Z</updated>
    
    <content type="html"><![CDATA[<h3 id="背景">背景</h3><p>在MySQL5.1中，如果配置项sync_binlog=1，并且innodb_flush_log_at_trx_commit=1，那么MySQL的TPS将会下降到几十每秒，完全不可接受。这是因为InnoDB提交事务时，不仅需要将REDO刷盘，还需要将Binlog刷盘，每个事务都需要2次sync操作。机械磁盘的IOPS也就为几百的水平，所以InnoDB的性能极差。</p><p>这个问题，在MySQL 5.6中得到了比较好的解决。在了解Binlog GroupCommit之前，需要先了解MySQLBinlog和InnoDB的两阶段提交。MySQL为了保证主库和从库的数据一致性，就必须保证Binlog和InnoDB的一致性，即如果一个事务写入了Binlog，InnoDB中就必须提交该事务；相反，如果一个事务没有写入Binlog，InnoDB就不能提交该事务。做法是：</p><p>InnoDB先执行Prepare，将Redo日志写磁盘。然后再将Binlog写磁盘，最后InnoDB再执行Commit，将事务标记为提交。这样，可以保证Binlog和InnoDB的一致性。具体原因，可以分三种情况考虑：</p><ol type="1"><li>果MySQL在InnoDBPrepare阶段Crash。MySQL在启动时做崩溃恢复，InnoDB会回滚这些事务，同时由于事务也没有写到binlog，InnoDB和Binlog一致。</li><li>如果MySQL在Binlog写磁盘阶段Crash。MySQL在启动时做崩溃恢复，在恢复时会扫描未成功提交的事务，和当时未成功关闭的binlog文件，如果事务已经Prepare了，并且也已经在Binlog中了，InnoDB会提交该事务；相反，如果事务已经在Prepare中了，但是不在Binlog中，InnoDB会回滚该事务。结果就是InnoDB和Binlog一致。</li><li>如果MySQL在InnoDB执行Commit阶段Crash，和情况2类似，由于事务已经成功Prepare，并且存在Binlog文件中，InnoDB在崩溃恢复时，仍然会提交该事务，确保Binlog和InnoDB一致。</li></ol><p><img src='/images/1436082376.83.2pc.png' />MySQL在实现时，将mysql_bin_log作为2阶段提交的协调者，可以参考MySQL的代码：sql/handler.cc:ha_commit_trans。内部分别调用tc_log-&gt;prepare()和tc_log-&gt;commit()实现2阶段提交，这里的tc_log就是MySQL源码中的全局对象mysql_bin_log。伪代码如下： <figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="built_in">ha_commit_trans</span>()</span><br><span class="line">  --&gt; tc_log-&gt;<span class="built_in">prepare</span>()</span><br><span class="line">        --&gt; <span class="built_in">ha_prepare_low</span>()</span><br><span class="line">              <span class="keyword">for</span> () &#123;</span><br><span class="line">                ht-&gt;<span class="built_in">prepare</span>() <span class="comment">//存储引擎 hton-&gt;prepare()</span></span><br><span class="line">              &#125;</span><br><span class="line">  --&gt; tc_log-&gt;<span class="built_in">commit</span>()</span><br><span class="line">        --&gt; MYSQL_BINLOG::<span class="built_in">ordered_commit</span>()<span class="comment">//做Group Commit</span></span><br><span class="line">              --&gt; MYSQL_BINLOG::<span class="built_in">process_commit_stage_queue</span>() <span class="comment">//Group Commit的Commit阶段，会调用InnoDB提交</span></span><br><span class="line">                    --&gt; <span class="built_in">ha_commit_low</span>()</span><br><span class="line">                          <span class="keyword">for</span> () &#123;</span><br><span class="line">                            ht-&gt;<span class="built_in">commit</span>(); <span class="comment">//存储引擎 hton-&gt;commit()</span></span><br><span class="line">                          &#125;</span><br></pre></td></tr></table></figure>两阶段提交的参与者分别为：binlog_hton和innobase_hton，它们实现了MySQL的存储引擎接口。如果你再深入调研一下，就会发现binlog_hton在2阶段提交时，啥也没干。所有binlog操作都是由协调者mysql_bin_log干的，包括GroupCommit，也都是在mysql_bin_log中实现的。下面我们就来分析一下，mysql_bin_log是如何做到GroupCommit的，也就是上面的函数ordered_commit()。</p><h3 id="实现">实现</h3><p>和Level DB的Group Commit类似，MySQL的GroupCommit也是维护了一个队列，第一个进入队列的线程就是Leader，负责写binlog。其他的线程是Flower，Flower不需要操作，只需要等待完成的通知即可。但是如果只用一个队列的话，在GroupCommit进行中的时候，后来的线程就得等待，还可以进一步优化，MySQL把这个过程分裂成了3个阶段：FLUSH_STAGE，SYNC_STAGE和COMMIT_STAGE。它们像流水线一样工作，每个阶段都会涉及一批事务，它们组成一个Group。可以这样理解，事务刚提交时，处于FLUSH阶段，同时处于FLUSH阶段的事务为一个队列，形成一个Group，只有队列的头，Leader在干活，FLUSH完成以后，Leader进入SYNC阶段（所有的Flower也都进入SYNC阶段）。这时，新提交的事务可以进入FLUSH阶段，它们又会产生一个新的Leader，如此不断的推进。每个阶段都需要一个队列，所以MySQL在GroupCommit时，需要3个队列。如下图所示，队列通过thd-&gt;next_to_commit连接：</p><p><img src='/images/1436082689.4.stage_queue.png' />MySQL把队列命名为Mutex_queue，这是一个C++的类，定义如下： <figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">class</span> <span class="title class_">Mutex_queue</span> &#123;</span><br><span class="line">    THD *m_first; <span class="comment">//队列的头指针</span></span><br><span class="line">    THD **m_last; <span class="comment">//队列尾指针的地址。如果队列为空，相当于&amp;m_first，否则，相当于&amp;last-&gt;next_to_commit</span></span><br><span class="line">    <span class="type">mysql_mutex_t</span> m_lock;</span><br><span class="line">&#125;;</span><br></pre></td></tr></table></figure>在GroupCommit时，事务的状态首先转为FLUSH_STAGE，然后为SYNC_STAGE，最后为COMMIT_STAGE。在状态转变时，都会调用如下函数Stage_manager::enroll_for：<figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="type">bool</span> <span class="title">Stage_manager::enroll_for</span><span class="params">(StageID stage, THD *thd, <span class="type">mysql_mutex_t</span> *stage_mutex)</span> </span>&#123;</span><br><span class="line"> </span><br><span class="line">  <span class="comment">// 只有队列的第一个元素为Leader，其他情况均为false</span></span><br><span class="line">  <span class="type">bool</span> leader= m_queue[stage].<span class="built_in">append</span>(thd);</span><br><span class="line"> </span><br><span class="line">  <span class="comment">// The stage mutex can be NULL if we are enrolling for the first stage.</span></span><br><span class="line">  <span class="keyword">if</span> (stage_mutex)</span><br><span class="line">    <span class="built_in">mysql_mutex_unlock</span>(stage_mutex);</span><br><span class="line"> </span><br><span class="line">  <span class="comment">/**</span></span><br><span class="line"><span class="comment">   * 如果不是Leader的话，只需等待Leader完成操作的通知</span></span><br><span class="line"><span class="comment">   * Leader完成以后，会设置thd-&gt;transaction.flags.pending = false</span></span><br><span class="line"><span class="comment">   */</span></span><br><span class="line">  <span class="keyword">if</span> (!leader) &#123;</span><br><span class="line">    <span class="built_in">mysql_mutex_lock</span>(&amp;m_lock_done);</span><br><span class="line"> </span><br><span class="line">    <span class="keyword">while</span> (thd-&gt;transaction.flags.pending)</span><br><span class="line">      <span class="built_in">mysql_cond_wait</span>(&amp;m_cond_done, &amp;m_lock_done);</span><br><span class="line"> </span><br><span class="line">    <span class="built_in">mysql_mutex_unlock</span>(&amp;m_lock_done);</span><br><span class="line">  &#125;</span><br><span class="line"> </span><br><span class="line">  <span class="keyword">return</span> leader;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>从上面的代码可以看出，Flower线程什么也不干，所有的事情都要靠Leader去做。上述代码有一个细节需要注意，先把自己添加到队列中，然后再释放锁stage_mutex，这个在后面会有解释。下面逐个分析一下，在每个阶段Leader线程所做的事情。</p><h3 id="flush阶段">FLUSH阶段</h3><p>因为InnoDB在事务执行过程中，要保证事务的原子性。对于INSERT/UPDATE/DELETE操作，会先将Binlog写事务日志（binlog_cache_mngr），事务提交时，也就是在FLUSH阶段，再把事务日志复制到binlog文件中，然后通知Dump线程去发送binlog，由于要写Binlog文件，这个过程需要锁定LOCK_log锁。这也就是FLUSH阶段要做的事情，可参考函数：MYSQL_BIN_LOG::process_flush_stage_queue。</p><p>在这个阶段，Leader线程遍历遍历FLUSH_STAGE链表，依次取出thd对应的事务日志，并写到binlog的IOCACHE中，然后flushIOCACHE。代码如下： <figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="type">int</span> <span class="title">MYSQL_BIN_LOG::ordered_commit</span><span class="params">(THD *thd, <span class="type">bool</span> all, <span class="type">bool</span> skip_commit)</span></span></span><br><span class="line"><span class="function"></span>&#123;</span><br><span class="line">   <span class="comment">//...</span></span><br><span class="line">    </span><br><span class="line">  <span class="comment">/*</span></span><br><span class="line"><span class="comment">    Stage #1: flushing transactions to binary log</span></span><br><span class="line"><span class="comment"> </span></span><br><span class="line"><span class="comment">    While flushing, we allow new threads to enter and will process</span></span><br><span class="line"><span class="comment">    them in due time. Once the queue was empty, we cannot reap</span></span><br><span class="line"><span class="comment">    anything more since it is possible that a thread entered and</span></span><br><span class="line"><span class="comment">    appointed itself leader for the flush phase.</span></span><br><span class="line"><span class="comment">  */</span></span><br><span class="line">  <span class="keyword">if</span> (<span class="built_in">change_stage</span>(thd, Stage_manager::FLUSH_STAGE, thd, <span class="literal">NULL</span>, &amp;LOCK_log))</span><br><span class="line">  &#123;</span><br><span class="line">    <span class="built_in">DBUG_PRINT</span>(<span class="string">&quot;return&quot;</span>, (<span class="string">&quot;Thread ID: %lu, commit_error: %d&quot;</span>,</span><br><span class="line">                          thd-&gt;thread_id, thd-&gt;commit_error));</span><br><span class="line">    <span class="built_in">DBUG_RETURN</span>(<span class="built_in">finish_commit</span>(thd));</span><br><span class="line">  &#125;</span><br><span class="line"> </span><br><span class="line">  THD *wait_queue= <span class="literal">NULL</span>;</span><br><span class="line">  flush_error= <span class="built_in">process_flush_stage_queue</span>(&amp;total_bytes, &amp;do_rotate, &amp;wait_queue);</span><br><span class="line"> </span><br><span class="line">  <span class="type">my_off_t</span> flush_end_pos= <span class="number">0</span>;</span><br><span class="line">  <span class="keyword">if</span> (flush_error == <span class="number">0</span> &amp;&amp; total_bytes &gt; <span class="number">0</span>)</span><br><span class="line">    flush_error= <span class="built_in">flush_cache_to_file</span>(&amp;flush_end_pos);</span><br><span class="line">     </span><br><span class="line">  <span class="comment">/*</span></span><br><span class="line"><span class="comment">    If the flush finished successfully, we can call the after_flush</span></span><br><span class="line"><span class="comment">    hook. Being invoked here, we have the guarantee that the hook is</span></span><br><span class="line"><span class="comment">    executed before the before/after_send_hooks on the dump thread</span></span><br><span class="line"><span class="comment">    preventing race conditions among these plug-ins.</span></span><br><span class="line"><span class="comment">  */</span></span><br><span class="line">  <span class="keyword">if</span> (flush_error == <span class="number">0</span>)</span><br><span class="line">  &#123;</span><br><span class="line">    <span class="type">const</span> <span class="type">char</span> *file_name_ptr= log_file_name + <span class="built_in">dirname_length</span>(log_file_name);</span><br><span class="line">    <span class="built_in">DBUG_ASSERT</span>(flush_end_pos != <span class="number">0</span>);</span><br><span class="line">    <span class="keyword">if</span> (<span class="built_in">RUN_HOOK</span>(binlog_storage, after_flush,</span><br><span class="line">                 (thd, file_name_ptr, flush_end_pos)))</span><br><span class="line">    &#123;</span><br><span class="line">      <span class="built_in">sql_print_error</span>(<span class="string">&quot;Failed to run &#x27;after_flush&#x27; hooks&quot;</span>);</span><br><span class="line">      flush_error= ER_ERROR_ON_WRITE;</span><br><span class="line">    &#125;</span><br><span class="line"> </span><br><span class="line">    <span class="built_in">signal_update</span>();</span><br><span class="line">    <span class="built_in">DBUG_EXECUTE_IF</span>(<span class="string">&quot;crash_commit_after_log&quot;</span>, <span class="built_in">DBUG_SUICIDE</span>(););</span><br><span class="line">  &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>在这个过程中有一个问题需要考虑，就是：一方面，Leader线程从链表中取出thd，将日志写binlogIOCACHE，另一方面，新提交的事务仍然会往FLUSH_STAGE链表中添加thd。如果MySQL的并发事务比较多，Leader线程写binlog的速度，小于新事务的提交速度，可能会造成事务停留在FLUSH阶段的时间过长。所以MySQL通过配置项binlog_max_flush_queue_time来控制这个时间，如果Leader线程在取THD时，发现超时了，Leader线程就将队列整个端走，再做处理。这样，当前已经处于FLUSH阶段的事务还用现在的Leader，而新提交的事务，会用新的Leader。因为LOCK_log锁的存在，所有新的Leader只能等当前的FLUSH执行完成才能开始执行。具体代码如下：<figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="type">int</span> <span class="title">MYSQL_BIN_LOG::process_flush_stage_queue</span><span class="params">(<span class="type">my_off_t</span> *total_bytes_var,</span></span></span><br><span class="line"><span class="params"><span class="function">                                         <span class="type">bool</span> *rotate_var,</span></span></span><br><span class="line"><span class="params"><span class="function">                                         THD **out_queue_var)</span></span></span><br><span class="line"><span class="function">  <span class="type">bool</span> has_more</span>= <span class="literal">true</span>;</span><br><span class="line">  THD *first_seen= <span class="literal">NULL</span>;</span><br><span class="line">  <span class="keyword">while</span> ((max_udelay == <span class="number">0</span> || <span class="built_in">my_micro_time</span>() &lt; start_utime + max_udelay) &amp;&amp; has_more)</span><br><span class="line">  &#123;</span><br><span class="line">    std::pair&lt;<span class="type">bool</span>,THD*&gt; current= stage_manager.<span class="built_in">pop_front</span>(Stage_manager::FLUSH_STAGE);</span><br><span class="line">    std::pair&lt;<span class="type">int</span>,<span class="type">my_off_t</span>&gt; result= <span class="built_in">flush_thread_caches</span>(current.second);</span><br><span class="line">    has_more= current.first;</span><br><span class="line">    total_bytes+= result.second;</span><br><span class="line">    <span class="keyword">if</span> (flush_error == <span class="number">1</span>)</span><br><span class="line">      flush_error= result.first;</span><br><span class="line">    <span class="keyword">if</span> (first_seen == <span class="literal">NULL</span>)</span><br><span class="line">      first_seen= current.second;</span><br><span class="line">  &#125;</span><br><span class="line">    <span class="comment">/*</span></span><br><span class="line"><span class="comment">    Either the queue is empty, or we ran out of time. If we ran out of</span></span><br><span class="line"><span class="comment">    time, we have to fetch the entire queue (and flush it) since</span></span><br><span class="line"><span class="comment">    otherwise the next batch will not have a leader.</span></span><br><span class="line"><span class="comment">   */</span></span><br><span class="line">  <span class="keyword">if</span> (has_more)</span><br><span class="line">  &#123;</span><br><span class="line">    THD *queue= stage_manager.<span class="built_in">fetch_queue_for</span>(Stage_manager::FLUSH_STAGE);</span><br><span class="line">    <span class="keyword">for</span> (THD *head= queue ; head ; head = head-&gt;next_to_commit)</span><br><span class="line">    &#123;</span><br><span class="line">      std::pair&lt;<span class="type">int</span>,<span class="type">my_off_t</span>&gt; result= <span class="built_in">flush_thread_caches</span>(head);</span><br><span class="line">      total_bytes+= result.second;</span><br><span class="line">      <span class="keyword">if</span> (flush_error == <span class="number">1</span>)</span><br><span class="line">        flush_error= result.first;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">if</span> (first_seen == <span class="literal">NULL</span>)</span><br><span class="line">      first_seen= queue;</span><br><span class="line">  &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure> 写完binlogIOCACHE后，还要将IOCACHE写文件，最后通知Dump线程读取binlog，FLUSH阶段完成。</p><h3 id="syn-c阶段">SYN C阶段</h3><p>SYNC阶段的任务比较简单,但是却非常耗时，就是将binlog文件sync到磁盘。这个操作由配置项sync_binlog= N来控制每隔N个binlog只sync一次。如果sync_binlog=1的话，MySQL在SYNC阶段不释放锁LOCK_log，而Dump线程为了读取binlog，必须先申请锁LOCK_log，所以可以保证主库先将binlogsync到磁盘，然后Dump线程才能读取Binlog，确保即使在主库操作系统Crash情况下，仍然保证主库和从库数据一致。其他情况会释放LOCK_log锁，这时Dump线程可以读取并发送binlog，同时新提交的事务也可以进入FLUSH阶段。所以SYNC阶段需要考虑有多个FLUSH阶段的Leader同时进入SYNC阶段的情况。MySQL将这些Leader合并为一个新的Leader，做法是：FLUSH阶段的Leader线程进入SYNC阶段前，需要将自己加入到SYNC_STAGE队列中，第一个进入SYNC_STAGE队列的线程为SYNC阶段的Leader，后进入的为Flower。由Leader完成后续操作，Flower线程只需等待通知即可。回忆前面的函数enroll_for()，在状态转变时，Leader先把自己添加到SYNC队列中，然后才释放锁stage_mutex，这里就是LOCK_log，其他事务才可以进入FLUSH阶段，这可以保证，第一个进入FLUSH阶段的Leader，在SYNC阶段仍然是Leader，同样，在COMMIT阶段还是Leader。这对于保证Binlog和InnoDB提交顺序一致非常重要。<img src='/images/1436088683.87.sync_stage.png' /> SYNC阶段的代码如下：<figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="type">int</span> <span class="title">MYSQL_BIN_LOG::ordered_commit</span><span class="params">(THD *thd, <span class="type">bool</span> all, <span class="type">bool</span> skip_commit)</span></span></span><br><span class="line"><span class="function"></span>&#123;</span><br><span class="line"></span><br><span class="line">   <span class="comment">// ...</span></span><br><span class="line"></span><br><span class="line">  <span class="comment">/*</span></span><br><span class="line"><span class="comment">    Stage #2: Syncing binary log file to disk</span></span><br><span class="line"><span class="comment">  */</span></span><br><span class="line">  <span class="type">bool</span> need_LOCK_log= (<span class="built_in">get_sync_period</span>() == <span class="number">1</span>); <span class="comment">//只有sync_binlog=1，才不释放LOCK_log锁</span></span><br><span class="line"></span><br><span class="line">  <span class="comment">/*</span></span><br><span class="line"><span class="comment">    LOCK_log is not released when sync_binlog is 1. It guarantees that the</span></span><br><span class="line"><span class="comment">    events are not be replicated by dump threads before they are synced to disk.</span></span><br><span class="line"><span class="comment">  */</span></span><br><span class="line">  <span class="comment">//不管怎样，都要申请锁LOCK_sync</span></span><br><span class="line">  <span class="keyword">if</span> (<span class="built_in">change_stage</span>(thd, Stage_manager::SYNC_STAGE, wait_queue,</span><br><span class="line">                   need_LOCK_log ? <span class="literal">NULL</span> : &amp;LOCK_log, &amp;LOCK_sync))</span><br><span class="line">  &#123;</span><br><span class="line">    <span class="built_in">DBUG_PRINT</span>(<span class="string">&quot;return&quot;</span>, (<span class="string">&quot;Thread ID: %lu, commit_error: %d&quot;</span>,</span><br><span class="line">                          thd-&gt;thread_id, thd-&gt;commit_error));</span><br><span class="line">    <span class="built_in">DBUG_RETURN</span>(<span class="built_in">finish_commit</span>(thd));</span><br><span class="line">  &#125;</span><br><span class="line">  THD *final_queue= stage_manager.<span class="built_in">fetch_queue_for</span>(Stage_manager::SYNC_STAGE);</span><br><span class="line">  <span class="keyword">if</span> (flush_error == <span class="number">0</span> &amp;&amp; total_bytes &gt; <span class="number">0</span>)</span><br><span class="line">  &#123;</span><br><span class="line">    <span class="built_in">DEBUG_SYNC</span>(thd, <span class="string">&quot;before_sync_binlog_file&quot;</span>);</span><br><span class="line">    std::pair&lt;<span class="type">bool</span>, <span class="type">bool</span>&gt; result= <span class="built_in">sync_binlog_file</span>(<span class="literal">false</span>);</span><br><span class="line">    flush_error= result.first;</span><br><span class="line">  &#125;</span><br><span class="line"></span><br><span class="line">  <span class="keyword">if</span> (need_LOCK_log)</span><br><span class="line">    <span class="built_in">mysql_mutex_unlock</span>(&amp;LOCK_log);</span><br><span class="line"></span><br><span class="line">  <span class="comment">//...</span></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure></p><h3 id="commit-阶段">COMMIT 阶段</h3><p>经过前面2个阶段，Binlog已经顺利sync到磁盘了，COMMIT阶段的任务就是让InnoDB存储引擎完成Commit。COMMIT阶段的逻辑通过MySQL的配置项binlog_order_commits控制。如果配置项为1，MySQL要保证InnoDB的提交顺序和Binlog的写入顺序一致，这个特性在InnoDB热备中使用。下面只分析binlog_order_commits=1的情况。</p><p>MySQL释放锁LOCK_sync，申请锁LOCK_commit。由于释放锁LOCK_sync，所以需要考虑多个线程同时完成SYNC阶段的情况，处理逻辑和SYNC阶段类似，将当前SYNC阶段的Leader合并，关于Leader的产生和SYNC阶段类似。Leader产生以后，遍历THD，完成事务提交，等所有事务都提交完成以后，再遍历thd，设置thd-&gt;transaction.flags.pending=false，最后广播通知Flower线程提交完成，自此，GroupCommit完成。</p><p>代码如下： <figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="type">int</span> <span class="title">MYSQL_BIN_LOG::ordered_commit</span><span class="params">(THD *thd, <span class="type">bool</span> all, <span class="type">bool</span> skip_commit)</span></span></span><br><span class="line"><span class="function"></span>&#123;</span><br><span class="line">   <span class="comment">//...</span></span><br><span class="line">    </span><br><span class="line">  <span class="comment">/*</span></span><br><span class="line"><span class="comment">    Stage #3: Commit all transactions in order.</span></span><br><span class="line"><span class="comment"> </span></span><br><span class="line"><span class="comment">    This stage is skipped if we do not need to order the commits and</span></span><br><span class="line"><span class="comment">    each thread have to execute the handlerton commit instead.</span></span><br><span class="line"><span class="comment"> </span></span><br><span class="line"><span class="comment">    Howver, since we are keeping the lock from the previous stage, we</span></span><br><span class="line"><span class="comment">    need to unlock it if we skip the stage.</span></span><br><span class="line"><span class="comment">   */</span></span><br><span class="line">  <span class="keyword">if</span> (opt_binlog_order_commits)</span><br><span class="line">  &#123;</span><br><span class="line">    <span class="keyword">if</span> (<span class="built_in">change_stage</span>(thd, Stage_manager::COMMIT_STAGE,</span><br><span class="line">                     final_queue, &amp;LOCK_sync, &amp;LOCK_commit))</span><br><span class="line">    &#123;</span><br><span class="line">      <span class="built_in">DBUG_PRINT</span>(<span class="string">&quot;return&quot;</span>, (<span class="string">&quot;Thread ID: %lu, commit_error: %d&quot;</span>,</span><br><span class="line">                            thd-&gt;thread_id, thd-&gt;commit_error));</span><br><span class="line">      <span class="built_in">DBUG_RETURN</span>(<span class="built_in">finish_commit</span>(thd));</span><br><span class="line">    &#125;</span><br><span class="line">    THD *commit_queue= stage_manager.<span class="built_in">fetch_queue_for</span>(Stage_manager::COMMIT_STAGE);</span><br><span class="line">    <span class="built_in">DBUG_EXECUTE_IF</span>(<span class="string">&quot;semi_sync_3-way_deadlock&quot;</span>,</span><br><span class="line">                    <span class="built_in">DEBUG_SYNC</span>(thd, <span class="string">&quot;before_process_commit_stage_queue&quot;</span>););</span><br><span class="line">    <span class="built_in">process_commit_stage_queue</span>(thd, commit_queue);</span><br><span class="line">    <span class="built_in">mysql_mutex_unlock</span>(&amp;LOCK_commit);</span><br><span class="line">    <span class="comment">/*</span></span><br><span class="line"><span class="comment">      Process after_commit after LOCK_commit is released for avoiding</span></span><br><span class="line"><span class="comment">      3-way deadlock among user thread, rotate thread and dump thread.</span></span><br><span class="line"><span class="comment">    */</span></span><br><span class="line">    <span class="built_in">process_after_commit_stage_queue</span>(thd, commit_queue);</span><br><span class="line">    final_queue= commit_queue;</span><br><span class="line">  &#125;</span><br><span class="line">  <span class="keyword">else</span></span><br><span class="line">    <span class="built_in">mysql_mutex_unlock</span>(&amp;LOCK_sync);</span><br><span class="line"> </span><br><span class="line"> <span class="comment">/* Commit done so signal all waiting threads */</span></span><br><span class="line">  stage_manager.<span class="built_in">signal_done</span>(final_queue);    </span><br><span class="line">  </span><br><span class="line"> <span class="comment">//...</span></span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure>Leader产生以后，Leader线程通过next_to_commit遍历thd，对每个thd完成事务提交ha_commit_low(),代码如下：<figure class="highlight cpp"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br></pre></td><td class="code"><pre><span class="line"><span class="function"><span class="type">void</span></span></span><br><span class="line"><span class="function"><span class="title">MYSQL_BIN_LOG::process_commit_stage_queue</span><span class="params">(THD *thd, THD *first)</span></span></span><br><span class="line"><span class="function"></span>&#123;</span><br><span class="line">  <span class="keyword">for</span> (THD *head= first ; head ; head = head-&gt;next_to_commit)</span><br><span class="line">  &#123;</span><br><span class="line">    <span class="keyword">if</span> (head-&gt;commit_error == THD::CE_NONE)</span><br><span class="line">    &#123;</span><br><span class="line">      excursion.<span class="built_in">try_to_attach_to</span>(head);</span><br><span class="line">      <span class="type">bool</span> all= head-&gt;transaction.flags.real_commit;</span><br><span class="line">      <span class="keyword">if</span> (head-&gt;transaction.flags.commit_low)</span><br><span class="line">      &#123;</span><br><span class="line">        <span class="comment">/* head is parked to have exited append() */</span></span><br><span class="line">        <span class="built_in">DBUG_ASSERT</span>(head-&gt;transaction.flags.ready_preempt);</span><br><span class="line">        <span class="comment">/*</span></span><br><span class="line"><span class="comment">          storage engine commit</span></span><br><span class="line"><span class="comment">        */</span></span><br><span class="line">        <span class="keyword">if</span> (<span class="built_in">ha_commit_low</span>(head, all, <span class="literal">false</span>))</span><br><span class="line">          head-&gt;commit_error= THD::CE_COMMIT_ERROR;</span><br><span class="line">      &#125;</span><br><span class="line">       </span><br><span class="line">    &#125;</span><br><span class="line">        <span class="comment">/*</span></span><br><span class="line"><span class="comment">      Decrement the prepared XID counter after storage engine commit.</span></span><br><span class="line"><span class="comment">      We also need decrement the prepared XID when encountering a</span></span><br><span class="line"><span class="comment">      flush error or session attach error for avoiding 3-way deadlock</span></span><br><span class="line"><span class="comment">      among user thread, rotate thread and dump thread.</span></span><br><span class="line"><span class="comment">    */</span></span><br><span class="line">    <span class="keyword">if</span> (head-&gt;transaction.flags.xid_written)</span><br><span class="line">      <span class="built_in">dec_prep_xids</span>(head);</span><br><span class="line">  &#125;</span><br><span class="line">&#125;</span><br><span class="line"> </span><br><span class="line"><span class="keyword">class</span> <span class="title class_">Stage_manager</span> &#123;</span><br><span class="line"><span class="keyword">public</span>:</span><br><span class="line">  <span class="comment">//遍历THD，标记提交完成，并广播通知</span></span><br><span class="line">  <span class="function"><span class="type">void</span> <span class="title">signal_done</span><span class="params">(THD *queue)</span> </span>&#123;</span><br><span class="line">    <span class="built_in">mysql_mutex_lock</span>(&amp;m_lock_done);</span><br><span class="line">    <span class="keyword">for</span> (THD *thd= queue ; thd ; thd = thd-&gt;next_to_commit)</span><br><span class="line">      thd-&gt;transaction.flags.pending= <span class="literal">false</span>;</span><br><span class="line">    <span class="built_in">mysql_mutex_unlock</span>(&amp;m_lock_done);</span><br><span class="line">    <span class="built_in">mysql_cond_broadcast</span>(&amp;m_cond_done);</span><br><span class="line">  &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure></p>]]></content>
    
    
      
      
    <summary type="html">&lt;h3 id=&quot;背景&quot;&gt;背景&lt;/h3&gt;
&lt;p&gt;在MySQL
5.1中，如果配置项sync_binlog=1，并且innodb_flush_log_at_trx_commit=1，那么MySQL的TPS将会下降到几十每秒，完全不可接受。这是因为InnoDB提交事务时，不仅需要将RE</summary>
      
    
    
    
    <category term="MySQL" scheme="http://www.wangyulong.cn/categories/MySQL/"/>
    
    
  </entry>
  
</feed>
