<?xml version="1.0" encoding="utf-8"?>
<feed xmlns="http://www.w3.org/2005/Atom">
  <title>Lin Blog</title>
  
  <subtitle>林柏格</subtitle>
  <link href="/blog/atom.xml" rel="self"/>
  
  <link href="https://breeze2.github.io/blog/"/>
  <updated>2019-09-30T02:22:37.082Z</updated>
  <id>https://breeze2.github.io/blog/</id>
  
  <author>
    <name>林毅锋</name>
    
  </author>
  
  <generator uri="http://hexo.io/">Hexo</generator>
  
  <entry>
    <title>源码编译Portable Pyton</title>
    <link href="https://breeze2.github.io/blog/practice-compile-portable-python.html"/>
    <id>https://breeze2.github.io/blog/practice-compile-portable-python.html</id>
    <published>2019-09-28T10:15:06.000Z</published>
    <updated>2019-09-30T02:22:37.082Z</updated>
    
    <content type="html"><![CDATA[<blockquote><p>Portable的意思是可拔插的，便携式的。Portable 程序在一台计算机上编译好后，移植到其他计算机上就可以直接运行。一般程序运行时都会依赖一些第三方库，只要把对这些库的依赖方式由静态依赖改成动态依赖，再把这些库和主体程序放到一起打包，就成为一个Portable 程序。<br>当然Portable 也是有限度的，不能跨操作系统移植，毕竟程序对操作系统的依赖关联太多，除非把整个系统也移植了😂。</p></blockquote><p>以<a href="https://www.python.org/downloads/release/python-365/" target="_blank" rel="noopener">Python-3.6.5</a> 为例，讲述一下源码编译一个Portable Pyton 的主要流程。</p><a id="more"></a><h2 id="Windows-平台"><a href="#Windows-平台" class="headerlink" title="Windows 平台"></a>Windows 平台</h2><p><a href="https://winpython.github.io/" target="_blank" rel="noopener">winpython</a> 是一个专门编译Windows 平台下Portable Pyton 的项目，可以直接下载<a href="https://github.com/winpython/winpython/releases/download/1.10.20180404/WinPython32-3.6.5.0Zero.exe" target="_blank" rel="noopener">WinPython32-3.6.5.0Zero.exe</a>或者<a href="https://github.com/winpython/winpython/releases/download/1.10.20180404/WinPython64-3.6.5.0Zero.exe" target="_blank" rel="noopener">WinPython64-3.6.5.0Zero.exe</a>，解压出来就是对应的32位Portable Pyton 或64位Portable Pyton.</p><h2 id="macOS-平台"><a href="#macOS-平台" class="headerlink" title="macOS 平台"></a>macOS 平台</h2><p>最好在<code>OS X10.9</code>系统下编译Portable Pyton，对其他更高级的macOS 兼容性会好一些。<br>python 的一些内置模块会依赖第三方库，主要是：</p><ul><li><a href="https://tukaani.org/xz/" target="_blank" rel="noopener">lzma</a></li><li><a href="https://www.openssl.org/" target="_blank" rel="noopener">openssl</a></li><li><a href="https://www.tcl.tk/" target="_blank" rel="noopener">tcl/tk</a></li></ul><p>所以，我们要先安装这些库。</p><h3 id="编译lzma"><a href="#编译lzma" class="headerlink" title="编译lzma"></a>编译lzma</h3><p>下载<a href="https://tukaani.org/xz/xz-5.2.3.tar.gz" target="_blank" rel="noopener">xz-5.2.3.tar.gz</a>，解压到<code>/tmp/xz-5.2.3</code>，然后：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> /tmp</span><br><span class="line">$ <span class="built_in">mkdir</span> local</span><br><span class="line">$ <span class="built_in">cd</span> xz-<span class="number">5</span>.<span class="number">2</span>.<span class="number">3</span></span><br><span class="line">$ ./configure --prefix=/tmp/local</span><br><span class="line">$ make &amp;&amp; make install</span><br></pre></td></tr></table></figure><h3 id="编译openssl"><a href="#编译openssl" class="headerlink" title="编译openssl"></a>编译openssl</h3><p>下载<a href="https://github.com/openssl/openssl/archive/OpenSSL_1_0_2t.zip" target="_blank" rel="noopener">openssl-OpenSSL_1_0_2t.zip</a>，解压到<code>/tmp/openssl-OpenSSL_1_0_2t</code>，然后：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> /tmp/openssl-OpenSSL_1_0_2t</span><br><span class="line">$ ./Configure darwin64-x86_64-cc --prefix=/tmp/local -shared</span><br><span class="line">$ make &amp;&amp; make install</span><br></pre></td></tr></table></figure><h3 id="编译tcl-tk"><a href="#编译tcl-tk" class="headerlink" title="编译tcl/tk"></a>编译tcl/tk</h3><p>下载<a href="https://sourceforge.net/projects/tcl/files/Tcl/8.5.19/tcl8519-src.zip/download" target="_blank" rel="noopener">tcl8519-src.zip</a>，解压到<code>/tmp/tk8519-src</code>；下载<a href="https://sourceforge.net/projects/tcl/files/Tcl/8.5.19/tk8519-src.zip/download" target="_blank" rel="noopener">tk8519-src.zip</a>，解压到<code>/tmp/tk8519-src</code>，然后：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> /tmp/tcl8519-src</span><br><span class="line">$ <span class="built_in">cd</span> unix/</span><br><span class="line">$ ./configure --prefix=/tmp/local</span><br><span class="line">$ make &amp;&amp; make install</span><br><span class="line">$ <span class="built_in">cd</span> /tmp/tk8519-src</span><br><span class="line">$ <span class="built_in">cd</span> unix/</span><br><span class="line">$ ./configure --prefix=/tmp/local --with-tcl=/tmp/tcl8519-src/unix --enable-aqua</span><br><span class="line">$ make &amp;&amp; make install</span><br></pre></td></tr></table></figure><h3 id="编译python"><a href="#编译python" class="headerlink" title="编译python"></a>编译python</h3><p>下载<a href="https://www.python.org/ftp/python/3.6.5/Python-3.6.5.tgz" target="_blank" rel="noopener">Python-3.6.5</a>，解压到<code>/tmp/Python-3.6.5</code>， 然后：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> /tmp</span><br><span class="line">$ <span class="built_in">mkdir</span> python</span><br><span class="line">$ <span class="built_in">cd</span> Python-<span class="number">3</span>.<span class="number">6</span>.<span class="number">5</span></span><br><span class="line">$ CPPFLAGS="-I/tmp/local/include" \</span><br><span class="line">LDFLAGS="-L/tmp/local/lib" \</span><br><span class="line">./configure --prefix=/tmp/python </span><br><span class="line">$ make &amp;&amp; make install</span><br></pre></td></tr></table></figure><h3 id="修改依赖路径"><a href="#修改依赖路径" class="headerlink" title="修改依赖路径"></a>修改依赖路径</h3><p>以上python 安装在<code>/tmp/python</code>，把python 依赖的第三方动态库，复制到<code>/tmp/python/lib</code>：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> /tmp/python/lib</span><br><span class="line">$ cp /tmp/local/lib/liblzma.<span class="number">5</span>.dylib ./</span><br><span class="line">$ cp /tmp/local/lib/libcrypto.<span class="number">1</span>.<span class="number">0</span>.<span class="number">0</span>.dylib ./</span><br><span class="line">$ cp /tmp/local/lib/libssl.<span class="number">1</span>.<span class="number">0</span>.<span class="number">0</span>.dylib ./</span><br><span class="line">$ cp /tmp/local/lib/libtcl8.<span class="number">5</span>.dylib ./</span><br><span class="line">$ cp /tmp/local/lib/libtk8.<span class="number">5</span>.dylib ./</span><br><span class="line">$ cp -R /tmp/local/lib/tcl8 ./</span><br><span class="line">$ cp -R /tmp/local/lib/tcl8.<span class="number">5</span> ./</span><br><span class="line">$ cp -R /tmp/local/lib/tk8.<span class="number">5</span> ./</span><br></pre></td></tr></table></figure><p>macOS下，<code>otool</code>工具可以查看动态库的依赖路径，<code>intall_name_tool</code>工具可以修改动态库的依赖路径。<br>我们要把所有的外部依赖（除了系统依赖）都修改为<code>/tmp/python/lib</code>内部依赖，绝对的依赖路径也改为相对的依赖路径：</p><h4 id="修改libssl-1-0-0-dylib："><a href="#修改libssl-1-0-0-dylib：" class="headerlink" title="修改libssl.1.0.0.dylib："></a>修改<code>libssl.1.0.0.dylib</code>：</h4><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> /tmp/python/lib</span><br><span class="line">$ otool -L libssl.<span class="number">1</span>.<span class="number">0</span>.<span class="number">0</span>.dylib</span><br><span class="line">$ sudo install_name_tool -change /tmp/local/lib/libcrypto.<span class="number">1</span>.<span class="number">0</span>.<span class="number">0</span>.dylib @executable_path/../lib/libcrypto.<span class="number">1</span>.<span class="number">0</span>.<span class="number">0</span>.dylib libssl.<span class="number">1</span>.<span class="number">0</span>.<span class="number">0</span>.dylib</span><br></pre></td></tr></table></figure><h4 id="修改-lzma-cpython-36m-darwin-so："><a href="#修改-lzma-cpython-36m-darwin-so：" class="headerlink" title="修改_lzma.cpython-36m-darwin.so："></a>修改<code>_lzma.cpython-36m-darwin.so</code>：</h4><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> /tmp/python/lib/python3.<span class="number">6</span>/lib-dynload</span><br><span class="line">$ otool -L _lzma.cpython-<span class="number">36</span>m-darwin.so</span><br><span class="line">$ install_name_tool -change /tmp/local/lib/liblzma.<span class="number">5</span>.dylib @executable_path/../lib/liblzma.<span class="number">5</span>.dylib _lzma.cpython-<span class="number">36</span>m-darwin.so</span><br></pre></td></tr></table></figure><h4 id="修改-ssl-cpython-36m-darwin-so："><a href="#修改-ssl-cpython-36m-darwin-so：" class="headerlink" title="修改_ssl.cpython-36m-darwin.so："></a>修改<code>_ssl.cpython-36m-darwin.so</code>：</h4><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> /tmp/python/lib/python3.<span class="number">6</span>/lib-dynload</span><br><span class="line">$ otool -L libssl.<span class="number">1</span>.<span class="number">0</span>.<span class="number">0</span>.dylib</span><br><span class="line">$ install_name_tool -change /tmp/local/lib/libssl.<span class="number">1</span>.<span class="number">0</span>.<span class="number">0</span>.dylib @executable_path/../lib/libssl.<span class="number">1</span>.<span class="number">0</span>.<span class="number">0</span>.dylib -change /tmp/local/lib/libcrypto.<span class="number">1</span>.<span class="number">0</span>.<span class="number">0</span>.dylib @executable_path/../lib/libcrypto.<span class="number">1</span>.<span class="number">0</span>.<span class="number">0</span>.dylib _ssl.cpython-<span class="number">36</span>m-darwin.so</span><br></pre></td></tr></table></figure><h4 id="修改-tkinter-cpython-36m-darwin-so："><a href="#修改-tkinter-cpython-36m-darwin-so：" class="headerlink" title="修改_tkinter.cpython-36m-darwin.so："></a>修改<code>_tkinter.cpython-36m-darwin.so</code>：</h4><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> /tmp/python/lib/python3.<span class="number">6</span>/lib-dynload</span><br><span class="line">$ otool -L libssl.<span class="number">1</span>.<span class="number">0</span>.<span class="number">0</span>.dylib</span><br><span class="line">$ install_name_tool -change /System/Library/Frameworks/Tcl.framework/Versions/<span class="number">8</span>.<span class="number">5</span>/Tcl @executable_path/../lib/libtcl8.<span class="number">5</span>.dylib -change /System/Library/Frameworks/Tk.framework/Versions/<span class="number">8</span>.<span class="number">5</span>/Tk @executable_path/../lib/libtk8.<span class="number">5</span>.dylib _tkinter.cpython-<span class="number">36</span>m-darwin.so</span><br></pre></td></tr></table></figure><h4 id="测试"><a href="#测试" class="headerlink" title="测试"></a>测试</h4><p>测试修改依赖路径后的python运行情况：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> /tmp/python</span><br><span class="line">$ ./bin/python -m pip lmza</span><br><span class="line">$ ./bin/python -m pip ssl</span><br><span class="line">$ ./bin/python -m pip tkinter</span><br></pre></td></tr></table></figure><h3 id="升级pip"><a href="#升级pip" class="headerlink" title="升级pip"></a>升级pip</h3><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> /tmp/python</span><br><span class="line">$ ./bin/python -m pip install --upgrade pip</span><br></pre></td></tr></table></figure><h3 id="最后"><a href="#最后" class="headerlink" title="最后"></a>最后</h3><p>我们得到了一个Portable Pyton <code>/tmp/python</code>，可以内嵌到其他程序中一起打包，分发到各个计算机运行。</p><h2 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h2><ul><li><a href="https://www.tcl.tk/doc/howto/compile.html" target="_blank" rel="noopener">How to Compile Tcl</a></li><li><a href="https://devguide.python.org/setup/" target="_blank" rel="noopener">Getting Started — Python Developer’s Guide </a></li><li><a href="https://tkdocs.com/tutorial/install.html" target="_blank" rel="noopener">TkDocs - Tk Tutorial - Installing Tk</a></li><li><a href="https://bitbucket.org/snippets/Zifix/88ny/" target="_blank" rel="noopener">Portable OpenSSL dylib on Mac OS X</a></li></ul>]]></content>
    
    <summary type="html">
    
      &lt;blockquote&gt;
&lt;p&gt;Portable的意思是可拔插的，便携式的。Portable 程序在一台计算机上编译好后，移植到其他计算机上就可以直接运行。一般程序运行时都会依赖一些第三方库，只要把对这些库的依赖方式由静态依赖改成动态依赖，再把这些库和主体程序放到一起打包，就成为一个Portable 程序。&lt;br&gt;当然Portable 也是有限度的，不能跨操作系统移植，毕竟程序对操作系统的依赖关联太多，除非把整个系统也移植了😂。&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;以&lt;a href=&quot;https://www.python.org/downloads/release/python-365/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;Python-3.6.5&lt;/a&gt; 为例，讲述一下源码编译一个Portable Pyton 的主要流程。&lt;/p&gt;
    
    </summary>
    
      <category term="实践" scheme="https://breeze2.github.io/blog/categories/%E5%AE%9E%E8%B7%B5/"/>
    
    
      <category term="practice" scheme="https://breeze2.github.io/blog/tags/practice/"/>
    
      <category term="python" scheme="https://breeze2.github.io/blog/tags/python/"/>
    
  </entry>
  
  <entry>
    <title>开发一个React + Electron应用</title>
    <link href="https://breeze2.github.io/blog/practice-develop-an-electron-app-with-react.html"/>
    <id>https://breeze2.github.io/blog/practice-develop-an-electron-app-with-react.html</id>
    <published>2019-02-16T10:50:06.000Z</published>
    <updated>2019-09-30T02:22:37.082Z</updated>
    
    <content type="html"><![CDATA[<p>最近用React + Electron开发了一个RSS阅读器，开源在：<a href="https://github.com/breeze2/breader" target="_blank" rel="noopener">https://github.com/breeze2/breader</a>，这里记录一下大致的开发过程。</p><p><img src="https://breeze2.github.io/breader/screenshot.jpg" alt="Breader"></p><a id="more"></a><h2 id="初始化"><a href="#初始化" class="headerlink" title="初始化"></a>初始化</h2><h3 id="创建项目"><a href="#创建项目" class="headerlink" title="创建项目"></a>创建项目</h3><p>以普通的React应用做基础，一步步初始化项目。预先安装<a href="https://yarnpkg.com/" target="_blank" rel="noopener">yarn</a>工具，用<code>yarn</code>来创建一个React应用项目，假设名字叫<code>demo</code>，再引入Electron依赖。</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> /<span class="built_in">PATH</span>/TO/PROJECTS</span><br><span class="line">$ yarn create react-app YOUR_APP_NAME</span><br><span class="line">$ <span class="built_in">cd</span> /<span class="built_in">PATH</span>/TO/PROJECTS/YOUR_APP_NAME</span><br><span class="line">$ yarn add electron --dev</span><br></pre></td></tr></table></figure><h3 id="配置入口文件"><a href="#配置入口文件" class="headerlink" title="配置入口文件"></a>配置入口文件</h3><p>创建项目后，大致的目录结构如下：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">|--&gt;demo</span><br><span class="line">    |--&gt;node_modules</span><br><span class="line">        |--...</span><br><span class="line">    |--&gt;public</span><br><span class="line">        |--index.html</span><br><span class="line">        |--...</span><br><span class="line">    |--&gt;src</span><br><span class="line">    |--package.json</span><br><span class="line">    |--...</span><br></pre></td></tr></table></figure><p>一般来说，React应用（测试环境下）的入口文件就是<code>public/index.html</code>，而Electron应用的入口文件最好也放在<code>public</code>目录下，并命名为<code>electron.js</code>（这样<a href="https://github.com/electron-userland/electron-builder" target="_blank" rel="noopener">electron-builder</a>可以自动识别）。</p><p>先在<code>package.json</code>中lectron应用的入口文件，添加<code>main</code>配置：</p><figure class="highlight"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">&#123;</span><br><span class="line">  ...</span><br><span class="line">  "main": "public/electron.js",</span><br><span class="line">  ...</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p><code>public/electron.js</code>代码：</p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">const</span> &#123; app, BrowserWindow &#125; = <span class="built_in">require</span>(<span class="string">'electron'</span>)</span><br><span class="line"><span class="keyword">const</span> path = <span class="built_in">require</span>(<span class="string">'path'</span>)</span><br><span class="line"></span><br><span class="line"><span class="keyword">let</span> mainWindow</span><br><span class="line"></span><br><span class="line"><span class="function"><span class="keyword">function</span> <span class="title">createWindow</span>(<span class="params"></span>) </span>&#123;</span><br><span class="line">    mainWindow = <span class="keyword">new</span> BrowserWindow(&#123;</span><br><span class="line">        width: <span class="number">960</span>,</span><br><span class="line">        height: <span class="number">600</span>,</span><br><span class="line">    &#125;)</span><br><span class="line">    mainWindow.loadFile(<span class="string">'index.html'</span>)</span><br><span class="line">    mainWindow.webContents.openDevTools()</span><br><span class="line">    mainWindow.on(<span class="string">'closed'</span>, <span class="function"><span class="keyword">function</span> (<span class="params"></span>) </span>&#123;</span><br><span class="line">        mainWindow = <span class="literal">null</span></span><br><span class="line">    &#125;)</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">app.on(<span class="string">'ready'</span>, createWindow)</span><br><span class="line"></span><br><span class="line">app.on(<span class="string">'window-all-closed'</span>, <span class="function"><span class="keyword">function</span> (<span class="params"></span>) </span>&#123;</span><br><span class="line">    <span class="keyword">if</span> (process.platform !== <span class="string">'darwin'</span>) &#123;</span><br><span class="line">        app.quit()</span><br><span class="line">    &#125;</span><br><span class="line">&#125;)</span><br><span class="line"></span><br><span class="line">app.on(<span class="string">'activate'</span>, <span class="function"><span class="keyword">function</span> (<span class="params"></span>) </span>&#123;</span><br><span class="line">    <span class="keyword">if</span> (mainWindow === <span class="literal">null</span>) &#123;</span><br><span class="line">        createWindow()</span><br><span class="line">    &#125;</span><br><span class="line">&#125;)</span><br></pre></td></tr></table></figure><p>在执行执行<code>yarn electron ./</code>命令就可以启动Electron应用了，不过React应用还没启动起来。</p><h3 id="启动应用"><a href="#启动应用" class="headerlink" title="启动应用"></a>启动应用</h3><p>一般用<code>yarn react-scripts start</code>命令，就可以将React应用挂载在本地3000端口上（ <a href="http://localhost:3000" target="_blank" rel="noopener">http://localhost:3000</a> ），用于开发调试。要用React结合Electron一起开发调试，先安装<a href="https://github.com/sindresorhus/electron-is-dev" target="_blank" rel="noopener">electron-is-dev</a>来识别当前是否开发环境。</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ yarn add electron-is-dev</span><br></pre></td></tr></table></figure><p>一般开发环境是加载<code>http://localhost:3000/</code>，正式环境是加载<code>../build/index.html</code>文件，所以修改<code>public/electron.js</code>代码（<code>createWindow</code>函数内）：</p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">const</span> isDev = <span class="built_in">require</span>(<span class="string">'electron-is-dev'</span>)</span><br><span class="line"></span><br><span class="line">...</span><br><span class="line">--    <span class="comment">// mainWindow.loadFile('index.html')</span></span><br><span class="line">++    <span class="keyword">if</span> (isDev) &#123;</span><br><span class="line">++        mainWindow.loadURL(<span class="string">'http://localhost:3000/'</span>)</span><br><span class="line">++    &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">++        mainWindow.loadFile(path.join(__dirname, <span class="string">'/../build/index.html'</span>))</span><br><span class="line">++    &#125;</span><br><span class="line">...</span><br></pre></td></tr></table></figure><p>然后，在项目目录下，打开两个终端，一个执行<code>yarn react-scripts start</code>，一个执行<code>yarn electron ./</code>，就可以开发调试React + Electron应用了。<br>不用羡慕<a href="https://github.com/SimulatedGREG/electron-vue" target="_blank" rel="noopener">electron-vue</a>，可以一句命令可以直接启动，其实原理是一样的。<br>要实现一句命令启动也不难，只要把两句命令合到一起执行就可以了。先安装工具库：</p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ yarn add concurrently --dev</span><br><span class="line">$ yarn add wait-on --dev</span><br></pre></td></tr></table></figure><p>修改<code>package.json</code>，添加一个<code>electron-dev</code>脚本命令：</p><figure class="highlight"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">&#123;</span><br><span class="line">  ...</span><br><span class="line">  "scripts": &#123;</span><br><span class="line">    ...</span><br><span class="line">    "electron-dev": "concurrently \"BROWSER=none react-scripts start\" \"wait-on http://localhost:3000 &amp;&amp; electron .\"",</span><br><span class="line">    ...</span><br><span class="line">  &#125;</span><br><span class="line">  ...</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>这样，执行一句<code>yarn electron-dev</code>就可以启动React + Electron应用了。</p><h2 id="JS运行时"><a href="#JS运行时" class="headerlink" title="JS运行时"></a>JS运行时</h2><p>一般来说，浏览器提供一种JS运行时，NodeJS提供另一种JS运行时，Electron则是将这两种运行时结合到一起来提供。不过，这两种运行时并不是完美地和睦相处，比如说使用<a href="https://webpack.js.org/" target="_blank" rel="noopener">webpack</a>时，通过webpack打包的JS代码中，不能直接通过<code>import</code>关键字或者<code>require</code>函数来引入NodeJS提供的功能接口，因为webpack覆盖了NodeJS自带的<code>require</code>函数。<br>不过Electron中，<code>window</code>对象的<code>require</code>属性会映射到NodeJS自带的<code>require</code>函数上，比如要调用NodeJS提供的<a href="https://nodejs.org/dist/latest-v10.x/docs/api/http.html" target="_blank" rel="noopener">http</a>接口，可以这样写：</p><figure class="highlight js"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">const</span> http = <span class="built_in">window</span>.require(<span class="string">'http'</span>)</span><br></pre></td></tr></table></figure><h2 id="重新编译NodeJS扩展"><a href="#重新编译NodeJS扩展" class="headerlink" title="重新编译NodeJS扩展"></a>重新编译NodeJS扩展</h2><p>通常，纯JS代码实现的工具库都可以在Electron环境中运行。不过有些NodeJS的工具库并不是纯JS代码实现的，比如<a href="https://github.com/mapbox/node-sqlite3" target="_blank" rel="noopener">node-sqlite3</a>。<code>node-sqlite3</code>是C++编写，算是NodeJS的扩展，而不是单纯的工具库。<code>node-sqlite3</code>需要针对Electron环境重新编译，才能在Electron环境中运行。</p><h3 id="electron-rebuild"><a href="#electron-rebuild" class="headerlink" title="electron-rebuild"></a>electron-rebuild</h3><p><a href="https://github.com/electron/electron-rebuild" target="_blank" rel="noopener">electron-rebuild</a>是专门Electron环境针对重新编译NodeJS扩展的一个工具。<br>在项目目录下，安装electron-rebuild：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ yarn add electron-rebuild --dev</span><br></pre></td></tr></table></figure><p>比如，重新编译node-sqlite3，只需要：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ yarn electron-rebuild -f -w sqlite3</span><br></pre></td></tr></table></figure><p>在MacOS系统上就是这么简单，在Windows系统就复杂一些。</p><h3 id="Windows系统上重新编译NodeJS扩展"><a href="#Windows系统上重新编译NodeJS扩展" class="headerlink" title="Windows系统上重新编译NodeJS扩展"></a>Windows系统上重新编译NodeJS扩展</h3><p>在Windows系统上也一样可以使用electron-rebuild工具，但必须预先配置好编译环境。</p><p>首先，安装window编译工具：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ npm install --global --production windows-build-tools</span><br><span class="line">$ npm config <span class="built_in">set</span> msvs_version <span class="number">2017</span></span><br></pre></td></tr></table></figure><p>然后，下载并安装<a href="https://www.python.org/downloads/release/python-2715/" target="_blank" rel="noopener">Python2</a>，必须Python2不能Python3，如果同时安装了Python2和Python3，须给<code>npm</code>指定Python2可执行文件路径，避免调用了Python3：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ npm config <span class="built_in">set</span> python /<span class="built_in">path</span>/to/executable/python2</span><br></pre></td></tr></table></figure><p>这样，才能在Windows系统上调用electron-rebuild。如果electron-rebuild执行过程中，遇上<code>CONNECT ERROR</code>可能是网络问题，可以换成淘宝源再试试。</p><p>所以尽量少用NodeJS扩展，可以免去跨平台时重新编译的麻烦。</p><h2 id="应用打包"><a href="#应用打包" class="headerlink" title="应用打包"></a>应用打包</h2><p>应用开发完成后，需要打包成<code>dmg</code>或<code>exe</code>等安装文件，可以使用<a href="https://www.electron.build/" target="_blank" rel="noopener">electron-builder</a></p><h3 id="electron-builder"><a href="#electron-builder" class="headerlink" title="electron-builder"></a>electron-builder</h3><p>electron-builder有很多配置选项，来调用各式各样的程序打包。这里只简单介绍一下MacOS系统安装程序和Windows系统NSIS安装程序的打包配置（<a href="https://sourceforge.net/projects/nsis/" target="_blank" rel="noopener">NSIS</a>是一个开源的 Windows 系统下安装程序制作工具）。</p><p>修改<code>package.json</code>，添加<code>build</code>配置：</p><figure class="highlight"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br></pre></td><td class="code"><pre><span class="line">&#123;</span><br><span class="line">  ...</span><br><span class="line">  "homepage": "./", // 因为最后React应用引用的JS、CSS等资源都是本地的，只要用当前的相对路径即可</span><br><span class="line">  "build": &#123;</span><br><span class="line">    "appId": "com.xxx.app", // 应用ID</span><br><span class="line">    "npmRebuild": true, // 打包前是否重新编译NodeJS扩展</span><br><span class="line">    "mac": &#123;</span><br><span class="line">      "category": "news", // 应用分类</span><br><span class="line">      "icon": "build/icon.png" // 应用icon路径</span><br><span class="line">    &#125;,</span><br><span class="line">    "win": &#123;</span><br><span class="line">      "icon": "build/icon.png", // 应用icon路径</span><br><span class="line">      "target": "nsis" // Windows安装文件的目标类型</span><br><span class="line">    &#125;,</span><br><span class="line">    "nsis": &#123;</span><br><span class="line">      "allowToChangeInstallationDirectory": true, // 是否允许修改安装路径</span><br><span class="line">      "allowElevation": false, // 是否允许提升权限</span><br><span class="line">      "createDesktopShortcut": true, // 是否创建桌面快捷方式</span><br><span class="line">      "menuCategory": true, //是否在菜单栏创建分类</span><br><span class="line">      "oneClick": false // 是否一键安装</span><br><span class="line">    &#125;,</span><br><span class="line">    "files": [</span><br><span class="line">      "build/**/*" // 引入的文件</span><br><span class="line">    ]</span><br><span class="line">  &#125;</span><br><span class="line">  ...</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>然后，就可以打包Electron应用了：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ yarn react-scripts build // 先用webpack打包React应用到`build`目录下</span><br><span class="line">$ yarn eletron-builder // 再用eletron-builder打包Electron应用</span><br></pre></td></tr></table></figure><p>当然，正式打包还需要<a href="https://electronjs.org/docs/tutorial/code-signing" target="_blank" rel="noopener">代码签名</a>，还有更多配置，请查看<a href="https://www.electron.build/configuration" target="_blank" rel="noopener">electron-builder配置说明</a>。</p><h2 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h2><ul><li><a href="https://medium.com/@kitze/%EF%B8%8F-from-react-to-an-electron-app-ready-for-production-a0468ecb1da3" target="_blank" rel="noopener">From React to an Electron app ready for production</a></li></ul>]]></content>
    
    <summary type="html">
    
      &lt;p&gt;最近用React + Electron开发了一个RSS阅读器，开源在：&lt;a href=&quot;https://github.com/breeze2/breader&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;https://github.com/breeze2/breader&lt;/a&gt;，这里记录一下大致的开发过程。&lt;/p&gt;
&lt;p&gt;&lt;img src=&quot;https://breeze2.github.io/breader/screenshot.jpg&quot; alt=&quot;Breader&quot;&gt;&lt;/p&gt;
    
    </summary>
    
      <category term="实践" scheme="https://breeze2.github.io/blog/categories/%E5%AE%9E%E8%B7%B5/"/>
    
    
      <category term="practice" scheme="https://breeze2.github.io/blog/tags/practice/"/>
    
      <category term="react" scheme="https://breeze2.github.io/blog/tags/react/"/>
    
      <category term="electron" scheme="https://breeze2.github.io/blog/tags/electron/"/>
    
  </entry>
  
  <entry>
    <title>PHP代码静态分析工具PHPStan</title>
    <link href="https://breeze2.github.io/blog/practice-php-static-analysis-tool-phpstan.html"/>
    <id>https://breeze2.github.io/blog/practice-php-static-analysis-tool-phpstan.html</id>
    <published>2018-08-08T17:31:23.000Z</published>
    <updated>2019-09-30T02:22:37.086Z</updated>
    
    <content type="html"><![CDATA[<blockquote><p>最近发现自己写的PHP代码运行结果总跟自己预想的不一样，排查时发现大多是语法错误，在运行之前错误已经种下。可能是自己粗心大意，或者说<code>php -l</code>检测太简单，不过的确是有一些语法错误埋藏得太深（毕竟PHP是动态语言），那么有没有办法，在代码代码正式运行之前，把语法错误全找出来呢？</p></blockquote><p>这里介绍一款PHP代码静态分析工具：<a href="https://github.com/phpstan/phpstan" target="_blank" rel="noopener">PHPStan</a>，不需要运行代码，也可以对代码进行严格的语法检测，尽量将代码运行错误率降到最低。</p><h2 id="PHPStan"><a href="#PHPStan" class="headerlink" title="PHPStan"></a>PHPStan</h2><h3 id="安装"><a href="#安装" class="headerlink" title="安装"></a>安装</h3><p>目前，PHPStan<code>V0.10.2</code>要求系统环境的PHP版本不低于7.1。用Composer全局安装：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ composer global require phpstan/phpstan</span><br></pre></td></tr></table></figure><a id="more"></a><h3 id="使用"><a href="#使用" class="headerlink" title="使用"></a>使用</h3><p>PHPStan静态分析的使用方法十分简单：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ phpstan analyse [-c|--configuration CONFIGURATION] [-l|--level LEVEL] [--no-progress] [--debug] [-a|--<span class="built_in">autoload</span>-file AUTOLOAD-FILE] [--errorFormat ERRORFORMAT] [--memory-limit MEMORY-LIMIT] [--] [&lt;paths&gt;]...</span><br></pre></td></tr></table></figure><ul><li>configuration：运行配置文件的路径；</li><li>level：严格级别，0-7，越大越严格；</li><li>no-progress：不显示进度；</li><li>debug：debug模式；</li><li>autoload-file：自动加载文件的路径；</li><li>errorFormat：错误格式；</li><li>memory-limit：内存限制；</li><li>paths：待分析的文件路径。</li></ul><p>比如，分析一个PHP文件：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ phpstan analyse --level=7 --<span class="built_in">autoload</span>-file=/PATH/TO/vendor/autoload.php /PATH/TO/someone.php</span><br></pre></td></tr></table></figure><h2 id="PHPStan-in-VSCode"><a href="#PHPStan-in-VSCode" class="headerlink" title="PHPStan in VSCode"></a>PHPStan in VSCode</h2><p>当然，语法分析应该是编辑器做的事，写完代码还要切换到命令终端执行<code>phpstan</code>，未免过于繁琐。所以这里推荐一款VSCode扩展：<a href="https://marketplace.visualstudio.com/items?itemName=breezelin.phpstan" target="_blank" rel="noopener">PHP Static Analysis</a>。</p><p><img src="/blog/assets/images/practice-php-static-analysis-tool-phpstan1.jpg" alt="PHP Static Analysis"></p><p>首先，用Composer全局安装PHPStan；然后，在VSCode的扩展管理中搜索<code>PHP Static Analysis</code>，安装第一个匹配的扩展；重载VSCode重载窗口后，扩展会自动分析VSCode下打开的PHP文件。</p><p>运行效果：<br><img src="/blog/assets/images/practice-php-static-analysis-tool-phpstan2.jpg" alt="运行效果"></p><p>比如，声明了一个变量未调用，调用了一个未声明的变量和调用了一个未定义的方法等等这样错误都会被检测出了。</p><p>不过，宽松一点地来说，其实<code>$this-&gt;array()</code>方法是存在的，只是通过魔术方法<code>__call()</code>实现的。</p><h2 id="PHPStan-with-Laravel"><a href="#PHPStan-with-Laravel" class="headerlink" title="PHPStan with Laravel"></a>PHPStan with Laravel</h2><p>高严格级别的PHPStan检测到调用未声明的类方法时，会报告类中方法不存在的错误，即使这个类定义了<code>__call()</code>或<code>__callStatic()</code>。</p><p>很多应用框架为了优雅，大量使用了魔术方法，比如<a href="https://laravel.com/" target="_blank" rel="noopener">Laravel</a>。<br>用PHPStan检测Laravel项目，自然会报告很多调用未声明类方法的错误，对于这个问题，可以借助<a href="https://github.com/barryvdh/laravel-ide-helper" target="_blank" rel="noopener">laravel-ide-helper</a>来降低误报。</p><h3 id="安装laravel-ide-helper"><a href="#安装laravel-ide-helper" class="headerlink" title="安装laravel-ide-helper"></a>安装laravel-ide-helper</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> /PATH/TO/LARAVEL_PROJECT</span><br><span class="line">$ composer require barryvdh/laravel-ide-helper</span><br></pre></td></tr></table></figure><h3 id="注入LaravelIdeHelper"><a href="#注入LaravelIdeHelper" class="headerlink" title="注入LaravelIdeHelper"></a>注入LaravelIdeHelper</h3><p>编辑<code>app/Providers/AppServiceProvider.php</code>里的注册方法：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line">    ...</span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">register</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="keyword">if</span> (<span class="keyword">$this</span>-&gt;app-&gt;environment() !== <span class="string">'production'</span>) &#123;</span><br><span class="line">            <span class="keyword">$this</span>-&gt;app-&gt;register(\Barryvdh\LaravelIdeHelper\IdeHelperServiceProvider::class);</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="comment">// ...</span></span><br><span class="line">    &#125;</span><br></pre></td></tr></table></figure><h3 id="生成-ide-helper-php"><a href="#生成-ide-helper-php" class="headerlink" title="生成_ide_helper.php"></a>生成_ide_helper.php</h3><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> /PATH/TO/LARAVEL_PROJECT</span><br><span class="line">$ php artisan ide-helper:generate</span><br></pre></td></tr></table></figure><p>这时，Laravel框架中的Facade类，原本通过<code>__callStatic()</code>获取的静态方法，全部在_ide_helper.php声明了，在PHPStan检测Laravel项目代码时引入<code>_ide_helper.php</code>文件，就可以减少误报。</p><h3 id="PHPStan配置"><a href="#PHPStan配置" class="headerlink" title="PHPStan配置"></a>PHPStan配置</h3><p>在Laravel项目的根目录下，新建<code>phpstan.neon</code>文件：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">parameters:</span><br><span class="line">    autoload_files:</span><br><span class="line">        - %currentWorkingDirectory%/_ide_helper.php</span><br></pre></td></tr></table></figure><p>在Laravel项目的根目录下，执行<code>phpstan</code>命令时，会自动使用<code>phpstan.neon</code>这个配置。</p><h2 id="最后"><a href="#最后" class="headerlink" title="最后"></a>最后</h2><p>代码的语法错误，应该在编写的时候及时发现，尽量减少正式运行时错误。</p>]]></content>
    
    <summary type="html">
    
      &lt;blockquote&gt;
&lt;p&gt;最近发现自己写的PHP代码运行结果总跟自己预想的不一样，排查时发现大多是语法错误，在运行之前错误已经种下。可能是自己粗心大意，或者说&lt;code&gt;php -l&lt;/code&gt;检测太简单，不过的确是有一些语法错误埋藏得太深（毕竟PHP是动态语言），那么有没有办法，在代码代码正式运行之前，把语法错误全找出来呢？&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;这里介绍一款PHP代码静态分析工具：&lt;a href=&quot;https://github.com/phpstan/phpstan&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;PHPStan&lt;/a&gt;，不需要运行代码，也可以对代码进行严格的语法检测，尽量将代码运行错误率降到最低。&lt;/p&gt;
&lt;h2 id=&quot;PHPStan&quot;&gt;&lt;a href=&quot;#PHPStan&quot; class=&quot;headerlink&quot; title=&quot;PHPStan&quot;&gt;&lt;/a&gt;PHPStan&lt;/h2&gt;&lt;h3 id=&quot;安装&quot;&gt;&lt;a href=&quot;#安装&quot; class=&quot;headerlink&quot; title=&quot;安装&quot;&gt;&lt;/a&gt;安装&lt;/h3&gt;&lt;p&gt;目前，PHPStan&lt;code&gt;V0.10.2&lt;/code&gt;要求系统环境的PHP版本不低于7.1。用Composer全局安装：&lt;/p&gt;
&lt;figure class=&quot;highlight bash&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;$ composer global require phpstan/phpstan&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;
    
    </summary>
    
      <category term="实践" scheme="https://breeze2.github.io/blog/categories/%E5%AE%9E%E8%B7%B5/"/>
    
    
      <category term="php" scheme="https://breeze2.github.io/blog/tags/php/"/>
    
      <category term="practice" scheme="https://breeze2.github.io/blog/tags/practice/"/>
    
  </entry>
  
  <entry>
    <title>用Swoole实现一个简单的MySQL连接池</title>
    <link href="https://breeze2.github.io/blog/swoole-a-simple-mysql-connection-pool.html"/>
    <id>https://breeze2.github.io/blog/swoole-a-simple-mysql-connection-pool.html</id>
    <published>2018-07-20T09:58:23.000Z</published>
    <updated>2019-09-30T02:22:37.086Z</updated>
    
    <content type="html"><![CDATA[<blockquote><p>最近Swoole发布了4.0.2版本，内置协程更加完善，利用Swoole\Coroutine\Channel和Swoole\Coroutine\MySQL可以轻松实现一个MySQL连接池。</p></blockquote><a id="more"></a><h2 id="实现代码"><a href="#实现代码" class="headerlink" title="实现代码"></a>实现代码</h2><p>直接看代码，脚本<code>mysql_connection_pool.php</code>：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br><span class="line">102</span><br><span class="line">103</span><br><span class="line">104</span><br><span class="line">105</span><br><span class="line">106</span><br><span class="line">107</span><br><span class="line">108</span><br><span class="line">109</span><br><span class="line">110</span><br><span class="line">111</span><br><span class="line">112</span><br><span class="line">113</span><br><span class="line">114</span><br><span class="line">115</span><br><span class="line">116</span><br><span class="line">117</span><br><span class="line">118</span><br><span class="line">119</span><br><span class="line">120</span><br><span class="line">121</span><br><span class="line">122</span><br><span class="line">123</span><br><span class="line">124</span><br><span class="line">125</span><br><span class="line">126</span><br><span class="line">127</span><br><span class="line">128</span><br><span class="line">129</span><br><span class="line">130</span><br><span class="line">131</span><br><span class="line">132</span><br><span class="line">133</span><br><span class="line">134</span><br><span class="line">135</span><br><span class="line">136</span><br><span class="line">137</span><br><span class="line">138</span><br><span class="line">139</span><br><span class="line">140</span><br><span class="line">141</span><br><span class="line">142</span><br><span class="line">143</span><br><span class="line">144</span><br><span class="line">145</span><br><span class="line">146</span><br><span class="line">147</span><br><span class="line">148</span><br><span class="line">149</span><br><span class="line">150</span><br><span class="line">151</span><br><span class="line">152</span><br><span class="line">153</span><br><span class="line">154</span><br><span class="line">155</span><br><span class="line">156</span><br><span class="line">157</span><br><span class="line">158</span><br><span class="line">159</span><br><span class="line">160</span><br><span class="line">161</span><br><span class="line">162</span><br><span class="line">163</span><br><span class="line">164</span><br><span class="line">165</span><br><span class="line">166</span><br><span class="line">167</span><br><span class="line">168</span><br><span class="line">169</span><br><span class="line">170</span><br><span class="line">171</span><br><span class="line">172</span><br><span class="line">173</span><br><span class="line">174</span><br><span class="line">175</span><br><span class="line">176</span><br><span class="line">177</span><br><span class="line">178</span><br><span class="line">179</span><br><span class="line">180</span><br><span class="line">181</span><br><span class="line">182</span><br><span class="line">183</span><br><span class="line">184</span><br><span class="line">185</span><br><span class="line">186</span><br><span class="line">187</span><br><span class="line">188</span><br><span class="line">189</span><br><span class="line">190</span><br><span class="line">191</span><br><span class="line">192</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="comment">// mysql_connection_pool.php</span></span><br><span class="line"><span class="keyword">use</span> <span class="title">Swoole</span>\<span class="title">Coroutine</span> <span class="title">as</span> <span class="title">Co</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">Swoole</span>\<span class="title">Coroutine</span>\<span class="title">Channel</span> <span class="title">as</span> <span class="title">CoChannel</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">Swoole</span>\<span class="title">Coroutine</span>\<span class="title">MySQL</span> <span class="title">as</span> <span class="title">CoMySQL</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">Swoole</span>\<span class="title">Http</span>\<span class="title">Server</span> <span class="title">as</span> <span class="title">HttpServer</span>;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">MySqlCoroutine</span> <span class="keyword">extends</span> <span class="title">CoMySQL</span></span></span><br><span class="line"><span class="class"></span>&#123;</span><br><span class="line">    <span class="keyword">protected</span> $used_at = <span class="keyword">null</span>;</span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">getUsedAt</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="keyword">return</span> <span class="keyword">$this</span>-&gt;used_at;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">setUsedAt</span><span class="params">($time)</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="keyword">$this</span>-&gt;used_at = $time;</span><br><span class="line">    &#125;</span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">isConnected</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="keyword">return</span> <span class="keyword">$this</span>-&gt;connected;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">MySqlManager</span></span></span><br><span class="line"><span class="class"></span>&#123;</span><br><span class="line">    <span class="keyword">protected</span> $max_number;</span><br><span class="line">    <span class="keyword">protected</span> $min_number;</span><br><span class="line">    <span class="keyword">protected</span> $config;</span><br><span class="line">    <span class="keyword">protected</span> $channel;</span><br><span class="line">    <span class="keyword">protected</span> $number;</span><br><span class="line">    <span class="keyword">private</span> $is_recycling = <span class="keyword">false</span>;</span><br><span class="line"></span><br><span class="line">    <span class="comment">/**</span></span><br><span class="line"><span class="comment">     * [__construct 构造函数]</span></span><br><span class="line"><span class="comment">     * <span class="doctag">@param</span> array   $config         [MySQL服务器信息]</span></span><br><span class="line"><span class="comment">     * <span class="doctag">@param</span> integer $max_number     [最大连接数]</span></span><br><span class="line"><span class="comment">     * <span class="doctag">@param</span> integer $min_number     [最小连接数]</span></span><br><span class="line"><span class="comment">     */</span></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">__construct</span><span class="params">(array $config, $max_number = <span class="number">150</span>, $min_number = <span class="number">50</span>)</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="keyword">$this</span>-&gt;max_number = $max_number;</span><br><span class="line">        <span class="keyword">$this</span>-&gt;min_number = $min_number;</span><br><span class="line">        <span class="keyword">$this</span>-&gt;config     = $config;</span><br><span class="line">        <span class="comment">// $this-&gt;channel        = new CoChannel($max_number);</span></span><br><span class="line">        <span class="keyword">$this</span>-&gt;number = <span class="number">0</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">initChannel</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="keyword">$this</span>-&gt;channel = <span class="keyword">new</span> CoChannel(<span class="keyword">$this</span>-&gt;max_number);</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">private</span> <span class="function"><span class="keyword">function</span> <span class="title">isFull</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="keyword">return</span> <span class="keyword">$this</span>-&gt;number === <span class="keyword">$this</span>-&gt;max_number;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">private</span> <span class="function"><span class="keyword">function</span> <span class="title">isEmpty</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="keyword">return</span> <span class="keyword">$this</span>-&gt;number === <span class="number">0</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">private</span> <span class="function"><span class="keyword">function</span> <span class="title">shouldRecover</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="keyword">return</span> <span class="keyword">$this</span>-&gt;number &gt; <span class="keyword">$this</span>-&gt;min_number;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">private</span> <span class="function"><span class="keyword">function</span> <span class="title">increase</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="keyword">return</span> <span class="keyword">$this</span>-&gt;number += <span class="number">1</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">private</span> <span class="function"><span class="keyword">function</span> <span class="title">decrease</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="keyword">return</span> <span class="keyword">$this</span>-&gt;number -= <span class="number">1</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">protected</span> <span class="function"><span class="keyword">function</span> <span class="title">build</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="keyword">if</span> (!<span class="keyword">$this</span>-&gt;isFull()) &#123;</span><br><span class="line">            printf(<span class="string">"%d do %s\n"</span>, time(), <span class="string">'build one'</span>);</span><br><span class="line">            <span class="keyword">$this</span>-&gt;increase();</span><br><span class="line">            $mysql = <span class="keyword">new</span> MySqlCoroutine();</span><br><span class="line">            $mysql-&gt;connect(<span class="keyword">$this</span>-&gt;config);</span><br><span class="line">            $mysql-&gt;setUsedAt(time());</span><br><span class="line">            <span class="keyword">return</span> $mysql;</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="keyword">return</span> <span class="keyword">false</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">protected</span> <span class="function"><span class="keyword">function</span> <span class="title">rebuild</span><span class="params">(MySqlCoroutine $mysql)</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        printf(<span class="string">"%d do %s\n"</span>, time(), <span class="string">'rebuild one'</span>);</span><br><span class="line">        $mysql-&gt;connect(<span class="keyword">$this</span>-&gt;config);</span><br><span class="line">        $mysql-&gt;setUsedAt(time());</span><br><span class="line">        <span class="keyword">return</span> $mysql;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">protected</span> <span class="function"><span class="keyword">function</span> <span class="title">destroy</span><span class="params">(MySqlCoroutine $mysql)</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="keyword">if</span> (!<span class="keyword">$this</span>-&gt;isEmpty()) &#123;</span><br><span class="line">            printf(<span class="string">"%d do %s\n"</span>, time(), <span class="string">'destroy one'</span>);</span><br><span class="line">            <span class="keyword">$this</span>-&gt;decrease();</span><br><span class="line">            <span class="keyword">return</span> <span class="keyword">true</span>;</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="keyword">return</span> <span class="keyword">false</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">push</span><span class="params">(MySqlCoroutine $mysql)</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="keyword">if</span> (!<span class="keyword">$this</span>-&gt;channel-&gt;isFull()) &#123;</span><br><span class="line">            printf(<span class="string">"%d do %s\n"</span>, time(), <span class="string">'push one'</span>);</span><br><span class="line">            <span class="keyword">$this</span>-&gt;channel-&gt;push($mysql);</span><br><span class="line">            <span class="keyword">return</span>;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">pop</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="keyword">if</span> ($mysql = <span class="keyword">$this</span>-&gt;build()) &#123;</span><br><span class="line">            <span class="keyword">return</span> $mysql;</span><br><span class="line">        &#125;</span><br><span class="line">        $mysql = <span class="keyword">$this</span>-&gt;channel-&gt;pop();</span><br><span class="line">        $now   = time();</span><br><span class="line">        printf(<span class="string">"%d do %s\n"</span>, time(), <span class="string">'pop one'</span>);</span><br><span class="line">        <span class="keyword">if</span> (!$mysql-&gt;isConnected()) &#123;</span><br><span class="line">            <span class="keyword">return</span> <span class="keyword">$this</span>-&gt;rebuild($mysql);</span><br><span class="line">        &#125;</span><br><span class="line">        $mysql-&gt;setUsedAt($now);</span><br><span class="line">        <span class="keyword">return</span> $mysql;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="comment">/**</span></span><br><span class="line"><span class="comment">     * [autoRecycling 自动回收连接]</span></span><br><span class="line"><span class="comment">     * <span class="doctag">@param</span>  integer $timeout [连接空置时限]</span></span><br><span class="line"><span class="comment">     * <span class="doctag">@param</span>  integer $sleep   [循环检查的时间间隔]</span></span><br><span class="line"><span class="comment">     * <span class="doctag">@return</span> null             [null]</span></span><br><span class="line"><span class="comment">     */</span></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">autoRecycling</span><span class="params">($timeout = <span class="number">200</span>, $sleep = <span class="number">20</span>)</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="keyword">if</span> (!<span class="keyword">$this</span>-&gt;is_recycling) &#123;</span><br><span class="line">            <span class="keyword">$this</span>-&gt;is_recycling = <span class="keyword">true</span>;</span><br><span class="line">            <span class="keyword">while</span> (<span class="number">1</span>) &#123;</span><br><span class="line">                Co::sleep($sleep);</span><br><span class="line">                <span class="keyword">if</span> (<span class="keyword">$this</span>-&gt;shouldRecover()) &#123;</span><br><span class="line">                    $mysql = <span class="keyword">$this</span>-&gt;channel-&gt;pop();</span><br><span class="line">                    $now   = time();</span><br><span class="line">                    <span class="keyword">if</span> ($now - $mysql-&gt;getUsedAt() &gt; $timeout) &#123;</span><br><span class="line">                        printf(<span class="string">"%d do %s\n"</span>, time(), <span class="string">'recover one'</span>);</span><br><span class="line">                        <span class="keyword">$this</span>-&gt;decrease();</span><br><span class="line">                    &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">                        !<span class="keyword">$this</span>-&gt;channel-&gt;isFull() &amp;&amp; <span class="keyword">$this</span>-&gt;channel-&gt;push($mysql);</span><br><span class="line">                    &#125;</span><br><span class="line">                &#125;</span><br><span class="line">            &#125;</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">$server = <span class="keyword">new</span> HttpServer(<span class="string">'127.0.0.1'</span>, <span class="number">9501</span>, SWOOLE_BASE);</span><br><span class="line"></span><br><span class="line">$server-&gt;set([</span><br><span class="line">    <span class="string">'worker_num'</span> =&gt; <span class="number">1</span>,</span><br><span class="line"></span><br><span class="line">]);</span><br><span class="line"></span><br><span class="line">$manager = <span class="keyword">new</span> MySqlManager([</span><br><span class="line">    <span class="string">'host'</span>     =&gt; <span class="string">'127.0.0.1'</span>,</span><br><span class="line">    <span class="string">'port'</span>     =&gt; <span class="number">3306</span>,</span><br><span class="line">    <span class="string">'user'</span>     =&gt; <span class="string">'root'</span>,</span><br><span class="line">    <span class="string">'password'</span> =&gt; <span class="string">''</span>,</span><br><span class="line">    <span class="string">'database'</span> =&gt; <span class="string">'test'</span>,</span><br><span class="line">    <span class="string">'timeout'</span>  =&gt; <span class="number">-1</span>,</span><br><span class="line"></span><br><span class="line">], <span class="number">4</span>, <span class="number">2</span>);</span><br><span class="line"></span><br><span class="line">$server-&gt;on(<span class="string">'workerStart'</span>, <span class="function"><span class="keyword">function</span> <span class="params">($server)</span> <span class="title">use</span> <span class="params">($manager)</span> </span>&#123;</span><br><span class="line">    $manager-&gt;initChannel();</span><br><span class="line">    $manager-&gt;autoRecycling(<span class="number">4</span>, <span class="number">2</span>); <span class="comment">// 启动自动回收</span></span><br><span class="line">&#125;);</span><br><span class="line"></span><br><span class="line">$server-&gt;on(<span class="string">'request'</span>, <span class="function"><span class="keyword">function</span> <span class="params">($request, $response)</span> <span class="title">use</span> <span class="params">($server, $manager)</span> </span>&#123;</span><br><span class="line">    $mysql = $manager-&gt;pop(); <span class="comment">// 取出一个MySQL连接</span></span><br><span class="line">    $mysql-&gt;query(<span class="string">'select sleep(1);'</span>);</span><br><span class="line">    $manager-&gt;push($mysql); <span class="comment">// 返回一个MySQL连接</span></span><br><span class="line">    $response-&gt;end(json_encode($server-&gt;stats()));</span><br><span class="line"></span><br><span class="line">&#125;);</span><br><span class="line"></span><br><span class="line">$server-&gt;start();</span><br></pre></td></tr></table></figure><h2 id="运行效果"><a href="#运行效果" class="headerlink" title="运行效果"></a>运行效果</h2><p>运行脚本：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ php mysql_connection_pool.php</span><br></pre></td></tr></table></figure><p>ab测试：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ ab -c 8 -n 8 http://127.0.0.1:9501/</span><br></pre></td></tr></table></figure><p>间隔10秒后，再次测试：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ ab -c 8 -n 8 http://127.0.0.1:9501/</span><br></pre></td></tr></table></figure><p>输出结果：</p><figure class="highlight bash"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br></pre></td><td class="code"><pre><span class="line">1532083141 <span class="keyword">do</span> build one</span><br><span class="line">1532083141 <span class="keyword">do</span> build one</span><br><span class="line">1532083141 <span class="keyword">do</span> build one</span><br><span class="line">1532083141 <span class="keyword">do</span> build one</span><br><span class="line">1532083142 <span class="keyword">do</span> push one</span><br><span class="line">1532083142 <span class="keyword">do</span> push one</span><br><span class="line">1532083142 <span class="keyword">do</span> push one</span><br><span class="line">1532083142 <span class="keyword">do</span> pop one</span><br><span class="line">1532083142 <span class="keyword">do</span> pop one</span><br><span class="line">1532083142 <span class="keyword">do</span> pop one</span><br><span class="line">1532083142 <span class="keyword">do</span> push one</span><br><span class="line">1532083142 <span class="keyword">do</span> pop one</span><br><span class="line">1532083143 <span class="keyword">do</span> push one</span><br><span class="line">1532083143 <span class="keyword">do</span> push one</span><br><span class="line">1532083143 <span class="keyword">do</span> push one</span><br><span class="line">1532083143 <span class="keyword">do</span> push one</span><br><span class="line">1532083147 <span class="keyword">do</span> recover one</span><br><span class="line">1532083149 <span class="keyword">do</span> recover one</span><br><span class="line">1532083160 <span class="keyword">do</span> build one</span><br><span class="line">1532083160 <span class="keyword">do</span> build one</span><br><span class="line">1532083160 <span class="keyword">do</span> pop one</span><br><span class="line">1532083160 <span class="keyword">do</span> rebuild one</span><br><span class="line">1532083160 <span class="keyword">do</span> pop one</span><br><span class="line">1532083160 <span class="keyword">do</span> rebuild one</span><br><span class="line">1532083161 <span class="keyword">do</span> push one</span><br><span class="line">1532083161 <span class="keyword">do</span> pop one</span><br><span class="line">1532083161 <span class="keyword">do</span> push one</span><br><span class="line">1532083161 <span class="keyword">do</span> push one</span><br><span class="line">1532083161 <span class="keyword">do</span> push one</span><br><span class="line">1532083161 <span class="keyword">do</span> pop one</span><br><span class="line">1532083161 <span class="keyword">do</span> pop one</span><br><span class="line">1532083161 <span class="keyword">do</span> pop one</span><br><span class="line">1532083162 <span class="keyword">do</span> push one</span><br><span class="line">1532083162 <span class="keyword">do</span> push one</span><br><span class="line">1532083162 <span class="keyword">do</span> push one</span><br><span class="line">1532083162 <span class="keyword">do</span> push one</span><br><span class="line">1532083166 <span class="keyword">do</span> recover one</span><br><span class="line">1532083168 <span class="keyword">do</span> recover one</span><br></pre></td></tr></table></figure><h2 id="代码解释"><a href="#代码解释" class="headerlink" title="代码解释"></a>代码解释</h2><p>首先代码里定义了两个类，分别是：</p><ul><li>MySqlCoroutine</li><li>MySqlManager</li></ul><h3 id="MySqlCoroutine"><a href="#MySqlCoroutine" class="headerlink" title="MySqlCoroutine"></a>MySqlCoroutine</h3><p>MySqlCoroutine继承自<code>Swoole\Coroutine\MySQL</code>，主要是为了增加一个属性<code>used_at</code>和两个与之配套的<code>get/set</code>方法：<code>getUsedAt</code>和<code>setUsedAt</code>。<code>used_at</code>是个时间戳，记录MySqlCoroutine的被调用的时间点。而<code>isConnected</code>方法是判断与服务器的连接是否有效。</p><h3 id="MySqlManager"><a href="#MySqlManager" class="headerlink" title="MySqlManager"></a>MySqlManager</h3><p>每个MySqlCoroutine实例与MySQL服务器维系一个连接，而MySqlManager来管理这些MySqlCoroutine，即管理整个连接池。<br>构建MySqlManager时，用到三个参数：</p><ul><li><code>config</code>：用于构建MySqlCoroutine；</li><li><code>max_number</code>：最大连接数；</li><li><code>min_number</code>：最小连接数。</li></ul><p>其实这里说的连接池，就是MySqlManager的<code>mysql_channel</code>属性，实际的数据类型是<code>Swoole\Coroutine\Channel</code>。而池内连接的数量，会用<code>number</code>属性记录下来。</p><h4 id="内部方法：build-rebuild-destroy"><a href="#内部方法：build-rebuild-destroy" class="headerlink" title="内部方法：build/rebuild/destroy"></a>内部方法：build/rebuild/destroy</h4><ul><li>build方法：如果当前连接的数量未达到最大值，则新建一个MySqlCoroutine实例作返回值，并且<code>number</code>属性加1；否则，返回false。</li><li>rebuild方法：传入一个MySqlCoroutine实例，返回一个新的MySqlCoroutine实例。</li><li>destroy方法：传入一个MySqlCoroutine实例，如果<code>number</code>大于0，那么<code>number</code>减1。</li></ul><h4 id="公有方法：pop-push-autoRecycling"><a href="#公有方法：pop-push-autoRecycling" class="headerlink" title="公有方法：pop/push/autoRecycling"></a>公有方法：pop/push/autoRecycling</h4><ul><li>pop方法：先执行build方法，若是返回MySqlCoroutine实例，则直接返回这个MySqlCoroutine实例；否则，从<code>mysql_channel</code>里<code>pop</code>（如果<code>mysql_channel</code>已经空了，当前协程会被挂起，等待其他协程往<code>mysql_channel</code>放入MySqlCoroutine实例），若是取出的MySqlCoroutine实例的连接已无效（连接可能已经被MySQL服务器自动断开），那就返回rebuild方法的值，否则直接返回这个MySqlCoroutine实例。</li><li>push方法：如果<code>mysql_channel</code>未满，就往<code>mysql_channel</code>放入一个MySqlCoroutine；否则，什么也不做。</li><li>autoRecycling方法：接收两个参数，<code>$timeou</code>和<code>$sleep</code>；每隔<code>$sleep</code>秒，就检查一次，当前连接的数量超过了最小值，那么就从<code>mysql_channel</code>里<code>pop</code>一个MySqlCoroutine实例，若是MySqlCoroutine实例上一次使用时间与现在时间间隔超过了<code>$timeou</code>秒，说明这个MySqlCoroutine实例没有被频繁使用，可以回收；所谓回收，就是从<code>mysql_channel</code>取出后，不再放回，并且<code>number</code>减1。</li></ul><h3 id="调用"><a href="#调用" class="headerlink" title="调用"></a>调用</h3><p>在<code>Swoole\Http\Server</code>的<code>workerStart</code>回调函数里，让MySqlManager实例开始自动回收MySqlCoroutine实例；而在<code>Swoole\Http\Server</code>的<code>request</code>回调函数里，需要MySqlCoroutine实例时，调用MySqlManager实例的pop方法从连接池里获取，用完再调用MySqlManager实例的push方法放回连接池。</p><h2 id="最后"><a href="#最后" class="headerlink" title="最后"></a>最后</h2><p>就这样，利用Swoole扩展，几十行PHP代码就能实现一个连接间可以并行使用，池内有最大/最小数量限制，有自动回收空闲连接机制的MySQL连接池。</p>]]></content>
    
    <summary type="html">
    
      &lt;blockquote&gt;
&lt;p&gt;最近Swoole发布了4.0.2版本，内置协程更加完善，利用Swoole\Coroutine\Channel和Swoole\Coroutine\MySQL可以轻松实现一个MySQL连接池。&lt;/p&gt;
&lt;/blockquote&gt;
    
    </summary>
    
      <category term="实践" scheme="https://breeze2.github.io/blog/categories/%E5%AE%9E%E8%B7%B5/"/>
    
    
      <category term="mysql" scheme="https://breeze2.github.io/blog/tags/mysql/"/>
    
      <category term="php" scheme="https://breeze2.github.io/blog/tags/php/"/>
    
      <category term="practice" scheme="https://breeze2.github.io/blog/tags/practice/"/>
    
      <category term="swoole" scheme="https://breeze2.github.io/blog/tags/swoole/"/>
    
  </entry>
  
  <entry>
    <title>在JPEG图片中嵌入HTML</title>
    <link href="https://breeze2.github.io/blog/practice-embed-html-in-jpeg.html"/>
    <id>https://breeze2.github.io/blog/practice-embed-html-in-jpeg.html</id>
    <published>2018-06-12T15:27:23.000Z</published>
    <updated>2019-09-30T02:22:37.086Z</updated>
    
    <content type="html"><![CDATA[<blockquote><p>最近看到一个有趣的网页：<a href="http://lcamtuf.coredump.cx/squirrel/" target="_blank" rel="noopener">http://lcamtuf.coredump.cx/squirrel/</a>，或者说一张有趣的图片——因为用网络浏览器打开它看到的是一个网页，用图片浏览器打开它看到的又是一张图片。</p></blockquote><p><img src="/blog/assets/images/practice-embed-html-in-jpeg1.jpg" alt="松鼠迷"></p><a id="more"></a><p>在服务端要对同一个请求地址实现不同响应是十分简单的，比如通过请求头<code>Accept</code>来判断：</p><ul><li><code>Accept=text/html</code>则返回超文本；</li><li><code>Accept=image/*</code>则返回图片；</li><li>……</li></ul><p>可是<a href="http://lcamtuf.coredump.cx/squirrel/" target="_blank" rel="noopener">http://lcamtuf.coredump.cx/squirrel/</a>这个网页（或者说图片）在脱离服务端的情况下，依然能够呈现出网页和图片两种文件内容，这是怎样实现的呢？</p><p>在前端没有秘密，打开网络浏览器的开发者工具查看一下网址的响应内容：</p><p><img src="/blog/assets/images/practice-embed-html-in-jpeg2.jpg" alt="响应内容"></p><p>响应内容中有熟悉HTML标签，也有一大堆乱码。这堆乱码应该是图片文件的字符读码，但是为什么在网页上看不到呢？<br>原来HTML中使用了<code>body { visibility: hidden; }</code>样式和<code>&lt;!--</code>注释标签（浏览器自动补全），乱码部分就这样被隐藏了。</p><p>HTML内容：</p><figure class="highlight html"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag">&lt;<span class="name">html</span>&gt;</span><span class="tag">&lt;<span class="name">body</span>&gt;</span><span class="tag">&lt;<span class="name">style</span>&gt;</span><span class="css"><span class="selector-tag">body</span> &#123; <span class="attribute">visibility</span>: hidden; &#125; <span class="selector-class">.n</span> &#123; <span class="attribute">visibility</span>: visible; <span class="attribute">position</span>: absolute; <span class="attribute">padding</span>: <span class="number">0</span> <span class="number">1ex</span> <span class="number">0</span> <span class="number">1ex</span>; <span class="attribute">margin</span>: <span class="number">0</span>; <span class="attribute">top</span>: <span class="number">0</span>; <span class="attribute">left</span>: <span class="number">0</span>; &#125; <span class="selector-tag">h1</span> &#123; <span class="attribute">margin-top</span>: <span class="number">0.4ex</span>; <span class="attribute">margin-bottom</span>: <span class="number">0.8ex</span>; &#125;</span><span class="tag">&lt;/<span class="name">style</span>&gt;</span><span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">n</span>&gt;</span><span class="tag">&lt;<span class="name">h1</span>&gt;</span><span class="tag">&lt;<span class="name">i</span>&gt;</span>Hello, squirrel fans!<span class="tag">&lt;/<span class="name">i</span>&gt;</span><span class="tag">&lt;/<span class="name">h1</span>&gt;</span>This is an embedded landing page for an image. You can link to this URL and get the HTML document you are viewing right now (soon to include essential squirrel facts); or embed the exact same URL as an image on your own squirrel-themed page:<span class="tag">&lt;<span class="name">p</span>&gt;</span><span class="tag">&lt;<span class="name">xmp</span>&gt;</span><span class="tag">&lt;<span class="name">a</span> <span class="attr">href</span>=<span class="string">"http://lcamtuf.coredump.cx/squirrel/"</span>&gt;</span>Click here!<span class="tag">&lt;/<span class="name">a</span>&gt;</span><span class="tag">&lt;/<span class="name">xmp</span>&gt;</span><span class="tag">&lt;<span class="name">xmp</span>&gt;</span><span class="tag">&lt;<span class="name">img</span> <span class="attr">src</span>=<span class="string">"http://lcamtuf.coredump.cx/squirrel/"</span>&gt;</span><span class="tag">&lt;/<span class="name">xmp</span>&gt;</span><span class="tag">&lt;<span class="name">p</span>&gt;</span>No server-side hacks involved - the magic happens in your browser. Let's try embedding the current page as an image right now (INCEPTION!):<span class="tag">&lt;<span class="name">p</span>&gt;</span><span class="tag">&lt;<span class="name">img</span> <span class="attr">src</span>=<span class="string">"#"</span> <span class="attr">style</span>=<span class="string">"border: 1px solid crimson"</span>&gt;</span><span class="tag">&lt;<span class="name">p</span>&gt;</span>Pretty radical, eh? Send money to: lcamtuf@coredump.cx<span class="comment">&lt;!--</span></span><br></pre></td></tr></table></figure><p>然而图片展示中，也没看到HTML的内容，又是为什么呢？</p><h2 id="JPEG相关"><a href="#JPEG相关" class="headerlink" title="JPEG相关"></a>JPEG相关</h2><p>首先可以确定这张图片是JPEG格式，因为内容开头有明显的<code>JFIF</code>标记（<a href="https://en.wikipedia.org/wiki/JPEG_File_Interchange_Format" target="_blank" rel="noopener">JFIF</a>，是JPEG最常见的文件存储格式，也是标准的JPEG文件转换格式）。<br>JPEG格式定义了一系列标记码，都是以0xFF开头，常见的有：</p><ul><li><code>0xFFD8</code>：SOI（Start Of Image），图片开始标记；</li><li><code>0xFFD9</code>：EOI（End Of Image），图片结束标记；</li><li><code>0xFFDA</code>：SOS（Start Of Scan），扫描开始标记；</li><li><code>0xFFDB</code>：DQT（Define Quantization Table），定义量化表；</li><li><code>0xFFC4</code>：DHT（Define Huffman Table），定义哈夫曼表；</li><li><code>0xFFEn</code>：APPn（Application Specific），应用程序信息；</li><li><code>0xFFFE</code>：COM（Comment），注释；</li><li>……</li></ul><p>图片的注释内容不会展示，很显然我们可以把HTML隐藏在图片注释中。<br>用十六进制读取这张图片，可以看到：</p><p><img src="/blog/assets/images/practice-embed-html-in-jpeg3.jpg" alt="十六进制读取"></p><p>这张图片中使用了注释标记<code>0xFFFE</code>，标记后面跟着<code>0x0372</code>。<code>0x0372</code>转成十进制是<code>882</code>，而HTML内容的长度刚好是880个字节（<code>0x0372</code>本身占2个字节），所以可以知道，HTML内容就是写在这张图片注释中。</p><h2 id="代码实现"><a href="#代码实现" class="headerlink" title="代码实现"></a>代码实现</h2><p>下面用PHP代码简单实现一个将HTML内容嵌入JPEG中的函数：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="function"><span class="keyword">function</span> <span class="title">embedHtmlInJpeg</span><span class="params">($jpeg_file, $html_str, $html_file)</span></span></span><br><span class="line"><span class="function"></span>&#123;</span><br><span class="line">    $length = strlen($html_str) + <span class="number">2</span>;</span><br><span class="line">    <span class="keyword">if</span> ($length &gt; <span class="number">256</span> * <span class="number">256</span> - <span class="number">1</span>) &#123;</span><br><span class="line">        <span class="keyword">return</span> <span class="keyword">false</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    $content = <span class="string">''</span>;</span><br><span class="line">    $reader  = fopen($jpeg_file, <span class="string">'rb'</span>);</span><br><span class="line">    $writer  = fopen($html_file, <span class="string">'wb'</span>);</span><br><span class="line">    $content = fread($reader, <span class="number">2</span>); <span class="comment">// read 0xFFD8</span></span><br><span class="line">    fwrite($writer, $content); <span class="comment">// write 0xFFD8</span></span><br><span class="line"></span><br><span class="line">    $header  = <span class="string">'FFFE'</span> . sprintf(<span class="string">'%04X'</span>, $length);</span><br><span class="line">    $header  = pack(<span class="string">'H*'</span>, $header);</span><br><span class="line">    $content = $header . $html_str;</span><br><span class="line">    fwrite($writer, $content); <span class="comment">// write 0xFFFE</span></span><br><span class="line"></span><br><span class="line">    <span class="keyword">while</span> (!feof($reader)) &#123;</span><br><span class="line">        $content = fread($reader, <span class="number">8192</span>);</span><br><span class="line">        fwrite($writer, $content); <span class="comment">// write else</span></span><br><span class="line">    &#125;</span><br><span class="line">    fclose($reader);</span><br><span class="line">    fclose($writer);</span><br><span class="line">    <span class="keyword">return</span> <span class="keyword">true</span>;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line"><span class="comment">// call it</span></span><br><span class="line">embedHtmlInJpeg(<span class="string">'lena.jpg'</span>,</span><br><span class="line">    <span class="string">'&lt;html&gt;&lt;body&gt;&lt;style&gt;body &#123; visibility: hidden; &#125; .n &#123; visibility: visible; position: absolute; padding: 0 1ex 0 1ex; margin: 0; top: 0; left: 0; &#125; h1 &#123; margin-top: 0.4ex; margin-bottom: 0.8ex; &#125;&lt;/style&gt;&lt;div class=n&gt;&lt;h1&gt;&lt;i&gt;This image is a page.&lt;/i&gt;&lt;/h1&gt;Just open it in new tab.&lt;p&gt;&lt;img src="#" style="border: 1px solid crimson"&gt;&lt;!--'</span>,</span><br><span class="line">    <span class="string">'lena.html'</span>);</span><br></pre></td></tr></table></figure><h2 id="这张图片是一个网页，不信你就在新标签页中打开它"><a href="#这张图片是一个网页，不信你就在新标签页中打开它" class="headerlink" title="这张图片是一个网页，不信你就在新标签页中打开它"></a>这张图片是一个网页，不信你就在新标签页中打开它</h2><p><a href="https://breeze2.github.io/blog/practice-embed-html-in-jpeg4.html" target="_blank"><img src="/blog/practice-embed-html-in-jpeg4.html" alt="Lena"></a></p><h2 id="实际应用"><a href="#实际应用" class="headerlink" title="实际应用"></a>实际应用</h2><p>将一段HTML文本嵌入到一张图片中，实际上，还没什么应用，哈哈哈😂。<br>如果能将<code>JS</code>或<code>Shell</code>脚本藏在图片中，并能后期执行，那就有意思了；而且本身是一个图片文件，可以避过一些安全软件的检查。</p>]]></content>
    
    <summary type="html">
    
      &lt;blockquote&gt;
&lt;p&gt;最近看到一个有趣的网页：&lt;a href=&quot;http://lcamtuf.coredump.cx/squirrel/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;http://lcamtuf.coredump.cx/squirrel/&lt;/a&gt;，或者说一张有趣的图片——因为用网络浏览器打开它看到的是一个网页，用图片浏览器打开它看到的又是一张图片。&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;&lt;img src=&quot;/blog/assets/images/practice-embed-html-in-jpeg1.jpg&quot; alt=&quot;松鼠迷&quot;&gt;&lt;/p&gt;
    
    </summary>
    
      <category term="实践" scheme="https://breeze2.github.io/blog/categories/%E5%AE%9E%E8%B7%B5/"/>
    
    
      <category term="php" scheme="https://breeze2.github.io/blog/tags/php/"/>
    
      <category term="practice" scheme="https://breeze2.github.io/blog/tags/practice/"/>
    
      <category term="html" scheme="https://breeze2.github.io/blog/tags/html/"/>
    
      <category term="frontend" scheme="https://breeze2.github.io/blog/tags/frontend/"/>
    
      <category term="jpeg" scheme="https://breeze2.github.io/blog/tags/jpeg/"/>
    
  </entry>
  
  <entry>
    <title>在Ubuntu上运行多实例MySQL</title>
    <link href="https://breeze2.github.io/blog/mysql-run-multiple-instances-on-ubuntu.html"/>
    <id>https://breeze2.github.io/blog/mysql-run-multiple-instances-on-ubuntu.html</id>
    <published>2018-03-18T19:02:13.000Z</published>
    <updated>2019-09-30T02:22:37.082Z</updated>
    
    <content type="html"><![CDATA[<p>一般情况下，都是一台服务器主机运行一个MySQL实例；不过在单机配置非常高的情况下，也可以考虑运行多个MySQL实例。毕竟MySQL是单进程（多线程）的运行模式，单实例MySQL用不上太多资源。</p><p>当然，高配置的单机应该考虑使用<a href="https://www.docker.com/" target="_blank" rel="noopener">Docker</a>实现多MySQL服务，只是这里介绍一下如何运行多实例MySQL。本以为在Ubuntu上实现很容易，原来也并不简单。</p><h2 id="运行环境"><a href="#运行环境" class="headerlink" title="运行环境"></a>运行环境</h2><ul><li>Ubuntu 16.04</li><li>MySQL 5.17</li></ul><a id="more"></a><h2 id="mysqld-multi"><a href="#mysqld-multi" class="headerlink" title="mysqld_multi"></a>mysqld_multi</h2><p>在安装好MySQL Server后，应该是可以使用<code>mysqld_multi</code>的。<br>查看配置示例命令：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">$ mysqld_multi --example</span><br><span class="line">[mysqld_multi]</span><br><span class="line">mysqld     = /usr/bin/mysqld_safe</span><br><span class="line">mysqladmin = /usr/bin/mysqladmin</span><br><span class="line">user       = multi_admin</span><br><span class="line">password   = my_password</span><br><span class="line"></span><br><span class="line">[mysqld2]</span><br><span class="line">socket     = /tmp/mysql.sock2</span><br><span class="line">port       = <span class="number">3307</span></span><br><span class="line">pid-file   = /var/lib/mysql2/hostname.pid2</span><br><span class="line">datadir    = /var/lib/mysql2</span><br><span class="line">language   = /usr/share/mysql/mysql/english</span><br><span class="line">user       = unix_user1</span><br><span class="line">...</span><br></pre></td></tr></table></figure><p>一般MySQL的配置文件是<code>/etc/mysql/my.cnf</code>或者<code>/etc/mysql/mysql.cnf</code>，在配置文件中最开始的地方添加：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">[mysqld_multi]</span><br><span class="line">mysqld     = /usr/bin/mysqld_safe</span><br><span class="line">mysqladmin = /usr/bin/mysqladmin</span><br></pre></td></tr></table></figure><p>查看当前<code>mysqld_multi</code>状态：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ mysqld_multi report                               </span><br><span class="line">Reporting MySQL servers</span><br><span class="line">No groups to be reported (check your GNRs)</span><br></pre></td></tr></table></figure><p>可以看到没有服务组群，现在添加一个——在MySQL的配置文件中，追加：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">[mysqld3307]</span><br><span class="line">socket     = /tmp/mysqld/mysqld3307.sock</span><br><span class="line">port       = 3307</span><br><span class="line">pid-file   = /tmp/mysqld/mysqld3307.pid</span><br><span class="line">datadir    = /var/lib/mysql/multi/data3307</span><br><span class="line">user       = mysql</span><br></pre></td></tr></table></figure><p>再查看当前<code>mysqld_multi</code>状态：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ mysqld_multi report                               </span><br><span class="line">Reporting MySQL servers</span><br><span class="line">MySQL server from group: mysqld3307 is not running</span><br></pre></td></tr></table></figure><p>可以看到服务组群<code>mysqld3307</code>（注意每个服务组群名称必须以<code>mysqld</code>开头）没有在运行。现在还不能直接运行<code>mysqld3307</code>服务，因为没有初始化数据目录<code>/var/lib/mysql/multi/data3307</code>。</p><h2 id="初始化数据库"><a href="#初始化数据库" class="headerlink" title="初始化数据库"></a>初始化数据库</h2><blockquote><p>过去MySQL初始化数据库使用的是<code>mysql_install_db</code>命令，现在是<code>mysqld --initialize</code>。</p></blockquote><p>在Ubuntu系统上有一个系统安全程序，叫AppArmor，它会限制各个应用程序访问系统资源的权限。Root用户运行程序时也碰到权限不足错误的话，那么应该就是AppArmor限制了。比如我们需要修改<code>mysqld</code>的权限，就得编辑对应的AppArmor配置文件，然后重启AppArmor服务：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ sudo vi /etc/apparmor.d/usr.sbin.mysqld</span><br><span class="line">$ sudo service apparmor restart</span><br></pre></td></tr></table></figure><p>上面服务组群<code>mysqld3307</code>的配置中用到的目录是<code>/tmp/mysqld</code>和<code>/var/lib/mysql/multi</code>，<code>mysqld</code>程序对<code>/tmp</code>和<code>/var/lib/mysql</code>都是具有读写权限的，所以不需要修改AppArmor配置文件。</p><p>现在来初始化一个新的数据库目录<code>/var/lib/mysql/multi/data3307</code>（以mysql用户身份执行）：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ sudo -u mysql <span class="built_in">mkdir</span> /tmp/mysqld</span><br><span class="line">$ sudo -u mysql <span class="built_in">mkdir</span> /var/lib/mysql/multi</span><br><span class="line">$ sudo -u mysql mysqld --initialize --user=mysql --datadir=/var/lib/mysql/multi/data3307</span><br></pre></td></tr></table></figure><p>初始化成功后，新数据库会自动建立root用户，并随机生成密码，这些会记录在MySQL日志中：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">$ sudo tail /var/log/mysql/error.log</span><br><span class="line"><span class="number">2018</span>-<span class="number">03</span>-<span class="number">18</span>T12:<span class="number">15</span>:<span class="number">25</span>.<span class="number">378684</span>Z <span class="number">0</span> [Warning] TIMESTAMP with implicit DEFAULT value is deprecated. Please use --explicit_defaults_for_timestamp server option (see documentation <span class="keyword">for</span> <span class="built_in">more</span> details).</span><br><span class="line"><span class="number">2018</span>-<span class="number">03</span>-<span class="number">18</span>T12:<span class="number">15</span>:<span class="number">26</span>.<span class="number">642564</span>Z <span class="number">0</span> [Warning] InnoDB: New log files created, LSN=<span class="number">45790</span></span><br><span class="line"><span class="number">2018</span>-<span class="number">03</span>-<span class="number">18</span>T12:<span class="number">15</span>:<span class="number">26</span>.<span class="number">900383</span>Z <span class="number">0</span> [Warning] InnoDB: Creating foreign key constraint system tables.</span><br><span class="line"><span class="number">2018</span>-<span class="number">03</span>-<span class="number">18</span>T12:<span class="number">15</span>:<span class="number">26</span>.<span class="number">986706</span>Z <span class="number">0</span> [Warning] No existing UUID has been found, so we assume that this is the first <span class="built_in">time</span> that this server has been started. Generating a new UUID: <span class="number">3544</span>d1ec-<span class="number">2</span>b6f-<span class="number">11</span>e8-bf0a-aaaa0007ba64.</span><br><span class="line"><span class="number">2018</span>-<span class="number">03</span>-<span class="number">18</span>T12:<span class="number">15</span>:<span class="number">26</span>.<span class="number">998294</span>Z <span class="number">0</span> [Warning] Gtid table is <span class="keyword">not</span> ready to be used. Table 'mysql.gtid_executed' cannot be opened.</span><br><span class="line"><span class="number">2018</span>-<span class="number">03</span>-<span class="number">18</span>T12:<span class="number">15</span>:<span class="number">26</span>.<span class="number">998765</span>Z <span class="number">1</span> [Note] A temporary password is generated <span class="keyword">for</span> root@localhost: <span class="number">4</span>kDUVrlO&gt;zT)</span><br></pre></td></tr></table></figure><p>我们先记住密码<code>4kDUVrlO&gt;zT)</code>。</p><h2 id="启动新实例"><a href="#启动新实例" class="headerlink" title="启动新实例"></a>启动新实例</h2><p>执行一下命令即可以mysql用户身份启动服务组群<code>mysqld3307</code>（不需要前缀<code>mysqld</code>）：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo -u mysql mysqld_multi <span class="built_in">start</span> <span class="number">3307</span></span><br></pre></td></tr></table></figure><p>查看当前<code>mysqld_multi</code>状态：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ mysqld_multi report                               </span><br><span class="line">Reporting MySQL servers</span><br><span class="line">MySQL server from group: mysqld3307 is running</span><br></pre></td></tr></table></figure><p>重置一下密码：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ mysqladmin -uroot -h <span class="number">127</span>.<span class="number">0</span>.<span class="number">0</span>.<span class="number">1</span> -P <span class="number">3307</span> -p'<span class="number">4</span>kDUVrlO&gt;zT)' password</span><br><span class="line">New password:</span><br><span class="line">Confirm new password:</span><br></pre></td></tr></table></figure><p>连接服务组群<code>mysqld3307</code>：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ mysql -uroot -h <span class="number">127</span>.<span class="number">0</span>.<span class="number">0</span>.<span class="number">1</span> -P <span class="number">3307</span> -p</span><br></pre></td></tr></table></figure><h2 id="关闭实例"><a href="#关闭实例" class="headerlink" title="关闭实例"></a>关闭实例</h2><p>原以为关闭服务组群<code>mysqld3307</code>，只需要：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo -u mysql mysqld_multi stop <span class="number">3307</span></span><br></pre></td></tr></table></figure><p>结果查看<code>mysqld_multi</code>状态，<code>mysqld3307</code>依然是运行中：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ mysqld_multi report                               </span><br><span class="line">Reporting MySQL servers</span><br><span class="line">MySQL server from group: mysqld3307 is running</span><br></pre></td></tr></table></figure><blockquote><p>可能是因为<code>mysqld3307</code>数据库中没有与<code>mysqld_multi</code>配置对应的用户，所以<code>mysqld_multi stop</code>命令没能关闭<code>mysqld3307</code>。</p></blockquote><p>我们可以用<code>mysqladmin</code>命令来关闭服务组群<code>mysqld3307</code>：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ mysqladmin -uroot -h <span class="number">127</span>.<span class="number">0</span>.<span class="number">0</span>.<span class="number">1</span> -P <span class="number">3307</span> -p shutdown</span><br></pre></td></tr></table></figure><h2 id="最后"><a href="#最后" class="headerlink" title="最后"></a>最后</h2><p>这里只介绍了监听3307端口的MySQL实例的建立过程，其他更多MySQL实例，如监听3308、3309端口，只需按照上述流程（添加配置，初始化数据目录，启动服务）执行即可。</p>]]></content>
    
    <summary type="html">
    
      &lt;p&gt;一般情况下，都是一台服务器主机运行一个MySQL实例；不过在单机配置非常高的情况下，也可以考虑运行多个MySQL实例。毕竟MySQL是单进程（多线程）的运行模式，单实例MySQL用不上太多资源。&lt;/p&gt;
&lt;p&gt;当然，高配置的单机应该考虑使用&lt;a href=&quot;https://www.docker.com/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;Docker&lt;/a&gt;实现多MySQL服务，只是这里介绍一下如何运行多实例MySQL。本以为在Ubuntu上实现很容易，原来也并不简单。&lt;/p&gt;
&lt;h2 id=&quot;运行环境&quot;&gt;&lt;a href=&quot;#运行环境&quot; class=&quot;headerlink&quot; title=&quot;运行环境&quot;&gt;&lt;/a&gt;运行环境&lt;/h2&gt;&lt;ul&gt;
&lt;li&gt;Ubuntu 16.04&lt;/li&gt;
&lt;li&gt;MySQL 5.17&lt;/li&gt;
&lt;/ul&gt;
    
    </summary>
    
      <category term="躬行" scheme="https://breeze2.github.io/blog/categories/%E8%BA%AC%E8%A1%8C/"/>
    
    
      <category term="mysql" scheme="https://breeze2.github.io/blog/tags/mysql/"/>
    
  </entry>
  
  <entry>
    <title>一些有趣的MySQL查询问题</title>
    <link href="https://breeze2.github.io/blog/note-mysql-in-leetcode.html"/>
    <id>https://breeze2.github.io/blog/note-mysql-in-leetcode.html</id>
    <published>2018-03-02T10:02:13.000Z</published>
    <updated>2019-09-30T02:22:37.082Z</updated>
    
    <content type="html"><![CDATA[<blockquote><p>在<a href="https://leetcode.com/" target="_blank" rel="noopener">LeetCode</a>上有一些有趣的MySQL查询问题，这里记录下来，方便以后查阅。</p></blockquote><a id="more"></a><h2 id="Department-Top-Three-Salaries"><a href="#Department-Top-Three-Salaries" class="headerlink" title="Department Top Three Salaries"></a>Department Top Three Salaries</h2><p>部门前三薪资问题：查询各个部门薪资前三的员工（薪资），并列排名只占一位。</p><h3 id="原题"><a href="#原题" class="headerlink" title="原题"></a>原题</h3><p>The Employee table holds all employees. Every employee has an Id, and there is also a column for the department Id.</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">+<span class="comment">----+-------+--------+--------------+</span></span><br><span class="line">| Id | Name  | Salary | DepartmentId |</span><br><span class="line">+<span class="comment">----+-------+--------+--------------+</span></span><br><span class="line">| 1  | Joe   | 70000  | 1            |</span><br><span class="line">| 2  | Henry | 80000  | 2            |</span><br><span class="line">| 3  | Sam   | 60000  | 2            |</span><br><span class="line">| 4  | Max   | 90000  | 1            |</span><br><span class="line">| 5  | Janet | 69000  | 1            |</span><br><span class="line">| 6  | Randy | 85000  | 1            |</span><br><span class="line">+<span class="comment">----+-------+--------+--------------+</span></span><br></pre></td></tr></table></figure><p>The Department table holds all departments of the company.</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">+<span class="comment">----+----------+</span></span><br><span class="line">| Id | Name     |</span><br><span class="line">+<span class="comment">----+----------+</span></span><br><span class="line">| 1  | IT       |</span><br><span class="line">| 2  | Sales    |</span><br><span class="line">+<span class="comment">----+----------+</span></span><br></pre></td></tr></table></figure><p>Write a SQL query to find employees who earn the top three salaries in each of the department. For the above tables, your SQL query should return the following rows.</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">+------------+----------+--------+</span><br><span class="line">| Department | Employee | Salary |</span><br><span class="line">+------------+----------+--------+</span><br><span class="line">| IT         | Max      | 90000  |</span><br><span class="line">| IT         | Randy    | 85000  |</span><br><span class="line">| IT         | Joe      | 70000  |</span><br><span class="line">| Sales      | Henry    | 80000  |</span><br><span class="line">| Sales      | Sam      | 60000  |</span><br><span class="line">+------------+----------+--------+</span><br></pre></td></tr></table></figure><h3 id="答案"><a href="#答案" class="headerlink" title="答案"></a>答案</h3><p>先说答案：</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">select</span> dep.Name <span class="keyword">as</span> <span class="string">"Department"</span>, emp.Name <span class="keyword">as</span> <span class="string">"Employee"</span>, emp.Salary <span class="keyword">as</span> <span class="string">"Salary"</span></span><br><span class="line"><span class="keyword">from</span> Department <span class="keyword">as</span> dep <span class="keyword">join</span> Employee <span class="keyword">as</span> emp <span class="keyword">on</span> dep.Id = emp.DepartmentId</span><br><span class="line"><span class="keyword">where</span> <span class="keyword">exists</span> (<span class="keyword">select</span> <span class="string">'x'</span> <span class="keyword">from</span> Employee <span class="keyword">as</span> emp1</span><br><span class="line">    <span class="keyword">where</span> emp1.Salary &gt; emp.Salary <span class="keyword">and</span> emp1.DepartmentId = emp.DepartmentId <span class="keyword">having</span> <span class="keyword">count</span>(<span class="keyword">distinct</span> emp1.Salary) &lt; <span class="number">3</span></span><br><span class="line">);</span><br></pre></td></tr></table></figure><p>首先Department和Employee联接是不可少的，因为结果包含两张表的字段；再用子查询（exists），保留同部门内高于本身薪资的不超过三个的员工（having），并列排名只占一位，需要去重（distinct）。</p><p>注意：以上<code>&#39;x&#39;</code>并无特殊意义，只是有值时表示存在，无值时表示不存在。；<code>having</code>应该配合<code>group by</code>使用，单独使用<code>having</code>时视当前数据为一组；<code>count()</code>函数也是，单独使用<code>having</code>时也相当于给当前数据分成一组。</p><p>另外：只用一个<code>select</code>（没有子查询）也能实现，就是将Employee和自身<code>join</code>，<code>on</code>薪资比自己高的员工；然后用<code>group by</code>去重，再结合<code>having count(distinct emp1.Salary) &lt; 3</code>，保留有效结果。这个<code>join</code>方法效率远低于上面子查询方法，所以不要对子查询有偏见。</p><h2 id="Human-Traffic-of-Stadium"><a href="#Human-Traffic-of-Stadium" class="headerlink" title="Human Traffic of Stadium"></a>Human Traffic of Stadium</h2><p>连续达标问题：查询连续3天体育馆人数高于100的日期。</p><h3 id="原题-1"><a href="#原题-1" class="headerlink" title="原题"></a>原题</h3><p>X city built a new stadium, each day many people visit it and the stats are saved as these columns: id, date, people<br>Please write a query to display the records which have 3 or more consecutive rows and the amount of people more than 100(inclusive).<br>For example, the table stadium:</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">+<span class="comment">------+------------+-----------+</span></span><br><span class="line">| id   | date       | people    |</span><br><span class="line">+<span class="comment">------+------------+-----------+</span></span><br><span class="line">| 1    | 2017-01-01 | 10        |</span><br><span class="line">| 2    | 2017-01-02 | 109       |</span><br><span class="line">| 3    | 2017-01-03 | 150       |</span><br><span class="line">| 4    | 2017-01-04 | 99        |</span><br><span class="line">| 5    | 2017-01-05 | 145       |</span><br><span class="line">| 6    | 2017-01-06 | 1455      |</span><br><span class="line">| 7    | 2017-01-07 | 199       |</span><br><span class="line">| 8    | 2017-01-08 | 188       |</span><br><span class="line">+<span class="comment">------+------------+-----------+</span></span><br></pre></td></tr></table></figure><p>For the sample data above, the output is:</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">+<span class="comment">------+------------+-----------+</span></span><br><span class="line">| id   | date       | people    |</span><br><span class="line">+<span class="comment">------+------------+-----------+</span></span><br><span class="line">| 5    | 2017-01-05 | 145       |</span><br><span class="line">| 6    | 2017-01-06 | 1455      |</span><br><span class="line">| 7    | 2017-01-07 | 199       |</span><br><span class="line">| 8    | 2017-01-08 | 188       |</span><br><span class="line">+<span class="comment">------+------------+-----------+</span></span><br></pre></td></tr></table></figure><p>Note:<br>Each day only have one row record, and the dates are increasing with id increasing.</p><h3 id="答案-1"><a href="#答案-1" class="headerlink" title="答案"></a>答案</h3><p>思路是先找出人数大于100的日期，然后在这些日期附近再找数大于100的日期，保留能够形成连续3天的日期。<br>提示里说表中每一天都有一条记录，而且日期是随id递增，所以我们只需要计算id，而不是日期。<br>用子查询的方法：</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">select</span> s.id, s.date, s.people </span><br><span class="line"><span class="keyword">from</span> stadium <span class="keyword">as</span> s </span><br><span class="line"><span class="keyword">where</span> s.people &gt;= <span class="number">100</span> <span class="keyword">and</span> <span class="keyword">exists</span>(<span class="keyword">select</span> <span class="string">'x'</span></span><br><span class="line">    <span class="keyword">from</span> stadium <span class="keyword">as</span> s1</span><br><span class="line">    <span class="keyword">where</span> s1.people &gt;= <span class="number">100</span></span><br><span class="line">        <span class="keyword">and</span> s1.id &gt;= s.id<span class="number">-2</span></span><br><span class="line">        <span class="keyword">and</span> s1.id &lt;= s.id+<span class="number">2</span></span><br><span class="line">        <span class="keyword">and</span> s1.id&lt;&gt;s.id</span><br><span class="line">    <span class="keyword">having</span> <span class="keyword">count</span>(s1.id) &gt; <span class="number">2</span></span><br><span class="line">        <span class="keyword">or</span> (<span class="keyword">count</span>(s1.id) = <span class="number">2</span> </span><br><span class="line">            <span class="keyword">and</span> (<span class="keyword">sum</span>(s1.id-s.id) <span class="keyword">not</span> <span class="keyword">in</span> (<span class="number">-1</span>,<span class="number">1</span>) <span class="keyword">and</span> <span class="keyword">sum</span>(<span class="keyword">abs</span>(s1.id-s.id)) &lt;&gt; <span class="number">4</span>)</span><br><span class="line">        )</span><br><span class="line">    );</span><br></pre></td></tr></table></figure><p>在当前日期的前2天和后2天找，找到人数大于100的日期个数大于2的话，那么当前日期必定可以形成连续3天人数大于100；找到人数大于100的日期个数等于2，且这两个日期与当前日期的距离和不是1或-1，绝对距离和不是4的话，那么当前日期也是可以形成连续3天人数大于100；其他情况都不行。</p><p>用<code>join</code>的方法：</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">select</span> <span class="keyword">distinct</span> t1.*</span><br><span class="line"><span class="keyword">from</span> stadium t1, stadium t2, stadium t3</span><br><span class="line"><span class="keyword">where</span> t1.people &gt;= <span class="number">100</span> <span class="keyword">and</span> t2.people &gt;= <span class="number">100</span> <span class="keyword">and</span> t3.people &gt;= <span class="number">100</span></span><br><span class="line">    <span class="keyword">and</span> ( (t1.id - t2.id = <span class="number">1</span> <span class="keyword">and</span> t1.id - t3.id = <span class="number">2</span> <span class="keyword">and</span> t2.id - t3.id =<span class="number">1</span>) <span class="comment">-- t1, t2, t3</span></span><br><span class="line">        <span class="keyword">or</span> (t2.id - t1.id = <span class="number">1</span> <span class="keyword">and</span> t2.id - t3.id = <span class="number">2</span> <span class="keyword">and</span> t1.id - t3.id =<span class="number">1</span>) <span class="comment">-- t2, t1, t3</span></span><br><span class="line">        <span class="keyword">or</span> (t3.id - t2.id = <span class="number">1</span> <span class="keyword">and</span> t2.id - t1.id =<span class="number">1</span> <span class="keyword">and</span> t3.id - t1.id = <span class="number">2</span>) <span class="comment">-- t3, t2, t1</span></span><br><span class="line">    )</span><br><span class="line"><span class="keyword">order</span> <span class="keyword">by</span> t1.id;</span><br></pre></td></tr></table></figure><p>同样，上面的子查询方法依然比这个join方法效率高。</p><h2 id="Trips-and-Users"><a href="#Trips-and-Users" class="headerlink" title="Trips and Users"></a>Trips and Users</h2><h3 id="原题-2"><a href="#原题-2" class="headerlink" title="原题"></a>原题</h3><p>用户取消叫车服务问题：统计一个时间段内每天用户取消叫车服务占所有叫车服务的比例。<br>The Trips table holds all taxi trips. Each trip has a unique Id, while Client_Id and Driver_Id are both foreign keys to the Users_Id at the Users table. Status is an ENUM type of (‘completed’, ‘cancelled_by_driver’, ‘cancelled_by_client’).</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">+<span class="comment">----+-----------+-----------+---------+--------------------+----------+</span></span><br><span class="line">| Id | Client_Id | Driver_Id | City_Id |        Status      |Request_at|</span><br><span class="line">+<span class="comment">----+-----------+-----------+---------+--------------------+----------+</span></span><br><span class="line">| 1  |     1     |    10     |    1    |     completed      |2013-10-01|</span><br><span class="line">| 2  |     2     |    11     |    1    | cancelled_by_driver|2013-10-01|</span><br><span class="line">| 3  |     3     |    12     |    6    |     completed      |2013-10-01|</span><br><span class="line">| 4  |     4     |    13     |    6    | cancelled_by_client|2013-10-01|</span><br><span class="line">| 5  |     1     |    10     |    1    |     completed      |2013-10-02|</span><br><span class="line">| 6  |     2     |    11     |    6    |     completed      |2013-10-02|</span><br><span class="line">| 7  |     3     |    12     |    6    |     completed      |2013-10-02|</span><br><span class="line">| 8  |     2     |    12     |    12   |     completed      |2013-10-03|</span><br><span class="line">| 9  |     3     |    10     |    12   |     completed      |2013-10-03| </span><br><span class="line">| 10 |     4     |    13     |    12   | cancelled_by_driver|2013-10-03|</span><br><span class="line">+<span class="comment">----+-----------+-----------+---------+--------------------+----------+</span></span><br></pre></td></tr></table></figure><p>The Users table holds all users. Each user has an unique Users_Id, and Role is an ENUM type of (‘client’, ‘driver’, ‘partner’).</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line">+<span class="comment">----------+--------+--------+</span></span><br><span class="line">| Users_Id | Banned |  Role  |</span><br><span class="line">+<span class="comment">----------+--------+--------+</span></span><br><span class="line">|    1     |   No   | client |</span><br><span class="line">|    2     |   Yes  | client |</span><br><span class="line">|    3     |   No   | client |</span><br><span class="line">|    4     |   No   | client |</span><br><span class="line">|    10    |   No   | driver |</span><br><span class="line">|    11    |   No   | driver |</span><br><span class="line">|    12    |   No   | driver |</span><br><span class="line">|    13    |   No   | driver |</span><br><span class="line">+<span class="comment">----------+--------+--------+</span></span><br></pre></td></tr></table></figure><p>Write a SQL query to find the cancellation rate of requests made by unbanned clients between Oct 1, 2013 and Oct 3, 2013. For the above tables, your SQL query should return the following rows with the cancellation rate being rounded to two decimal places.</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">+<span class="comment">------------+-------------------+</span></span><br><span class="line">|     Day    | Cancellation Rate |</span><br><span class="line">+<span class="comment">------------+-------------------+</span></span><br><span class="line">| 2013-10-01 |       0.33        |</span><br><span class="line">| 2013-10-02 |       0.00        |</span><br><span class="line">| 2013-10-03 |       0.50        |</span><br><span class="line">+<span class="comment">------------+-------------------+</span></span><br></pre></td></tr></table></figure><h3 id="答案-2"><a href="#答案-2" class="headerlink" title="答案"></a>答案</h3><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">select</span> Request_at <span class="keyword">as</span> <span class="string">"Day"</span>, <span class="keyword">round</span>(<span class="number">1</span>-<span class="keyword">sum</span>(t.Status=<span class="string">"completed"</span>)/<span class="keyword">count</span>(t.id), <span class="number">2</span>) <span class="keyword">as</span> <span class="string">"Cancellation Rate"</span></span><br><span class="line"><span class="keyword">from</span> Trips <span class="keyword">as</span> t <span class="keyword">join</span> <span class="keyword">users</span> <span class="keyword">as</span> u <span class="keyword">on</span> u.Users_Id = t.Client_Id <span class="keyword">and</span> u.Banned = <span class="string">"No"</span></span><br><span class="line"><span class="keyword">where</span> t.Request_at &gt;=<span class="string">'2013-10-01'</span> <span class="keyword">and</span> t.Request_at&lt;=<span class="string">'2013-10-03'</span></span><br><span class="line"><span class="keyword">group</span> <span class="keyword">by</span> t.Request_at;</span><br></pre></td></tr></table></figure><h2 id="Median-Employee-Salary"><a href="#Median-Employee-Salary" class="headerlink" title="Median Employee Salary"></a>Median Employee Salary</h2><h3 id="原题-3"><a href="#原题-3" class="headerlink" title="原题"></a>原题</h3><p>中位薪资问题：查询各家公司中薪资排中位的的员工（薪资）。<br>The Employee table holds all employees. The employee table has three columns: Employee Id, Company Name, and Salary.</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line">+<span class="comment">-----+------------+--------+</span></span><br><span class="line">|Id   | Company    | Salary |</span><br><span class="line">+<span class="comment">-----+------------+--------+</span></span><br><span class="line">|1    | A          | 2341   |</span><br><span class="line">|2    | A          | 341    |</span><br><span class="line">|3    | A          | 15     |</span><br><span class="line">|4    | A          | 15314  |</span><br><span class="line">|5    | A          | 451    |</span><br><span class="line">|6    | A          | 513    |</span><br><span class="line">|7    | B          | 15     |</span><br><span class="line">|8    | B          | 13     |</span><br><span class="line">|9    | B          | 1154   |</span><br><span class="line">|10   | B          | 1345   |</span><br><span class="line">|11   | B          | 1221   |</span><br><span class="line">|12   | B          | 234    |</span><br><span class="line">|13   | C          | 2345   |</span><br><span class="line">|14   | C          | 2645   |</span><br><span class="line">|15   | C          | 2645   |</span><br><span class="line">|16   | C          | 2652   |</span><br><span class="line">|17   | C          | 65     |</span><br><span class="line">+<span class="comment">-----+------------+--------+</span></span><br></pre></td></tr></table></figure><p>Write a SQL query to find the median salary of each company. Bonus points if you can solve it without using any built-in SQL functions.</p><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line">+<span class="comment">-----+------------+--------+</span></span><br><span class="line">|Id   | Company    | Salary |</span><br><span class="line">+<span class="comment">-----+------------+--------+</span></span><br><span class="line">|5    | A          | 451    |</span><br><span class="line">|6    | A          | 513    |</span><br><span class="line">|12   | B          | 234    |</span><br><span class="line">|9    | B          | 1154   |</span><br><span class="line">|14   | C          | 2645   |</span><br><span class="line">+<span class="comment">-----+------------+--------+</span></span><br></pre></td></tr></table></figure><h3 id="答案-3"><a href="#答案-3" class="headerlink" title="答案"></a>答案</h3><figure class="highlight sql"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">select</span> emp.Id, emp.Company, emp.Salary</span><br><span class="line"><span class="keyword">from</span> Employee <span class="keyword">as</span> emp</span><br><span class="line"><span class="keyword">join</span> (<span class="keyword">select</span> Company, <span class="keyword">count</span>(*) <span class="keyword">as</span> EmployeeNum <span class="keyword">from</span> Employee <span class="keyword">group</span> <span class="keyword">by</span> Company) <span class="keyword">as</span> emp1</span><br><span class="line"><span class="keyword">on</span> emp1.Company = emp.Company</span><br><span class="line"><span class="keyword">where</span> <span class="keyword">exists</span> (<span class="keyword">select</span> <span class="string">'x'</span> <span class="keyword">from</span> Employee <span class="keyword">as</span> emp2</span><br><span class="line">    <span class="keyword">where</span> emp2.Company = emp.Company <span class="keyword">and</span> emp2.Salary &lt; emp.Salary</span><br><span class="line">    <span class="keyword">having</span> <span class="keyword">count</span>(emp2.Id) <span class="keyword">in</span> ( <span class="keyword">floor</span>((EmployeeNum<span class="number">-1</span>)/<span class="number">2</span>), <span class="keyword">floor</span>(EmployeeNum/<span class="number">2</span>) )</span><br><span class="line">);</span><br></pre></td></tr></table></figure>]]></content>
    
    <summary type="html">
    
      &lt;blockquote&gt;
&lt;p&gt;在&lt;a href=&quot;https://leetcode.com/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;LeetCode&lt;/a&gt;上有一些有趣的MySQL查询问题，这里记录下来，方便以后查阅。&lt;/p&gt;
&lt;/blockquote&gt;
    
    </summary>
    
      <category term="笔记" scheme="https://breeze2.github.io/blog/categories/%E7%AC%94%E8%AE%B0/"/>
    
    
      <category term="mysql" scheme="https://breeze2.github.io/blog/tags/mysql/"/>
    
      <category term="leetcode" scheme="https://breeze2.github.io/blog/tags/leetcode/"/>
    
  </entry>
  
  <entry>
    <title>Swoole+Lumen：同步编程风格调用MySQL异步查询</title>
    <link href="https://breeze2.github.io/blog/swoole-lumen-asynchronous-mysql-query-with-synchronous-programming-style.html"/>
    <id>https://breeze2.github.io/blog/swoole-lumen-asynchronous-mysql-query-with-synchronous-programming-style.html</id>
    <published>2018-03-02T09:23:02.000Z</published>
    <updated>2019-09-30T02:22:37.086Z</updated>
    
    <content type="html"><![CDATA[<blockquote><p>网络编程一直是PHP的短板，尽管<a href="https://www.swoole.com/" target="_blank" rel="noopener">Swoole</a>扩展弥补了这个缺陷，但是其编程风格偏向了NodeJS或GoLang，与原本的同步编程风格迥然相异。目前PHP的大部分主流应用框架依然是同步编程风格，所以一直在探索Swoole与同步编程结合的途径。<br><a href="https://github.com/breeze2/lumen-swoole-http" target="_blank" rel="noopener">lumen-swoole-http</a>正是连接同步编程Lumen和异步编程Swoole的一座桥梁，有兴趣可以关注一下。</p></blockquote><h2 id="LNMP的不足"><a href="#LNMP的不足" class="headerlink" title="LNMP的不足"></a>LNMP的不足</h2><p>LNMP是经典的Web应用架构组合，虽然（Linux、NginX、MySQL和PHP-FPM）四者各种是优秀的系统或软件，但是组合到一起的总体性能并不尽人意，明显的不是<code>1+1+1+1&gt;4</code>，而是<code>4+3+2+1&lt;1</code>。Linux系统无可厚非，主要问题出现在：</p><h3 id="从NginX到PHP-FPM"><a href="#从NginX到PHP-FPM" class="headerlink" title="从NginX到PHP-FPM"></a>从NginX到PHP-FPM</h3><p>NginX利用IO多路复用机制<a href="https://en.wikipedia.org/wiki/Epoll" target="_blank" rel="noopener">epoll</a>，极大地减少了IO阻塞等待，可以轻松应对<a href="https://en.wikipedia.org/wiki/C10k_problem" target="_blank" rel="noopener">C10K</a>。可是每次NginX将用户请求传递给PHP-FPM时，PHP-FPM总是需要从新加载PHP项目代码：创建执行环境，读取PHP文件和代码解析、编译等操作一次又一次的重复执行，造成不小的消耗。</p><h3 id="从PHP-FPM到MySQL"><a href="#从PHP-FPM到MySQL" class="headerlink" title="从PHP-FPM到MySQL"></a>从PHP-FPM到MySQL</h3><p>由于PHP代码本身是同步执行，PHP-FPM连接MySQL查询数据时，只能空闲等待MySQL返回查询结果。一个查询语句执行时间可能会需要几秒钟，期间PHP-FPM若是能暂时放下当前用户慢查询请求，而去处理其他用户请求，效率必然有所提高。</p><a id="more"></a><h2 id="Swoole-HTTP服务器"><a href="#Swoole-HTTP服务器" class="headerlink" title="Swoole HTTP服务器"></a>Swoole HTTP服务器</h2><p><a href="https://wiki.swoole.com/wiki/page/326.html" target="_blank" rel="noopener">Swoole HTTP服务器</a>也采用了epoll机制，运行性能与NginX相比，虽不及，犹未远。不过Swoole HTTP服务器嵌入PHP中作为其一部分，可以直接运行PHP，完全可以取代NginX + PHP-FPM组合。</p><p>以目前流行的为框架<a href="https://lumen.laravel.com/" target="_blank" rel="noopener">Lumen</a>（Laravel的子框架）为例，用Swoole HTTP服务器运行Lumen项目十分简单，只需要在<code>$worker-&gt;onRequest($request, $response)</code>（收到用户请求）时将<code>$request</code>传给Lumen处理，<code>$response</code>再将Lumen的处理结果返回给用户，而且<code>$worker</code>的整个生命周期里只会加载一次Lumen项目代码，没有多余的磁盘IO和PHP代码编译的开销。</p><h3 id="压力测试"><a href="#压力测试" class="headerlink" title="压力测试"></a>压力测试</h3><p>在4GB+4Core的虚拟机下，测试HTTP服务器的静态输出：</p><ul><li><p>2000客户端并发500000请求，不开启HTTP Keepalive，平均QPS：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">NginX + HTML               QPS：25883.44</span><br><span class="line">NginX + PHP-FPM + Lumen    QPS：828.36</span><br><span class="line">Swoole + Lumen             QPS：13647.75</span><br></pre></td></tr></table></figure></li><li><p>2000客户端并发500000请求，开启HTTP Keepalive，平均QPS：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">NginX + HTML               QPS：86843.11</span><br><span class="line">NginX + PHP-FPM + Lumen    QPS：894.06</span><br><span class="line">Swoole + Lumen             QPS：18183.43</span><br></pre></td></tr></table></figure></li></ul><p>可以看出，<code>Swoole + Lumen</code>组合的执行效率远高于<code>NginX + PHP-FPM + Lumen</code>组合。</p><h2 id="异步MySQL客户端"><a href="#异步MySQL客户端" class="headerlink" title="异步MySQL客户端"></a>异步MySQL客户端</h2><blockquote><p>以上都是铺垫，以下才是整篇文章的重点😂😂😂</p></blockquote><p>一个PHP应用要做的事不会是单纯的数据计算和数据输出，更多的是与数据库数据交互。以MySQL数据库为例，在只有一个PHP进程的情况，有10个用户同时请求执行<code>select sleep(1);</code>（耗时1秒）查询语句，若是使用MySQL同步查询，那么总耗时至少是10秒；若是使用MySQL异步查询，那么总耗时可能压缩到1到2秒内。</p><p>在PHP应用中能够实现数据库异步查询，才能更大的突破性能瓶颈。</p><p>虽然Swoole提供了<a href="https://wiki.swoole.com/wiki/page/517.html" target="_blank" rel="noopener">异步MySQL客户端</a>，但是其异步编程风格与Lumen这种同步编程风格的项目框架冲突，那么有没有可能在同步编程风格代码中调用异步MySQL客户端呢？</p><blockquote><p>一开始我觉得这是不可能的，直到看了这篇文章：<a href="http://nikic.github.io/2012/12/22/Cooperative-multitasking-using-coroutines-in-PHP.html" target="_blank" rel="noopener">Cooperative multitasking using coroutines (in PHP!)</a>。当然，我看的是中文版：<a href="http://www.laruence.com/2015/05/28/3038.html" target="_blank" rel="noopener"> 在PHP中使用协程实现多任务调度</a>，文中提到了PHP5.5加入的一个新功能：<a href="http://php.net/manual/en/language.generators.syntax.php" target="_blank" rel="noopener">yield</a>。</p></blockquote><h3 id="Yield"><a href="#Yield" class="headerlink" title="Yield"></a>Yield</h3><p><code>yield</code>是个动词，意思是“生成”，PHP中<code>yield</code>生出的东西叫<code>Generator</code>，意思是“生成器”😂😂😂。</p><p>个人理解是：yield将当前执行的上下文作为当前函数的结果返回（yield必须在函数中使用）。</p><p>在系统层面，各个进程的运行秩序由CPU调度；而有了yield，在PHP进程内，程序员可以自由调度各个代码块的执行顺序。比如，当“发现”当前用户请求的MySQL查询将会花费较多的时间，那么可以将当前执行上下文记录起来，交给异步MySQL客户端处理（与用户请求相关的<code>$request</code>和<code>$response</code>也传递过去），而主进程继续处理下一个用户请求。</p><h3 id="约定声明"><a href="#约定声明" class="headerlink" title="约定声明"></a>约定声明</h3><p>前面用了“发现”这个词，当然程序不可能智能地发现还没执行的查询语句将会是个慢查询，我们需要一些约定和声明。<br>Lumen框架是经典的MVC模式，我们约定C即Controller是处理用户请求的最后一步——Controller接受用户请求<code>$request</code>并返回响应<code>$response</code>。同时我们声明一个类，叫<code>SlowQuery</code>，这个类十分简单（具体请参见<a href="https://github.com/breeze2/lumen-swoole-http/blob/master/src/Database/SlowQuery.php" target="_blank" rel="noopener">SlowQuery.php</a>）：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="keyword">namespace</span> <span class="title">BL</span>\<span class="title">SwooleHttp</span>\<span class="title">Database</span>;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">SlowQuery</span></span></span><br><span class="line"><span class="class"></span>&#123;</span><br><span class="line">    <span class="keyword">public</span> $sql = <span class="string">''</span>;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">__construct</span><span class="params">($sql)</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="keyword">$this</span>-&gt;sql    = $sql;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>比如，Lumen项目中有这么一个Controller：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="keyword">namespace</span> <span class="title">App</span>\<span class="title">Http</span>\<span class="title">Controllers</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">App</span>\<span class="title">Http</span>\<span class="title">Controllers</span>\<span class="title">Controller</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">DB</span>;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">TestController</span> <span class="keyword">extends</span> <span class="title">Controller</span></span></span><br><span class="line"><span class="class"></span>&#123;</span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">test</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        $a = DB::select(<span class="string">'select sleep(1);'</span>);</span><br><span class="line">        response()-&gt;json($a);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>上面的<code>DB::select</code>使用的同步MySQL客户端查询，我们用<code>SlowQuery</code>对象替换它：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="keyword">namespace</span> <span class="title">App</span>\<span class="title">Http</span>\<span class="title">Controllers</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">App</span>\<span class="title">Http</span>\<span class="title">Controllers</span>\<span class="title">Controller</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">BL</span>\<span class="title">SwooleHttp</span>\<span class="title">Database</span>\<span class="title">SlowQuery</span>;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">TestController</span> <span class="keyword">extends</span> <span class="title">Controller</span></span></span><br><span class="line"><span class="class"></span>&#123;</span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">test</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        $a = <span class="keyword">yield</span> <span class="keyword">new</span> SlowQuery(<span class="string">'select sleep(1);'</span>);</span><br><span class="line">        response()-&gt;json($a);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>以Swoole HTTP服务器运行Lumen项目时，我们一定会获取Controller的返回结果。Controller的返回结果一般可以直接包装成Lumen响应返回给用户的，但返回结果若是一个生成器Generator对象，而且其当前值是一个慢查询SlowQuery对象的话，那么我们可以取出SlowQuery对象的sql属性，交由异步MySQL客户端执行；在异步查询的回调函数中将查询结果放回Generator对象存储的上下文中运行，得到最后结果才返回给用户；而主进程没有阻塞，可以继续处理其他用户请求。</p><p>当然，如果想用<a href="http://laravel.com/docs/eloquent" target="_blank" rel="noopener">Eloquent ORM</a>，那也很简单：我们先继承Lumen的Model，封装成一个新的Model类（具体参见<a href="https://github.com/breeze2/lumen-swoole-http/blob/master/src/Database/Model.php" target="_blank" rel="noopener">Model.php</a>），应用中的数据模型都继承于新的Model，Controller就可以这样写：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="keyword">namespace</span> <span class="title">App</span>\<span class="title">Http</span>\<span class="title">Controllers</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">App</span>\<span class="title">Http</span>\<span class="title">Controllers</span>\<span class="title">Controller</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">App</span>\<span class="title">Models</span>\<span class="title">User</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">DB</span>;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">TestController</span> <span class="keyword">extends</span> <span class="title">Controller</span></span></span><br><span class="line"><span class="class"></span>&#123;</span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">test</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        $a = <span class="keyword">yield</span> User::select(DB::raw(<span class="string">'sleep(1)'</span>))-&gt;yieldGet(); <span class="comment">// 注意User须继承自\BL\SwooleHttp\Database\Model</span></span><br><span class="line">        response()-&gt;json($a);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>以上三个Controller最终产出的用户响应都是一样的，不过后两者使用的是异步MySQL客户端，效率更高。</p><h3 id="任务调度器"><a href="#任务调度器" class="headerlink" title="任务调度器"></a>任务调度器</h3><p>当然，我们还需要一个任务调度器来执行这些生成器，任务调度器的实现方法<a href="http://www.laruence.com/2015/05/28/3038.html" target="_blank" rel="noopener"> 在PHP中使用协程实现多任务调度</a>文中“多任务协作”章节里有介绍，这里不展开。<br>Lumen框架中的代码保持了同步编程风格，而任务调度器中使用了异步编程风格来调用异步MySQL客户端。任务调度器是在Swoole HTTP服务器层面使用的，具体参见<a href="https://github.com/breeze2/lumen-swoole-http/blob/master/src/Service.php" target="_blank" rel="noopener">Service.php</a>。</p><h3 id="连接限制"><a href="#连接限制" class="headerlink" title="连接限制"></a>连接限制</h3><p>其实，每开启一个Swoole异步MySQL客户端，主进程就会新建一个线程连接MySQL，若是建立太多连接（线程），会增加自身服务器的压力，也会增加MySQL数据库服务器的压力。<br>这种利用yield来调用异步MySQL客户端处理慢查询而产生的线程，暂且称它为“慢查询协程”。<br>为了限制数据库连接数量，我们可以设置一个全局变量记录可新建慢查询协程的数量<code>MAX_COROUTINE</code>，开启一个异步MySQL客户端时让其减一，关闭一个异步MySQL客户端时让其加一；当用户请求慢查询时，<code>MAX_COROUTINE</code>大于0则由异步MySQL客户端处理，<code>MAX_COROUTINE</code>等于0时则由主进程“硬着头皮”自己处理。</p><h3 id="压力测试-1"><a href="#压力测试-1" class="headerlink" title="压力测试"></a>压力测试</h3><p>在4GB+4Core的虚拟机下，测试HTTP服务器与数据库读写：</p><h4 id="一般的快速查询和快速写入测试："><a href="#一般的快速查询和快速写入测试：" class="headerlink" title="一般的快速查询和快速写入测试："></a>一般的快速查询和快速写入测试：</h4><ul><li><p>200并发50000请求读，利用HTTP Keepalive，平均QPS：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">NginX + PHP-FPM + Lumen + MySQL    QPS：521.56</span><br><span class="line">Swoole + Lumen + MySQL             QPS：7509.99</span><br></pre></td></tr></table></figure></li><li><p>200并发50000请求写，利用HTTP Keepalive，平均QPS：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">NginX + PHP-FPM + Lumen + MySQL    QPS：449.44</span><br><span class="line">Swoole + Lumen + MySQL             QPS：1253.93</span><br></pre></td></tr></table></figure></li></ul><h4 id="慢查询协程测试："><a href="#慢查询协程测试：" class="headerlink" title="慢查询协程测试："></a>慢查询协程测试：</h4><ul><li>16worker的Swoole HTTP服务器，并发执行<code>select sleep(1);</code>请求的最大效率是15.72rps；</li><li>16worker x 10coroutine的Swoole HTTP服务器，并发执行<code>select sleep(1);</code>请求的最大效率是151.93rps。</li></ul><blockquote><p>这里为什么说最大效率呢？因为当并发量远大于worker数目 x coroutine数目时，可开启慢查询协程的Swoole HTTP服务器的效率会逐渐跌向普通Swoole HTTP服务器。</p></blockquote><p><code>select sleep(1);</code>查询语句耗时1秒，每个用户请求都需要1秒时间来处理；不过，16进程的、每个进程可开启10个慢查询协程的Swoole HTTP服务器的每秒最多可以处理160个用户请求，而16进程的普通Swoole HTTP服务器每秒最多只能处理16个用户请求。</p><h3 id="延伸"><a href="#延伸" class="headerlink" title="延伸"></a>延伸</h3><p>其实利用yield，我们还可以实现各种各样的“协程”。比如，<a href="https://www.oschina.net/news/93248/swoole-2-1-released" target="_blank" rel="noopener">Swoole2.1版本</a>已经开始支持go函数与通道，后续我们可能还可以将Lumen Controller中一些IO阻塞的操作的上下文移至go函数里执行，这样既保留了同步编程的风格，由达到异步执行的性能。</p><h2 id="最后"><a href="#最后" class="headerlink" title="最后"></a>最后</h2><p>以上理论，已经在<a href="https://github.com/breeze2/lumen-swoole-http" target="_blank" rel="noopener">lumen-swoole-http</a>项目中实现。<br><code>lumen-swoole-http</code>是连接同步编程Lumen和异步编程Swoole的一座桥梁，可以帮助原生PHP的Lumen应用项目快速迁移到Swoole HTTP服务器上；当然也可以快速迁移回去😂。<br>有兴趣的同学可以尝试使用：</p><ul><li><a href="https://breeze2.github.io/lumen-swoole-http/#/1_installation">安装</a></li><li><a href="https://breeze2.github.io/lumen-swoole-http/#/1_usage">使用</a></li><li><a href="https://breeze2.github.io/lumen-swoole-http/#/1_configuration">配置</a></li><li><a href="https://breeze2.github.io/lumen-swoole-http/#/3_coroutine_for_slow_query">慢查询协程</a></li><li>…</li></ul>]]></content>
    
    <summary type="html">
    
      &lt;blockquote&gt;
&lt;p&gt;网络编程一直是PHP的短板，尽管&lt;a href=&quot;https://www.swoole.com/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;Swoole&lt;/a&gt;扩展弥补了这个缺陷，但是其编程风格偏向了NodeJS或GoLang，与原本的同步编程风格迥然相异。目前PHP的大部分主流应用框架依然是同步编程风格，所以一直在探索Swoole与同步编程结合的途径。&lt;br&gt;&lt;a href=&quot;https://github.com/breeze2/lumen-swoole-http&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;lumen-swoole-http&lt;/a&gt;正是连接同步编程Lumen和异步编程Swoole的一座桥梁，有兴趣可以关注一下。&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;LNMP的不足&quot;&gt;&lt;a href=&quot;#LNMP的不足&quot; class=&quot;headerlink&quot; title=&quot;LNMP的不足&quot;&gt;&lt;/a&gt;LNMP的不足&lt;/h2&gt;&lt;p&gt;LNMP是经典的Web应用架构组合，虽然（Linux、NginX、MySQL和PHP-FPM）四者各种是优秀的系统或软件，但是组合到一起的总体性能并不尽人意，明显的不是&lt;code&gt;1+1+1+1&amp;gt;4&lt;/code&gt;，而是&lt;code&gt;4+3+2+1&amp;lt;1&lt;/code&gt;。Linux系统无可厚非，主要问题出现在：&lt;/p&gt;
&lt;h3 id=&quot;从NginX到PHP-FPM&quot;&gt;&lt;a href=&quot;#从NginX到PHP-FPM&quot; class=&quot;headerlink&quot; title=&quot;从NginX到PHP-FPM&quot;&gt;&lt;/a&gt;从NginX到PHP-FPM&lt;/h3&gt;&lt;p&gt;NginX利用IO多路复用机制&lt;a href=&quot;https://en.wikipedia.org/wiki/Epoll&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;epoll&lt;/a&gt;，极大地减少了IO阻塞等待，可以轻松应对&lt;a href=&quot;https://en.wikipedia.org/wiki/C10k_problem&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;C10K&lt;/a&gt;。可是每次NginX将用户请求传递给PHP-FPM时，PHP-FPM总是需要从新加载PHP项目代码：创建执行环境，读取PHP文件和代码解析、编译等操作一次又一次的重复执行，造成不小的消耗。&lt;/p&gt;
&lt;h3 id=&quot;从PHP-FPM到MySQL&quot;&gt;&lt;a href=&quot;#从PHP-FPM到MySQL&quot; class=&quot;headerlink&quot; title=&quot;从PHP-FPM到MySQL&quot;&gt;&lt;/a&gt;从PHP-FPM到MySQL&lt;/h3&gt;&lt;p&gt;由于PHP代码本身是同步执行，PHP-FPM连接MySQL查询数据时，只能空闲等待MySQL返回查询结果。一个查询语句执行时间可能会需要几秒钟，期间PHP-FPM若是能暂时放下当前用户慢查询请求，而去处理其他用户请求，效率必然有所提高。&lt;/p&gt;
    
    </summary>
    
      <category term="求索" scheme="https://breeze2.github.io/blog/categories/%E6%B1%82%E7%B4%A2/"/>
    
    
      <category term="php" scheme="https://breeze2.github.io/blog/tags/php/"/>
    
      <category term="swoole" scheme="https://breeze2.github.io/blog/tags/swoole/"/>
    
      <category term="lumen" scheme="https://breeze2.github.io/blog/tags/lumen/"/>
    
      <category term="think" scheme="https://breeze2.github.io/blog/tags/think/"/>
    
  </entry>
  
  <entry>
    <title>在Lumen项目中自定义后置异步协程</title>
    <link href="https://breeze2.github.io/blog/swoole-lumen-define-post-processing-coroutine.html"/>
    <id>https://breeze2.github.io/blog/swoole-lumen-define-post-processing-coroutine.html</id>
    <published>2018-03-01T23:12:02.000Z</published>
    <updated>2019-09-30T02:22:37.086Z</updated>
    
    <content type="html"><![CDATA[<blockquote><p><a href="https://blog.breezelin.cn/swoole-lumen-use-async-mysql-client-in-lumen.html" target="_blank" rel="noopener">上一篇文章</a>介绍了利用yield特性实现后置的MySQL异步查询协程，按照这种“后置执行”的思路，其实还可以实现更多其他的后置异步协程。这里介绍如何自定义一个后置异步协程，将最大开发自由度交由Lumen框架使用者。</p></blockquote><h2 id="CustomAsyncProcess"><a href="#CustomAsyncProcess" class="headerlink" title="CustomAsyncProcess"></a>CustomAsyncProcess</h2><p><a href="https://breeze2.github.io/lumen-swoole-http/">lumen-swoole-http</a>中已经定义了抽象类<code>CustomAsyncProcess</code>，在处理用户请求时，若是捕获到<code>CustomAsyncProcess</code>类的对象，程序便会按照<code>CustomAsyncProcess</code>对象的指定方法执行。部分代码：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line">    ...</span><br><span class="line">    $current_value = $last_generator-&gt;current();</span><br><span class="line">    <span class="keyword">if</span> ($current_value <span class="keyword">instanceof</span> CustomAsyncProcess) &#123;</span><br><span class="line">        <span class="keyword">if</span> ($worker-&gt;canDoCoroutine()) &#123;</span><br><span class="line">            $worker-&gt;upCoroutineNum();</span><br><span class="line">            <span class="keyword">return</span> $current_value-&gt;runAsyncTask($request, $response, $worker, <span class="keyword">$this</span>-&gt;scheduler, $last_generator);</span><br><span class="line">        &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">            <span class="keyword">return</span> $current_value-&gt;runNormalTask($request, $response, $worker, <span class="keyword">$this</span>-&gt;scheduler, $last_generator);</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">    ...</span><br></pre></td></tr></table></figure><a id="more"></a><h2 id="一个简单的后置协程"><a href="#一个简单的后置协程" class="headerlink" title="一个简单的后置协程"></a>一个简单的后置协程</h2><p>我们先来实现一个简单的后置协程<code>EasyProcess</code>，继承自<code>CustomAsyncProcess</code>，需要实现两个抽象方法：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="keyword">namespace</span> <span class="title">App</span>\<span class="title">http</span>\<span class="title">AfterCoroutines</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">BL</span>\<span class="title">SwooleHttp</span>\<span class="title">Service</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">BL</span>\<span class="title">SwooleHttp</span>\<span class="title">Coroutine</span>\<span class="title">SimpleSerialScheduler</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">Generator</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">swoole_http_request</span> <span class="title">as</span> <span class="title">SwooleHttpRequest</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">swoole_http_response</span> <span class="title">as</span> <span class="title">SwooleHttpResponse</span>;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">EasyProcess</span> <span class="keyword">extends</span> \<span class="title">BL</span>\<span class="title">SwooleHttp</span>\<span class="title">Coroutine</span>\<span class="title">CustomAsyncProcess</span></span></span><br><span class="line"><span class="class"></span>&#123;</span><br><span class="line">    <span class="comment">// 因为有协程数量限制，达到最大协程数量时，会调用runNormalTask方法</span></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">runNormalTask</span><span class="params">(SwooleHttpRequest $request, SwooleHttpResponse $response, Service $worker, SimpleSerialScheduler $scheduler, Generator $last_generator)</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        $value = <span class="number">2</span>;</span><br><span class="line">        <span class="comment">// 给最后一层生成器，传入$value值，并递归导出整个生成器套层的最终值$final</span></span><br><span class="line">        $final = <span class="keyword">$this</span>-&gt;fullRunScheduler($scheduler, $last_generator, $value);</span><br><span class="line">        $http_response = $final;</span><br><span class="line">        <span class="comment">// 注意：runNormalTask方法中使用makeNormalResponse方法响应用户请求</span></span><br><span class="line">        <span class="keyword">$this</span>-&gt;makeNormalResponse($request, $response, $worker, $http_response);</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// 没达到最大协程数量时，会调用runAsyncTask方法</span></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">runAsyncTask</span><span class="params">(SwooleHttpRequest $request, SwooleHttpResponse $response, Service $worker, SimpleSerialScheduler $scheduler, Generator $last_generator)</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        $value = <span class="number">1</span>;</span><br><span class="line">        <span class="comment">// 给最后一层生成器，传入$value值，并递归导出整个生成器套层的最终值$final</span></span><br><span class="line">        $final = <span class="keyword">$this</span>-&gt;fullRunScheduler($scheduler, $last_generator, $value);</span><br><span class="line">        $http_response = $final;</span><br><span class="line">        <span class="comment">// 注意：runAsyncTask方法中使用makeAsyncResponse方法响应用户请求</span></span><br><span class="line">        <span class="keyword">$this</span>-&gt;makeAsyncResponse($request, $response, $worker, $http_response);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>然后就可以在Controller中使用这个后置协程：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="keyword">namespace</span> <span class="title">App</span>\<span class="title">Http</span>\<span class="title">Controllers</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">App</span>\<span class="title">http</span>\<span class="title">AfterCoroutines</span>\<span class="title">EasyProcess</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">App</span>\<span class="title">Http</span>\<span class="title">Controllers</span>\<span class="title">Controller</span>;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">TestController</span> <span class="keyword">extends</span> <span class="title">Controller</span></span></span><br><span class="line"><span class="class"></span>&#123;</span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">test</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        $a = <span class="keyword">yield</span> <span class="keyword">new</span> EasyProcess();</span><br><span class="line">        response()-&gt;json($a);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>当访问<code>TestController@test</code>对应的路由时，得到的返回结果有可能是1，也有可能是2，视当时协程数量而定。1或2这个结果在<code>TestController@test</code>层面上是未曾知的，是由<code>EasyProcess</code>这个后置协程产出的。</p><p><code>EasyProcess</code>不算一个异步协程，无论<code>runNormalTask</code>方法或者<code>runAsyncTask</code>方法，都是同步执行的。若想实现异步执行，必须使用Swoole提供的异步客户端，详细请查看Swoole官方文档<a href="https://wiki.swoole.com/wiki/page/p-async.html" target="_blank" rel="noopener">AsyncIO</a>。</p><p>下面介绍如何实现一个后置的异步HTTP客户端协程。</p><h2 id="一个后置的异步HTTP客户端协程"><a href="#一个后置的异步HTTP客户端协程" class="headerlink" title="一个后置的异步HTTP客户端协程"></a>一个后置的异步HTTP客户端协程</h2><p>假设我们处理某个用户请求时，需要从远端链接<code>http://www.domain.com/api/data</code>获取一些数据，期间可能需要耗时几秒钟，那么怎样用后置异步协程实现来避免阻塞呢？</p><p>我们先来实现一个简单的后置协程<code>AysncHttpProcess</code>，继承自<code>CustomAsyncProcess</code>，需要实现两个抽象方法：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="keyword">namespace</span> <span class="title">App</span>\<span class="title">http</span>\<span class="title">AfterCoroutines</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">BL</span>\<span class="title">SwooleHttp</span>\<span class="title">Service</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">BL</span>\<span class="title">SwooleHttp</span>\<span class="title">Coroutine</span>\<span class="title">SimpleSerialScheduler</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">Generator</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">swoole_http_request</span> <span class="title">as</span> <span class="title">SwooleHttpRequest</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">swoole_http_response</span> <span class="title">as</span> <span class="title">SwooleHttpResponse</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">Swoole</span>\<span class="title">Http2</span>\<span class="title">Client</span> <span class="title">as</span> <span class="title">SwooleHttpClient</span>;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">AysncHttpProcess</span> <span class="keyword">extends</span> \<span class="title">BL</span>\<span class="title">SwooleHttp</span>\<span class="title">Coroutine</span>\<span class="title">CustomAsyncProcess</span></span></span><br><span class="line"><span class="class"></span>&#123;</span><br><span class="line">    public function $url;</span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">__construct</span><span class="params">($url)</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="keyword">$this</span>-&gt;url = $url;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// 因为有协程数量限制，达到最大协程数量时，会调用runNormalTask方法</span></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">runNormalTask</span><span class="params">(SwooleHttpRequest $request, SwooleHttpResponse $response, Service $worker, SimpleSerialScheduler $scheduler, Generator $last_generator)</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        $data = file_get_contents(<span class="keyword">$this</span>-&gt;url);</span><br><span class="line">        <span class="comment">// 给最后一层生成器，传入$value值，并递归导出整个生成器套层的最终值$final</span></span><br><span class="line">        $final = <span class="keyword">$this</span>-&gt;fullRunScheduler($scheduler, $last_generator, $data);</span><br><span class="line">        $http_response = $final;</span><br><span class="line">        <span class="comment">// 注意：runNormalTask方法中使用makeNormalResponse方法响应用户请求</span></span><br><span class="line">        <span class="keyword">$this</span>-&gt;makeNormalResponse($request, $response, $worker, $http_response);</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="comment">// 没达到最大协程数量时，会调用runAsyncTask方法</span></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">runAsyncTask</span><span class="params">(SwooleHttpRequest $request, SwooleHttpResponse $response, Service $worker, SimpleSerialScheduler $scheduler, Generator $last_generator)</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        $client = <span class="keyword">new</span> SwooleHttpClient();</span><br><span class="line">        $caller = <span class="keyword">$this</span>;</span><br><span class="line">        $client-&gt;get(<span class="keyword">$this</span>-&gt;url, <span class="function"><span class="keyword">function</span> <span class="params">($o)</span> <span class="title">use</span><span class="params">($client, $caller)</span> </span>&#123;</span><br><span class="line">            $data = $o-&gt;body;</span><br><span class="line">            <span class="comment">// 给最后一层生成器，传入$value值，并递归导出整个生成器套层的最终值$final</span></span><br><span class="line">            $final = $caller-&gt;fullRunScheduler($scheduler, $last_generator, $value);</span><br><span class="line">            $http_response = $final;</span><br><span class="line">            <span class="comment">// 注意：runAsyncTask方法中使用makeAsyncResponse方法响应用户请求</span></span><br><span class="line">            $caller-&gt;makeAsyncResponse($request, $response, $worker, $http_response);</span><br><span class="line">            $client-&gt;close();</span><br><span class="line">        &#125;);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>然后就可以在Controller中使用这个后置协程：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="keyword">namespace</span> <span class="title">App</span>\<span class="title">Http</span>\<span class="title">Controllers</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">App</span>\<span class="title">http</span>\<span class="title">AfterCoroutines</span>\<span class="title">AysncHttpProcess</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">App</span>\<span class="title">Http</span>\<span class="title">Controllers</span>\<span class="title">Controller</span>;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">TestController</span> <span class="keyword">extends</span> <span class="title">Controller</span></span></span><br><span class="line"><span class="class"></span>&#123;</span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">test</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        $a = <span class="keyword">yield</span> <span class="keyword">new</span> AysncHttpProcess(<span class="string">'http://www.domain.com/api/data'</span>);</span><br><span class="line">        response()-&gt;json($a);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>就这样，在协程数量允许情况下，我们可以使用后置的异步HTTP客户端协程获取远端数据，并且避免了阻塞。</p><h2 id="最后"><a href="#最后" class="headerlink" title="最后"></a>最后</h2><p>三篇文章：</p><ol><li><a href="https://blog.breezelin.cn/swoole-lumen-run-lumen-with-swoole.html" target="_blank" rel="noopener">用Swoole HTTP服务器运行Lumen项目的实现方法</a></li><li><a href="https://blog.breezelin.cn/swoole-lumen-use-async-mysql-client-in-lumen.html" target="_blank" rel="noopener">在Lumen项目中使用Swoole异步MySQL客户端的实现方法</a></li><li><a href>在Lumen项目中自定义后置异步协程</a></li></ol><p>至此，已经将<a href="https://breeze2.github.io/lumen-swoole-http/">lumen-swoole-http</a>的设计理念介绍完了，主要是两点：</p><ul><li>利用Swoole HTTP服务器运行Lumen项目，提高执行效率；</li><li>利用后置异步协程，保持同步编程风格，避免主进程阻塞。</li></ul><blockquote><p>Swoole填补了PHP网络编程的缺陷，引入NodeJS、GoLang等语言的并行执行特性，使用了不同的编程风格。对此，不应该一味抗拒，而是学会融会贯通——学无止境，知识没有界限，PHPer也应该学习其他语言的设计理念，特别是为高并发而生的GoLang。</p></blockquote>]]></content>
    
    <summary type="html">
    
      &lt;blockquote&gt;
&lt;p&gt;&lt;a href=&quot;https://blog.breezelin.cn/swoole-lumen-use-async-mysql-client-in-lumen.html&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;上一篇文章&lt;/a&gt;介绍了利用yield特性实现后置的MySQL异步查询协程，按照这种“后置执行”的思路，其实还可以实现更多其他的后置异步协程。这里介绍如何自定义一个后置异步协程，将最大开发自由度交由Lumen框架使用者。&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;CustomAsyncProcess&quot;&gt;&lt;a href=&quot;#CustomAsyncProcess&quot; class=&quot;headerlink&quot; title=&quot;CustomAsyncProcess&quot;&gt;&lt;/a&gt;CustomAsyncProcess&lt;/h2&gt;&lt;p&gt;&lt;a href=&quot;https://breeze2.github.io/lumen-swoole-http/&quot;&gt;lumen-swoole-http&lt;/a&gt;中已经定义了抽象类&lt;code&gt;CustomAsyncProcess&lt;/code&gt;，在处理用户请求时，若是捕获到&lt;code&gt;CustomAsyncProcess&lt;/code&gt;类的对象，程序便会按照&lt;code&gt;CustomAsyncProcess&lt;/code&gt;对象的指定方法执行。部分代码：&lt;/p&gt;
&lt;figure class=&quot;highlight php&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;2&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;3&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;4&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;5&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;6&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;7&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;8&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;9&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;10&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;11&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;12&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;meta&quot;&gt;&amp;lt;?php&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    ...&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    $current_value = $last_generator-&amp;gt;current();&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;keyword&quot;&gt;if&lt;/span&gt; ($current_value &lt;span class=&quot;keyword&quot;&gt;instanceof&lt;/span&gt; CustomAsyncProcess) &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;        &lt;span class=&quot;keyword&quot;&gt;if&lt;/span&gt; ($worker-&amp;gt;canDoCoroutine()) &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            $worker-&amp;gt;upCoroutineNum();&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            &lt;span class=&quot;keyword&quot;&gt;return&lt;/span&gt; $current_value-&amp;gt;runAsyncTask($request, $response, $worker, &lt;span class=&quot;keyword&quot;&gt;$this&lt;/span&gt;-&amp;gt;scheduler, $last_generator);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;        &amp;#125; &lt;span class=&quot;keyword&quot;&gt;else&lt;/span&gt; &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            &lt;span class=&quot;keyword&quot;&gt;return&lt;/span&gt; $current_value-&amp;gt;runNormalTask($request, $response, $worker, &lt;span class=&quot;keyword&quot;&gt;$this&lt;/span&gt;-&amp;gt;scheduler, $last_generator);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;        &amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    ...&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;
    
    </summary>
    
      <category term="求索" scheme="https://breeze2.github.io/blog/categories/%E6%B1%82%E7%B4%A2/"/>
    
    
      <category term="php" scheme="https://breeze2.github.io/blog/tags/php/"/>
    
      <category term="swoole" scheme="https://breeze2.github.io/blog/tags/swoole/"/>
    
      <category term="lumen" scheme="https://breeze2.github.io/blog/tags/lumen/"/>
    
      <category term="think" scheme="https://breeze2.github.io/blog/tags/think/"/>
    
  </entry>
  
  <entry>
    <title>在Lumen项目中使用Swoole异步MySQL客户端的实现方法</title>
    <link href="https://breeze2.github.io/blog/swoole-lumen-use-async-mysql-client-in-lumen.html"/>
    <id>https://breeze2.github.io/blog/swoole-lumen-use-async-mysql-client-in-lumen.html</id>
    <published>2018-03-01T16:03:02.000Z</published>
    <updated>2019-09-30T02:22:37.086Z</updated>
    
    <content type="html"><![CDATA[<blockquote><p><a href="https://blog.breezelin.cn/swoole-lumen-run-lumen-with-swoole.html" target="_blank" rel="noopener">上一篇文章</a>介绍了用Swoole HTTP服务器替代NginX + PHP-FPM来运行Lumen项目的方法，提高运行效率。不过，在处理用户请求过程中还是会存在很多IO阻塞情况，比如MySQL数据查询。有没有可能在Lumen中使用异步MySQL客户端，以此避免IO阻塞呢？</p></blockquote><h2 id="异步MySQL客户端"><a href="#异步MySQL客户端" class="headerlink" title="异步MySQL客户端"></a>异步MySQL客户端</h2><p>Swoole1.8.6增加了内置异步MySQL客户端的支持，无需依赖其他第三方库，使用方法也非常简单，（具体参考Swoole官方文档<a href="https://wiki.swoole.com/wiki/page/517.html" target="_blank" rel="noopener">异步MySQL客户端</a>）：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line">$db = <span class="keyword">new</span> \swoole_mysql;</span><br><span class="line">$mysql_config = <span class="keyword">array</span>(</span><br><span class="line">    <span class="string">'host'</span> =&gt; <span class="string">'192.168.56.102'</span>,</span><br><span class="line">    <span class="string">'port'</span> =&gt; <span class="number">3306</span>,</span><br><span class="line">    <span class="string">'user'</span> =&gt; <span class="string">'test'</span>,</span><br><span class="line">    <span class="string">'password'</span> =&gt; <span class="string">'test'</span>,</span><br><span class="line">    <span class="string">'database'</span> =&gt; <span class="string">'test'</span>,</span><br><span class="line">    <span class="string">'charset'</span> =&gt; <span class="string">'utf8'</span>, <span class="comment">//指定字符集</span></span><br><span class="line">    <span class="string">'timeout'</span> =&gt; <span class="number">2</span>,</span><br><span class="line">);</span><br><span class="line"></span><br><span class="line">$db-&gt;connect($mysql_config, <span class="function"><span class="keyword">function</span> <span class="params">($db, $r)</span> </span>&#123;</span><br><span class="line">    <span class="keyword">if</span> ($r) &#123;</span><br><span class="line">        $sql = <span class="string">'show tables'</span>;</span><br><span class="line">            $db-&gt;query($sql, <span class="function"><span class="keyword">function</span><span class="params">($db, $r)</span> </span>&#123;</span><br><span class="line">            <span class="keyword">if</span> ($r) &#123;</span><br><span class="line">                var_dump($r);</span><br><span class="line">            &#125;</span><br><span class="line">            $db-&gt;close();</span><br><span class="line">        &#125;);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;);</span><br></pre></td></tr></table></figure><a id="more"></a><p>这是典型回调处理的异步编程风格，而Lumen本身是同步编程风格，在编程层面，两者不能融合。比如，有这么一个<code>Controller</code>：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="keyword">namespace</span> <span class="title">App</span>\<span class="title">Http</span>\<span class="title">Controllers</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">App</span>\<span class="title">Http</span>\<span class="title">Controllers</span>\<span class="title">Controller</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">DB</span>;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">TestController</span> <span class="keyword">extends</span> <span class="title">Controller</span></span></span><br><span class="line"><span class="class"></span>&#123;</span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">test</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        $a = DB::query(<span class="string">'select * from users;'</span>);</span><br><span class="line">        response()-&gt;json($a);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>若是改写成：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="keyword">namespace</span> <span class="title">App</span>\<span class="title">Http</span>\<span class="title">Controllers</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">App</span>\<span class="title">Http</span>\<span class="title">Controllers</span>\<span class="title">Controller</span>;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">TestController</span> <span class="keyword">extends</span> <span class="title">Controller</span></span></span><br><span class="line"><span class="class"></span>&#123;</span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">test</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        $a = <span class="keyword">null</span>;</span><br><span class="line">        $db = <span class="keyword">new</span> \swoole_mysql;</span><br><span class="line">        $db-&gt;connect($mysql_config, <span class="function"><span class="keyword">function</span> <span class="params">($db, $r)</span> <span class="title">use</span> <span class="params">($a)</span> </span>&#123;</span><br><span class="line">            <span class="keyword">if</span> ($r) &#123;</span><br><span class="line">                $db-&gt;query(<span class="string">'select * from users;'</span>, <span class="function"><span class="keyword">function</span><span class="params">($db, $r)</span> <span class="title">use</span> <span class="params">($a)</span> </span>&#123;</span><br><span class="line">                    <span class="keyword">if</span> ($r) &#123;</span><br><span class="line">                        $a = json_decode(json_encode($r));</span><br><span class="line">                    &#125;</span><br><span class="line">                    $db-&gt;close();</span><br><span class="line">                &#125;);</span><br><span class="line">            &#125;</span><br><span class="line">        &#125;);</span><br><span class="line">        response()-&gt;json($a);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>这样返回的响应永远是<code>null</code>，因为<code>response()-&gt;json($a)</code>会在<code>$db-&gt;query()</code>之前被执行。</p><blockquote><p>一开始我也觉得Lumen项目里永远没办法使用异步MySQL客户端了，直到看了这篇文章：<a href="http://nikic.github.io/2012/12/22/Cooperative-multitasking-using-coroutines-in-PHP.html" target="_blank" rel="noopener">Cooperative multitasking using coroutines (in PHP!)</a>。当然，我看的是中文版：<a href="http://www.laruence.com/2015/05/28/3038.html" target="_blank" rel="noopener"> 在PHP中使用协程实现多任务调度</a>，文中提到了PHP5.5加入的一个新功能：<a href="http://php.net/manual/en/language.generators.syntax.php" target="_blank" rel="noopener">yield</a>。</p></blockquote><h2 id="yield"><a href="#yield" class="headerlink" title="yield"></a>yield</h2><p><code>yield</code>是个动词，意思是“生成”，PHP中yield生出的东西叫<code>Generator</code>，译作“生成器”。<br>yield可以做什么呢？yield可以将当前执行的上下文作为当前函数的结果返回（yield必须在函数中使用）。<br>有了yield，又能怎样呢？</p><p>首先，我们声明一个类，叫<code>SlwoQuery</code>：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="keyword">namespace</span> <span class="title">BL</span>\<span class="title">SwooleHttp</span>\<span class="title">Database</span>;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">SlowQuery</span></span></span><br><span class="line"><span class="class"></span>&#123;</span><br><span class="line">    <span class="keyword">public</span> $sql = <span class="string">''</span>;</span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">__construct</span><span class="params">($sql)</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="keyword">$this</span>-&gt;sql = $sql;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>结合上一篇文章《<a href="https://blog.breezelin.cn/swoole-lumen-run-lumen-with-swoole.html" target="_blank" rel="noopener">用Swoole HTTP服务器运行Lumen项目的实现方法</a>》，我们修改一下<code>Service</code>类的<code>onRequest</code>方法：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">onRequest</span><span class="params">(\swoole_http_request $request,\swoole_http_response $response)</span> </span>&#123;</span><br><span class="line">        $app = <span class="keyword">$this</span>-&gt;app;</span><br><span class="line">        <span class="comment">// 处理用户请求</span></span><br><span class="line">        $http_request = <span class="keyword">$this</span>-&gt;parseRequest($request);</span><br><span class="line">        $http_response = $app-&gt;dispatch($http_request);</span><br><span class="line"></span><br><span class="line">        <span class="keyword">if</span> ($http_response <span class="keyword">instanceof</span> \Generator) &#123;</span><br><span class="line">            $gen = $http_response-&gt;current();</span><br><span class="line">            $gen_queue = <span class="keyword">new</span> \SplQueue();</span><br><span class="line">            <span class="keyword">while</span>($gen <span class="keyword">instanceof</span> Generator) &#123;</span><br><span class="line">                $gen_queue-&gt;push($gen); $gen = $gen-&gt;current();</span><br><span class="line">            &#125;</span><br><span class="line">            $last_gen = $gen_queue-&gt;pop();</span><br><span class="line">            $value = $last_gen-&gt;current();</span><br><span class="line">            <span class="keyword">if</span> ($value <span class="keyword">instanceof</span> \BL\SwooleHttp\Database\SlowQuery) &#123;</span><br><span class="line">                $db = <span class="keyword">new</span> \swoole_mysql;</span><br><span class="line">                $caller = <span class="keyword">$this</span>;</span><br><span class="line">                <span class="comment">// 关键部分</span></span><br><span class="line">                $db-&gt;connect($mysql_config, <span class="function"><span class="keyword">function</span> <span class="params">($db, $r)</span> <span class="title">use</span> <span class="params">($caller, $request, $response, $gen_queue, $last_gen, $value)</span> </span>&#123;</span><br><span class="line">                    <span class="keyword">if</span> ($r) &#123;</span><br><span class="line">                        $db-&gt;query($value-&gt;sql, <span class="function"><span class="keyword">function</span><span class="params">($db, $r)</span> <span class="title">use</span> <span class="params">($caller, $request, $response, $gen_queue, $last_gen)</span> </span>&#123;</span><br><span class="line">                            <span class="keyword">if</span> ($r) &#123;</span><br><span class="line">                                <span class="comment">// 关键部分</span></span><br><span class="line">                                $r = json_decode(json_encode($r));</span><br><span class="line">                                $last_gen-&gt;send($r);</span><br><span class="line">                                $ret = $last_gen-&gt;getReturn();</span><br><span class="line">                                <span class="keyword">while</span>(!$gen_queue-&gt;isEmpty()) &#123;</span><br><span class="line">                                    $gen = $gen_queue-&gt;pop(); $gen-&gt;send($ret); $ret = $gen-&gt;getReturn();</span><br><span class="line">                                &#125;</span><br><span class="line">                                $caller-&gt;makeResponse($response, $ret);</span><br><span class="line">                            &#125;</span><br><span class="line">                            $db-&gt;close();</span><br><span class="line">                        &#125;);</span><br><span class="line">                    &#125;</span><br><span class="line">                &#125;);</span><br><span class="line">            &#125;</span><br><span class="line">        &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">            <span class="comment">// 响应用户请求</span></span><br><span class="line">            <span class="keyword">$this</span>-&gt;makeResponse($response, $http_response);</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br></pre></td></tr></table></figure><p>代码是长了些，因为没做分拆和封装，主要关注<code>$db-&gt;connect()</code>和<code>$caller-&gt;makeResponse()</code>的出现位置。整块代码的意思是：</p><ol><li>如果Lumen处理用户请求的返回结果是一个生成器，那么就从这个生成器的函数套层里寻找（SlowQuery）；</li><li>如果当前生成器的当前值也是一个生成器，那么就往更深一层里寻找；</li><li>如果当前生成器的当前值是一个<code>SlowQuery</code>对象，那么将<code>SlowQuery</code>对象的<code>sql</code>属性，交给异步MySQL客户<code>swoole_mysql</code>查询数据；</li><li>将查询数据传入当前生成器，获得当前层函数的返回结果；</li><li>将函数返回结果传入上一层生成器，获取上一层函数的返回结果，重复直到没有更高层生成器；</li><li>才将最顶层层的函数返回结果作为响应输出给用户。<br>虽然整个过程很绕，但是，在Lumen的Controller层面却是十分的直接了然：<figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="keyword">namespace</span> <span class="title">App</span>\<span class="title">Http</span>\<span class="title">Controllers</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">App</span>\<span class="title">Http</span>\<span class="title">Controllers</span>\<span class="title">Controller</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">BL</span>\<span class="title">SwooleHttp</span>\<span class="title">Database</span>\<span class="title">SlowQuery</span>;</span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">TestController</span> <span class="keyword">extends</span> <span class="title">Controller</span></span></span><br><span class="line"><span class="class"></span>&#123;</span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">test</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        $a = <span class="keyword">yield</span> <span class="keyword">new</span> SlowQuery(<span class="string">'select * from users;'</span>);</span><br><span class="line">        response()-&gt;json($a);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure></li></ol><p>就这样，实现了在同步编程风格的Lumen项目中使用上了Swoole的异步MySQL客户端。实现的思路，就像<a href="https://lumen.laravel.com/docs/middleware" target="_blank" rel="noopener">后置中间件</a>，保持控制器代码的整洁，脏活累活放在中间件里执行。</p><h2 id="效率提升"><a href="#效率提升" class="headerlink" title="效率提升"></a>效率提升</h2><p>假如用户请求执行一个数据库慢查询语句（可能耗时1秒，如<code>select sleep(1);</code>），单个PHP进程使用同步MySQL客户端处理1个这样的用户请求，至少需要1秒，处理10个，则至少需要10秒；而使用异步MySQL客户端，处理1个，可能需要1秒多，处理10个，可能也只是需要1秒。异步执行带来的效率提升，是不言而喻的。</p><p>不过，使用异步MySQL客户端是会消耗系统资源的，不能大量使用；而且非慢查询的查询语句，根本不需要使用异步MySQL客户端来执行，比如<code>select * from users.id = 1;</code>，id有主键索引，查询时间不会太长。</p><h2 id="最后"><a href="#最后" class="headerlink" title="最后"></a>最后</h2><p>yield可以将整个程序的各个代码块的执行秩序交由程序员自行调度，确实可以实现很多意向不到的效果。像以上这种“后置执行”的思路，不仅仅是可以使用异步MySQL客户端，还可以使用异步Redis客户端，异步HTTP客户端，还有更多各种各样的应用。</p><p>下一篇文章介绍如何自定义后置异步协程。</p>]]></content>
    
    <summary type="html">
    
      &lt;blockquote&gt;
&lt;p&gt;&lt;a href=&quot;https://blog.breezelin.cn/swoole-lumen-run-lumen-with-swoole.html&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;上一篇文章&lt;/a&gt;介绍了用Swoole HTTP服务器替代NginX + PHP-FPM来运行Lumen项目的方法，提高运行效率。不过，在处理用户请求过程中还是会存在很多IO阻塞情况，比如MySQL数据查询。有没有可能在Lumen中使用异步MySQL客户端，以此避免IO阻塞呢？&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;异步MySQL客户端&quot;&gt;&lt;a href=&quot;#异步MySQL客户端&quot; class=&quot;headerlink&quot; title=&quot;异步MySQL客户端&quot;&gt;&lt;/a&gt;异步MySQL客户端&lt;/h2&gt;&lt;p&gt;Swoole1.8.6增加了内置异步MySQL客户端的支持，无需依赖其他第三方库，使用方法也非常简单，（具体参考Swoole官方文档&lt;a href=&quot;https://wiki.swoole.com/wiki/page/517.html&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;异步MySQL客户端&lt;/a&gt;）：&lt;/p&gt;
&lt;figure class=&quot;highlight php&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;2&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;3&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;4&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;5&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;6&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;7&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;8&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;9&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;10&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;11&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;12&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;13&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;14&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;15&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;16&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;17&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;18&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;19&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;20&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;21&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;22&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;23&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;&lt;span class=&quot;meta&quot;&gt;&amp;lt;?php&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;$db = &lt;span class=&quot;keyword&quot;&gt;new&lt;/span&gt; \swoole_mysql;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;$mysql_config = &lt;span class=&quot;keyword&quot;&gt;array&lt;/span&gt;(&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;string&quot;&gt;&#39;host&#39;&lt;/span&gt; =&amp;gt; &lt;span class=&quot;string&quot;&gt;&#39;192.168.56.102&#39;&lt;/span&gt;,&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;string&quot;&gt;&#39;port&#39;&lt;/span&gt; =&amp;gt; &lt;span class=&quot;number&quot;&gt;3306&lt;/span&gt;,&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;string&quot;&gt;&#39;user&#39;&lt;/span&gt; =&amp;gt; &lt;span class=&quot;string&quot;&gt;&#39;test&#39;&lt;/span&gt;,&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;string&quot;&gt;&#39;password&#39;&lt;/span&gt; =&amp;gt; &lt;span class=&quot;string&quot;&gt;&#39;test&#39;&lt;/span&gt;,&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;string&quot;&gt;&#39;database&#39;&lt;/span&gt; =&amp;gt; &lt;span class=&quot;string&quot;&gt;&#39;test&#39;&lt;/span&gt;,&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;string&quot;&gt;&#39;charset&#39;&lt;/span&gt; =&amp;gt; &lt;span class=&quot;string&quot;&gt;&#39;utf8&#39;&lt;/span&gt;, &lt;span class=&quot;comment&quot;&gt;//指定字符集&lt;/span&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;string&quot;&gt;&#39;timeout&#39;&lt;/span&gt; =&amp;gt; &lt;span class=&quot;number&quot;&gt;2&lt;/span&gt;,&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;$db-&amp;gt;connect($mysql_config, &lt;span class=&quot;function&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;function&lt;/span&gt; &lt;span class=&quot;params&quot;&gt;($db, $r)&lt;/span&gt; &lt;/span&gt;&amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &lt;span class=&quot;keyword&quot;&gt;if&lt;/span&gt; ($r) &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;        $sql = &lt;span class=&quot;string&quot;&gt;&#39;show tables&#39;&lt;/span&gt;;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            $db-&amp;gt;query($sql, &lt;span class=&quot;function&quot;&gt;&lt;span class=&quot;keyword&quot;&gt;function&lt;/span&gt;&lt;span class=&quot;params&quot;&gt;($db, $r)&lt;/span&gt; &lt;/span&gt;&amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            &lt;span class=&quot;keyword&quot;&gt;if&lt;/span&gt; ($r) &amp;#123;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;                var_dump($r);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            &amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;            $db-&amp;gt;close();&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;        &amp;#125;);&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;    &amp;#125;&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;&amp;#125;);&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;
    
    </summary>
    
      <category term="求索" scheme="https://breeze2.github.io/blog/categories/%E6%B1%82%E7%B4%A2/"/>
    
    
      <category term="mysql" scheme="https://breeze2.github.io/blog/tags/mysql/"/>
    
      <category term="php" scheme="https://breeze2.github.io/blog/tags/php/"/>
    
      <category term="swoole" scheme="https://breeze2.github.io/blog/tags/swoole/"/>
    
      <category term="lumen" scheme="https://breeze2.github.io/blog/tags/lumen/"/>
    
      <category term="think" scheme="https://breeze2.github.io/blog/tags/think/"/>
    
  </entry>
  
  <entry>
    <title>用Swoole HTTP服务器运行Lumen项目的实现方法</title>
    <link href="https://breeze2.github.io/blog/swoole-lumen-run-lumen-with-swoole.html"/>
    <id>https://breeze2.github.io/blog/swoole-lumen-run-lumen-with-swoole.html</id>
    <published>2018-03-01T12:03:02.000Z</published>
    <updated>2019-09-30T02:22:37.086Z</updated>
    
    <content type="html"><![CDATA[<blockquote><p>LNMP虽是传统的Web应用架构组合，无奈NginX + PHP-FPM的搭配运行效率实在太低；而Swoole HTTP服务器具有NginX级别的性能，且本身嵌入在PHP中，完全可以替代NginX + PHP-FPM。于是一直在探索用Swoole HTTP服务器运行传统PHP应用的途径。这里主要介绍一下Swoole HTTP服务器运行Lumen项目的实现方法：</p></blockquote><h2 id="Swoole-HTTP服务器"><a href="#Swoole-HTTP服务器" class="headerlink" title="Swoole HTTP服务器"></a>Swoole HTTP服务器</h2><p>Swoole1.7.7增加了内置HTTP服务器的支持，使用了跟NginX一样的IO多路复用机制epoll，可以达到NginX级别的性能，并且只需几行代码即可写出一个异步非阻塞多进程的HTTP服务器（epoll属于同步非阻塞，这里说“异步非阻塞”主要是因为“多进程”，单个进程内部依然是同步非阻塞）。</p><a id="more"></a><p>开启一个Swoole HTTP服务器的面向对象编程代码样板大致如下（具体参考Swoole官方文档<a href="https://wiki.swoole.com/wiki/page/326.html" target="_blank" rel="noopener">HttpServer</a>）：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"></span><br><span class="line"><span class="class"><span class="keyword">class</span> <span class="title">Service</span> </span>&#123;</span><br><span class="line">    <span class="keyword">public</span> $app;</span><br><span class="line">    <span class="keyword">public</span> $server;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">__construct</span><span class="params">($host, $port)</span> </span>&#123;</span><br><span class="line">        <span class="keyword">$this</span>-&gt;server = <span class="keyword">new</span> \swoole_http_server($host, $port);</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">start</span><span class="params">()</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="comment">// $this-&gt;server-&gt;on('start', array($this, 'onStart'));</span></span><br><span class="line">        <span class="comment">// $this-&gt;server-&gt;on('shutdown', array($this, 'onShutdown'));</span></span><br><span class="line">        <span class="comment">// $this-&gt;server-&gt;on('workerStop', array($this, 'onWorkerStop'));</span></span><br><span class="line">        <span class="keyword">$this</span>-&gt;server-&gt;on(<span class="string">'workerStart'</span>, <span class="keyword">array</span>(<span class="keyword">$this</span>, <span class="string">'onWorkerStart'</span>));</span><br><span class="line">        <span class="keyword">$this</span>-&gt;server-&gt;on(<span class="string">'request'</span>, <span class="keyword">array</span>(<span class="keyword">$this</span>, <span class="string">'onRequest'</span>));</span><br><span class="line">        <span class="keyword">$this</span>-&gt;server-&gt;start();</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">onWorkerStart</span><span class="params">($serv, $worker_id)</span> </span>&#123;</span><br><span class="line">        <span class="comment">// 应用初始化</span></span><br><span class="line">        <span class="keyword">$this</span>-&gt;app = <span class="number">1</span>;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">onRequest</span><span class="params">(\swoole_http_request $request,\swoole_http_response $response)</span> </span>&#123;</span><br><span class="line">        $app = <span class="keyword">$this</span>-&gt;app;</span><br><span class="line">        <span class="comment">// 处理用户请求</span></span><br><span class="line">        $get = json_encode($request-&gt;get);</span><br><span class="line">        <span class="comment">// 响应用户请求</span></span><br><span class="line">        $response-&gt;end(<span class="string">"App is &#123;$app&#125;. Get &#123;$get&#125;"</span>);</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">$s = <span class="keyword">new</span> Service(<span class="string">"127.0.0.1"</span>, <span class="number">9080</span>);</span><br><span class="line">$s-&gt;start();</span><br></pre></td></tr></table></figure><p>直接用<code>php</code>命令执行以上代码，浏览器访问<code>http://127.0.0.1:9080</code>，会得到：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">App is 1. Get null</span><br></pre></td></tr></table></figure><p>浏览器访问<code>http://127.0.0.1:9080/?user=guest</code>，会得到：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">App is 1. Get &#123;&quot;user&quot;:&quot;guest&quot;&#125;</span><br></pre></td></tr></table></figure><p>就是这么简单。</p><h2 id="加载Lumen项目"><a href="#加载Lumen项目" class="headerlink" title="加载Lumen项目"></a>加载Lumen项目</h2><p>Lumen项目的入口文件是项目路径下的<code>public/index.php</code>，里面只有简单的两行代码：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line">$app = <span class="keyword">require</span> <span class="keyword">__DIR__</span>.<span class="string">'/../bootstrap/app.php'</span>;</span><br><span class="line">$app-&gt;run();</span><br></pre></td></tr></table></figure><p>第一行代码就是加载整个Lumen框架代码，第二行代码则是处理请求，生成响应。</p><p>所以，在Swoole HTTP服务器中加载Lumen项目，也很简单，只需修改<code>onWorkerStart</code>方法：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">onWorkerStart</span><span class="params">($serv, $worker_id)</span> </span>&#123;</span><br><span class="line">        <span class="comment">// 应用初始化</span></span><br><span class="line">        <span class="keyword">$this</span>-&gt;app = <span class="keyword">require</span> <span class="string">'/THE/FULL/PATH/TO/bootstrap/app.php'</span>;</span><br><span class="line">    &#125;</span><br></pre></td></tr></table></figure><h2 id="处理用户请求"><a href="#处理用户请求" class="headerlink" title="处理用户请求"></a>处理用户请求</h2><p>虽然可以加载Lumen框架，但是要怎样才能用Lumen框架来处理用户请求呢？</p><p>由于Lumen框架的解耦程度非常高，我们可以很轻松地将<code>swoole_http_request</code>对象<code>$request</code>，转换成Lumen框架熟悉的<code>Illuminate\Http\Request</code>对象，这样Lumen框架就能处理用户请求了。<br>这里实现一个<code>parseRequest</code>方法，接收<code>swoole_http_request</code>类参数，返回<code>\Illuminate\Http\Request</code>类结果：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line">    <span class="keyword">protected</span> <span class="function"><span class="keyword">function</span> <span class="title">parseRequest</span><span class="params">(\swoole_http_request $request)</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        $get     = <span class="keyword">isset</span>($request-&gt;get) ? $request-&gt;get : <span class="keyword">array</span>();</span><br><span class="line">        $post    = <span class="keyword">isset</span>($request-&gt;post) ? $request-&gt;post : <span class="keyword">array</span>();</span><br><span class="line">        $cookie  = <span class="keyword">isset</span>($request-&gt;cookie) ? $request-&gt;cookie : <span class="keyword">array</span>();</span><br><span class="line">        $server  = <span class="keyword">isset</span>($request-&gt;server) ? $request-&gt;server : <span class="keyword">array</span>();</span><br><span class="line">        $header  = <span class="keyword">isset</span>($request-&gt;header) ? $request-&gt;header : <span class="keyword">array</span>();</span><br><span class="line">        $files   = <span class="keyword">isset</span>($request-&gt;files) ? $request-&gt;files : <span class="keyword">array</span>();</span><br><span class="line">        $fastcgi = <span class="keyword">array</span>();</span><br><span class="line"></span><br><span class="line">        $new_server = <span class="keyword">array</span>();</span><br><span class="line">        <span class="keyword">foreach</span> ($server <span class="keyword">as</span> $key =&gt; $value) &#123;</span><br><span class="line">            $new_server[strtoupper($key)] = $value;</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="keyword">foreach</span> ($header <span class="keyword">as</span> $key =&gt; $value) &#123;</span><br><span class="line">            $new_server[<span class="string">'HTTP_'</span> . strtoupper($key)] = $value;</span><br><span class="line">        &#125;</span><br><span class="line"></span><br><span class="line">        $content = $request-&gt;rawContent() ?: <span class="keyword">null</span>;</span><br><span class="line"></span><br><span class="line">        $http_request = <span class="keyword">new</span> \Illuminate\Http\Request($get, $post, $fastcgi, $cookie, $files, $new_server, $content);</span><br><span class="line"></span><br><span class="line">        <span class="keyword">return</span> $http_request;</span><br><span class="line">    &#125;</span><br></pre></td></tr></table></figure><h2 id="生成请求响应"><a href="#生成请求响应" class="headerlink" title="生成请求响应"></a>生成请求响应</h2><p>Lumen框架处理用户请求，十分方便，只需调用应用的<code>dispatch</code>方法即可。不过<code>dispatch</code>方法返回结果一般是<code>Symfony\Component\HttpFoundation\Response</code>对象，需要做一些转化才能交由<code>swoole_http_response</code>对象给用户输出响应。<br>这里实现一个<code>makeResponse</code>方法：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line">    <span class="keyword">protected</span> <span class="function"><span class="keyword">function</span> <span class="title">parseRequest</span><span class="params">(\swoole_http_response $request, \Symfony\Component\HttpFoundation\Response $http_response)</span></span></span><br><span class="line"><span class="function">    </span>&#123;</span><br><span class="line">        <span class="comment">// status</span></span><br><span class="line">        $response-&gt;status($http_response-&gt;getStatusCode());</span><br><span class="line">        <span class="comment">// headers</span></span><br><span class="line">        <span class="keyword">foreach</span> ($http_response-&gt;headers-&gt;allPreserveCase() <span class="keyword">as</span> $name =&gt; $values) &#123;</span><br><span class="line">            <span class="keyword">foreach</span> ($values <span class="keyword">as</span> $value) &#123;</span><br><span class="line">                $response-&gt;header($name, $value);</span><br><span class="line">            &#125;</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="comment">// cookies</span></span><br><span class="line">        <span class="keyword">foreach</span> ($http_response-&gt;headers-&gt;getCookies() <span class="keyword">as</span> $cookie) &#123;</span><br><span class="line">            $response-&gt;rawcookie(</span><br><span class="line">                $cookie-&gt;getName(),</span><br><span class="line">                $cookie-&gt;getValue(),</span><br><span class="line">                $cookie-&gt;getExpiresTime(),</span><br><span class="line">                $cookie-&gt;getPath(),</span><br><span class="line">                $cookie-&gt;getDomain(),</span><br><span class="line">                $cookie-&gt;isSecure(),</span><br><span class="line">                $cookie-&gt;isHttpOnly()</span><br><span class="line">            );</span><br><span class="line">        &#125;</span><br><span class="line">        <span class="comment">// content</span></span><br><span class="line">        $content = $http_response-&gt;getContent();</span><br><span class="line">        <span class="comment">// send content</span></span><br><span class="line">        $response-&gt;end($content);</span><br><span class="line">    &#125;</span><br></pre></td></tr></table></figure><p>有了<code>parseRequest</code>方法和<code>makeResponse</code>方法，要实现以Swoole HTTP服务器运行Lumen项目来处理用户请求只要几行代码。<code>onRequest</code>方法这样修改：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">onRequest</span><span class="params">(\swoole_http_request $request,\swoole_http_response $response)</span> </span>&#123;</span><br><span class="line">        $app = <span class="keyword">$this</span>-&gt;app;</span><br><span class="line">        <span class="comment">// 处理用户请求</span></span><br><span class="line">        $http_request = <span class="keyword">$this</span>-&gt;parseRequest($request);</span><br><span class="line">        $http_response = $app-&gt;dispatch($http_request);</span><br><span class="line">        <span class="comment">// 响应用户请求</span></span><br><span class="line">        <span class="keyword">$this</span>-&gt;makeResponse($response, $http_response);</span><br><span class="line">    &#125;</span><br></pre></td></tr></table></figure><h2 id="最后"><a href="#最后" class="headerlink" title="最后"></a>最后</h2><p>就这样，不需要修改Lumen项目原本任何代码，就能让其运行在Swoole HTTP服务器上。<br>当然，用户请求各种各样，有请求文件上传，有请求静态资源等等；响应类型也是各种各样，有Gzip压缩，有JSON格式，有设置Cookie等待。要实现一个健全的<code>Swoole+Lumen</code>网关服务，以上代码还须更多优化，具体可以参考<a href="https://github.com/breeze2/lumen-swoole-http" target="_blank" rel="noopener">Service.php</a>。</p><blockquote><p>按照以上思路，要实现以Swoole HTTP服务器运行Laravel项目也不难，不过，为了效率选择Swoole，自然会为了效率选择Lumen而不是Laravel。</p></blockquote><p><a href="https://blog.breezelin.cn/swoole-lumen-use-async-mysql-client-in-lumen.html" target="_blank" rel="noopener">下一篇文章</a>介绍如何在Lumen中使用异步MySQL客户端，减少IO阻塞。</p>]]></content>
    
    <summary type="html">
    
      &lt;blockquote&gt;
&lt;p&gt;LNMP虽是传统的Web应用架构组合，无奈NginX + PHP-FPM的搭配运行效率实在太低；而Swoole HTTP服务器具有NginX级别的性能，且本身嵌入在PHP中，完全可以替代NginX + PHP-FPM。于是一直在探索用Swoole HTTP服务器运行传统PHP应用的途径。这里主要介绍一下Swoole HTTP服务器运行Lumen项目的实现方法：&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;Swoole-HTTP服务器&quot;&gt;&lt;a href=&quot;#Swoole-HTTP服务器&quot; class=&quot;headerlink&quot; title=&quot;Swoole HTTP服务器&quot;&gt;&lt;/a&gt;Swoole HTTP服务器&lt;/h2&gt;&lt;p&gt;Swoole1.7.7增加了内置HTTP服务器的支持，使用了跟NginX一样的IO多路复用机制epoll，可以达到NginX级别的性能，并且只需几行代码即可写出一个异步非阻塞多进程的HTTP服务器（epoll属于同步非阻塞，这里说“异步非阻塞”主要是因为“多进程”，单个进程内部依然是同步非阻塞）。&lt;/p&gt;
    
    </summary>
    
      <category term="求索" scheme="https://breeze2.github.io/blog/categories/%E6%B1%82%E7%B4%A2/"/>
    
    
      <category term="php" scheme="https://breeze2.github.io/blog/tags/php/"/>
    
      <category term="swoole" scheme="https://breeze2.github.io/blog/tags/swoole/"/>
    
      <category term="lumen" scheme="https://breeze2.github.io/blog/tags/lumen/"/>
    
      <category term="think" scheme="https://breeze2.github.io/blog/tags/think/"/>
    
  </entry>
  
  <entry>
    <title>搭建Docker Registry私有镜像仓库</title>
    <link href="https://breeze2.github.io/blog/practice-docker-registry.html"/>
    <id>https://breeze2.github.io/blog/practice-docker-registry.html</id>
    <published>2018-01-16T11:01:06.000Z</published>
    <updated>2019-09-30T02:22:37.082Z</updated>
    
    <content type="html"><![CDATA[<p><a href="https://store.docker.com/" target="_blank" rel="noopener">Docker Store</a>是Docker官方提供的公共的镜像仓库，但有时会需要在局部内共享镜像，那么可以利用<a href="https://docs.docker.com/registry/" target="_blank" rel="noopener">Docker Registry</a>工具搭建一个私有的镜像仓库。</p><p>直接运行官方<a href="https://store.docker.com/images/registry" target="_blank" rel="noopener">registry</a>镜像，即可开启镜像仓库服务：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ docker run -d -p <span class="number">5000</span>:<span class="number">5000</span> --restart=always --name registry registry</span><br></pre></td></tr></table></figure><p>若要定制使用，还需更多配置。</p><a id="more"></a><h2 id="Docker安装"><a href="#Docker安装" class="headerlink" title="Docker安装"></a>Docker安装</h2><p>简单介绍一下Docker的安装：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">$ curl -fsSL https://download.docker.com/linux/ubuntu/gpg | sudo apt-key add -</span><br><span class="line">$ sudo add-apt-repository "deb [arch=amd64] https://download.docker.com/linux/ubuntu $(lsb_release -cs) stable"</span><br><span class="line">$ sudo apt-get update</span><br><span class="line">$ sudo apt-get install -y docker-ce</span><br><span class="line">$ sudo apt-get install -y docker-compose</span><br><span class="line">$ sudo usermod -aG docker $&#123;USER&#125;</span><br></pre></td></tr></table></figure><h2 id="Docker-Compose安装"><a href="#Docker-Compose安装" class="headerlink" title="Docker Compose安装"></a>Docker Compose安装</h2><p>Docker Compose的安装：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ sudo curl -L https://github.com/docker/compose/releases/download/<span class="number">1</span>.<span class="number">18</span>.<span class="number">0</span>/docker-compose-`uname -s`-`uname -m` -o /usr/local/bin/docker-compose</span><br><span class="line">$ sudo chmod +x /usr/local/bin/docker-compose</span><br></pre></td></tr></table></figure><h2 id="Docker-Compose编排"><a href="#Docker-Compose编排" class="headerlink" title="Docker Compose编排"></a>Docker Compose编排</h2><p>进入工作目录，编辑<code>docker-compose.yml</code>：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> ~/workspace</span><br><span class="line">$ <span class="built_in">cd</span> docker/registry</span><br><span class="line">$ vi docker-compose.yml</span><br></pre></td></tr></table></figure><p>基于官方镜像<code>registry</code>制作，<code>/etc/docker/registry</code>是容器内配置信息路径，映射到本地<code>./config</code>；<code>/var/lib/registry</code>是容器内数据保存路径，映射到本地<code>./data</code>：</p><figure class="highlight yml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">version:</span> <span class="string">"3"</span></span><br><span class="line"><span class="attr">services:</span></span><br><span class="line"><span class="attr">  registry:</span></span><br><span class="line"><span class="attr">    image:</span> <span class="string">registry</span></span><br><span class="line"><span class="attr">    ports:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"5000:5000"</span></span><br><span class="line"><span class="attr">    volumes:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"./config:/etc/docker/registry"</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"./data:/var/lib/registry"</span></span><br></pre></td></tr></table></figure><h2 id="Docker-Registry配置"><a href="#Docker-Registry配置" class="headerlink" title="Docker Registry配置"></a>Docker Registry配置</h2><p>设置HTTP认证：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">mkdir</span> -p config/auth</span><br><span class="line">$ htpasswd -Bbn $USERNAME $PASSWORD &gt; ./config/auth/htpasswd # or</span><br><span class="line">$ docker run --rm --entrypoint htpasswd registry -Bbn $USERNAME $PASSWORD &gt; ./config/auth/htpasswd</span><br></pre></td></tr></table></figure><p>配置Registry：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ vi config/config.yml</span><br></pre></td></tr></table></figure><p><code>config/config.yml</code>：</p><figure class="highlight yml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">version:</span> <span class="number">0.1</span></span><br><span class="line"><span class="attr">log:</span></span><br><span class="line"><span class="attr">  fields:</span></span><br><span class="line"><span class="attr">    service:</span> <span class="string">registry</span></span><br><span class="line"><span class="attr">storage:</span></span><br><span class="line"><span class="attr">  cache:</span></span><br><span class="line"><span class="attr">    blobdescriptor:</span> <span class="string">inmemory</span></span><br><span class="line"><span class="attr">  filesystem:</span></span><br><span class="line"><span class="attr">    rootdirectory:</span> <span class="string">/var/lib/registry</span></span><br><span class="line"><span class="attr">auth:</span></span><br><span class="line"><span class="attr">  htpasswd:</span></span><br><span class="line"><span class="attr">    realm:</span> <span class="string">basic-realm</span></span><br><span class="line"><span class="attr">    path:</span> <span class="string">/etc/docker/registry/auth/htpasswd</span></span><br><span class="line"><span class="attr">http:</span></span><br><span class="line"><span class="attr">  addr:</span> <span class="string">:5000</span></span><br><span class="line"><span class="attr">  headers:</span></span><br><span class="line"><span class="attr">    X-Content-Type-Options:</span> <span class="string">[nosniff]</span></span><br><span class="line"><span class="attr">health:</span></span><br><span class="line"><span class="attr">  storagedriver:</span></span><br><span class="line"><span class="attr">    enabled:</span> <span class="literal">true</span></span><br><span class="line"><span class="attr">    interval:</span> <span class="number">10</span><span class="string">s</span></span><br><span class="line"><span class="attr">    threshold:</span> <span class="number">3</span></span><br></pre></td></tr></table></figure><h2 id="运行与测试"><a href="#运行与测试" class="headerlink" title="运行与测试"></a>运行与测试</h2><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br></pre></td><td class="code"><pre><span class="line">$ docker-compose up -d</span><br><span class="line">$ docker login <span class="number">127</span>.<span class="number">0</span>.<span class="number">0</span>.<span class="number">1</span>:<span class="number">5000</span></span><br><span class="line">$ docker pull ubuntu:<span class="number">16</span>.<span class="number">04</span></span><br><span class="line">$ docker tag ubuntu:<span class="number">16</span>.<span class="number">04</span> <span class="number">127</span>.<span class="number">0</span>.<span class="number">0</span>.<span class="number">1</span>/username/ubuntu:<span class="number">16</span>.<span class="number">04</span></span><br><span class="line">$ docker push <span class="number">127</span>.<span class="number">0</span>.<span class="number">0</span>.<span class="number">1</span>/username/ubuntu:<span class="number">16</span>.<span class="number">04</span></span><br><span class="line">$ docker pull <span class="number">127</span>.<span class="number">0</span>.<span class="number">0</span>.<span class="number">1</span>/username/ubuntu:<span class="number">16</span>.<span class="number">04</span></span><br><span class="line">$ docker logout</span><br><span class="line">$ docker push <span class="number">127</span>.<span class="number">0</span>.<span class="number">0</span>.<span class="number">1</span>/username/ubuntu:<span class="number">16</span>.<span class="number">04</span></span><br></pre></td></tr></table></figure><h2 id="HTTPS"><a href="#HTTPS" class="headerlink" title="*HTTPS"></a>*HTTPS</h2><p>虽然Docker Registry默认监听端口是<code>5000</code>，但实际上服务是基于HTTP，可以通过HTTPS保障传输数据安全。Docker默认不允许非HTTPS连接下推送镜像（<code>127.0.0.1</code>本地网络地址例外），要取消这个限制须修改Docker配置——比如连接<code>192.168.1.1:5000</code>服务，在<code>/etc/default/docker</code>文件中添加<code>DOCKER_OPTS=&quot;--insecure-registries=192.168.1.1:5000&quot;</code>，然后重启Docker：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo service docker restart</span><br></pre></td></tr></table></figure><p>当然，最好还是给Docker Registry设置HTTPS：可以用NginX代理本地<code>5000</code>端口，然后在NginX上设置HTTPS；也可以在Docker Registry内部设置HTTPS，详细过程请阅读<br><a href="https://yeasy.gitbooks.io/docker_practice/content/repository/registry_auth.html" target="_blank" rel="noopener">《私有仓库高级配置》</a>。</p><h2 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h2><ul><li><a href="https://docs.docker.com/registry/deploying/" target="_blank" rel="noopener">Deploy a registry server</a></li><li><a href="https://docs.docker.com/registry/configuration/" target="_blank" rel="noopener">Configuring a registry</a></li></ul>]]></content>
    
    <summary type="html">
    
      &lt;p&gt;&lt;a href=&quot;https://store.docker.com/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;Docker Store&lt;/a&gt;是Docker官方提供的公共的镜像仓库，但有时会需要在局部内共享镜像，那么可以利用&lt;a href=&quot;https://docs.docker.com/registry/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;Docker Registry&lt;/a&gt;工具搭建一个私有的镜像仓库。&lt;/p&gt;
&lt;p&gt;直接运行官方&lt;a href=&quot;https://store.docker.com/images/registry&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;registry&lt;/a&gt;镜像，即可开启镜像仓库服务：&lt;/p&gt;
&lt;figure class=&quot;highlight cmd&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;$ docker run -d -p &lt;span class=&quot;number&quot;&gt;5000&lt;/span&gt;:&lt;span class=&quot;number&quot;&gt;5000&lt;/span&gt; --restart=always --name registry registry&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;

&lt;p&gt;若要定制使用，还需更多配置。&lt;/p&gt;
    
    </summary>
    
      <category term="实践" scheme="https://breeze2.github.io/blog/categories/%E5%AE%9E%E8%B7%B5/"/>
    
    
      <category term="practice" scheme="https://breeze2.github.io/blog/tags/practice/"/>
    
      <category term="docker" scheme="https://breeze2.github.io/blog/tags/docker/"/>
    
      <category term="registry" scheme="https://breeze2.github.io/blog/tags/registry/"/>
    
  </entry>
  
  <entry>
    <title>源码安装GitLab（MySQL &amp; HTTPS）</title>
    <link href="https://breeze2.github.io/blog/practice-gitlab-installation-from-source.html"/>
    <id>https://breeze2.github.io/blog/practice-gitlab-installation-from-source.html</id>
    <published>2018-01-12T18:26:13.000Z</published>
    <updated>2019-09-30T02:22:37.086Z</updated>
    
    <content type="html"><![CDATA[<p>介绍一下在Ubuntu16.04系统下源码安装GitLab10.04且以MySQL作为数据库、SSL加密传输的步骤：</p><a id="more"></a><h2 id="开始"><a href="#开始" class="headerlink" title="开始"></a>开始</h2><h3 id="Dependencies"><a href="#Dependencies" class="headerlink" title="Dependencies"></a>Dependencies</h3><p>安装依赖：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo apt-get install -y build-essential zlib1g-dev libyaml-dev libssl-dev libgdbm-dev libre2-dev libreadline-dev libncurses5-dev libffi-dev curl openssh-server checkinstall libxml2-dev libxslt-dev libcurl4-openssl-dev libicu-dev logrotate rsync python-docutils pkg-config cmake</span><br></pre></td></tr></table></figure><p>为GitLab创建一个系统用户git：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo adduser --disabled-login --gecos 'GitLab' git</span><br></pre></td></tr></table></figure><p>$ sudo apt-get install -y postfix</p><h3 id="Git"><a href="#Git" class="headerlink" title="Git"></a>Git</h3><p>安装最新版Git：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ sudo add-apt-repository ppa:git-core/ppa</span><br><span class="line">$ sudo apt-get update</span><br><span class="line">$ sudo apt-get install git</span><br></pre></td></tr></table></figure><h3 id="Ruby"><a href="#Ruby" class="headerlink" title="Ruby"></a>Ruby</h3><p>安装最新版Ruby：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ sudo apt-add-repository ppa:brightbox/ruby-ng</span><br><span class="line">$ sudo apt-get update</span><br><span class="line">$ sudo apt-get install ruby</span><br><span class="line">$ sudo apt-get install ruby-dev</span><br></pre></td></tr></table></figure><p>安装Bundler：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo gem install bundler --no-ri --no-rdoc</span><br></pre></td></tr></table></figure><h3 id="Go"><a href="#Go" class="headerlink" title="Go"></a>Go</h3><p>安装Go1.9.2：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ curl --remote-name --progress https://dl.google.com/go/go1.<span class="number">9</span>.<span class="number">2</span>.linux-amd64.tar.gz</span><br><span class="line">$ sudo tar -C /usr/local -xzf go1.<span class="number">9</span>.<span class="number">2</span>.linux-amd64.tar.gz</span><br><span class="line">$ sudo ln -sf /usr/local/go/bin/&#123;go,godoc,gofmt&#125; /usr/local/bin/</span><br><span class="line">$ rm go1.<span class="number">9</span>.<span class="number">2</span>.linux-amd64.tar.gz</span><br></pre></td></tr></table></figure><h3 id="Node"><a href="#Node" class="headerlink" title="Node"></a>Node</h3><p>安装NodeJS8.x：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ curl --location https://deb.nodesource.com/setup_8.x | sudo bash -</span><br><span class="line">$ sudo apt-get install -y nodejs</span><br></pre></td></tr></table></figure><p>安装yarn：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ curl --silent --show-error https://dl.yarnpkg.com/debian/pubkey.gpg | sudo apt-key add -</span><br><span class="line"><span class="built_in">echo</span> "deb https://dl.yarnpkg.com/debian/ stable main" | sudo tee /etc/apt/sources.list.d/yarn.list</span><br><span class="line">$ sudo apt-get update</span><br><span class="line">$ sudo apt-get install yarn</span><br></pre></td></tr></table></figure><h3 id="MySQL"><a href="#MySQL" class="headerlink" title="MySQL"></a>MySQL</h3><p>安装MySQL5.7，先从<a href="https://dev.mysql.com/downloads/repo/apt/" target="_blank" rel="noopener">MySQL官网</a>下载APT配置器(mysql-apt-config_0.8.9-1_all.deb)[<a href="https://dev.mysql.com/downloads/file/?id=474129]，然后：" target="_blank" rel="noopener">https://dev.mysql.com/downloads/file/?id=474129]，然后：</a></p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ sudo dpkg -i mysql-apt-config_0.<span class="number">8</span>.<span class="number">9</span>-<span class="number">1</span>_all.deb</span><br><span class="line">$ sudo apt-get update</span><br><span class="line">$ sudo apt-get install -y mysql-server mysql-client libmysqlclient-dev</span><br></pre></td></tr></table></figure><p>MySQL安全配置：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ </span><br><span class="line">$ mysql --version</span><br><span class="line">$ sudo mysql_secure_installation</span><br></pre></td></tr></table></figure><p>创建GitLab用户和数据库：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line">$ mysql -u root -p</span><br><span class="line">mysql&gt; CREATE USER &apos;git&apos;@&apos;localhost&apos; IDENTIFIED BY &apos;$password&apos;;</span><br><span class="line"></span><br><span class="line"># Create the GitLab production database</span><br><span class="line">mysql&gt; CREATE DATABASE IF NOT EXISTS `gitlabhq_production` DEFAULT CHARACTER SET `utf8` COLLATE `utf8_general_ci`;</span><br><span class="line"></span><br><span class="line"># Grant the GitLab user necessary permissions on the database</span><br><span class="line">mysql&gt; GRANT SELECT, INSERT, UPDATE, DELETE, CREATE, CREATE TEMPORARY TABLES, DROP, INDEX, ALTER, LOCK TABLES, REFERENCES, TRIGGER ON `gitlabhq_production`.* TO &apos;git&apos;@&apos;localhost&apos;;</span><br><span class="line"></span><br><span class="line"># Quit the database session</span><br><span class="line">mysql&gt; \q</span><br></pre></td></tr></table></figure><p>测试用户是否可以访问数据库：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo -u git -H mysql -u git -p -D gitlabhq_production</span><br></pre></td></tr></table></figure><h3 id="Redis"><a href="#Redis" class="headerlink" title="Redis"></a>Redis</h3><p>安装Redis3.0.6：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo apt-get install redis</span><br></pre></td></tr></table></figure><h3 id="GitLab"><a href="#GitLab" class="headerlink" title="GitLab"></a>GitLab</h3><h4 id="克隆GitLab源码"><a href="#克隆GitLab源码" class="headerlink" title="克隆GitLab源码"></a>克隆GitLab源码</h4><p>克隆GitLab源码，并配置GitLab（修改域名等等）：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> /home/git</span><br><span class="line">$ sudo -u git -H git clone https://gitlab.com/gitlab-org/gitlab-ce.git -b <span class="number">10</span>-<span class="number">4</span>-stable gitlab</span><br><span class="line">$ <span class="built_in">cd</span> /home/git/gitlab</span><br><span class="line">$ sudo -u git -H cp config/gitlab.yml.example config/gitlab.yml</span><br><span class="line">$ sudo -u git -H vi config/gitlab.yml</span><br></pre></td></tr></table></figure><h4 id="配置GitLab"><a href="#配置GitLab" class="headerlink" title="配置GitLab"></a>配置GitLab</h4><p>配置密钥文件：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ sudo -u git -H cp config/secrets.yml.example config/secrets.yml</span><br><span class="line">$ sudo -u git -H chmod <span class="number">0600</span> config/secrets.yml</span><br></pre></td></tr></table></figure><p>配置Unicorn（修改监听端口等等）：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ sudo -u git -H cp config/unicorn.rb.example config/unicorn.rb</span><br><span class="line">$ sudo -u git -H vi config/unicorn.rb</span><br></pre></td></tr></table></figure><p>设置目录权限：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br></pre></td><td class="code"><pre><span class="line">$ sudo chown -R git log/</span><br><span class="line">$ sudo chown -R git tmp/</span><br><span class="line">$ sudo chmod -R u+rwX,go-w log/</span><br><span class="line">$ sudo chmod -R u+rwX tmp/</span><br><span class="line"></span><br><span class="line">$ sudo chmod -R u+rwX tmp/pids/</span><br><span class="line">$ sudo chmod -R u+rwX tmp/sockets/</span><br><span class="line"></span><br><span class="line">$ sudo -u git -H <span class="built_in">mkdir</span> public/uploads/</span><br><span class="line">$ sudo chmod <span class="number">0700</span> public/uploads</span><br><span class="line"></span><br><span class="line">$ sudo chmod -R u+rwX builds/</span><br><span class="line">$ sudo chmod -R u+rwX shared/artifacts/</span><br><span class="line">$ sudo chmod -R ug+rwX shared/pages/</span><br></pre></td></tr></table></figure><p>配置Rack Attack：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo -u git -H cp config/initializers/rack_attack.rb.example config/initializers/rack_attack.rb</span><br></pre></td></tr></table></figure><p>配置Resque（修改Redis链接等等）：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ sudo -u git -H cp config/resque.yml.example config/resque.yml</span><br><span class="line">$ sudo -u git -H vi config/resque.yml</span><br></pre></td></tr></table></figure><p>配置Git用户：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br></pre></td><td class="code"><pre><span class="line"># Configure Git global settings <span class="keyword">for</span> git user</span><br><span class="line"># 'autocrlf' is needed <span class="keyword">for</span> the web editor</span><br><span class="line">$ sudo -u git -H git config --global core.autocrlf input</span><br><span class="line"></span><br><span class="line"># Disable 'git gc --auto' because GitLab already runs 'git gc' when needed</span><br><span class="line">$ sudo -u git -H git config --global gc.auto <span class="number">0</span></span><br><span class="line"></span><br><span class="line"># Enable packfile bitmaps</span><br><span class="line">$ sudo -u git -H git config --global repack.writeBitmaps true</span><br><span class="line"></span><br><span class="line"># Enable push options</span><br><span class="line">$ sudo -u git -H git config --global receive.advertisePushOptions true</span><br></pre></td></tr></table></figure><h4 id="数据库设置"><a href="#数据库设置" class="headerlink" title="数据库设置"></a>数据库设置</h4><p>配置MySQL（修改账户密码等等）：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ sudo -u git cp config/database.yml.mysql config/database.yml</span><br><span class="line">$ sudo -u git -H chmod o-rwx config/database.yml</span><br><span class="line">$ sudo -u git -H vi config/database.yml</span><br></pre></td></tr></table></figure><h4 id="安装Gems"><a href="#安装Gems" class="headerlink" title="安装Gems"></a>安装Gems</h4><p>安装Gems：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"># <span class="keyword">If</span> you use MySQL (note, the option says "without ... postgres")</span><br><span class="line"># <span class="keyword">If</span> you want to use Kerberos <span class="keyword">for</span> user authentication, then omit kerberos <span class="keyword">in</span> the --without option</span><br><span class="line">$ sudo -u git -H bundle install --deployment --without development test postgres aws kerberos</span><br></pre></td></tr></table></figure><h4 id="安装GitLab-Shell"><a href="#安装GitLab-Shell" class="headerlink" title="安装GitLab Shell"></a>安装GitLab Shell</h4><p>安装GitLab Shell，并配置（修改GitLab链接等等）：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line"># Run the installation task <span class="keyword">for</span> gitlab-shell (<span class="built_in">replace</span> `REDIS_URL` <span class="keyword">if</span> needed):</span><br><span class="line">$ sudo -u git -H bundle exec rake gitlab:shell:install RAILS_ENV=production SKIP_STORAGE_VALIDATION=true</span><br><span class="line"></span><br><span class="line"># By default, the gitlab-shell config is generated from your main GitLab config.</span><br><span class="line"># You can review (and modify) the gitlab-shell config as follows:</span><br><span class="line">$ sudo -u git -H vi /home/git/gitlab-shell/config.yml</span><br></pre></td></tr></table></figure><h4 id="安装Gitlab-Workhorse"><a href="#安装Gitlab-Workhorse" class="headerlink" title="安装Gitlab Workhorse"></a>安装Gitlab Workhorse</h4><p>安装Gitlab Workhorse：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ sudo -u git -H bundle exec rake "gitlab:workhorse:install[/home/git/gitlab-workhorse]" RAILS_ENV=production</span><br><span class="line"></span><br><span class="line">$ sudo -u git -H bundle exec rake gitlab:setup RAILS_ENV=production GITLAB_ROOT_PASSWORD='$root_password' GITLAB_ROOT_EMAIL='$root_email'</span><br></pre></td></tr></table></figure><h4 id="secrets-yml"><a href="#secrets-yml" class="headerlink" title="secrets.yml"></a>secrets.yml</h4><p><code>/home/git/gitlab/config/secrets.yml</code>文件存储着各类数据加密的钥匙，应备份在一个安全的地方。</p><h4 id="安装启动脚本"><a href="#安装启动脚本" class="headerlink" title="安装启动脚本"></a>安装启动脚本</h4><p>安装在系统中启动GitLab服务的脚步：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ sudo cp lib/support/init.d/gitlab /etc/init.d/gitlab</span><br><span class="line">$ sudo cp lib/support/init.d/gitlab.default.example /etc/default/gitlab</span><br><span class="line">$ sudo update-rc.d gitlab defaults <span class="number">21</span></span><br></pre></td></tr></table></figure><h4 id="安装Gitaly"><a href="#安装Gitaly" class="headerlink" title="安装Gitaly"></a>安装Gitaly</h4><p>安装Gitaly，并配置：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br></pre></td><td class="code"><pre><span class="line">$ sudo -u git -H bundle exec rake "gitlab:gitaly:install[/home/git/gitaly]" RAILS_ENV=production</span><br><span class="line">$ sudo chmod <span class="number">0700</span> /home/git/gitlab/tmp/sockets/private</span><br><span class="line">$ sudo chown git /home/git/gitlab/tmp/sockets/private</span><br><span class="line">$ sudo -u git -H vi /home/git/gitaly/config.toml</span><br></pre></td></tr></table></figure><h4 id="设置Logrotate"><a href="#设置Logrotate" class="headerlink" title="设置Logrotate"></a>设置Logrotate</h4><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo cp lib/support/logrotate/gitlab /etc/logrotate.d/gitlab</span><br></pre></td></tr></table></figure><h4 id="编译GetText-PO"><a href="#编译GetText-PO" class="headerlink" title="编译GetText PO"></a>编译GetText PO</h4><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo -u git -H bundle exec rake gettext:compile RAILS_ENV=production</span><br></pre></td></tr></table></figure><h4 id="MySQL字符串限制"><a href="#MySQL字符串限制" class="headerlink" title="MySQL字符串限制"></a>MySQL字符串限制</h4><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo -u git -H bundle exec rake add_limits_mysql RAILS_ENV=production</span><br></pre></td></tr></table></figure><h4 id="编译前端资源"><a href="#编译前端资源" class="headerlink" title="编译前端资源"></a>编译前端资源</h4><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ sudo -u git -H yarn install --production --pure-lockfile</span><br><span class="line">$ sudo -u git -H bundle exec rake gitlab:assets:compile RAILS_ENV=production NODE_ENV=production</span><br></pre></td></tr></table></figure><h4 id="启动GitLab服务"><a href="#启动GitLab服务" class="headerlink" title="启动GitLab服务"></a>启动GitLab服务</h4><p>启动或重启GitLab服务：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ sudo service gitlab <span class="built_in">start</span></span><br><span class="line"># or</span><br><span class="line">$ sudo /etc/init.d/gitlab restart</span><br></pre></td></tr></table></figure><h4 id="获取服务配置"><a href="#获取服务配置" class="headerlink" title="获取服务配置"></a>获取服务配置</h4><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo -u git -H bundle exec rake gitlab:env:info RAILS_ENV=production</span><br></pre></td></tr></table></figure><h4 id="检测服务状态"><a href="#检测服务状态" class="headerlink" title="检测服务状态"></a>检测服务状态</h4><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo -u git -H bundle exec rake gitlab:check RAILS_ENV=production</span><br></pre></td></tr></table></figure><h3 id="Nginx"><a href="#Nginx" class="headerlink" title="Nginx"></a>Nginx</h3><p>安装最新版NginX：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ sudo add-apt-repository ppa:nginx/stable</span><br><span class="line">$ sudo apt-get update</span><br><span class="line">$ sudo apt-install nginx</span><br></pre></td></tr></table></figure><p>配置NginX代理（修改域名等等）：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ sudo cp lib/support/nginx/gitlab /etc/nginx/sites-available/gitlab</span><br><span class="line">$ sudo ln -s /etc/nginx/sites-available/gitlab /etc/nginx/sites-enabled/gitlab</span><br><span class="line">$ sudo vi /etc/nginx/sites-available/gitlab</span><br></pre></td></tr></table></figure><p>重启NginX服务：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo service nginx <span class="built_in">start</span></span><br></pre></td></tr></table></figure><h3 id="Letsencrypt"><a href="#Letsencrypt" class="headerlink" title="*Letsencrypt"></a>*Letsencrypt</h3><p>安装Letsencrypt：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo apt-get install letsencrypt</span><br></pre></td></tr></table></figure><p>申请密钥和证书：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo letsencrypt sudo letsencrypt certonly --webroot -w /home/git/gitlab/public -d '$domainname'</span><br></pre></td></tr></table></figure><p>配置NginX代理（修改域名、密钥位置、证书位置等等）：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">sudo cp lib/support/nginx/gitlab-ssl /etc/nginx/sites-available/gitlab</span><br><span class="line">sudo ln -s /etc/nginx/sites-available/gitlab /etc/nginx/sites-enabled/gitlab</span><br><span class="line">sudo vi /etc/nginx/sites-available/gitlab</span><br></pre></td></tr></table></figure><p>重启NginX服务：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sudo service nginx <span class="built_in">start</span></span><br></pre></td></tr></table></figure><h2 id="参考"><a href="#参考" class="headerlink" title="参考"></a>参考</h2><ul><li><a href="https://docs.gitlab.com/ce/install/installation.html" target="_blank" rel="noopener">Installation from source</a></li><li><a href="https://docs.gitlab.com/ce/install/database_mysql.html" target="_blank" rel="noopener">Database MySQL</a></li></ul>]]></content>
    
    <summary type="html">
    
      &lt;p&gt;介绍一下在Ubuntu16.04系统下源码安装GitLab10.04且以MySQL作为数据库、SSL加密传输的步骤：&lt;/p&gt;
    
    </summary>
    
      <category term="实践" scheme="https://breeze2.github.io/blog/categories/%E5%AE%9E%E8%B7%B5/"/>
    
    
      <category term="mysql" scheme="https://breeze2.github.io/blog/tags/mysql/"/>
    
      <category term="practice" scheme="https://breeze2.github.io/blog/tags/practice/"/>
    
      <category term="gitlab" scheme="https://breeze2.github.io/blog/tags/gitlab/"/>
    
      <category term="nginx" scheme="https://breeze2.github.io/blog/tags/nginx/"/>
    
  </entry>
  
  <entry>
    <title>Web页面中滚动穿透的解决方法</title>
    <link href="https://breeze2.github.io/blog/hack-scroll-event-propagation.html"/>
    <id>https://breeze2.github.io/blog/hack-scroll-event-propagation.html</id>
    <published>2018-01-08T18:48:30.000Z</published>
    <updated>2019-09-30T02:22:37.082Z</updated>
    
    <content type="html"><![CDATA[<p>最近做一个移动端的Web应用，因为是单页模式，所以少不了各种各样的弹出框，侧边栏，结果发现这些元素在滚动到边界时，滚动事件会传递给父元素，导致父元素开始滚动（在PC端也是一样存在这样的问题，而在移动端更为突出，有时即使不在边界也会导致父元素滚动，自身却不动）。</p><p>在网上搜寻一番后，找到很多关于这个问题的讨论，解决方法大致可以分为两类：</p><ol><li>（JS）监听元素<code>el</code>滚动事件<code>event</code>，当<code>el.scrollTop</code>或者<code>el.scrollLeft</code>到达边界值的时候，用<code>event.preventDefault()</code>方法取消浏览器默认操作和用<code>event.preventDefault()</code>方法停止事件往上传播；</li><li>（CSS）在表层元素（弹出框，侧边栏等等）显示时，修改底层元素（一般是网页主体<code>body</code>）的样式，如<code>position:fixed</code>、<code>overflow:hidden</code>等等，使其不可滚动。</li></ol><p>具体可以参考<a href="https://github.com/pod4g/tool/wiki/%E7%A7%BB%E5%8A%A8%E7%AB%AF%E6%BB%9A%E5%8A%A8%E7%A9%BF%E9%80%8F%E9%97%AE%E9%A2%98" target="_blank" rel="noopener">移动端滚动穿透问题</a>，其中提到的“终极完美解决方案”便属上面的第二类。</p><a id="more"></a><h2 id="疑虑"><a href="#疑虑" class="headerlink" title="疑虑"></a>疑虑</h2><p>虽说是“终极完美解决方案”，但其实我是不相信的，因为修改了元素定位方式，元素需要重绘，内容显示位置也会改变，即使记录内容位置后续恢复，画面也少不免抖动，所以怎么可能是“完美”呢？</p><h2 id="实践"><a href="#实践" class="headerlink" title="实践"></a>实践</h2><p>尽管有所疑虑，还是要实践一下。于是简单地写一个测试样例了：</p><figure class="highlight html"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;!DOCTYPE html&gt;</span></span><br><span class="line"><span class="tag">&lt;<span class="name">html</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;<span class="name">head</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">title</span>&gt;</span>Test<span class="tag">&lt;/<span class="name">title</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">style</span> <span class="attr">type</span>=<span class="string">"text/css"</span>&gt;</span></span><br><span class="line"><span class="css">        <span class="selector-tag">html</span>, <span class="selector-tag">body</span> &#123;<span class="attribute">width</span>: <span class="number">100%</span>;&#125;</span></span><br><span class="line"><span class="css">        <span class="selector-tag">body</span> &#123;<span class="attribute">background-color</span>: <span class="number">#9e9e9e</span>; <span class="attribute">height</span>: <span class="number">9000px</span>; <span class="attribute">margin</span>: <span class="number">0</span>; <span class="attribute">padding</span>: <span class="number">0</span>; <span class="attribute">z-index</span>: <span class="number">1</span>;&#125;</span></span><br><span class="line"><span class="css">        <span class="selector-class">.main</span> &#123;<span class="attribute">background-color</span>: <span class="number">#00bcd4</span>; <span class="attribute">height</span>: <span class="number">6000px</span>; <span class="attribute">margin</span>: <span class="number">0</span>;&#125;</span></span><br><span class="line"><span class="css">        <span class="selector-class">.nav</span> &#123;<span class="attribute">background-color</span>: <span class="number">#2196f3</span>; <span class="attribute">width</span>: <span class="number">200px</span>; <span class="attribute">position</span>: fixed; <span class="attribute">top</span>: <span class="number">0</span>; <span class="attribute">bottom</span>: <span class="number">0</span>; <span class="attribute">right</span>: <span class="number">0</span>; <span class="attribute">overflow</span>: scroll; <span class="attribute">display</span>: none; <span class="attribute">z-index</span>: <span class="number">9</span>;&#125;</span></span><br><span class="line">    <span class="tag">&lt;/<span class="name">style</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">head</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;<span class="name">body</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">"main"</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">div</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">li</span>&gt;</span>testing testing<span class="tag">&lt;/<span class="name">li</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">li</span>&gt;</span>testing testing<span class="tag">&lt;/<span class="name">li</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">li</span>&gt;</span>testing testing<span class="tag">&lt;/<span class="name">li</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">li</span>&gt;</span>testing testing<span class="tag">&lt;/<span class="name">li</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">li</span>&gt;</span>testing testing<span class="tag">&lt;/<span class="name">li</span>&gt;</span></span><br><span class="line">            <span class="comment">&lt;!-- more and more     --&gt;</span></span><br><span class="line">        <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">class</span>=<span class="string">"nav"</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">div</span> <span class="attr">style</span>=<span class="string">"height: 2000px"</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">li</span>&gt;</span>testing testing<span class="tag">&lt;/<span class="name">li</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">li</span>&gt;</span>testing testing<span class="tag">&lt;/<span class="name">li</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">li</span>&gt;</span>testing testing<span class="tag">&lt;/<span class="name">li</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">li</span>&gt;</span>testing testing<span class="tag">&lt;/<span class="name">li</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">li</span>&gt;</span>testing testing<span class="tag">&lt;/<span class="name">li</span>&gt;</span></span><br><span class="line">            <span class="comment">&lt;!-- more and more     --&gt;</span>   </span><br><span class="line">        <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">script</span> <span class="attr">type</span>=<span class="string">"text/javascript"</span>&gt;</span></span><br><span class="line"><span class="javascript">        <span class="keyword">var</span> main = <span class="built_in">document</span>.querySelector(<span class="string">'.main'</span>);</span></span><br><span class="line"><span class="javascript">        <span class="keyword">var</span> nav  = <span class="built_in">document</span>.querySelector(<span class="string">'.nav'</span>);</span></span><br><span class="line"><span class="actionscript">        <span class="keyword">var</span> flag = <span class="number">0</span>;</span></span><br><span class="line"><span class="actionscript">        main.addEventListener(<span class="string">'click'</span>, <span class="function"><span class="keyword">function</span> <span class="params">()</span> </span>&#123;</span></span><br><span class="line"><span class="actionscript">            nav.style.display = <span class="string">'block'</span>;</span></span><br><span class="line"><span class="javascript">            flag = <span class="built_in">document</span>.body.scrollTop;</span></span><br><span class="line"><span class="javascript">            <span class="built_in">document</span>.body.style.position = <span class="string">'fixed'</span>;</span></span><br><span class="line"><span class="javascript">            <span class="built_in">document</span>.body.style.top = -flag + <span class="string">'px'</span>;</span></span><br><span class="line">        &#125;);</span><br><span class="line"><span class="actionscript">        nav.addEventListener(<span class="string">'click'</span>, <span class="function"><span class="keyword">function</span> <span class="params">()</span> </span>&#123;</span></span><br><span class="line"><span class="javascript">            <span class="built_in">document</span>.body.style.position = <span class="string">'static'</span>;</span></span><br><span class="line"><span class="javascript">            <span class="built_in">document</span>.body.style.top = <span class="string">'auto'</span>;</span></span><br><span class="line"><span class="javascript">            <span class="built_in">document</span>.body.scrollTop = flag;</span></span><br><span class="line"><span class="actionscript">            nav.style.display = <span class="string">'none'</span>;</span></span><br><span class="line">        &#125;);</span><br><span class="line">    <span class="tag">&lt;/<span class="name">script</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">body</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">html</span>&gt;</span></span><br></pre></td></tr></table></figure><p>样例页面中，点击<code>.main</code>时，将<code>.nav</code>设为可见，且将<code>body</code>的定位设为<code>fixed</code>，向上偏移到<code>body</code>本来的滚动高度，这样<code>.nav</code>可以滚动而<code>.main</code>不能滚动，就不会出现滚动穿透的问题；再点击<code>.nav</code>，将<code>.nav</code>设为不可见，且将<code>body</code>的定位设为<code>static</code>，取消<code>body</code>的向上偏移并滚回本来的滚动高度。</p><h3 id="有个问题"><a href="#有个问题" class="headerlink" title="有个问题"></a>有个问题</h3><p>样例页面在Safari浏览器中运行正常，在Chrome浏览器中却有问题，因为Safari浏览器认为<code>body</code>元素是滚动主体，而Chrome浏览器认为<code>html</code>元素才是滚动主体。<code>document</code>中获取<code>body</code>元素的键值是<code>body</code>，而获取<code>html</code>元素的键值是<code>documentElement</code>，所以要在Chrome浏览器中运行正常的脚步代码应该是：</p><figure class="highlight javascript"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">var</span> main = <span class="built_in">document</span>.querySelector(<span class="string">'.main'</span>);</span><br><span class="line"><span class="keyword">var</span> nav  = <span class="built_in">document</span>.querySelector(<span class="string">'.nav'</span>);</span><br><span class="line"><span class="keyword">var</span> flag = <span class="number">0</span>;</span><br><span class="line">main.addEventListener(<span class="string">'click'</span>, <span class="function"><span class="keyword">function</span> (<span class="params"></span>) </span>&#123;</span><br><span class="line">    nav.style.display = <span class="string">'block'</span>;</span><br><span class="line">    flag = <span class="built_in">document</span>.documentElement.scrollTop;</span><br><span class="line">    <span class="built_in">document</span>.documentElement.style.position = <span class="string">'fixed'</span>;</span><br><span class="line">    <span class="built_in">document</span>.documentElement.style.top = -flag + <span class="string">'px'</span>;</span><br><span class="line">&#125;);</span><br><span class="line">nav.addEventListener(<span class="string">'click'</span>, <span class="function"><span class="keyword">function</span> (<span class="params"></span>) </span>&#123;</span><br><span class="line">    <span class="built_in">document</span>.documentElement.style.position = <span class="string">'static'</span>;</span><br><span class="line">    <span class="built_in">document</span>.documentElement.style.top = <span class="string">'auto'</span>;</span><br><span class="line">    <span class="built_in">document</span>.documentElement.scrollTop = flag;</span><br><span class="line">    nav.style.display = <span class="string">'none'</span>;</span><br><span class="line">&#125;);</span><br></pre></td></tr></table></figure><h3 id="运行效果"><a href="#运行效果" class="headerlink" title="运行效果"></a>运行效果</h3><p>在<code>.nav</code>出现或消失时，认真看可以开到<code>.main</code>有些小抖动，但并不明显，在今天移动端硬件（CPU，内存）和软件（浏览器）优化足够的情况下，这个滚动穿透的解决方法确实是可以放心使用的。</p><h2 id="最后"><a href="#最后" class="headerlink" title="最后"></a>最后</h2><p>开卷有疑，实践证明。</p>]]></content>
    
    <summary type="html">
    
      &lt;p&gt;最近做一个移动端的Web应用，因为是单页模式，所以少不了各种各样的弹出框，侧边栏，结果发现这些元素在滚动到边界时，滚动事件会传递给父元素，导致父元素开始滚动（在PC端也是一样存在这样的问题，而在移动端更为突出，有时即使不在边界也会导致父元素滚动，自身却不动）。&lt;/p&gt;
&lt;p&gt;在网上搜寻一番后，找到很多关于这个问题的讨论，解决方法大致可以分为两类：&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;（JS）监听元素&lt;code&gt;el&lt;/code&gt;滚动事件&lt;code&gt;event&lt;/code&gt;，当&lt;code&gt;el.scrollTop&lt;/code&gt;或者&lt;code&gt;el.scrollLeft&lt;/code&gt;到达边界值的时候，用&lt;code&gt;event.preventDefault()&lt;/code&gt;方法取消浏览器默认操作和用&lt;code&gt;event.preventDefault()&lt;/code&gt;方法停止事件往上传播；&lt;/li&gt;
&lt;li&gt;（CSS）在表层元素（弹出框，侧边栏等等）显示时，修改底层元素（一般是网页主体&lt;code&gt;body&lt;/code&gt;）的样式，如&lt;code&gt;position:fixed&lt;/code&gt;、&lt;code&gt;overflow:hidden&lt;/code&gt;等等，使其不可滚动。&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;具体可以参考&lt;a href=&quot;https://github.com/pod4g/tool/wiki/%E7%A7%BB%E5%8A%A8%E7%AB%AF%E6%BB%9A%E5%8A%A8%E7%A9%BF%E9%80%8F%E9%97%AE%E9%A2%98&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;移动端滚动穿透问题&lt;/a&gt;，其中提到的“终极完美解决方案”便属上面的第二类。&lt;/p&gt;
    
    </summary>
    
      <category term="方案" scheme="https://breeze2.github.io/blog/categories/%E6%96%B9%E6%A1%88/"/>
    
    
      <category term="js" scheme="https://breeze2.github.io/blog/tags/js/"/>
    
      <category term="html5" scheme="https://breeze2.github.io/blog/tags/html5/"/>
    
      <category term="css" scheme="https://breeze2.github.io/blog/tags/css/"/>
    
      <category term="scheme" scheme="https://breeze2.github.io/blog/tags/scheme/"/>
    
  </entry>
  
  <entry>
    <title>基于Redis的任务调度设计方案</title>
    <link href="https://breeze2.github.io/blog/scheme-redis-task-queue.html"/>
    <id>https://breeze2.github.io/blog/scheme-redis-task-queue.html</id>
    <published>2018-01-08T09:23:02.000Z</published>
    <updated>2019-09-30T02:22:37.086Z</updated>
    
    <content type="html"><![CDATA[<blockquote><p>一个网关服务器就跟快餐店一样，总是希望客人来得快、去得也快，这样在相同时间内才可以服务更多的客人。如果快餐店的服务员在一个顾客点餐、等餐和结账时都全程跟陪的话，那么这个服务员大部分时间都是在空闲的等待。应该有专门的服务员负责点餐，专门的服务员负责送餐，专门的服务员负责结账，这样才能提高效率。同样道理，网关服务器中也需要分工明确。举个例子：</p></blockquote><p>假设有一个申请发送重置密码邮件的网关接口，须知道发送一封邮件可能会花费上好几秒钟，如果网关服务器直接在线上给用户发送重置密码邮件，高并发的情况下就很容易造成网络拥挤。但实际上，网关服务器并非一定要等待邮件发送成功后才能响应用户，完全可以先告知用户邮件会发送的，而后再在线下把邮件发送出去（就像快餐店里点餐的服务员跟顾客说先去找位置坐，饭菜做好后会有人给他送过去）。</p><p>那么是谁来把邮件发送出去呢？</p><h2 id="任务队列"><a href="#任务队列" class="headerlink" title="任务队列"></a>任务队列</h2><p>为了网关接口能够尽快响应用户请求，无需即时知道结果的耗时操作可以交由任务队列机制来处理。<br>任务队列机制中包含两种角色，一个是任务生产者，一个是任务消费者，而任务队列是两者之间的纽带：</p><ul><li>生产者往队列里放入任务；</li><li>消费者从队列里取出任务。</li></ul><p>任务队列的整体运行流程是：任务生产者把当前操作的关键信息（后续可以根据这些信息还原出当前操作）抽象出来，比如发送重置密码的邮件，我们只需要当前用户邮箱和用户名就可以了；任务生产者把任务放进队列，实际就是把任务的关键信息存储起来，这里会用到MySQL、Redis之类数据存储工具，常用的是Redis；而任务消费者就不断地从数据库中取出任务信息，逐一执行。</p><p>任务生产者的工作是任务分发，一般由线上的网关服务程序执行；任务消费者的工作是任务调度，一般由线下的程序执行，这样即使任务耗时再多，也不阻塞网关服务。</p><p>这里主要讨论的是任务调度（任务消费者）的程序设计。</p><a id="more"></a><h2 id="简单直接"><a href="#简单直接" class="headerlink" title="简单直接"></a>简单直接</h2><p>假设我们用Redis列表List存储任务信息，列表键名是<code>queues:default</code>，任务发布就是往列表<code>queues:default</code>后追加数据：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="comment">// PHP伪代码</span></span><br><span class="line">    Redis::rpush(<span class="string">'queues:default'</span>, serialize($task));</span><br></pre></td></tr></table></figure><p>那么任务调度可以这样简单直接的实现：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="comment">// PHP伪代码</span></span><br><span class="line"><span class="class"><span class="keyword">Class</span> <span class="title">Worker</span> </span>&#123;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">schedule</span><span class="params">()</span> </span>&#123;</span><br><span class="line">        <span class="keyword">while</span>(<span class="number">1</span>) &#123;</span><br><span class="line">            $seri = Redis::lpop(<span class="string">'queues:default'</span>);</span><br><span class="line">            <span class="keyword">if</span>($seri) &#123;</span><br><span class="line">                $task = unserialize($seri);</span><br><span class="line">                <span class="keyword">$this</span>-&gt;handle($task);</span><br><span class="line">                <span class="keyword">continue</span>;</span><br><span class="line">            &#125;</span><br><span class="line">            sleep(<span class="number">1</span>);</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">handle</span><span class="params">($task)</span> </span>&#123;</span><br><span class="line">        <span class="comment">// do something time-consuming</span></span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br><span class="line"></span><br><span class="line">$worker = <span class="keyword">new</span> Worker;</span><br><span class="line">$worker-&gt;schedule();</span><br></pre></td></tr></table></figure><h2 id="意外保险"><a href="#意外保险" class="headerlink" title="意外保险"></a>意外保险</h2><p>上面代码是直接从<code>queues:default</code>列表中移出第一个任务（lpop），因为<code>handle($task)</code>函数是一个耗时的操作，过程中若是遇到什么意外导致了整个程序退出，这个任务可能还没执行完成，可是任务信息已经完全丢失了。保险起见，对<code>schedule()</code>函数进行以下修改：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line">...</span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">schedule</span><span class="params">()</span> </span>&#123;</span><br><span class="line">        <span class="keyword">while</span>(<span class="number">1</span>) &#123;</span><br><span class="line">            $seri = Redis::lindex(<span class="string">'queues:default'</span>, <span class="number">0</span>);</span><br><span class="line">            <span class="keyword">if</span>($seri) &#123;</span><br><span class="line">                $task = unserialize($seri);</span><br><span class="line">                <span class="keyword">$this</span>-&gt;handle($task);</span><br><span class="line">                Redis::lpop(<span class="string">'queues:default'</span>);</span><br><span class="line">                <span class="keyword">continue</span>;</span><br><span class="line">            &#125;</span><br><span class="line">            sleep(<span class="number">1</span>);</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">...</span><br></pre></td></tr></table></figure><p>即在任务完成后才将任务信息从列表中移除。</p><h2 id="延时执行"><a href="#延时执行" class="headerlink" title="延时执行"></a>延时执行</h2><p><code>queues:default</code>列表中的任务都是需要即时执行的，但是有些任务是需要间隔一段时间后或者在某个时间点上执行，那么可以引入一个有序集合，命名为<code>queues:default:delayed</code>，来存放这些任务。任务发布时需要指明执行的时间点<code>$time</code>：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="comment">// PHP伪代码</span></span><br><span class="line">    Redis::zadd(<span class="string">'queues:default:delayed'</span>, $time, serialize($task));</span><br></pre></td></tr></table></figure><p>任务调度时，如果<code>queues:default</code>列表已经空了，就从<code>queues:default:delayed</code>集合中取出到达执行时间的任务放入<code>queues:default</code>列表中：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line">...</span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">schedule</span><span class="params">()</span> </span>&#123;</span><br><span class="line">        <span class="keyword">while</span>(<span class="number">1</span>) &#123;</span><br><span class="line">            $seri = Redis::lindex(<span class="string">'queues:default'</span>, <span class="number">0</span>);</span><br><span class="line">            <span class="keyword">if</span>($seri) &#123;</span><br><span class="line">                $task = unserialize($seri);</span><br><span class="line">                <span class="keyword">$this</span>-&gt;handle($task);</span><br><span class="line">                Redis::lpop(<span class="string">'queues:default'</span>);</span><br><span class="line">                <span class="keyword">continue</span>;</span><br><span class="line">            &#125;</span><br><span class="line">            $seri_arr = Redis::zremrangebyscore(<span class="string">'queues:default:delayed'</span>, <span class="number">0</span>, time());</span><br><span class="line">            <span class="keyword">if</span>($seri_arr) &#123;</span><br><span class="line">                Redis::rpush(<span class="string">'queues:default'</span>, $seri_arr);</span><br><span class="line">                <span class="keyword">continue</span>;</span><br><span class="line">            &#125;</span><br><span class="line">            sleep(<span class="number">1</span>);</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line">...</span><br></pre></td></tr></table></figure><h2 id="任务超时"><a href="#任务超时" class="headerlink" title="任务超时"></a>任务超时</h2><p>预估任务正常执行所需的最大时间值，若是任务执行超过了这个时间，可能是过程中遇到一些意外，如果任由它继续卡着，那么后面的任务就会无法被执行了。<br>首先我们给任务设定一个时限属性<code>timeout</code>，然后在执行任务前先给进程本身设置一个闹钟信号，<code>timeout</code>后收到信号说明任务执行超时，需要退出当前进程（用<a href="http://supervisord.org/" target="_blank" rel="noopener">supervisor</a>守护进程时，进程自身退出，supervisor会自动再拉起）。<br>注意：<code>pcntl_alarm($timeout)</code>会覆盖之前闹钟信号，而<code>pcntl_alarm(0)</code>会取消闹钟信号；任务超时后，当前任务放入<code>queues:default:delayed</code>集合中延时执行，以免再次阻塞队列。</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line">...</span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">schedule</span><span class="params">()</span> </span>&#123;</span><br><span class="line">        <span class="keyword">while</span>(<span class="number">1</span>) &#123;</span><br><span class="line">            $seri = Redis::lindex(<span class="string">'queues:default'</span>, <span class="number">0</span>);</span><br><span class="line">            <span class="keyword">if</span>($seri) &#123;</span><br><span class="line">                $task = unserialize($seri);</span><br><span class="line">                <span class="keyword">$this</span>-&gt;timeoutHanle($task);</span><br><span class="line">                <span class="keyword">$this</span>-&gt;handle($task);</span><br><span class="line">                Redis::lpop(<span class="string">'queues:default'</span>);</span><br><span class="line">                <span class="keyword">continue</span>;</span><br><span class="line">            &#125;</span><br><span class="line">            $seri_arr = Redis::zremrangebyscore(<span class="string">'queues:default:delayed'</span>, <span class="number">0</span>, time());</span><br><span class="line">            <span class="keyword">if</span>($seri_arr) &#123;</span><br><span class="line">                Redis::rpush(<span class="string">'queues:default'</span>, $seri_arr);</span><br><span class="line">                <span class="keyword">continue</span>;</span><br><span class="line">            &#125;</span><br><span class="line">            pcntl_alarm(<span class="number">0</span>);</span><br><span class="line">            sleep(<span class="number">1</span>);</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">timeoutHanle</span><span class="params">($task)</span> </span>&#123;</span><br><span class="line">        $timeout = (int)$task-&gt;timeout;</span><br><span class="line">        <span class="keyword">if</span> ($timeout &gt; <span class="number">0</span>) &#123;</span><br><span class="line">            pcntl_signal(SIGALRM, <span class="function"><span class="keyword">function</span> <span class="params">()</span> </span>&#123;</span><br><span class="line">                $seri = Redis::lpop(<span class="string">'queues:default'</span>);</span><br><span class="line">                Redis::zadd(<span class="string">'queues:default:delayed'</span>, time()+<span class="number">10</span>), $seri);</span><br><span class="line">                posix_kill(getmypid(), SIGKILL);</span><br><span class="line">            &#125;);</span><br><span class="line">        &#125;</span><br><span class="line">        pcntl_alarm($timeout);</span><br><span class="line">    &#125;</span><br><span class="line">...</span><br></pre></td></tr></table></figure><h2 id="并发执行"><a href="#并发执行" class="headerlink" title="并发执行"></a>并发执行</h2><p>上面代码，直观上没什么问题，但是在多进程并发执行的时候，有些任务可能会被重复执行，是因为没能及时将当前执行的任务从<code>queues:default</code>列表中移出，其他进程也可以读取到。为了避免重复执行的问题，我们需要引入一个有序集合SortedSet存放正在执行的任务，命名为<code>queues:default:reserved</code>。<br>首先任务是从<code>queues:default</code>列表中直接移出，然后开始执行任务前先把任务放进<code>queues:default:reserved</code>集合中，任务完成了再从<code>queues:default:reserved</code>集合中移出。<br>再结合任务超时，假设一个任务执行时间不可能超过<code>60*60</code>秒（可以按需调整），在<code>queues:default</code>列表为空的时候，<code>queues:default:reserved</code>集合中有任务已经存放超过了<code>60*60</code>秒，那么有可能是某些进程在执行任务是意外退出了，所以把这些任务放到<code>queues:default:delayed</code>集合中稍后执行。</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line">...</span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">schedule</span><span class="params">()</span> </span>&#123;</span><br><span class="line">        <span class="keyword">while</span>(<span class="number">1</span>) &#123;</span><br><span class="line">            $seri = Redis::lpop(<span class="string">'queues:default'</span>, <span class="number">0</span>);</span><br><span class="line">            <span class="keyword">if</span>($seri) &#123;</span><br><span class="line">                Redis::zadd(<span class="string">'queues:default:reserved'</span>, time()+<span class="number">10</span>, $seri);</span><br><span class="line">                $task = unserialize($seri);                </span><br><span class="line">                <span class="keyword">$this</span>-&gt;timeoutHanle($task);</span><br><span class="line">                <span class="keyword">$this</span>-&gt;handle($task);</span><br><span class="line">                Redis::zrem(<span class="string">'queues:default:reserved'</span>, $seri);</span><br><span class="line">                <span class="keyword">continue</span>;</span><br><span class="line">            &#125;</span><br><span class="line">            $seri_arr = Redis::zremrangebyscore(<span class="string">'queues:default:delayed'</span>, <span class="number">0</span>, time());</span><br><span class="line">            <span class="keyword">if</span>($seri_arr) &#123;</span><br><span class="line">                Redis::rpush(<span class="string">'queues:default'</span>, $seri_arr);</span><br><span class="line">                <span class="keyword">continue</span>;</span><br><span class="line">            &#125;</span><br><span class="line">            $seri_arr = Redis::zremrangebyscore(<span class="string">'queues:default:reserved'</span>, <span class="number">0</span>, time()<span class="number">-60</span>*<span class="number">60</span>);</span><br><span class="line">            <span class="keyword">if</span>($seri_arr) &#123;</span><br><span class="line">                <span class="keyword">foreach</span>($seri_arr <span class="keyword">as</span> $seri) &#123;</span><br><span class="line">                    Redis::zadd(<span class="string">'queues:default:delayed'</span>, time()+<span class="number">10</span>, $seri);</span><br><span class="line">                &#125;</span><br><span class="line">            &#125;</span><br><span class="line"></span><br><span class="line">            sleep(<span class="number">1</span>);</span><br><span class="line">        &#125;</span><br><span class="line">    &#125;</span><br><span class="line"></span><br><span class="line">    <span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">timeoutHanle</span><span class="params">($task)</span> </span>&#123;</span><br><span class="line">        $timeout = (int)$task-&gt;timeout;</span><br><span class="line">        <span class="keyword">if</span> ($timeout &gt; <span class="number">0</span>) &#123;</span><br><span class="line">            pcntl_signal(SIGALRM, <span class="function"><span class="keyword">function</span> <span class="params">()</span> <span class="title">use</span> <span class="params">($task)</span> </span>&#123;</span><br><span class="line">                $seri = serialize($task);</span><br><span class="line">                Redis::zrem(<span class="string">'queues:default:reserved'</span>, $seri);</span><br><span class="line">                Redis::zadd(<span class="string">'queues:default:delayed'</span>, time()+<span class="number">10</span>), $seri);</span><br><span class="line">                posix_kill(getmypid(), SIGKILL);</span><br><span class="line">            &#125;);</span><br><span class="line">        &#125;</span><br><span class="line">        pcntl_alarm($timeout);</span><br><span class="line">    &#125;</span><br><span class="line">...</span><br></pre></td></tr></table></figure><h2 id="其他"><a href="#其他" class="headerlink" title="其他"></a>其他</h2><h3 id="失败重试"><a href="#失败重试" class="headerlink" title="失败重试"></a>失败重试</h3><p>以上代码没有检验任务是否执行成功，应该有任务失败的处理机制：比如给任务设定一个最多重试次数属性<code>retry_times</code>，任务每执行一次<code>retry_times</code>，任务执行失败时，若是<code>retry_times</code>等于0，则将任务放入<code>queues:default:failed</code>列表中不再执行；否则放入放到<code>queues:default:delayed</code>集合中稍后执行。</p><h3 id="休眠时间"><a href="#休眠时间" class="headerlink" title="休眠时间"></a>休眠时间</h3><p>以上代码是进程忙时连续执行，闲时休眠一秒，可以按需调整优化。</p><h3 id="事件监听"><a href="#事件监听" class="headerlink" title="事件监听"></a>事件监听</h3><p>若是需要在任务执行成功或失败时进行某些操作，可以给任务设定成功操作方法<code>afterSucceeded()</code>或失败操作方法<code>afterFailed()</code>，在相应的时候回调。</p><h2 id="最后"><a href="#最后" class="headerlink" title="最后"></a>最后</h2><p>以上讲述了一个任务调度程序的逐步演变，设计方案很大程度上参考了<a href="https://laravel.com/docs/5.5/queues" target="_blank" rel="noopener">Laravel Queue</a>。<br>用工具，知其然，知其所以然。</p>]]></content>
    
    <summary type="html">
    
      &lt;blockquote&gt;
&lt;p&gt;一个网关服务器就跟快餐店一样，总是希望客人来得快、去得也快，这样在相同时间内才可以服务更多的客人。如果快餐店的服务员在一个顾客点餐、等餐和结账时都全程跟陪的话，那么这个服务员大部分时间都是在空闲的等待。应该有专门的服务员负责点餐，专门的服务员负责送餐，专门的服务员负责结账，这样才能提高效率。同样道理，网关服务器中也需要分工明确。举个例子：&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;假设有一个申请发送重置密码邮件的网关接口，须知道发送一封邮件可能会花费上好几秒钟，如果网关服务器直接在线上给用户发送重置密码邮件，高并发的情况下就很容易造成网络拥挤。但实际上，网关服务器并非一定要等待邮件发送成功后才能响应用户，完全可以先告知用户邮件会发送的，而后再在线下把邮件发送出去（就像快餐店里点餐的服务员跟顾客说先去找位置坐，饭菜做好后会有人给他送过去）。&lt;/p&gt;
&lt;p&gt;那么是谁来把邮件发送出去呢？&lt;/p&gt;
&lt;h2 id=&quot;任务队列&quot;&gt;&lt;a href=&quot;#任务队列&quot; class=&quot;headerlink&quot; title=&quot;任务队列&quot;&gt;&lt;/a&gt;任务队列&lt;/h2&gt;&lt;p&gt;为了网关接口能够尽快响应用户请求，无需即时知道结果的耗时操作可以交由任务队列机制来处理。&lt;br&gt;任务队列机制中包含两种角色，一个是任务生产者，一个是任务消费者，而任务队列是两者之间的纽带：&lt;/p&gt;
&lt;ul&gt;
&lt;li&gt;生产者往队列里放入任务；&lt;/li&gt;
&lt;li&gt;消费者从队列里取出任务。&lt;/li&gt;
&lt;/ul&gt;
&lt;p&gt;任务队列的整体运行流程是：任务生产者把当前操作的关键信息（后续可以根据这些信息还原出当前操作）抽象出来，比如发送重置密码的邮件，我们只需要当前用户邮箱和用户名就可以了；任务生产者把任务放进队列，实际就是把任务的关键信息存储起来，这里会用到MySQL、Redis之类数据存储工具，常用的是Redis；而任务消费者就不断地从数据库中取出任务信息，逐一执行。&lt;/p&gt;
&lt;p&gt;任务生产者的工作是任务分发，一般由线上的网关服务程序执行；任务消费者的工作是任务调度，一般由线下的程序执行，这样即使任务耗时再多，也不阻塞网关服务。&lt;/p&gt;
&lt;p&gt;这里主要讨论的是任务调度（任务消费者）的程序设计。&lt;/p&gt;
    
    </summary>
    
      <category term="方案" scheme="https://breeze2.github.io/blog/categories/%E6%96%B9%E6%A1%88/"/>
    
    
      <category term="scheme" scheme="https://breeze2.github.io/blog/tags/scheme/"/>
    
      <category term="php" scheme="https://breeze2.github.io/blog/tags/php/"/>
    
      <category term="redis" scheme="https://breeze2.github.io/blog/tags/redis/"/>
    
      <category term="queue" scheme="https://breeze2.github.io/blog/tags/queue/"/>
    
  </entry>
  
  <entry>
    <title>动态更新时效性缓存的一个解决方案</title>
    <link href="https://breeze2.github.io/blog/scheme-refresh-timeliness-cache.html"/>
    <id>https://breeze2.github.io/blog/scheme-refresh-timeliness-cache.html</id>
    <published>2018-01-08T09:23:02.000Z</published>
    <updated>2019-09-30T02:22:37.086Z</updated>
    
    <content type="html"><![CDATA[<p>最近看到一篇文章<a href="http://blog.jobbole.com/111076/" target="_blank" rel="noopener">一张优惠券引发的血案</a>，文章作者实现一个获取优惠券列表的RPC接口，接口中的执行流程大概是先在缓存里获取优惠券列表，有则返回，没有则在数据库里获取，将结果写入缓存再返回。然而，在高并发请求的情况下，缓存优惠券列表中出现大量的重复数据。<br>主要原因是：</p><ol><li>程序只适应单例执行，没有考虑到多例并发的情况；</li><li>优惠券列表在缓存中存储结构是数组，而更新方法是直接追加；</li><li>直接在RPC接口中执行更新缓存。</li></ol><p>文章作者最后通过反复判断来解决这个问题，但并不是一个好办法。其实这样需要动态更新缓存场景在开发中十分常见，所以在这里探讨一下相对完善的解决方案。</p><a id="more"></a><h2 id="场景"><a href="#场景" class="headerlink" title="场景"></a>场景</h2><p>假设有一些时效性的数据（比如最新资讯、今日天气等等），这些数据在一定时间内不会改变，用户经常会获取，而直接从数据库中读取相对耗时，那么这些数据很应该放在缓存中。<br>因为这些数据具有时效性，所以需要及时更新。用定时更新的话（一般都不会每个1秒更新一次），会有一些间隔；用动态更新的话（由用户触发更新），在用户请求获取这些数据的时候检查是否需要更新，是则更新并将新的数据返回给用户。</p><h2 id="实现"><a href="#实现" class="headerlink" title="实现"></a>实现</h2><h3 id="数据结构"><a href="#数据结构" class="headerlink" title="数据结构"></a>数据结构</h3><p>在缓存中存储数据时，应该选择合适的数据结构。</p><ol><li>如果数据本身是有规则元素的集合，可以考虑采用数组结构存储，方便扩展；</li><li>如果数据本身是有规则元素的集合且元素各有主键，采用映射结构存储，可以单独对元素进行更新；</li><li>如果数据本身毫无规则，可以封装成JSON格式，采用字符串结构存储，更新时是全量更新。</li></ol><p>另外，需要记录数据上一次更新的时间戳，即将这个时间戳也放进缓存，以此来判断数据是否有效，是否需要更新。当然，像Redis这类成熟的存储工具，可以直接利用其提供的TTL（time to live）来判断。</p><h3 id="任务队列"><a href="#任务队列" class="headerlink" title="任务队列"></a>任务队列</h3><p>首先先实现一个更新数据的任务，这里更新是指全量更新，伪代码如下：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="keyword">static</span> <span class="function"><span class="keyword">function</span> <span class="title">updateTheDataTask</span><span class="params">()</span> </span>&#123;</span><br><span class="line">    $ttl  = Redis::getTheDataTTL();</span><br><span class="line">    <span class="keyword">if</span>($ttl &gt; NEED_TO_UPDATE) &#123;</span><br><span class="line">        <span class="keyword">return</span> <span class="keyword">false</span>;</span><br><span class="line">    &#125;</span><br><span class="line">    $data = MySQL::getTheData();</span><br><span class="line">    Redis::setTheData($data); <span class="comment">// update expires of the data at the same time</span></span><br><span class="line">    sleep(<span class="number">2</span>);</span><br><span class="line">    Queue::cancelOtherUpdateTheDataTasks();</span><br><span class="line">    <span class="keyword">return</span> <span class="keyword">true</span>;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p>然后，运行一个执行数据更新任务的队列进程，注意须是单进程，而非多进程，这样可以确保在一个时间点上只执行一个数据更新任务。其中做TTL判断，是为了过滤没必要任务执行；更新成功后休眠两秒（其实可以休眠更久点），主要等待Redis同步，以免后续任务获取到旧的数据；取消掉后续的数据更新任务是因为根本没必要执行了。</p><p>任务队列可以用<a href="http://gearman.org/" target="_blank" rel="noopener">gearman</a>实现，而队列进程可以用<a href="http://supervisord.org/" target="_blank" rel="noopener">supervisord</a>来守护，这里不展开介绍。</p><h3 id="网关接口"><a href="#网关接口" class="headerlink" title="网关接口"></a>网关接口</h3><p>有了上面的任务队列，下面实现面向用户的获取数据接口就容易多了，伪代码如下：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">public</span> <span class="function"><span class="keyword">function</span> <span class="title">theData</span><span class="params">()</span> </span>&#123;</span><br><span class="line">    $data = Redis::getTheData();</span><br><span class="line">    <span class="keyword">if</span>($data) &#123;</span><br><span class="line">        <span class="keyword">return</span> $data;</span><br><span class="line">    &#125; <span class="keyword">else</span> &#123;</span><br><span class="line">        $data = MySQL::getTheData();</span><br><span class="line">        Queue::pushUpdateTheDataTask();</span><br><span class="line">        <span class="keyword">return</span> $data;</span><br><span class="line">    &#125;</span><br><span class="line">&#125;</span><br></pre></td></tr></table></figure><p><code>Queue::pushUpdateTheDataTask()</code>只是将数据更新任务放入队列，由队列处理进程去执行。这里强调一下，数据更新虽然是由用户触发，但是绝不在面向用户的网关接口里执行，因为面向用户的网关接口可能会有大量并发请求，接口内不应该做耗时操作，所以才建立任务队列来执行数据更新，减轻网关压力。</p><h3 id="效果"><a href="#效果" class="headerlink" title="效果"></a>效果</h3><p>这个方案在大量并发访问网关接口的情况下，缓存的数据可以保持准确，过程没有过多的无用操作，也节省了执行时间。</p><h2 id="最后"><a href="#最后" class="headerlink" title="最后"></a>最后</h2><p>《一张优惠券引发的血案》图文并茂，文中作者对进程、代码块、分布式锁等方面进行了讨论，最后也提供解决方法，然而读者更需要的是独立思考——同样的问题，可以有不同的、更好的解决方法。</p><p>开卷有益，贵在思考。</p>]]></content>
    
    <summary type="html">
    
      &lt;p&gt;最近看到一篇文章&lt;a href=&quot;http://blog.jobbole.com/111076/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;一张优惠券引发的血案&lt;/a&gt;，文章作者实现一个获取优惠券列表的RPC接口，接口中的执行流程大概是先在缓存里获取优惠券列表，有则返回，没有则在数据库里获取，将结果写入缓存再返回。然而，在高并发请求的情况下，缓存优惠券列表中出现大量的重复数据。&lt;br&gt;主要原因是：&lt;/p&gt;
&lt;ol&gt;
&lt;li&gt;程序只适应单例执行，没有考虑到多例并发的情况；&lt;/li&gt;
&lt;li&gt;优惠券列表在缓存中存储结构是数组，而更新方法是直接追加；&lt;/li&gt;
&lt;li&gt;直接在RPC接口中执行更新缓存。&lt;/li&gt;
&lt;/ol&gt;
&lt;p&gt;文章作者最后通过反复判断来解决这个问题，但并不是一个好办法。其实这样需要动态更新缓存场景在开发中十分常见，所以在这里探讨一下相对完善的解决方案。&lt;/p&gt;
    
    </summary>
    
      <category term="方案" scheme="https://breeze2.github.io/blog/categories/%E6%96%B9%E6%A1%88/"/>
    
    
      <category term="scheme" scheme="https://breeze2.github.io/blog/tags/scheme/"/>
    
      <category term="mysql" scheme="https://breeze2.github.io/blog/tags/mysql/"/>
    
      <category term="php" scheme="https://breeze2.github.io/blog/tags/php/"/>
    
      <category term="redis" scheme="https://breeze2.github.io/blog/tags/redis/"/>
    
  </entry>
  
  <entry>
    <title>用Docker搭建RabbitMQ高可用集群</title>
    <link href="https://breeze2.github.io/blog/practice-rabbitmq-ha-docker-compose.html"/>
    <id>https://breeze2.github.io/blog/practice-rabbitmq-ha-docker-compose.html</id>
    <published>2017-12-04T18:39:06.000Z</published>
    <updated>2019-09-30T02:22:37.086Z</updated>
    
    <content type="html"><![CDATA[<blockquote><p><a href="https://www.rabbitmq.com/" target="_blank" rel="noopener">RabbitMQ</a>是基于<a href="https://www.amqp.org/" target="_blank" rel="noopener">高级消息队列协议（AMQP）</a>实现的开源消息代理软件，主要提供消息队列服务。这里介绍用Docker Compose搭建RabbitMQ高可用集群的过程。</p></blockquote><p>RabbitMQ自身提供部署集群的功能，通过命令：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">$ rabbitmqctl -n rabbit@rmqha_node1 stop_app</span><br><span class="line">$ rabbitmqctl -n rabbit@rmqha_node1 join_cluster --ram rabbit@rmqha_node0</span><br><span class="line">$ rabbitmqctl -n rabbit@rmqha_node1 start_app</span><br></pre></td></tr></table></figure><p>就可以很容易的将节点rabbit@rmqha_node1加入到集群rabbit@rmqha_node0中。<code>--ram</code>选项表示节点以内存存储方式运行，读写速度快，重启后内容会丢失；不加<code>--ram</code>选项，节点则以磁盘存储方式运行，虽然读写速度慢，但是内容一般可以持久保持。</p><p>在同一个RabbitMQ集群中，节点之间并没有主从之分，所有节点会同步相同的队列结构，队列内容（消息）则各自不同，不过消息会在节点间传递。这样的集群只是提高了应对大量并发请求的能力，整体可用性还是很低，因为某个节点宕机后，寄存在该节点上的消息不可用，而在其他节点上也没有这些消息的备份，若是该节点无法恢复，那么这些消息就丢失了。</p><p>为了解决这个问题，RabbitMQ提供镜像队列功能，通过命令：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ rabbitmqctl set_policy ha-all "^" '&#123;"ha-<span class="built_in">mode</span>":"all"&#125;'</span><br></pre></td></tr></table></figure><p>可以设置镜像队列，<code>&quot;^&quot;</code>表示匹配所有队列，即所有队列在各个节点上都会有备份。在集群中，只需要在一个节点上设置镜像队列，设置操作会同步到其他节点。</p><a id="more"></a><h2 id="Docker-Compose编排"><a href="#Docker-Compose编排" class="headerlink" title="Docker Compose编排"></a>Docker Compose编排</h2><p>这个编排主要实现一个磁盘节点、两个内存节点的RabbitMQ集群和一个HAProxy代理。</p><h3 id="目录结构"><a href="#目录结构" class="headerlink" title="目录结构"></a>目录结构</h3><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">L--rabbitmq-ha-docker                    //主目录</span><br><span class="line">    L--scripts                           //本地（Docker宿主）使用的一些脚本</span><br><span class="line">        L--rmqha_set_policy.sh           //设置各个数据库账号和开启主从复制</span><br><span class="line">    L--volumes                           //各个容器的挂载数据卷</span><br><span class="line">        L--rmqha_proxy</span><br><span class="line">            L--haproxy.cfg               //HAProxy配置</span><br><span class="line">        L--rmqha_slave</span><br><span class="line">            L--cluster_entrypoint.sh     //入口文件</span><br><span class="line">    L--parameters.env                    //账号密码等环境参数</span><br><span class="line">    L--docker-compose.yml                //编排配置</span><br></pre></td></tr></table></figure><h3 id="docker-compose-yml"><a href="#docker-compose-yml" class="headerlink" title="docker-compose.yml"></a>docker-compose.yml</h3><figure class="highlight yml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br><span class="line">92</span><br><span class="line">93</span><br><span class="line">94</span><br><span class="line">95</span><br><span class="line">96</span><br><span class="line">97</span><br><span class="line">98</span><br><span class="line">99</span><br><span class="line">100</span><br><span class="line">101</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">version:</span> <span class="string">"2"</span></span><br><span class="line"><span class="attr">services:</span></span><br><span class="line"></span><br><span class="line"><span class="attr">  master:</span></span><br><span class="line"><span class="attr">    image:</span> <span class="attr">rabbitmq:3.6-management</span></span><br><span class="line"><span class="attr">    container_name:</span> <span class="string">rmqha_node0</span></span><br><span class="line"><span class="attr">    restart:</span> <span class="string">always</span></span><br><span class="line"><span class="attr">    mem_limit:</span> <span class="number">256</span><span class="string">m</span></span><br><span class="line"><span class="attr">    networks:</span></span><br><span class="line"><span class="attr">      net1:</span></span><br><span class="line"><span class="attr">        ipv4_address:</span> <span class="number">10.9</span><span class="number">.0</span><span class="number">.10</span></span><br><span class="line"><span class="attr">    hostname:</span> <span class="string">rmqha_node0</span></span><br><span class="line"><span class="attr">    ports:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"55672:15672"</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"56720:5672"</span></span><br><span class="line"><span class="attr">    env_file:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">./parameters.env</span></span><br><span class="line"><span class="attr">    environment:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">CONTAINER_NAME=rmqha_node0</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">RABBITMQ_HOSTNAME=rmqha_node0</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">RABBITMQ_NODENAME=rabbit</span></span><br><span class="line"></span><br><span class="line"><span class="attr">  slave1:</span></span><br><span class="line"><span class="attr">    image:</span> <span class="attr">rabbitmq:3.6-management</span></span><br><span class="line"><span class="attr">    container_name:</span> <span class="string">rmqha_node1</span></span><br><span class="line"><span class="attr">    restart:</span> <span class="string">always</span></span><br><span class="line"><span class="attr">    depends_on:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">master</span></span><br><span class="line"><span class="attr">    mem_limit:</span> <span class="number">256</span><span class="string">m</span></span><br><span class="line"><span class="attr">    networks:</span></span><br><span class="line"><span class="attr">      net1:</span></span><br><span class="line"><span class="attr">        ipv4_address:</span> <span class="number">10.9</span><span class="number">.0</span><span class="number">.11</span></span><br><span class="line"><span class="attr">    hostname:</span> <span class="string">rmqha_node1</span></span><br><span class="line">    <span class="comment"># ports:</span></span><br><span class="line">    <span class="comment">#   - "56721:5672"</span></span><br><span class="line"><span class="attr">    volumes:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"./volumes/rmqha_slave/cluster_entrypoint.sh:/usr/local/bin/cluster_entrypoint.sh"</span></span><br><span class="line"><span class="attr">    entrypoint:</span> <span class="string">"/usr/local/bin/cluster_entrypoint.sh"</span></span><br><span class="line"><span class="attr">    command:</span> <span class="string">"rabbitmq-server"</span></span><br><span class="line"><span class="attr">    env_file:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">./parameters.env</span></span><br><span class="line"><span class="attr">    environment:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">CONTAINER_NAME=rmqha_node1</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">RABBITMQ_HOSTNAME=rmqha_node1</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">RABBITMQ_NODENAME=rabbit</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">RMQHA_RAM_NODE=true</span></span><br><span class="line"></span><br><span class="line"><span class="attr">  slave2:</span></span><br><span class="line"><span class="attr">    image:</span> <span class="attr">rabbitmq:3.6-management</span></span><br><span class="line"><span class="attr">    container_name:</span> <span class="string">rmqha_node2</span></span><br><span class="line"><span class="attr">    restart:</span> <span class="string">always</span></span><br><span class="line"><span class="attr">    depends_on:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">master</span></span><br><span class="line"><span class="attr">    mem_limit:</span> <span class="number">256</span><span class="string">m</span></span><br><span class="line"><span class="attr">    networks:</span></span><br><span class="line"><span class="attr">      net1:</span></span><br><span class="line"><span class="attr">        ipv4_address:</span> <span class="number">10.9</span><span class="number">.0</span><span class="number">.12</span></span><br><span class="line"><span class="attr">    hostname:</span> <span class="string">rmqha_node2</span></span><br><span class="line">    <span class="comment"># ports:</span></span><br><span class="line">    <span class="comment">#   - "56722:5672"</span></span><br><span class="line"><span class="attr">    volumes:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"./volumes/rmqha_slave/cluster_entrypoint.sh:/usr/local/bin/cluster_entrypoint.sh"</span></span><br><span class="line"><span class="attr">    entrypoint:</span> <span class="string">"/usr/local/bin/cluster_entrypoint.sh"</span></span><br><span class="line"><span class="attr">    command:</span> <span class="string">"rabbitmq-server"</span></span><br><span class="line"><span class="attr">    env_file:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">./parameters.env</span></span><br><span class="line"><span class="attr">    environment:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">CONTAINER_NAME=rmqha_node2</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">RABBITMQ_HOSTNAME=rmqha_node2</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">RABBITMQ_NODENAME=rabbit</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">RMQHA_RAM_NODE=true</span></span><br><span class="line">  </span><br><span class="line"><span class="attr">  haproxy:</span></span><br><span class="line"><span class="attr">    image:</span> <span class="attr">haproxy:1.8</span></span><br><span class="line"><span class="attr">    container_name:</span> <span class="string">rmqha_proxy</span></span><br><span class="line"><span class="attr">    restart:</span> <span class="string">always</span></span><br><span class="line"><span class="attr">    depends_on:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">master</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">slave1</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">slave2</span></span><br><span class="line"><span class="attr">    mem_limit:</span> <span class="number">256</span><span class="string">m</span></span><br><span class="line"><span class="attr">    networks:</span></span><br><span class="line"><span class="attr">      net1:</span></span><br><span class="line"><span class="attr">        ipv4_address:</span> <span class="number">10.9</span><span class="number">.0</span><span class="number">.19</span></span><br><span class="line"><span class="attr">    hostname:</span> <span class="string">rmqha_proxy</span></span><br><span class="line"><span class="attr">    ports:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"56729:5672"</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"51080:1080"</span></span><br><span class="line"><span class="attr">    volumes:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"./volumes/rmqha_proxy/haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro"</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"./volumes/rmqha_proxy:/root/rmqha_proxy"</span></span><br><span class="line"><span class="attr">    environment:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">CONTAINER_NAME=rmqha_proxy</span></span><br><span class="line"></span><br><span class="line"><span class="attr">networks:</span></span><br><span class="line"><span class="attr">  net1:</span></span><br><span class="line"><span class="attr">    driver:</span> <span class="string">bridge</span></span><br><span class="line"><span class="attr">    ipam:</span></span><br><span class="line"><span class="attr">      config:</span></span><br><span class="line"><span class="attr">        - subnet:</span> <span class="number">10.9</span><span class="number">.0</span><span class="number">.0</span><span class="string">/16</span></span><br><span class="line"><span class="attr">          gateway:</span> <span class="number">10.9</span><span class="number">.0</span><span class="number">.1</span></span><br></pre></td></tr></table></figure><p>这里配置了四个容器服务，一个<code>haproxy</code>，负责代理各个RabbitMQ服务；三个<code>rabbitmq</code>，组成RabbitMQ集群。每个容器服务都指定了静态IP，即使服务重启也不会出现IP错乱问题，特殊的网络端口映射后面会介绍。</p><h3 id="环境参数"><a href="#环境参数" class="headerlink" title="环境参数"></a>环境参数</h3><p>parameters.env</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br></pre></td><td class="code"><pre><span class="line">RMQHA_MASTER_NODE=rabbit</span><br><span class="line">RMQHA_MASTER_HOST=rmqha_node0</span><br><span class="line">RABBITMQ_DEFAULT_USER=guest</span><br><span class="line">RABBITMQ_DEFAULT_PASS=guest</span><br><span class="line">RABBITMQ_NODENAME=rabbit</span><br><span class="line">RABBITMQ_ERLANG_COOKIE=myerlangcookie</span><br></pre></td></tr></table></figure><h3 id="RabbitMQ启动"><a href="#RabbitMQ启动" class="headerlink" title="RabbitMQ启动"></a>RabbitMQ启动</h3><p>这里RabbitMQ容器是使用<a href="https://hub.docker.com/_/rabbitmq/" target="_blank" rel="noopener">Docker官方镜像</a>生成的，节点rmqha_node0可以直接启动；而节点rmqha_node1和rmqha_node2需要加入到rmqha_node0集群里，所以需要需改入口文件。<br>volumes/rmqha_slave/cluster_entrypoint.sh</p><figure class="highlight shell"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">#</span><span class="bash">!/bin/bash</span></span><br><span class="line">set -e</span><br><span class="line"></span><br><span class="line">if [ -e "/root/is_not_first_time" ]; then</span><br><span class="line">    exec "$@"</span><br><span class="line">else</span><br><span class="line">    /usr/local/bin/docker-entrypoint.sh rabbitmq-server -detached # 先按官方入口文件启动且是后台运行</span><br><span class="line"></span><br><span class="line">    rabbitmqctl -n "$RABBITMQ_NODENAME@$RABBITMQ_HOSTNAME" stop_app # 停止应用</span><br><span class="line">    rabbitmqctl -n "$RABBITMQ_NODENAME@$RABBITMQ_HOSTNAME" join_cluster $&#123;RMQHA_RAM_NODE:+--ram&#125; "$RMQHA_MASTER_NODE@$RMQHA_MASTER_HOST" # 加入rmqha_node0集群</span><br><span class="line">    rabbitmqctl -n "$RABBITMQ_NODENAME@$RABBITMQ_HOSTNAME" start_app # 启动应用</span><br><span class="line">    rabbitmqctl stop # 停止所有服务</span><br><span class="line"></span><br><span class="line">    touch /root/is_not_first_time</span><br><span class="line">    sleep 2s</span><br><span class="line">    exec "$@"</span><br><span class="line">fi</span><br></pre></td></tr></table></figure><h3 id="HAProxy配置"><a href="#HAProxy配置" class="headerlink" title="HAProxy配置"></a>HAProxy配置</h3><p>这里HAProxy容器也是使用<a href="https://hub.docker.com/_/haproxy/" target="_blank" rel="noopener">Docker官方镜像</a>生成的，启动前需要先准备配置文件。<br>volumes/rmqha_proxy/haproxy.cfg</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br></pre></td><td class="code"><pre><span class="line">global</span><br><span class="line">    log 127.0.0.1 local0</span><br><span class="line">    maxconn 4096</span><br><span class="line"></span><br><span class="line">defaults</span><br><span class="line">    log     global</span><br><span class="line">    mode    tcp</span><br><span class="line">    option  tcplog</span><br><span class="line">    retries 3</span><br><span class="line">    option  redispatch</span><br><span class="line">    maxconn 2000</span><br><span class="line">    timeout connect 5000</span><br><span class="line">    timeout client 50000</span><br><span class="line">    timeout server 50000</span><br><span class="line"></span><br><span class="line"># ssl for rabbitmq</span><br><span class="line"># frontend ssl_rabbitmq</span><br><span class="line">    # bind *:5673 ssl crt /root/rmqha_proxy/rmqha.pem</span><br><span class="line">    # mode tcp</span><br><span class="line">    # default_backend rabbitmq</span><br><span class="line"></span><br><span class="line">listen stats</span><br><span class="line">    bind *:1080 # haproxy容器1080端口显示代理统计页面，映射到宿主51080端口</span><br><span class="line">    mode http</span><br><span class="line">    stats enable</span><br><span class="line">    stats hide-version</span><br><span class="line">    stats realm Haproxy\ Statistics</span><br><span class="line">    stats uri /</span><br><span class="line">    stats auth admin:admin</span><br><span class="line"></span><br><span class="line">listen rabbitmq</span><br><span class="line">    bind *:5672 # haproxy容器5672端口代理多个rabbitmq服务，映射到宿主56729端口</span><br><span class="line">    mode tcp</span><br><span class="line">    balance roundrobin</span><br><span class="line">    timeout client 1h</span><br><span class="line">    timeout server 1h</span><br><span class="line">    option  clitcpka</span><br><span class="line">    # server  rmqha_node0 rmqha_node0:5672  check inter 5s rise 2 fall 3</span><br><span class="line">    server  rmqha_node1 rmqha_node1:5672  check inter 5s rise 2 fall 3</span><br><span class="line">    server  rmqha_node2 rmqha_node2:5672  check inter 5s rise 2 fall 3</span><br></pre></td></tr></table></figure><h2 id="实际运行"><a href="#实际运行" class="headerlink" title="实际运行"></a>实际运行</h2><p>在主目录下执行<code>docker-compose up -d</code>构建并运行整个Docker服务。</p><h3 id="镜像队列"><a href="#镜像队列" class="headerlink" title="镜像队列"></a>镜像队列</h3><p>，在主目录下执行：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sh ./scripts/rmqha_set_policy.sh</span><br></pre></td></tr></table></figure><p>实际上是执行了：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ docker exec -it rmqha_node0 rabbitmqctl set_policy ha-all &apos;^&apos; &apos;&#123;&quot;ha-mode&quot;:&quot;all&quot;&#125;&apos;</span><br></pre></td></tr></table></figure><p>即在rmqha_node0集群中将所有队列设置为镜像队列，这个命令只需执行一次，除非重新构建整个Docker服务。</p><h2 id="测试"><a href="#测试" class="headerlink" title="测试"></a>测试</h2><p>用两个PHP脚本可以对RabbitMQ进行简单的测试，不过需要用到<code>php-amqplib</code>库。</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ composer require php-amqplib/php-amqplib</span><br></pre></td></tr></table></figure><p>发送消息脚本：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="comment">// send.php</span></span><br><span class="line"><span class="keyword">require_once</span> <span class="keyword">__DIR__</span> . <span class="string">'/vendor/autoload.php'</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">PhpAmqpLib</span>\<span class="title">Connection</span>\<span class="title">AMQPStreamConnection</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">PhpAmqpLib</span>\<span class="title">Message</span>\<span class="title">AMQPMessage</span>;</span><br><span class="line"></span><br><span class="line">$connection = <span class="keyword">new</span> AMQPStreamConnection(<span class="string">'127.0.0.1'</span>, <span class="number">56729</span>, <span class="string">'guest'</span>, <span class="string">'guest'</span>, <span class="string">'/'</span>); <span class="comment">// 连接rmqha_proxy</span></span><br><span class="line">$channel    = $connection-&gt;channel();</span><br><span class="line">$channel-&gt;queue_declare(<span class="string">'task_queue'</span>, <span class="keyword">false</span>, <span class="keyword">true</span>, <span class="keyword">false</span>, <span class="keyword">false</span>);</span><br><span class="line">$data = <span class="string">"Hello World!"</span>;</span><br><span class="line"></span><br><span class="line">$msg = <span class="keyword">new</span> AMQPMessage($data,</span><br><span class="line">    <span class="keyword">array</span>(<span class="string">'delivery_mode'</span> =&gt; AMQPMessage::DELIVERY_MODE_PERSISTENT)</span><br><span class="line">);</span><br><span class="line">$channel-&gt;basic_publish($msg, <span class="string">''</span>, <span class="string">'task_queue'</span>);</span><br><span class="line"><span class="keyword">echo</span> <span class="string">" [x] Sent "</span>, $data, <span class="string">"\n"</span>;</span><br><span class="line">$channel-&gt;close();</span><br><span class="line">$connection-&gt;close();</span><br></pre></td></tr></table></figure><p>接受消息脚本：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="comment">// receive.php</span></span><br><span class="line"><span class="keyword">require_once</span> <span class="keyword">__DIR__</span> . <span class="string">'/vendor/autoload.php'</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">PhpAmqpLib</span>\<span class="title">Connection</span>\<span class="title">AMQPStreamConnection</span>;</span><br><span class="line"></span><br><span class="line">$connection = <span class="keyword">new</span> AMQPStreamConnection(<span class="string">'127.0.0.1'</span>, <span class="number">56720</span>, <span class="string">'guest'</span>, <span class="string">'guest'</span>, <span class="string">'/'</span>); <span class="comment">// 连接rmqha_node0</span></span><br><span class="line">$channel    = $connection-&gt;channel();</span><br><span class="line">$channel-&gt;queue_declare(<span class="string">'task_queue'</span>, <span class="keyword">false</span>, <span class="keyword">true</span>, <span class="keyword">false</span>, <span class="keyword">false</span>);</span><br><span class="line"><span class="keyword">echo</span> <span class="string">' [*] Waiting for messages. To exit press CTRL+C'</span>, <span class="string">"\n"</span>;</span><br><span class="line">$callback = <span class="function"><span class="keyword">function</span> <span class="params">($msg)</span> </span>&#123;</span><br><span class="line">    <span class="keyword">echo</span> <span class="string">" [x] Received "</span>, $msg-&gt;body, <span class="string">"\n"</span>;</span><br><span class="line">    sleep(<span class="number">1</span>);</span><br><span class="line">    <span class="keyword">echo</span> <span class="string">" [x] Done"</span>, <span class="string">"\n"</span>;</span><br><span class="line">    $msg-&gt;delivery_info[<span class="string">'channel'</span>]-&gt;basic_ack($msg-&gt;delivery_info[<span class="string">'delivery_tag'</span>]);</span><br><span class="line">&#125;;</span><br><span class="line">$channel-&gt;basic_qos(<span class="keyword">null</span>, <span class="number">1</span>, <span class="keyword">null</span>);</span><br><span class="line">$channel-&gt;basic_consume(<span class="string">'task_queue'</span>, <span class="string">''</span>, <span class="keyword">false</span>, <span class="keyword">false</span>, <span class="keyword">false</span>, <span class="keyword">false</span>, $callback);</span><br><span class="line"></span><br><span class="line"><span class="keyword">while</span> (count($channel-&gt;callbacks)) &#123;</span><br><span class="line">    $channel-&gt;wait();</span><br><span class="line">&#125;</span><br><span class="line">$channel-&gt;close();</span><br><span class="line">$connection-&gt;close();</span><br></pre></td></tr></table></figure><h2 id="SSL设置"><a href="#SSL设置" class="headerlink" title="SSL设置"></a>SSL设置</h2><p>为了RabbitMQ服务在网络传输中不泄漏信息，可以给HAProxy设置SSL传输（比对各个RabbitMQ服务设置SSL传输的性能消耗要小），这里简单的介绍一下自签证书：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">$ <span class="built_in">cd</span> ./volumes/rmqha_proxy/</span><br><span class="line">$ openssl genrsa -out rmqha.key <span class="number">1024</span> # 随机生成一个私钥</span><br><span class="line">$ openssl req -new -key rmqha.key -out rmqha.csr # 根据私钥生成证书签署请求</span><br><span class="line">$ openssl x509 -req -days <span class="number">365</span> -<span class="keyword">in</span> rmqha.csr -signkey rmqha.key -out rmqha.crt # 自己签署证书</span><br><span class="line">$ cat rmqha.crt rmqha.key|tee rmqha.pem # 将私钥和证书合并到一个文件中</span><br></pre></td></tr></table></figure><p>注意：生成证书签署请求时，需要填写一些信息，<code>Common Name (eg, fully qualified host name)</code>应该写<code>127.0.0.1</code>。<br>HAProxy配置中添加：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"># ssl for rabbitmq</span><br><span class="line">frontend ssl_rabbitmq</span><br><span class="line">    bind *:5673 ssl crt /root/rmqha_proxy/rmqha.pem # haproxy容器5673端口代理rabbitmq服务，需要映射到宿主端口</span><br><span class="line">    mode tcp</span><br><span class="line">    default_backend rabbitmq</span><br></pre></td></tr></table></figure><p>PHP中通过SSL连接HAProxy代理服务示例：</p><figure class="highlight php"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;?php</span></span><br><span class="line"><span class="keyword">require_once</span> <span class="keyword">__DIR__</span> . <span class="string">'/vendor/autoload.php'</span>;</span><br><span class="line"><span class="keyword">use</span> <span class="title">PhpAmqpLib</span>\<span class="title">Connection</span>\<span class="title">AMQPSSLConnection</span>;</span><br><span class="line">$connection = <span class="keyword">new</span> AMQPSSLConnection(<span class="string">'127.0.0.1'</span>, <span class="number">56730</span>, <span class="string">'guest'</span>, <span class="string">'guest'</span>, <span class="string">'/'</span>, [<span class="string">'cafile'</span>=&gt;<span class="string">'./rmqha.crt'</span>]); <span class="comment">// 假设SSL的监听端口是56730，rmqha.crt是上面生成的自签证书</span></span><br><span class="line">$channel    = $connection-&gt;channel();</span><br></pre></td></tr></table></figure><h2 id="最后"><a href="#最后" class="headerlink" title="最后"></a>最后</h2><h3 id="建议"><a href="#建议" class="headerlink" title="建议"></a>建议</h3><ol><li>消息生产者可以通过HAProxy代理连接RabbitMQ服务，实现负载均衡；</li><li>消息处理者应该直接连接RabbitMQ服务主机，并且跟随RabbitMQ服务主机启动服务、停止服务。</li></ol><h3 id="扩展阅读"><a href="#扩展阅读" class="headerlink" title="扩展阅读"></a>扩展阅读</h3><ul><li><a href="https://addops.cn/post/rabbitmq-ha-mirror.html" target="_blank" rel="noopener">Rabbitmq高可用-镜像队列模式</a></li><li><a href="http://www.cnblogs.com/flat_peach/archive/2013/04/07/3004008.html" target="_blank" rel="noopener">Rabbitmq集群高可用测试</a></li></ul>]]></content>
    
    <summary type="html">
    
      &lt;blockquote&gt;
&lt;p&gt;&lt;a href=&quot;https://www.rabbitmq.com/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;RabbitMQ&lt;/a&gt;是基于&lt;a href=&quot;https://www.amqp.org/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;高级消息队列协议（AMQP）&lt;/a&gt;实现的开源消息代理软件，主要提供消息队列服务。这里介绍用Docker Compose搭建RabbitMQ高可用集群的过程。&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;RabbitMQ自身提供部署集群的功能，通过命令：&lt;/p&gt;
&lt;figure class=&quot;highlight cmd&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;2&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;3&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;$ rabbitmqctl -n rabbit@rmqha_node1 stop_app&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;$ rabbitmqctl -n rabbit@rmqha_node1 join_cluster --ram rabbit@rmqha_node0&lt;/span&gt;&lt;br&gt;&lt;span class=&quot;line&quot;&gt;$ rabbitmqctl -n rabbit@rmqha_node1 start_app&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;

&lt;p&gt;就可以很容易的将节点rabbit@rmqha_node1加入到集群rabbit@rmqha_node0中。&lt;code&gt;--ram&lt;/code&gt;选项表示节点以内存存储方式运行，读写速度快，重启后内容会丢失；不加&lt;code&gt;--ram&lt;/code&gt;选项，节点则以磁盘存储方式运行，虽然读写速度慢，但是内容一般可以持久保持。&lt;/p&gt;
&lt;p&gt;在同一个RabbitMQ集群中，节点之间并没有主从之分，所有节点会同步相同的队列结构，队列内容（消息）则各自不同，不过消息会在节点间传递。这样的集群只是提高了应对大量并发请求的能力，整体可用性还是很低，因为某个节点宕机后，寄存在该节点上的消息不可用，而在其他节点上也没有这些消息的备份，若是该节点无法恢复，那么这些消息就丢失了。&lt;/p&gt;
&lt;p&gt;为了解决这个问题，RabbitMQ提供镜像队列功能，通过命令：&lt;/p&gt;
&lt;figure class=&quot;highlight cmd&quot;&gt;&lt;table&gt;&lt;tr&gt;&lt;td class=&quot;gutter&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;1&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;td class=&quot;code&quot;&gt;&lt;pre&gt;&lt;span class=&quot;line&quot;&gt;$ rabbitmqctl set_policy ha-all &quot;^&quot; &#39;&amp;#123;&quot;ha-&lt;span class=&quot;built_in&quot;&gt;mode&lt;/span&gt;&quot;:&quot;all&quot;&amp;#125;&#39;&lt;/span&gt;&lt;br&gt;&lt;/pre&gt;&lt;/td&gt;&lt;/tr&gt;&lt;/table&gt;&lt;/figure&gt;

&lt;p&gt;可以设置镜像队列，&lt;code&gt;&amp;quot;^&amp;quot;&lt;/code&gt;表示匹配所有队列，即所有队列在各个节点上都会有备份。在集群中，只需要在一个节点上设置镜像队列，设置操作会同步到其他节点。&lt;/p&gt;
    
    </summary>
    
      <category term="实践" scheme="https://breeze2.github.io/blog/categories/%E5%AE%9E%E8%B7%B5/"/>
    
    
      <category term="practice" scheme="https://breeze2.github.io/blog/tags/practice/"/>
    
      <category term="docker" scheme="https://breeze2.github.io/blog/tags/docker/"/>
    
      <category term="rabbitmq" scheme="https://breeze2.github.io/blog/tags/rabbitmq/"/>
    
      <category term="haproxy" scheme="https://breeze2.github.io/blog/tags/haproxy/"/>
    
  </entry>
  
  <entry>
    <title>JS实现人脸换妆和页面截图</title>
    <link href="https://breeze2.github.io/blog/practice-js-face-makeup-and-html2canvas.html"/>
    <id>https://breeze2.github.io/blog/practice-js-face-makeup-and-html2canvas.html</id>
    <published>2017-11-20T09:46:23.000Z</published>
    <updated>2019-09-30T02:22:37.086Z</updated>
    
    <content type="html"><![CDATA[<blockquote><p>最近要做一个H5活动页面，大概功能是给图片人脸换妆，自然也少不了上传原始图片和导出最终图片等功能。如果图片加工（即人脸换妆）放在服务器处理，那么并发量高的时候，肯定产生大量阻塞。于是想把图片加工放在客户端处理，用HTML堆砌出图片的最终效果，但是怎样把HTML导出成图片呢？</p></blockquote><h2 id="图片上传"><a href="#图片上传" class="headerlink" title="图片上传"></a>图片上传</h2><p>图片上传可以参考<a href="https://blog.breezelin.site/2017/08/12/scheme-nginx-php-js-upload-process/" target="_blank" rel="noopener">HTTP文件上传的一个后端完善方案（NginX）</a>，或者使用现有的云存储服务，如<a href="https://www.qiniu.com/" target="_blank" rel="noopener">七牛</a>等等。</p><h2 id="五官定位"><a href="#五官定位" class="headerlink" title="五官定位"></a>五官定位</h2><p>人脸识别和五官定位可以利用<a href="https://opencv.org/" target="_blank" rel="noopener">OpenVC</a>自行实现，或者使用成熟的云识别服务，如<a href="https://www.faceplusplus.com.cn/" target="_blank" rel="noopener">Face++</a>等等。这里使用腾讯云的人脸识别服务。</p><a id="more"></a><h2 id="人脸换妆"><a href="#人脸换妆" class="headerlink" title="人脸换妆"></a>人脸换妆</h2><p>假设有一张图片，识别结果如下：<br><img src="/blog/assets/images/practice-js-face-makeup-and-html2canvas1.png" alt="识别结果"></p><blockquote><p>图片来源：<a href="https://cloud.tencent.com/act/event/ci_demo.html" target="_blank" rel="noopener">万象优图-腾讯云</a></p></blockquote><p>根据五官定位点，在适当位置给人脸添上妆饰，比如小胡子：<br><img src="/blog/assets/images/practice-js-face-makeup-and-html2canvas2.png" alt="小胡子"></p><p>源码：</p><figure class="highlight html"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br></pre></td><td class="code"><pre><span class="line"><span class="meta">&lt;!DOCTYPE html&gt;</span></span><br><span class="line"><span class="tag">&lt;<span class="name">html</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;<span class="name">head</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">title</span>&gt;</span>Demo<span class="tag">&lt;/<span class="name">title</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">head</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;<span class="name">body</span> <span class="attr">style</span>=<span class="string">"padding: 0; margin: 0;"</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">div</span> <span class="attr">style</span>=<span class="string">"width: 100%; text-align: center;"</span>&gt;</span></span><br><span class="line">        <span class="tag">&lt;<span class="name">div</span> <span class="attr">style</span>=<span class="string">"width: 336px; margin-left: auto; margin-right: auto; position: relative;"</span> <span class="attr">id</span>=<span class="string">"div1"</span>&gt;</span></span><br><span class="line">            <span class="tag">&lt;<span class="name">img</span> <span class="attr">src</span>=<span class="string">"./face.png"</span> <span class="attr">style</span>=<span class="string">"width: 100%;"</span> <span class="attr">id</span>=<span class="string">"face"</span> /&gt;</span></span><br><span class="line">        <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;/<span class="name">div</span>&gt;</span></span><br><span class="line">    <span class="tag">&lt;<span class="name">script</span> <span class="attr">type</span>=<span class="string">"text/javascript"</span>&gt;</span></span><br><span class="line"><span class="actionscript">        <span class="keyword">var</span> FACE = &#123;</span></span><br><span class="line"><span class="actionscript">            <span class="string">"session_id"</span>:<span class="string">""</span>,</span></span><br><span class="line"><span class="actionscript">            <span class="string">"face_shape"</span>:[</span></span><br><span class="line">                &#123;</span><br><span class="line"><span class="actionscript">                    <span class="string">"face_profile"</span>:[&#123;<span class="string">"x"</span>:<span class="number">49</span>,<span class="string">"y"</span>:<span class="number">231</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">59</span>,<span class="string">"y"</span>:<span class="number">249</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">71</span>,<span class="string">"y"</span>:<span class="number">267</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">84</span>,<span class="string">"y"</span>:<span class="number">284</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">100</span>,<span class="string">"y"</span>:<span class="number">298</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">118</span>,<span class="string">"y"</span>:<span class="number">310</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">138</span>,<span class="string">"y"</span>:<span class="number">318</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">158</span>,<span class="string">"y"</span>:<span class="number">324</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">178</span>,<span class="string">"y"</span>:<span class="number">329</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">198</span>,<span class="string">"y"</span>:<span class="number">329</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">217</span>,<span class="string">"y"</span>:<span class="number">323</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">232</span>,<span class="string">"y"</span>:<span class="number">309</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">239</span>,<span class="string">"y"</span>:<span class="number">291</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">244</span>,<span class="string">"y"</span>:<span class="number">271</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">249</span>,<span class="string">"y"</span>:<span class="number">252</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">252</span>,<span class="string">"y"</span>:<span class="number">232</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">253</span>,<span class="string">"y"</span>:<span class="number">213</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">251</span>,<span class="string">"y"</span>:<span class="number">194</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">247</span>,<span class="string">"y"</span>:<span class="number">175</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">240</span>,<span class="string">"y"</span>:<span class="number">157</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">232</span>,<span class="string">"y"</span>:<span class="number">142</span>&#125;],</span></span><br><span class="line"><span class="actionscript">                    <span class="string">"left_eye"</span>:[&#123;<span class="string">"x"</span>:<span class="number">89</span>,<span class="string">"y"</span>:<span class="number">209</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">100</span>,<span class="string">"y"</span>:<span class="number">210</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">110</span>,<span class="string">"y"</span>:<span class="number">207</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">120</span>,<span class="string">"y"</span>:<span class="number">202</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">128</span>,<span class="string">"y"</span>:<span class="number">196</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">117</span>,<span class="string">"y"</span>:<span class="number">190</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">105</span>,<span class="string">"y"</span>:<span class="number">191</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">95</span>,<span class="string">"y"</span>:<span class="number">198</span>&#125;],</span></span><br><span class="line"><span class="actionscript">                    <span class="string">"right_eye"</span>:[&#123;<span class="string">"x"</span>:<span class="number">209</span>,<span class="string">"y"</span>:<span class="number">153</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">204</span>,<span class="string">"y"</span>:<span class="number">161</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">196</span>,<span class="string">"y"</span>:<span class="number">167</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">186</span>,<span class="string">"y"</span>:<span class="number">170</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">176</span>,<span class="string">"y"</span>:<span class="number">172</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">179</span>,<span class="string">"y"</span>:<span class="number">161</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">187</span>,<span class="string">"y"</span>:<span class="number">154</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">198</span>,<span class="string">"y"</span>:<span class="number">150</span>&#125;],</span></span><br><span class="line"><span class="actionscript">                    <span class="string">"left_eyebrow"</span>:[&#123;<span class="string">"x"</span>:<span class="number">58</span>,<span class="string">"y"</span>:<span class="number">190</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">72</span>,<span class="string">"y"</span>:<span class="number">180</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">89</span>,<span class="string">"y"</span>:<span class="number">175</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">106</span>,<span class="string">"y"</span>:<span class="number">172</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">122</span>,<span class="string">"y"</span>:<span class="number">167</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">105</span>,<span class="string">"y"</span>:<span class="number">161</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">86</span>,<span class="string">"y"</span>:<span class="number">165</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">68</span>,<span class="string">"y"</span>:<span class="number">174</span>&#125;],</span></span><br><span class="line"><span class="actionscript">                    <span class="string">"right_eyebrow"</span>:[&#123;<span class="string">"x"</span>:<span class="number">214</span>,<span class="string">"y"</span>:<span class="number">120</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">200</span>,<span class="string">"y"</span>:<span class="number">125</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">187</span>,<span class="string">"y"</span>:<span class="number">133</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">175</span>,<span class="string">"y"</span>:<span class="number">143</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">162</span>,<span class="string">"y"</span>:<span class="number">150</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">169</span>,<span class="string">"y"</span>:<span class="number">135</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">182</span>,<span class="string">"y"</span>:<span class="number">123</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">198</span>,<span class="string">"y"</span>:<span class="number">115</span>&#125;],</span></span><br><span class="line"><span class="actionscript">                    <span class="string">"mouth"</span>:[&#123;<span class="string">"x"</span>:<span class="number">149</span>,<span class="string">"y"</span>:<span class="number">279</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">165</span>,<span class="string">"y"</span>:<span class="number">290</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">184</span>,<span class="string">"y"</span>:<span class="number">294</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">202</span>,<span class="string">"y"</span>:<span class="number">289</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">215</span>,<span class="string">"y"</span>:<span class="number">276</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">219</span>,<span class="string">"y"</span>:<span class="number">258</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">219</span>,<span class="string">"y"</span>:<span class="number">239</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">208</span>,<span class="string">"y"</span>:<span class="number">241</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">196</span>,<span class="string">"y"</span>:<span class="number">246</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">189</span>,<span class="string">"y"</span>:<span class="number">253</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">178</span>,<span class="string">"y"</span>:<span class="number">256</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">163</span>,<span class="string">"y"</span>:<span class="number">266</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">165</span>,<span class="string">"y"</span>:<span class="number">282</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">181</span>,<span class="string">"y"</span>:<span class="number">281</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">197</span>,<span class="string">"y"</span>:<span class="number">277</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">207</span>,<span class="string">"y"</span>:<span class="number">266</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">214</span>,<span class="string">"y"</span>:<span class="number">254</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">209</span>,<span class="string">"y"</span>:<span class="number">245</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">200</span>,<span class="string">"y"</span>:<span class="number">252</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">191</span>,<span class="string">"y"</span>:<span class="number">258</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">177</span>,<span class="string">"y"</span>:<span class="number">266</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">163</span>,<span class="string">"y"</span>:<span class="number">273</span>&#125;],</span></span><br><span class="line"><span class="actionscript">                    <span class="string">"nose"</span>:[&#123;<span class="string">"x"</span>:<span class="number">180</span>,<span class="string">"y"</span>:<span class="number">231</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">155</span>,<span class="string">"y"</span>:<span class="number">187</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">154</span>,<span class="string">"y"</span>:<span class="number">201</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">154</span>,<span class="string">"y"</span>:<span class="number">216</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">153</span>,<span class="string">"y"</span>:<span class="number">230</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">152</span>,<span class="string">"y"</span>:<span class="number">247</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">170</span>,<span class="string">"y"</span>:<span class="number">248</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">186</span>,<span class="string">"y"</span>:<span class="number">245</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">196</span>,<span class="string">"y"</span>:<span class="number">235</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">203</span>,<span class="string">"y"</span>:<span class="number">219</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">190</span>,<span class="string">"y"</span>:<span class="number">211</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">178</span>,<span class="string">"y"</span>:<span class="number">203</span>&#125;,&#123;<span class="string">"x"</span>:<span class="number">167</span>,<span class="string">"y"</span>:<span class="number">195</span>&#125;]</span></span><br><span class="line">                &#125;</span><br><span class="line">            ],</span><br><span class="line"><span class="actionscript">            <span class="string">"image_height"</span>:<span class="number">430</span>,</span></span><br><span class="line"><span class="actionscript">            <span class="string">"image_width"</span>:<span class="number">336</span></span></span><br><span class="line"><span class="actionscript">        &#125;; <span class="comment">// 人脸识别结果</span></span></span><br><span class="line"><span class="actionscript">        <span class="keyword">var</span> BEARD = &#123;</span></span><br><span class="line"><span class="actionscript">            <span class="string">"src"</span>:<span class="string">"./beard.png"</span>,</span></span><br><span class="line"><span class="actionscript">            <span class="string">"image_height"</span>:<span class="number">222</span>,</span></span><br><span class="line"><span class="actionscript">            <span class="string">"image_width"</span>:<span class="number">567</span></span></span><br><span class="line"><span class="actionscript">        &#125;; <span class="comment">// 胡子图片信息</span></span></span><br><span class="line"></span><br><span class="line"><span class="actionscript">        <span class="function"><span class="keyword">function</span> <span class="title">parseEyesData</span><span class="params">(leye, reye)</span></span>&#123;</span></span><br><span class="line"><span class="actionscript">            <span class="keyword">var</span> lx = ly = <span class="number">0</span>;</span></span><br><span class="line"><span class="actionscript">            <span class="keyword">var</span> rx = ry = <span class="number">0</span>;</span></span><br><span class="line"><span class="actionscript">            <span class="keyword">for</span>(<span class="keyword">var</span> i <span class="keyword">in</span> leye) &#123;</span></span><br><span class="line"><span class="actionscript">                lx += leye[i][<span class="string">'x'</span>]; ly += leye[i][<span class="string">'y'</span>];</span></span><br><span class="line">            &#125;</span><br><span class="line">            lx = lx/leye.length; ly = ly/leye.length;</span><br><span class="line"><span class="actionscript">            <span class="keyword">for</span>(<span class="keyword">var</span> i <span class="keyword">in</span> reye) &#123;</span></span><br><span class="line"><span class="actionscript">                rx += reye[i][<span class="string">'x'</span>]; ry += reye[i][<span class="string">'y'</span>];</span></span><br><span class="line">            &#125;</span><br><span class="line">            rx = rx/leye.length; ry = ry/leye.length;</span><br><span class="line"><span class="actionscript">            <span class="keyword">return</span> &#123;</span></span><br><span class="line"><span class="javascript">                angle: <span class="built_in">Math</span>.atan((ly-ry)/(lx-rx))/<span class="built_in">Math</span>.PI*(<span class="number">180</span>),</span></span><br><span class="line"><span class="javascript">                span: <span class="built_in">Math</span>.sqrt(<span class="built_in">Math</span>.pow(lx-rx, <span class="number">2</span>)+<span class="built_in">Math</span>.pow(ly-ry, <span class="number">2</span>))</span></span><br><span class="line">            &#125;</span><br><span class="line"><span class="actionscript">        &#125; <span class="comment">// 根据眼睛数据，获取脸倾角和眼心距</span></span></span><br><span class="line"></span><br><span class="line"><span class="javascript">        <span class="keyword">var</span> div1 = <span class="built_in">document</span>.getElementById(<span class="string">'div1'</span>);</span></span><br><span class="line"><span class="actionscript">        <span class="keyword">var</span> edata = parseEyesData(FACE[<span class="string">'face_shape'</span>][<span class="number">0</span>][<span class="string">'left_eye'</span>], FACE[<span class="string">'face_shape'</span>][<span class="number">0</span>][<span class="string">'right_eye'</span>]);</span></span><br><span class="line"><span class="javascript">        <span class="keyword">var</span> beard = <span class="built_in">document</span>.createElement(<span class="string">'img'</span>);</span></span><br><span class="line"><span class="actionscript">        <span class="keyword">var</span> scale = div1.offsetWidth/FACE[<span class="string">'image_width'</span>];</span></span><br><span class="line">        </span><br><span class="line"><span class="actionscript">        <span class="keyword">var</span> point1 = FACE[<span class="string">'face_shape'</span>][<span class="number">0</span>][<span class="string">'nose'</span>][<span class="number">0</span>]; <span class="comment">// 鼻尖</span></span></span><br><span class="line"><span class="actionscript">        <span class="keyword">var</span> point2 = FACE[<span class="string">'face_shape'</span>][<span class="number">0</span>][<span class="string">'nose'</span>][<span class="number">7</span>]; <span class="comment">// 鼻低</span></span></span><br><span class="line"><span class="actionscript">        <span class="keyword">var</span> point3 = FACE[<span class="string">'face_shape'</span>][<span class="number">0</span>][<span class="string">'mouth'</span>][<span class="number">9</span>]; <span class="comment">//上唇上中点</span></span></span><br><span class="line"><span class="actionscript">        <span class="keyword">var</span> beardx = ((point1[<span class="string">'x'</span>]+point2[<span class="string">'x'</span>])/<span class="number">2</span>+point3[<span class="string">'x'</span>])/<span class="number">2</span>*scale; <span class="comment">// 大概算出人中X坐标</span></span></span><br><span class="line"><span class="actionscript">        <span class="keyword">var</span> beardy = ((point1[<span class="string">'y'</span>]+point2[<span class="string">'y'</span>])/<span class="number">2</span>+point3[<span class="string">'y'</span>])/<span class="number">2</span>*scale; <span class="comment">// 大概算出人中Y坐标</span></span></span><br><span class="line"><span class="actionscript">        <span class="keyword">var</span> beardw = edata[<span class="string">'span'</span>]*scale; <span class="comment">// 胡子长度取眼心距值</span></span></span><br><span class="line">        beard.width = beardw;</span><br><span class="line"><span class="actionscript">        beard.src = <span class="string">'./beard.png'</span>;</span></span><br><span class="line"><span class="actionscript">        beard.style.transform = <span class="string">'rotate('</span>+edata[<span class="string">'angle'</span>]+<span class="string">'deg)'</span>;</span></span><br><span class="line"><span class="actionscript">        beard.style.position = <span class="string">'absolute'</span>;</span></span><br><span class="line"><span class="actionscript">        beard.style.top = (beardy-beardw/<span class="number">2</span>/(BEARD[<span class="string">'image_width'</span>]/BEARD[<span class="string">'image_height'</span>]))+<span class="string">'px'</span>;</span></span><br><span class="line"><span class="actionscript">        beard.style.left = (beardx-beardw/<span class="number">2</span>)+<span class="string">'px'</span>;</span></span><br><span class="line">        </span><br><span class="line">        div1.appendChild(beard);</span><br><span class="line">    <span class="tag">&lt;/<span class="name">script</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">body</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;/<span class="name">html</span>&gt;</span></span><br></pre></td></tr></table></figure><p>最终效果：<br><img src="/blog/assets/images/practice-js-face-makeup-and-html2canvas3.png" alt="最终效果"></p><p>这只是HTML，怎样保存成图片呢？</p><h2 id="导出图片"><a href="#导出图片" class="headerlink" title="导出图片"></a>导出图片</h2><p>把HTML保存成图片，在服务器上可以用<a href="http://phantomjs.org/" target="_blank" rel="noopener">PhantomJS</a>实现，在浏览器上呢？其实也是可以实现的，首先将HTML（DOM结构）写入Canvas，Canvas再导出DataURL，最后DataURL赋予img标签的src属性，便得到最终图片，详情请阅读<a href="https://developer.mozilla.org/en-US/docs/Web/API/Canvas_API/Drawing_DOM_objects_into_a_canvas" target="_blank" rel="noopener">Drawing DOM objects into a canvas</a>。</p><p>这里使用插件<a href="https://github.com/niklasvh/html2canvas" target="_blank" rel="noopener">html2canvas</a>来实现HTML写入Canvas功能，类似的插件还有<a href="https://github.com/cburgmer/rasterizeHTML.js" target="_blank" rel="noopener">rasterizeHTML.js</a>等等。</p><p>源码：</p><figure class="highlight html"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag">&lt;<span class="name">script</span> <span class="attr">type</span>=<span class="string">"text/javascript"</span> <span class="attr">src</span>=<span class="string">"./html2canvas.min.js"</span>&gt;</span><span class="tag">&lt;/<span class="name">script</span>&gt;</span></span><br><span class="line"><span class="tag">&lt;<span class="name">script</span> <span class="attr">type</span>=<span class="string">"text/javascript"</span>&gt;</span></span><br><span class="line">    ...</span><br><span class="line"></span><br><span class="line"><span class="javascript">    <span class="keyword">var</span> can1 = <span class="built_in">document</span>.createElement(<span class="string">'canvas'</span>);</span></span><br><span class="line"><span class="javascript">    <span class="keyword">var</span> img1 = <span class="built_in">document</span>.createElement(<span class="string">'img'</span>);</span></span><br><span class="line"><span class="actionscript">    <span class="keyword">var</span> ctxt = can1.getContext(<span class="string">'2d'</span>);</span></span><br><span class="line"><span class="actionscript">    <span class="keyword">var</span> level = <span class="number">2</span>;</span></span><br><span class="line"><span class="javascript">    can1.width = <span class="built_in">document</span>.body.offsetWidth*level;</span></span><br><span class="line">    can1.height = div1.offsetHeight*level;</span><br><span class="line"><span class="actionscript">    can1.style.width = can1.width + <span class="string">'px'</span>;</span></span><br><span class="line"><span class="actionscript">    can1.style.height = can1.height + <span class="string">'px'</span>;</span></span><br><span class="line"><span class="actionscript">    ctxt.scale(level, level); <span class="comment">// 使用两倍图，保持清晰度</span></span></span><br><span class="line"></span><br><span class="line"><span class="actionscript">    setTimeout(<span class="function"><span class="keyword">function</span> <span class="params">()</span> </span>&#123;</span></span><br><span class="line"><span class="actionscript">        html2canvas(div1, &#123;canvas:can1&#125;).then(<span class="function"><span class="keyword">function</span><span class="params">(can1)</span> </span>&#123;</span></span><br><span class="line"><span class="javascript">            <span class="keyword">var</span> can2 = <span class="built_in">document</span>.createElement(<span class="string">'canvas'</span>);</span></span><br><span class="line"><span class="javascript">            <span class="keyword">var</span> face = <span class="built_in">document</span>.getElementById(<span class="string">'face'</span>);</span></span><br><span class="line"><span class="actionscript">            <span class="keyword">var</span> ctxt = can1.getContext(<span class="string">'2d'</span>);</span></span><br><span class="line"><span class="actionscript">            <span class="keyword">var</span> level = <span class="number">2</span>;</span></span><br><span class="line">            can2.width = face.offsetWidth*level;</span><br><span class="line">            can2.height = face.offsetHeight*level;</span><br><span class="line"><span class="actionscript">            can2.style.width = face.offsetWidth + <span class="string">'px'</span>;</span></span><br><span class="line"><span class="actionscript">            can2.style.height = face.offsetWidth + <span class="string">'px'</span>;</span></span><br><span class="line">            ctxt.scale(level, level);</span><br><span class="line"><span class="actionscript">            can2.getContext(<span class="string">'2d'</span>).drawImage(can1,</span></span><br><span class="line">            div1.offsetLeft*level, 0, face.offsetWidth*level, face.offsetHeight*level,</span><br><span class="line">                0, 0, face.offsetWidth*level, face.offsetHeight*level</span><br><span class="line"><span class="actionscript">            ); <span class="comment">// 使用can2是为了裁掉与人脸图片无关的边缘</span></span></span><br><span class="line"><span class="actionscript">            <span class="keyword">var</span> data = can2.toDataURL();</span></span><br><span class="line">            img1.src = data;</span><br><span class="line"><span class="javascript">            <span class="built_in">document</span>.body.appendChild(img1); <span class="comment">// 把图片放入当前页面中</span></span></span><br><span class="line"><span class="actionscript">            can2 = <span class="literal">null</span>;</span></span><br><span class="line"><span class="actionscript">            can1 = <span class="literal">null</span>;</span></span><br><span class="line">        &#125;);</span><br><span class="line"><span class="actionscript">    &#125;, <span class="number">2000</span>); <span class="comment">// 需要等待人脸图片加载完成再执行</span></span></span><br><span class="line"><span class="tag">&lt;/<span class="name">script</span>&gt;</span></span><br></pre></td></tr></table></figure><p>在PC端可以用a标签下载图片，href属性值应该是canvas导出的DataURL：</p><figure class="highlight html"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line"><span class="tag">&lt;<span class="name">a</span> <span class="attr">href</span>=<span class="string">"DataURL"</span> <span class="attr">download</span>=<span class="string">"最终效果"</span> <span class="attr">id</span>=<span class="string">"a1"</span>&gt;</span>下载<span class="tag">&lt;/<span class="name">a</span>&gt;</span></span><br></pre></td></tr></table></figure><p>在移动端则需要显示图片，并引导用户长按保存图片。</p><h2 id="最后"><a href="#最后" class="headerlink" title="最后"></a>最后</h2><p>注意：如果最终图片未能写入canvas，可能是图片跨域问题，或者图片未加载完就开始写了。<br>另外：可以分析图片人脸的表情、姿态、年纪、心情等属性，搭配更合适的妆饰，效果更佳；原始图片与妆饰图片可能会有色调、分辨率等不相配的问题，可以使用CSS3滤镜调整，使得最终画面更融洽。</p>]]></content>
    
    <summary type="html">
    
      &lt;blockquote&gt;
&lt;p&gt;最近要做一个H5活动页面，大概功能是给图片人脸换妆，自然也少不了上传原始图片和导出最终图片等功能。如果图片加工（即人脸换妆）放在服务器处理，那么并发量高的时候，肯定产生大量阻塞。于是想把图片加工放在客户端处理，用HTML堆砌出图片的最终效果，但是怎样把HTML导出成图片呢？&lt;/p&gt;
&lt;/blockquote&gt;
&lt;h2 id=&quot;图片上传&quot;&gt;&lt;a href=&quot;#图片上传&quot; class=&quot;headerlink&quot; title=&quot;图片上传&quot;&gt;&lt;/a&gt;图片上传&lt;/h2&gt;&lt;p&gt;图片上传可以参考&lt;a href=&quot;https://blog.breezelin.site/2017/08/12/scheme-nginx-php-js-upload-process/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;HTTP文件上传的一个后端完善方案（NginX）&lt;/a&gt;，或者使用现有的云存储服务，如&lt;a href=&quot;https://www.qiniu.com/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;七牛&lt;/a&gt;等等。&lt;/p&gt;
&lt;h2 id=&quot;五官定位&quot;&gt;&lt;a href=&quot;#五官定位&quot; class=&quot;headerlink&quot; title=&quot;五官定位&quot;&gt;&lt;/a&gt;五官定位&lt;/h2&gt;&lt;p&gt;人脸识别和五官定位可以利用&lt;a href=&quot;https://opencv.org/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;OpenVC&lt;/a&gt;自行实现，或者使用成熟的云识别服务，如&lt;a href=&quot;https://www.faceplusplus.com.cn/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;Face++&lt;/a&gt;等等。这里使用腾讯云的人脸识别服务。&lt;/p&gt;
    
    </summary>
    
      <category term="实践" scheme="https://breeze2.github.io/blog/categories/%E5%AE%9E%E8%B7%B5/"/>
    
    
      <category term="js" scheme="https://breeze2.github.io/blog/tags/js/"/>
    
      <category term="practice" scheme="https://breeze2.github.io/blog/tags/practice/"/>
    
  </entry>
  
  <entry>
    <title>用Docker搭建MySQL MHA高可用集群</title>
    <link href="https://breeze2.github.io/blog/practice-mysql-mha-docker-compose.html"/>
    <id>https://breeze2.github.io/blog/practice-mysql-mha-docker-compose.html</id>
    <published>2017-10-13T15:39:06.000Z</published>
    <updated>2019-09-30T02:22:37.086Z</updated>
    
    <content type="html"><![CDATA[<p><a href="http://yoshinorimatsunobu.blogspot.ca/2011/07/announcing-mysql-mha-mysql-master-high.html" target="_blank" rel="noopener">MySQL MHA（Master High Availability）</a>是目前相对成熟的一个MySQL高可用解决方案。<br>这篇文章主要介绍用Docker Compose编排MySQL MHA高可用集群的实践过程。<br>Docker Compose编排中使用了两个现有镜像，分别是<a href="https://hub.docker.com/r/breeze2/mha4mysql-manager/" target="_blank" rel="noopener">breeze2/mha4mysql-manager</a>和<a href="https://hub.docker.com/r/breeze2/mha4mysql-node/" target="_blank" rel="noopener">breeze2/mha4mysql-node</a>，这两个镜像的相关信息可以查看<a href="https://blog.breezelin.site/2017/10/13/practice-make-mha4mysql-docker-image/" target="_blank" rel="noopener">《制作mha4mysql的Docker镜像》</a>。<br>本次实践源码上传在GitHub的<a href="https://github.com/breeze2/mysql-mha-docker" target="_blank" rel="noopener">breeze2/mysql-mha-docker</a>。</p><a id="more"></a><h2 id="Docker-Compose编排"><a href="#Docker-Compose编排" class="headerlink" title="Docker Compose编排"></a>Docker Compose编排</h2><p>这个编排主要实现一主一备一从的MySQL MHA高可用集群。</p><h3 id="目录结构"><a href="#目录结构" class="headerlink" title="目录结构"></a>目录结构</h3><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line">L--mysql-mha-docker                //主目录</span><br><span class="line">    L--scripts                     //本地（Docker宿主）使用的一些脚本</span><br><span class="line">        L--mha_check_repl.sh</span><br><span class="line">        L--...</span><br><span class="line">    L--services                    //需要build的服务（目前是空）</span><br><span class="line">    L--volumes                     //各个容器的挂载数据卷</span><br><span class="line">        L--mha_manager</span><br><span class="line">        L--mha_node0</span><br><span class="line">        L--mha_node1</span><br><span class="line">        L--mha_node2</span><br><span class="line">        L--mha_share               //各个容器共享的目录</span><br><span class="line">          L--scripts               //各个容器共用的一些脚本</span><br><span class="line">          L--sshkeys               //各个容器的ssh public key</span><br><span class="line">    L--parameters.env              //账号密码等环境参数</span><br><span class="line">    L--docker-compose.yml          //编排配置</span><br></pre></td></tr></table></figure><h3 id="docker-compose-yml"><a href="#docker-compose-yml" class="headerlink" title="docker-compose.yml"></a>docker-compose.yml</h3><figure class="highlight yml"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br><span class="line">22</span><br><span class="line">23</span><br><span class="line">24</span><br><span class="line">25</span><br><span class="line">26</span><br><span class="line">27</span><br><span class="line">28</span><br><span class="line">29</span><br><span class="line">30</span><br><span class="line">31</span><br><span class="line">32</span><br><span class="line">33</span><br><span class="line">34</span><br><span class="line">35</span><br><span class="line">36</span><br><span class="line">37</span><br><span class="line">38</span><br><span class="line">39</span><br><span class="line">40</span><br><span class="line">41</span><br><span class="line">42</span><br><span class="line">43</span><br><span class="line">44</span><br><span class="line">45</span><br><span class="line">46</span><br><span class="line">47</span><br><span class="line">48</span><br><span class="line">49</span><br><span class="line">50</span><br><span class="line">51</span><br><span class="line">52</span><br><span class="line">53</span><br><span class="line">54</span><br><span class="line">55</span><br><span class="line">56</span><br><span class="line">57</span><br><span class="line">58</span><br><span class="line">59</span><br><span class="line">60</span><br><span class="line">61</span><br><span class="line">62</span><br><span class="line">63</span><br><span class="line">64</span><br><span class="line">65</span><br><span class="line">66</span><br><span class="line">67</span><br><span class="line">68</span><br><span class="line">69</span><br><span class="line">70</span><br><span class="line">71</span><br><span class="line">72</span><br><span class="line">73</span><br><span class="line">74</span><br><span class="line">75</span><br><span class="line">76</span><br><span class="line">77</span><br><span class="line">78</span><br><span class="line">79</span><br><span class="line">80</span><br><span class="line">81</span><br><span class="line">82</span><br><span class="line">83</span><br><span class="line">84</span><br><span class="line">85</span><br><span class="line">86</span><br><span class="line">87</span><br><span class="line">88</span><br><span class="line">89</span><br><span class="line">90</span><br><span class="line">91</span><br></pre></td><td class="code"><pre><span class="line"><span class="attr">version:</span> <span class="string">"2"</span></span><br><span class="line"><span class="attr">services:</span></span><br><span class="line"></span><br><span class="line"><span class="attr">  master:</span></span><br><span class="line"><span class="attr">    image:</span> <span class="string">breeze2/mha4mysql-node:0.57</span></span><br><span class="line"><span class="attr">    container_name:</span> <span class="string">mha_node0</span></span><br><span class="line"><span class="attr">    restart:</span> <span class="string">always</span></span><br><span class="line"><span class="attr">    mem_limit:</span> <span class="number">256</span><span class="string">m</span></span><br><span class="line"><span class="attr">    networks:</span></span><br><span class="line"><span class="attr">      net1:</span></span><br><span class="line"><span class="attr">        ipv4_address:</span> <span class="number">10.5</span><span class="number">.0</span><span class="number">.10</span></span><br><span class="line"><span class="attr">    ports:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"33060:3306"</span></span><br><span class="line"><span class="attr">    volumes:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"./volumes/mha_share/:/root/mha_share/"</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"./volumes/mha_node0/lib/:/var/lib/mysql/"</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"./volumes/mha_node0/conf/:/etc/mysql/conf.d/"</span></span><br><span class="line"><span class="attr">    env_file:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">./parameters.env</span></span><br><span class="line"><span class="attr">    environment:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">CONTAINER_NAME=mha_node0</span></span><br><span class="line"></span><br><span class="line"><span class="attr">  slave1:</span></span><br><span class="line"><span class="attr">    image:</span> <span class="string">breeze2/mha4mysql-node:0.57</span></span><br><span class="line"><span class="attr">    container_name:</span> <span class="string">mha_node1</span></span><br><span class="line"><span class="attr">    restart:</span> <span class="string">always</span></span><br><span class="line"><span class="attr">    depends_on:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">master</span></span><br><span class="line"><span class="attr">    mem_limit:</span> <span class="number">256</span><span class="string">m</span></span><br><span class="line"><span class="attr">    networks:</span></span><br><span class="line"><span class="attr">      net1:</span></span><br><span class="line"><span class="attr">        ipv4_address:</span> <span class="number">10.5</span><span class="number">.0</span><span class="number">.11</span></span><br><span class="line"><span class="attr">    ports:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"33061:3306"</span></span><br><span class="line"><span class="attr">    volumes:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"./volumes/mha_share/:/root/mha_share/"</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"./volumes/mha_node1/lib/:/var/lib/mysql/"</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"./volumes/mha_node1/conf/:/etc/mysql/conf.d/"</span></span><br><span class="line"><span class="attr">    env_file:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">./parameters.env</span></span><br><span class="line"><span class="attr">    environment:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">CONTAINER_NAME=mha_node1</span></span><br><span class="line"><span class="attr">  slave2:</span></span><br><span class="line"><span class="attr">    image:</span> <span class="string">breeze2/mha4mysql-node:0.57</span></span><br><span class="line"><span class="attr">    container_name:</span> <span class="string">mha_node2</span></span><br><span class="line"><span class="attr">    depends_on:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">master</span></span><br><span class="line"><span class="attr">    restart:</span> <span class="string">always</span></span><br><span class="line"><span class="attr">    mem_limit:</span> <span class="number">256</span><span class="string">m</span></span><br><span class="line"><span class="attr">    networks:</span></span><br><span class="line"><span class="attr">      net1:</span></span><br><span class="line"><span class="attr">        ipv4_address:</span> <span class="number">10.5</span><span class="number">.0</span><span class="number">.12</span></span><br><span class="line"><span class="attr">    ports:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"33062:3306"</span></span><br><span class="line"><span class="attr">    volumes:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"./volumes/mha_share/:/root/mha_share/"</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"./volumes/mha_node2/lib/:/var/lib/mysql/"</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"./volumes/mha_node2/conf/:/etc/mysql/conf.d/"</span></span><br><span class="line"><span class="attr">    env_file:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">./parameters.env</span></span><br><span class="line"><span class="attr">    environment:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">CONTAINER_NAME=mha_node2</span></span><br><span class="line"></span><br><span class="line"><span class="attr">  manager:</span></span><br><span class="line"><span class="attr">    image:</span> <span class="string">breeze2/mha4mysql-manager:0.57</span></span><br><span class="line"><span class="attr">    container_name:</span> <span class="string">mha_manager</span></span><br><span class="line"><span class="attr">    depends_on:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">master</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">slave1</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">slave2</span></span><br><span class="line"><span class="attr">    restart:</span> <span class="string">always</span></span><br><span class="line"><span class="attr">    mem_limit:</span> <span class="number">256</span><span class="string">m</span></span><br><span class="line"><span class="attr">    networks:</span></span><br><span class="line"><span class="attr">      net1:</span></span><br><span class="line"><span class="attr">        ipv4_address:</span> <span class="number">10.5</span><span class="number">.0</span><span class="number">.9</span></span><br><span class="line"><span class="attr">    volumes:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"./volumes/mha_share/:/root/mha_share/"</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"./volumes/mha_manager/conf:/etc/mha"</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">"./volumes/mha_manager/work:/usr/local/mha"</span></span><br><span class="line"><span class="attr">    entrypoint:</span> <span class="string">"tailf /dev/null"</span></span><br><span class="line"><span class="attr">    env_file:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">./parameters.env</span></span><br><span class="line"><span class="attr">    environment:</span></span><br><span class="line"><span class="bullet">      -</span> <span class="string">CONTAINER_NAME=mha_manager</span></span><br><span class="line"><span class="attr">networks:</span></span><br><span class="line"><span class="attr">  net1:</span></span><br><span class="line"><span class="attr">    driver:</span> <span class="string">bridge</span></span><br><span class="line"><span class="attr">    ipam:</span></span><br><span class="line"><span class="attr">      config:</span></span><br><span class="line"><span class="attr">        - subnet:</span> <span class="number">10.5</span><span class="number">.0</span><span class="number">.0</span><span class="string">/16</span></span><br><span class="line"><span class="attr">          gateway:</span> <span class="number">10.5</span><span class="number">.0</span><span class="number">.1</span></span><br></pre></td></tr></table></figure><p>这里配置了四个容器服务，一个<code>breeze2/mha4mysql-manager</code>，负责管理各个数据库结点；三个<code>breeze2/mha4mysql-node</code>，其中一个是主库，一个是备用主库（兼作从库），还有一个是（纯）从库。每个容器服务都指定了静态IP，即使服务重启也不会出现IP错乱问题。</p><h3 id="环境参数"><a href="#环境参数" class="headerlink" title="环境参数"></a>环境参数</h3><p>parameters.env</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">ROOT_PASSWORD=123456</span><br><span class="line">MYSQL_ROOT_PASSWORD=123456</span><br><span class="line">MYSQL_DATABASE=testing</span><br><span class="line">MYSQL_User=testing</span><br><span class="line">MYSQL_PASSWORD=testing</span><br><span class="line">MHA_SHARE_SCRIPTS_PATH=/root/mha_share/scripts</span><br><span class="line">MHA_SHARE_SSHKEYS_PATH=/root/mha_share/sshkeys</span><br></pre></td></tr></table></figure><h3 id="数据库配置"><a href="#数据库配置" class="headerlink" title="数据库配置"></a>数据库配置</h3><p>这里简单的配置一下各个数据库数据复制备份相关的参数<br>主库，mha_node0/conf/my.cnf</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">[mysqld]</span><br><span class="line">server-id=1</span><br><span class="line">log-bin=mysql-bin</span><br><span class="line">binlog-do-db=testing</span><br><span class="line">binlog-ignore-db=mysql</span><br><span class="line">replicate-do-db=testing</span><br><span class="line">replicate-ignore-db=mysql</span><br><span class="line">auto_increment_increment=2</span><br><span class="line">auto_increment_offset=1</span><br><span class="line">expire_logs_days=7</span><br></pre></td></tr></table></figure><p>备库，mha_node1/conf/my.cnf</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br></pre></td><td class="code"><pre><span class="line">[mysqld]</span><br><span class="line">server-id=2</span><br><span class="line">log-bin=mysql-bin</span><br><span class="line">binlog-do-db=testing</span><br><span class="line">binlog-ignore-db=mysql</span><br><span class="line">replicate-do-db=testing</span><br><span class="line">replicate-ignore-db=mysql</span><br><span class="line">auto_increment_increment=2</span><br><span class="line">auto_increment_offset=2</span><br><span class="line">expire_logs_days=7</span><br></pre></td></tr></table></figure><p>从库，mha_node2/conf/my.cnf</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br></pre></td><td class="code"><pre><span class="line">[mysqld]</span><br><span class="line">server-id=3</span><br><span class="line">replicate-do-db=testing</span><br><span class="line">replicate-ignore-db=mysql</span><br><span class="line">expire_logs_days=7</span><br></pre></td></tr></table></figure><h3 id="MHA配置"><a href="#MHA配置" class="headerlink" title="MHA配置"></a>MHA配置</h3><p>manager/conf/app1.conf</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br></pre></td><td class="code"><pre><span class="line">[server default]</span><br><span class="line">user=root</span><br><span class="line">password=123456</span><br><span class="line">ssh_user=root</span><br><span class="line"></span><br><span class="line">manager_workdir=/usr/local/mha</span><br><span class="line">remote_workdir=/usr/local/mha</span><br><span class="line"></span><br><span class="line">repl_user=myslave</span><br><span class="line">repl_password=myslave</span><br><span class="line"></span><br><span class="line">[server0]</span><br><span class="line">hostname=10.5.0.10</span><br><span class="line"></span><br><span class="line">[server1]</span><br><span class="line">hostname=10.5.0.11</span><br><span class="line"></span><br><span class="line">[server2]</span><br><span class="line">hostname=10.5.0.12</span><br></pre></td></tr></table></figure><h2 id="实际运行"><a href="#实际运行" class="headerlink" title="实际运行"></a>实际运行</h2><p>在主目录下执行<code>docker-compose up -d</code>构建并运行整个Docker服务。</p><h3 id="SSH设置"><a href="#SSH设置" class="headerlink" title="SSH设置"></a>SSH设置</h3><p>MHA要求各个主机能够相互SSH登录，所以整体服务首次启动成功后，在主目录下先执行一些命令：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ sh ./scripts/ssh_start.sh</span><br><span class="line">$ sh ./scripts/ssh_share.sh</span><br></pre></td></tr></table></figure><p><code>ssh_start.sh</code>作用是在各个容器上开启SSH服务，<code>ssh_share.sh</code>作用是在容器内生成SSH公密钥，再把公钥共享到其他容器。常用命令调用的脚本在<code>scripts</code>和<code>volumes/mha_share/scripts</code>下，这些脚本都很简单，一看就明。<br>若是整体服务重新启动，只需重新开启SSH服务即可：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sh ./scripts/ssh_start.sh</span><br></pre></td></tr></table></figure><p>在manager容器上检测SSH是否配置成功：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ docker exec -it mha_manager /bin/bash</span><br><span class="line">root@mha_manager# masterha_check_ssh --conf=/etc/mha/app1.conf</span><br></pre></td></tr></table></figure><p>若是成功，会显示</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">Mon Oct <span class="number">16</span> <span class="number">14</span>:<span class="number">53</span>:<span class="number">59</span> <span class="number">2017</span> - [debug]   ok.</span><br><span class="line">Mon Oct <span class="number">16</span> <span class="number">14</span>:<span class="number">53</span>:<span class="number">59</span> <span class="number">2017</span> - [info] All SSH connection tests passed successfully.</span><br></pre></td></tr></table></figure><h3 id="开启数据主从复制"><a href="#开启数据主从复制" class="headerlink" title="开启数据主从复制"></a>开启数据主从复制</h3><p>首先对主库、备考创建和授权复制账号，对备库、从库设置主库信息和开始复制，以下命令会完成这些操作：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ sh ./scripts/mysql_set_mbs.sh</span><br></pre></td></tr></table></figure><p>可以在主库上的testing数据库里创建一张表，写入一些数据，看看备库、从库会不会同步。</p><p>在manager容器上检测REPL是否配置成功：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ docker exec -it mha_manager /bin/bash</span><br><span class="line">root@mha_manager# masterha_check_repl --conf=/etc/mha/app1.conf</span><br></pre></td></tr></table></figure><p>若是成功，会显示</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br></pre></td><td class="code"><pre><span class="line">Mon Oct <span class="number">16</span> <span class="number">15</span>:<span class="number">01</span>:<span class="number">35</span> <span class="number">2017</span> - [info] Got <span class="keyword">exit</span> code <span class="number">0</span> (<span class="keyword">Not</span> master dead).</span><br><span class="line"></span><br><span class="line">MySQL Replication Health is OK.</span><br></pre></td></tr></table></figure><h3 id="开启MHA监控"><a href="#开启MHA监控" class="headerlink" title="开启MHA监控"></a>开启MHA监控</h3><p>SSH和REPL检测没问题后，可以在manager容器上开启MHA监控：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ docker exec -it mha_manager /bin/bash</span><br><span class="line">root@mha_manager# masterha_manager --conf=/etc/mha/app1.conf</span><br></pre></td></tr></table></figure><p><code>masterha_manager</code>进程会一直监视主库状态是否可用，若是主库宕机，<code>masterha_manager</code>会将备库与从库的Relay Log进行比较，把最新的数据整合到备库，然后把备库提升为新主库，从库跟随复制新主库，最后<code>masterha_manager</code>进程会退出，不再监控。</p><p>我们可以在本地（Docker宿主）暂停主库（mha_node0容器）：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br></pre></td><td class="code"><pre><span class="line">$ docker <span class="built_in">pause</span> mha_node0</span><br></pre></td></tr></table></figure><p>然后，manager容器上<code>masterha_manager</code>确认主库失联后，开始切换主库，成功后会显示：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br></pre></td><td class="code"><pre><span class="line">----- Failover Report -----</span><br><span class="line"></span><br><span class="line"><span class="function">app1: <span class="title">MySQL</span> <span class="title">Master</span> <span class="title">failover</span> 10.5.0.10(10.5.0.10:3306) <span class="title">to</span> 10.5.0.11(10.5.0.11:3306) <span class="title">succeeded</span></span></span><br><span class="line"><span class="function"></span></span><br><span class="line"><span class="function"><span class="title">Master</span> 10.5.0.10(10.5.0.10:3306) <span class="title">is</span> <span class="title">down</span>!</span></span><br><span class="line"><span class="function"></span></span><br><span class="line"><span class="function"><span class="title">Check</span> <span class="title">MHA</span> <span class="title">Manager</span> <span class="title">logs</span> <span class="title">at</span> 3<span class="title">d2d8185510b</span> <span class="title">for</span> <span class="title">details</span>.</span></span><br><span class="line"><span class="function"></span></span><br><span class="line"><span class="function"><span class="title">Started</span> <span class="title">automated</span>(<span class="title">non</span>-<span class="title">interactive</span>) <span class="title">failover</span>.</span></span><br><span class="line"><span class="function"><span class="title">The</span> <span class="title">latest</span> <span class="title">slave</span> 10.5.0.11(10.5.0.11:3306) <span class="title">has</span> <span class="title">all</span> <span class="title">relay</span> <span class="title">logs</span> <span class="title">for</span> <span class="title">recovery</span>.</span></span><br><span class="line"><span class="function"><span class="title">Selected</span> 10.5.0.11(10.5.0.11:3306) <span class="title">as</span> <span class="title">a</span> <span class="title">new</span> <span class="title">master</span>.</span></span><br><span class="line"><span class="function">10.5.0.11(10.5.0.11:3306): <span class="title">OK</span>: <span class="title">Applying</span> <span class="title">all</span> <span class="title">logs</span> <span class="title">succeeded</span>.</span></span><br><span class="line"><span class="function">10.5.0.12(10.5.0.12:3306): <span class="title">This</span> <span class="title">host</span> <span class="title">has</span> <span class="title">the</span> <span class="title">latest</span> <span class="title">relay</span> <span class="title">log</span> <span class="title">events</span>.</span></span><br><span class="line"><span class="function"><span class="title">Generating</span> <span class="title">relay</span> <span class="title">diff</span> <span class="title">files</span> <span class="title">from</span> <span class="title">the</span> <span class="title">latest</span> <span class="title">slave</span> <span class="title">succeeded</span>.</span></span><br><span class="line"><span class="function">10.5.0.12(10.5.0.12:3306): <span class="title">OK</span>: <span class="title">Applying</span> <span class="title">all</span> <span class="title">logs</span> <span class="title">succeeded</span>. <span class="title">Slave</span> <span class="title">started</span>, <span class="title">replicating</span> <span class="title">from</span> 10.5.0.11(10.5.0.11:3306)</span></span><br><span class="line"><span class="function">10.5.0.11(10.5.0.11:3306): <span class="title">Resetting</span> <span class="title">slave</span> <span class="title">info</span> <span class="title">succeeded</span>.</span></span><br><span class="line"><span class="function"><span class="title">Master</span> <span class="title">failover</span> <span class="title">to</span> 10.5.0.11(10.5.0.11:3306) <span class="title">completed</span> <span class="title">successfully</span>.</span></span><br></pre></td></tr></table></figure><p>可以到从库（mha_node2容器），查看复制状态，可以看到跟随主库是10.5.0.11，即新主库（原备库）：</p><figure class="highlight plain"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br></pre></td><td class="code"><pre><span class="line">$ docker exec -it mha_manager /bin/bash</span><br><span class="line">root@mha_manager# mysql -u root -p$MYSQL_ROOT_PASSWORD</span><br><span class="line">mysql&gt; show slave status;</span><br><span class="line">*************************** 1. row ***************************</span><br><span class="line">               Slave_IO_State: Waiting for master to send event</span><br><span class="line">                  Master_Host: 10.5.0.11</span><br><span class="line">                  Master_User: myslave</span><br></pre></td></tr></table></figure><h2 id="最后"><a href="#最后" class="headerlink" title="最后"></a>最后</h2><p>在数据库服务器集群中，使用MHA，可以在主库宕机后快速切换新主库，可是应用服务器并不知道主库已经被切换。所以，<code>masterha_manager</code>进程切换主库成功后应该通知应用服务器新的主库IP，或者使用虚拟IP静默过渡。若是应用与数据库通过中间件（比如ProxySQL）来连接的，那么只需在中间件上修改主库IP，对应用影响不大。</p><blockquote><p>注意：因为本compose中MySQL服务与SSH服务没有同一启动（SSH服务是容器启动后才开启的），所以本compose只能当作线下模拟练习，未能在正式线上应用。比如其中某个结点容器重启，SSH服务并没有自动重启，进而整个MHA集群不可用。</p></blockquote>]]></content>
    
    <summary type="html">
    
      &lt;p&gt;&lt;a href=&quot;http://yoshinorimatsunobu.blogspot.ca/2011/07/announcing-mysql-mha-mysql-master-high.html&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;MySQL MHA（Master High Availability）&lt;/a&gt;是目前相对成熟的一个MySQL高可用解决方案。&lt;br&gt;这篇文章主要介绍用Docker Compose编排MySQL MHA高可用集群的实践过程。&lt;br&gt;Docker Compose编排中使用了两个现有镜像，分别是&lt;a href=&quot;https://hub.docker.com/r/breeze2/mha4mysql-manager/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;breeze2/mha4mysql-manager&lt;/a&gt;和&lt;a href=&quot;https://hub.docker.com/r/breeze2/mha4mysql-node/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;breeze2/mha4mysql-node&lt;/a&gt;，这两个镜像的相关信息可以查看&lt;a href=&quot;https://blog.breezelin.site/2017/10/13/practice-make-mha4mysql-docker-image/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;《制作mha4mysql的Docker镜像》&lt;/a&gt;。&lt;br&gt;本次实践源码上传在GitHub的&lt;a href=&quot;https://github.com/breeze2/mysql-mha-docker&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;breeze2/mysql-mha-docker&lt;/a&gt;。&lt;/p&gt;
    
    </summary>
    
      <category term="实践" scheme="https://breeze2.github.io/blog/categories/%E5%AE%9E%E8%B7%B5/"/>
    
    
      <category term="mysql" scheme="https://breeze2.github.io/blog/tags/mysql/"/>
    
      <category term="practice" scheme="https://breeze2.github.io/blog/tags/practice/"/>
    
      <category term="docker" scheme="https://breeze2.github.io/blog/tags/docker/"/>
    
      <category term="mha4mysql" scheme="https://breeze2.github.io/blog/tags/mha4mysql/"/>
    
  </entry>
  
  <entry>
    <title>制作mha4mysql的Docker镜像</title>
    <link href="https://breeze2.github.io/blog/practice-make-mha4mysql-docker-image.html"/>
    <id>https://breeze2.github.io/blog/practice-make-mha4mysql-docker-image.html</id>
    <published>2017-10-13T10:16:08.000Z</published>
    <updated>2019-09-30T02:22:37.086Z</updated>
    
    <content type="html"><![CDATA[<blockquote><p>本来想记录一下用Docker构建MySQL MHA集群的实践过程的，但是整个篇幅有点长，所以这里先来介绍一下怎样制作mha4mysql的Docker镜像。</p></blockquote><p>mha4mysql是日本工程师<a href="https://www.percona.com/live/mysql-conference-2014/users/yoshinori-matsunobu" target="_blank" rel="noopener">Yoshinori Matsunobu</a>开发的一款MySQL高可用软件。mha4mysql分为两部分，一是管理器部分<a href="https://github.com/yoshinorim/mha4mysql-manager" target="_blank" rel="noopener">mha4mysql-manager</a>，二是结点部分<a href="https://github.com/yoshinorim/mha4mysql-node" target="_blank" rel="noopener">mha4mysql-node</a>。mha4mysql-node要运行在每台受管理的MySQL服务器上；而mha4mysql-manager所在服务器则不需要MySQL，但需要mha4mysql-node。因为mha4mysql-manager依赖mha4mysql-node，即安装mha4mysql-manager前必须先安装mha4mysql-node。</p><p>下面讲解一下，基于<a href="https://hub.docker.com/_/debian/" target="_blank" rel="noopener">debian:jessie</a>制作mha4mysql-manager的Docker镜像和<br>基于<a href="https://hub.docker.com/_/mysql/" target="_blank" rel="noopener">mysql:5.7</a>制作mha4mysql-node的Docker镜像。</p><a id="more"></a><h2 id="mha4mysql-manager"><a href="#mha4mysql-manager" class="headerlink" title="mha4mysql-manager"></a>mha4mysql-manager</h2><p>mha4mysql-manager的Dockerfile:</p><figure class="highlight dockerfile"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br><span class="line">16</span><br><span class="line">17</span><br><span class="line">18</span><br><span class="line">19</span><br><span class="line">20</span><br><span class="line">21</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">FROM</span> debian:jessie</span><br><span class="line"></span><br><span class="line"><span class="keyword">COPY</span><span class="bash"> ./mha4mysql-manager.tar.gz /tmp/</span></span><br><span class="line"><span class="keyword">COPY</span><span class="bash"> ./mha4mysql-node.tar.gz /tmp/</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">RUN</span><span class="bash"> build_deps=<span class="string">'ssh sshpass perl libdbi-perl libmodule-install-perl libdbd-mysql-perl libconfig-tiny-perl liblog-dispatch-perl libparallel-forkmanager-perl make'</span> \</span></span><br><span class="line"><span class="bash">    &amp;&amp; apt-get update \</span></span><br><span class="line"><span class="bash">    &amp;&amp; apt-get -y --force-yes install <span class="variable">$build_deps</span> \</span></span><br><span class="line"><span class="bash">    &amp;&amp; tar -zxf /tmp/mha4mysql-node.tar.gz -C /opt \</span></span><br><span class="line"><span class="bash">    &amp;&amp; <span class="built_in">cd</span> /opt/mha4mysql-node \</span></span><br><span class="line"><span class="bash">    &amp;&amp; perl Makefile.PL \</span></span><br><span class="line"><span class="bash">    &amp;&amp; make \</span></span><br><span class="line"><span class="bash">    &amp;&amp; make install \</span></span><br><span class="line"><span class="bash">    &amp;&amp; tar -zxf /tmp/mha4mysql-manager.tar.gz -C /opt \</span></span><br><span class="line"><span class="bash">    &amp;&amp; <span class="built_in">cd</span> /opt/mha4mysql-manager \</span></span><br><span class="line"><span class="bash">    &amp;&amp; perl Makefile.PL \</span></span><br><span class="line"><span class="bash">    &amp;&amp; make \</span></span><br><span class="line"><span class="bash">    &amp;&amp; make install \</span></span><br><span class="line"><span class="bash">    &amp;&amp; <span class="built_in">cd</span> /opt \</span></span><br><span class="line"><span class="bash">    &amp;&amp; rm -rf /opt/mha4mysql-* \</span></span><br><span class="line"><span class="bash">    &amp;&amp; apt-get clean</span></span><br></pre></td></tr></table></figure><p>注释：</p><ol><li>基于debian:jessie镜像二次制作；</li><li>将mha4mysql-manager和mha4mysql-node的当前最新版本（v0.57）打包，复制到镜像内；</li><li><code>build_deps</code>是mha4mysql-manager和mha4mysql-node的安装依赖、运行依赖；</li><li>先拆包安装mha4mysql-node，安装命令：<code>perl Makefile.PL &amp;&amp; make &amp;&amp; make install</code>；</li><li>才能拆包安装mha4mysql-manager，安装命令：<code>perl Makefile.PL &amp;&amp; make &amp;&amp; make install</code>；</li><li>清理一些无用文件。</li></ol><h2 id="mha4mysql-node"><a href="#mha4mysql-node" class="headerlink" title="mha4mysql-node"></a>mha4mysql-node</h2><p>mha4mysql-node的Dockerfile:</p><figure class="highlight dockerfile"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br><span class="line">3</span><br><span class="line">4</span><br><span class="line">5</span><br><span class="line">6</span><br><span class="line">7</span><br><span class="line">8</span><br><span class="line">9</span><br><span class="line">10</span><br><span class="line">11</span><br><span class="line">12</span><br><span class="line">13</span><br><span class="line">14</span><br><span class="line">15</span><br></pre></td><td class="code"><pre><span class="line"><span class="keyword">FROM</span> mysql:<span class="number">5.7</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">COPY</span><span class="bash"> ./mha4mysql-node.tar.gz /tmp/</span></span><br><span class="line"></span><br><span class="line"><span class="keyword">RUN</span><span class="bash"> build_deps=<span class="string">'ssh sshpass perl libdbi-perl libmodule-install-perl libdbd-mysql-perl make'</span> \</span></span><br><span class="line"><span class="bash">    &amp;&amp; apt-get update \</span></span><br><span class="line"><span class="bash">    &amp;&amp; apt-get -y --force-yes install <span class="variable">$build_deps</span> \</span></span><br><span class="line"><span class="bash">    &amp;&amp; tar -zxf /tmp/mha4mysql-node.tar.gz -C /opt \</span></span><br><span class="line"><span class="bash">    &amp;&amp; <span class="built_in">cd</span> /opt/mha4mysql-node \</span></span><br><span class="line"><span class="bash">    &amp;&amp; perl Makefile.PL \</span></span><br><span class="line"><span class="bash">    &amp;&amp; make \</span></span><br><span class="line"><span class="bash">    &amp;&amp; make install \</span></span><br><span class="line"><span class="bash">    &amp;&amp; <span class="built_in">cd</span> /opt \</span></span><br><span class="line"><span class="bash">    &amp;&amp; rm -rf /opt/mha4mysql-* \</span></span><br><span class="line"><span class="bash">    &amp;&amp; apt-get clean</span></span><br></pre></td></tr></table></figure><p>注释：</p><ol><li>基于mysql:5.7镜像二次制作；</li><li>将mha4mysql-node的当前最新版本（v0.57）打包，复制到镜像内；</li><li><code>build_deps</code>是mha4mysql-node的安装依赖、运行依赖；</li><li>拆包安装mha4mysql-manager，安装命令：<code>perl Makefile.PL &amp;&amp; make &amp;&amp; make install</code>；</li><li>清理一些无用文件。</li></ol><h2 id="注意"><a href="#注意" class="headerlink" title="注意"></a>注意</h2><h3 id="v0-57-bug"><a href="#v0-57-bug" class="headerlink" title="v0.57 bug"></a>v0.57 bug</h3><p>目前mha4mysql-manager和mha4mysql-node的v0.57并不兼容，新版的libdbd-mysql-perl的调用语法，比如新版要求连接数据库格式是：<br><code>my $dsn = &quot;DBI:mysql:;host=$opt{host};port=$opt{port}&quot;;</code><br>而v0.57源码是：<br><code>my $dsn = &quot;DBI:mysql:;host=[$opt{host}];port=$opt{port}&quot;;</code><br>多了一对中括号会导致连接数据库失败。<br>这里镜像中的mha4mysql-manager和mha4mysql-node的源码包是已经对上面问题进行修正的。</p><h3 id="单容器多服务"><a href="#单容器多服务" class="headerlink" title="单容器多服务"></a>单容器多服务</h3><p>按照Docker的思想，是一个容器只做一件事，但是mha4mysql结点需要mysql服务和ssh服务，这里的临时解决办法是容器先运行mysql服务，之后执行<code>docker exec -it $CONTAINER_NAME /bin/bash service ssh start</code>。<br>目前实现Dokcer单容器多服务的hack方法是利用supervisor，只做supervisor这一件事，而supervisor做多件事。如果后期真的需要使用supervisor，也可以基于本镜像进行二次制作。</p><h2 id="最后"><a href="#最后" class="headerlink" title="最后"></a>最后</h2><p>mha4mysql-manager和mha4mysql-node的Dockerfile分别放在GitHub<a href="https://github.com/breeze2/mha4mysql-manager-docker" target="_blank" rel="noopener">breeze2/mha4mysql-manager-docker</a>和<a href="https://github.com/breeze2/mha4mysql-node-docker" target="_blank" rel="noopener">breeze2/mha4mysql-node-docker</a>上；镜像可以用docker命令直接拉取：</p><figure class="highlight cmd"><table><tr><td class="gutter"><pre><span class="line">1</span><br><span class="line">2</span><br></pre></td><td class="code"><pre><span class="line">$ docker pull breeze2/mha4mysql-manager:<span class="number">0</span>.<span class="number">57</span></span><br><span class="line">$ docker pull breeze2/mha4mysql-node:<span class="number">0</span>.<span class="number">57</span></span><br></pre></td></tr></table></figure>]]></content>
    
    <summary type="html">
    
      &lt;blockquote&gt;
&lt;p&gt;本来想记录一下用Docker构建MySQL MHA集群的实践过程的，但是整个篇幅有点长，所以这里先来介绍一下怎样制作mha4mysql的Docker镜像。&lt;/p&gt;
&lt;/blockquote&gt;
&lt;p&gt;mha4mysql是日本工程师&lt;a href=&quot;https://www.percona.com/live/mysql-conference-2014/users/yoshinori-matsunobu&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;Yoshinori Matsunobu&lt;/a&gt;开发的一款MySQL高可用软件。mha4mysql分为两部分，一是管理器部分&lt;a href=&quot;https://github.com/yoshinorim/mha4mysql-manager&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;mha4mysql-manager&lt;/a&gt;，二是结点部分&lt;a href=&quot;https://github.com/yoshinorim/mha4mysql-node&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;mha4mysql-node&lt;/a&gt;。mha4mysql-node要运行在每台受管理的MySQL服务器上；而mha4mysql-manager所在服务器则不需要MySQL，但需要mha4mysql-node。因为mha4mysql-manager依赖mha4mysql-node，即安装mha4mysql-manager前必须先安装mha4mysql-node。&lt;/p&gt;
&lt;p&gt;下面讲解一下，基于&lt;a href=&quot;https://hub.docker.com/_/debian/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;debian:jessie&lt;/a&gt;制作mha4mysql-manager的Docker镜像和&lt;br&gt;基于&lt;a href=&quot;https://hub.docker.com/_/mysql/&quot; target=&quot;_blank&quot; rel=&quot;noopener&quot;&gt;mysql:5.7&lt;/a&gt;制作mha4mysql-node的Docker镜像。&lt;/p&gt;
    
    </summary>
    
      <category term="实践" scheme="https://breeze2.github.io/blog/categories/%E5%AE%9E%E8%B7%B5/"/>
    
    
      <category term="mysql" scheme="https://breeze2.github.io/blog/tags/mysql/"/>
    
      <category term="practice" scheme="https://breeze2.github.io/blog/tags/practice/"/>
    
      <category term="docker" scheme="https://breeze2.github.io/blog/tags/docker/"/>
    
      <category term="mha4mysql" scheme="https://breeze2.github.io/blog/tags/mha4mysql/"/>
    
  </entry>
  
</feed>
