<rss xmlns:atom="http://www.w3.org/2005/Atom" version="2.0"><channel><title>PyTorch - Tag - Xiaopeng Xu</title><link>https://xu-xp.com/tags/pytorch/</link><description>PyTorch - Tag - Xiaopeng Xu</description><generator>Hugo -- gohugo.io</generator><language>en</language><managingEditor>xiaopeng.xu@kaust.edu.sa (Xiaopeng Xu)</managingEditor><webMaster>xiaopeng.xu@kaust.edu.sa (Xiaopeng Xu)</webMaster><lastBuildDate>Tue, 05 Dec 2023 00:00:00 +0000</lastBuildDate><atom:link href="https://xu-xp.com/tags/pytorch/" rel="self" type="application/rss+xml"/><item><title>PyTorch 常用命令</title><link>https://xu-xp.com/posts/pytorch_commons/</link><pubDate>Tue, 05 Dec 2023 00:00:00 +0000</pubDate><author>xiaopeng.xu@kaust.edu.sa (Xiaopeng Xu)</author><guid>https://xu-xp.com/posts/pytorch_commons/</guid><description><![CDATA[<h2 id="tensor-操作">Tensor 操作</h2>
<p>文档：<a href="https://pytorch.org/docs/stable/index.html" target="_blank" rel="noopener noreffer ">https://pytorch.org/docs/stable/index.html</a></p>
<h3 id="tensor-相关属性">Tensor 相关属性</h3>
<h4 id="查看-device">查看 device</h4>
<div class="code-block code-line-numbers open" style="counter-reset: code-block 0">
    <div class="code-header language-Python">
        <span class="code-title"><i class="arrow fas fa-angle-right" aria-hidden="true"></i></span>
        <span class="ellipses"><i class="fas fa-ellipsis-h" aria-hidden="true"></i></span>
        <span class="copy" title="Copy to clipboard"><i class="far fa-copy" aria-hidden="true"></i></span>
    </div><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-Python" data-lang="Python"><span class="line"><span class="cl"><span class="n">example_tensor</span><span class="o">.</span><span class="n">device</span></span></span></code></pre></div></div>
<h4 id="查看-shape">查看 shape</h4>
<div class="code-block code-line-numbers open" style="counter-reset: code-block 0">
    <div class="code-header language-Python">
        <span class="code-title"><i class="arrow fas fa-angle-right" aria-hidden="true"></i></span>
        <span class="ellipses"><i class="fas fa-ellipsis-h" aria-hidden="true"></i></span>
        <span class="copy" title="Copy to clipboard"><i class="far fa-copy" aria-hidden="true"></i></span>
    </div><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-Python" data-lang="Python"><span class="line"><span class="cl"><span class="n">example_tensor</span><span class="o">.</span><span class="n">shape</span>
</span></span><span class="line"><span class="cl"><span class="n">example_tensor</span><span class="o">.</span><span class="n">size</span><span class="p">(</span><span class="mi">1</span><span class="p">)</span></span></span></code></pre></div></div>
<h4 id="查看元素个数">查看元素个数</h4>
<div class="code-block code-line-numbers open" style="counter-reset: code-block 0">
    <div class="code-header language-Python">
        <span class="code-title"><i class="arrow fas fa-angle-right" aria-hidden="true"></i></span>
        <span class="ellipses"><i class="fas fa-ellipsis-h" aria-hidden="true"></i></span>
        <span class="copy" title="Copy to clipboard"><i class="far fa-copy" aria-hidden="true"></i></span>
    </div><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-Python" data-lang="Python"><span class="line"><span class="cl"><span class="n">example_tensor</span><span class="o">.</span><span class="n">numel</span><span class="p">()</span></span></span></code></pre></div></div>
<h3 id="tensor-元素索引">Tensor 元素索引</h3>
<div class="code-block code-line-numbers open" style="counter-reset: code-block 0">
    <div class="code-header language-Python">
        <span class="code-title"><i class="arrow fas fa-angle-right" aria-hidden="true"></i></span>
        <span class="ellipses"><i class="fas fa-ellipsis-h" aria-hidden="true"></i></span>
        <span class="copy" title="Copy to clipboard"><i class="far fa-copy" aria-hidden="true"></i></span>
    </div><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-Python" data-lang="Python"><span class="line"><span class="cl"><span class="n">example_scalar</span> <span class="o">=</span> <span class="n">example_tensor</span><span class="p">[</span><span class="mi">1</span><span class="p">,</span> <span class="mi">1</span><span class="p">,</span> <span class="mi">0</span><span class="p">]</span>
</span></span><span class="line"><span class="cl"><span class="n">example_scalar</span><span class="o">.</span><span class="n">item</span><span class="p">()</span>
</span></span><span class="line"><span class="cl"><span class="n">example_tensor</span><span class="p">[:,</span> <span class="mi">0</span><span class="p">,</span> <span class="mi">0</span><span class="p">]</span></span></span></code></pre></div></div>
<h3 id="tensor-初始化">Tensor 初始化</h3>
<h4 id="0-1-或随机数">0, 1 或随机数</h4>
<div class="code-block code-line-numbers open" style="counter-reset: code-block 0">
    <div class="code-header language-Python">
        <span class="code-title"><i class="arrow fas fa-angle-right" aria-hidden="true"></i></span>
        <span class="ellipses"><i class="fas fa-ellipsis-h" aria-hidden="true"></i></span>
        <span class="copy" title="Copy to clipboard"><i class="far fa-copy" aria-hidden="true"></i></span>
    </div><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-Python" data-lang="Python"><span class="line"><span class="cl"><span class="n">torch</span><span class="o">.</span><span class="n">ones_like</span><span class="p">(</span><span class="n">example_tensor</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="n">torch</span><span class="o">.</span><span class="n">zeros_like</span><span class="p">(</span><span class="n">example_tensor</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="n">torch</span><span class="o">.</span><span class="n">randn_like</span><span class="p">(</span><span class="n">example_tensor</span><span class="p">)</span>
</span></span><span class="line"><span class="cl"><span class="n">torch</span><span class="o">.</span><span class="n">randn</span><span class="p">(</span><span class="mi">2</span><span class="p">,</span> <span class="mi">2</span><span class="p">,</span> <span class="n">device</span><span class="o">=</span><span class="s1">&#39;cpu&#39;</span><span class="p">)</span> <span class="c1"># Alternatively, for a GPU tensor, you&#39;d use device=&#39;cuda&#39;</span>
</span></span><span class="line"><span class="cl"><span class="n">torch</span><span class="o">.</span><span class="n">arange</span><span class="p">(</span><span class="mi">4</span> <span class="o">*</span> <span class="mi">5</span> <span class="o">*</span> <span class="mi">6</span><span class="p">)</span><span class="o">.</span><span class="n">view</span><span class="p">(</span><span class="mi">4</span><span class="p">,</span> <span class="mi">5</span><span class="p">,</span> <span class="mi">6</span><span class="p">)</span> <span class="c1"># shape [4,5,6]</span></span></span></code></pre></div></div>
<h4 id="torchclone-复制-tensor"><code>torch.clone</code> 复制 tensor</h4>
<div class="code-block code-line-numbers open" style="counter-reset: code-block 0">
    <div class="code-header language-Python">
        <span class="code-title"><i class="arrow fas fa-angle-right" aria-hidden="true"></i></span>
        <span class="ellipses"><i class="fas fa-ellipsis-h" aria-hidden="true"></i></span>
        <span class="copy" title="Copy to clipboard"><i class="far fa-copy" aria-hidden="true"></i></span>
    </div><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-Python" data-lang="Python"><span class="line"><span class="cl"><span class="n">real_clone</span> <span class="o">=</span> <span class="n">real</span><span class="o">.</span><span class="n">clone</span><span class="p">()</span></span></span></code></pre></div></div>
<h4 id="fone_hot-生成多分类-tensor"><code>F.one_hot</code> 生成多分类 tensor</h4>
<div class="code-block code-line-numbers open" style="counter-reset: code-block 0">
    <div class="code-header language-Python">
        <span class="code-title"><i class="arrow fas fa-angle-right" aria-hidden="true"></i></span>
        <span class="ellipses"><i class="fas fa-ellipsis-h" aria-hidden="true"></i></span>
        <span class="copy" title="Copy to clipboard"><i class="far fa-copy" aria-hidden="true"></i></span>
    </div><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-Python" data-lang="Python"><span class="line"><span class="cl"><span class="n">F</span><span class="o">.</span><span class="n">one_hot</span><span class="p">(</span><span class="n">labels</span> <span class="o">%</span> <span class="n">n_classes</span><span class="p">)</span></span></span></code></pre></div></div>
<h4 id="nninit-填充-tensor"><code>nn.init.*</code> 填充 tensor</h4>
<div class="code-block code-line-numbers open" style="counter-reset: code-block 0">
    <div class="code-header language-Python">
        <span class="code-title"><i class="arrow fas fa-angle-right" aria-hidden="true"></i></span>
        <span class="ellipses"><i class="fas fa-ellipsis-h" aria-hidden="true"></i></span>
        <span class="copy" title="Copy to clipboard"><i class="far fa-copy" aria-hidden="true"></i></span>
    </div><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-Python" data-lang="Python"><span class="line"><span class="cl"><span class="n">nn</span><span class="o">.</span><span class="n">init</span><span class="o">.</span><span class="n">xavier_uniform_</span><span class="p">(</span><span class="n">tensor</span><span class="p">)</span> <span class="c1"># fill with uniform distribution, values scaled by &#39;gain&#39;</span>
</span></span><span class="line"><span class="cl"><span class="n">nn</span><span class="o">.</span><span class="n">init</span><span class="o">.</span><span class="n">constant_</span><span class="p">(</span><span class="n">bias</span><span class="p">,</span> <span class="mi">0</span><span class="p">)</span> <span class="c1"># fill with constant</span>
</span></span><span class="line"><span class="cl"><span class="n">nn</span><span class="o">.</span><span class="n">init</span><span class="o">.</span><span class="n">orthogonal_</span><span class="p">(</span><span class="n">param</span><span class="p">)</span> <span class="c1"># with a (semi) orthogonal matrix</span></span></span></code></pre></div></div>
<h3 id="基础函数">基础函数</h3>
<h4 id="逐元素加减乘-mul-">逐元素加减乘 mul, *</h4>
<div class="code-block code-line-numbers open" style="counter-reset: code-block 0">
    <div class="code-header language-Python">
        <span class="code-title"><i class="arrow fas fa-angle-right" aria-hidden="true"></i></span>
        <span class="ellipses"><i class="fas fa-ellipsis-h" aria-hidden="true"></i></span>
        <span class="copy" title="Copy to clipboard"><i class="far fa-copy" aria-hidden="true"></i></span>
    </div><div class="highlight"><pre tabindex="0" class="chroma"><code class="language-Python" data-lang="Python"><span class="line"><span class="cl"><span class="p">(</span><span class="n">example_tensor</span> <span class="o">-</span> <span class="mi">5</span><span class="p">)</span> <span class="o">*</span> <span class="mi">2</span> <span class="c1"># 顺序有影响</span>
</span></span><span class="line"><span class="cl"><span class="n">torch</span><span class="o">.</span><span class="n">mul</span><span class="p">(</span><span class="n">tensor_x</span><span class="p">,</span> <span class="n">tensor_x</span><span class="p">)</span> <span class="c1"># 逐元素积</span>
</span></span><span class="line"><span class="cl"><span class="n">tensor_x</span> <span class="o">*</span> <span class="n">tensor_y</span> <span class="c1"># 逐元素积 = torch.mul(x, y)</span></span></span></code></pre></div></div>
<h4 id="tensor-乘法-matmul--bmm-dot">Tensor 乘法 matmul, @, bmm, dot</h4>
<p><a href="https://blog.csdn.net/foneone/article/details/103876519" target="_blank" rel="noopener noreffer ">https://blog.csdn.net/foneone/article/details/103876519</a></p>]]></description></item><item><title>Pytorch HPO 超参优化</title><link>https://xu-xp.com/posts/pytorch_hpo/</link><pubDate>Wed, 24 May 2023 00:00:00 +0000</pubDate><author>xiaopeng.xu@kaust.edu.sa (Xiaopeng Xu)</author><guid>https://xu-xp.com/posts/pytorch_hpo/</guid><description><![CDATA[<p>超参优化是深度学习中的常见问题，有好多工具用于解决相关问题。其中，有一些博客对其做了比较好的综述，见这里：<a href="https://neptune.ai/blog/best-tools-for-model-tuning-and-hyperparameter-optimization" target="_blank" rel="noopener noreffer ">https://neptune.ai/blog/best-tools-for-model-tuning-and-hyperparameter-optimization</a>。</p>
<p>总结如下：</p>
<ul>
<li>
<p><a href="https://docs.google.com/spreadsheets/d/e/2PACX-1vQQngvMHiqVNND4-pBHq-2_S4SfJFnBn1qg4V_RsnG28qBJ4EV3soATwpq-YaShLDz2437FcH11-JLp/pubhtml?gid=0&amp;single=true" target="_blank" rel="noopener noreffer ">https://docs.google.com/spreadsheets/d/e/2PACX-1vQQngvMHiqVNND4-pBHq-2_S4SfJFnBn1qg4V_RsnG28qBJ4EV3soATwpq-YaShLDz2437FcH11-JLp/pubhtml?gid=0&amp;single=true</a>  Google Vizer, AWS Sage Maker, 和 Azure Machine Learning 付费不开源，可以忽略。</p>]]></description></item><item><title>Torch Geometric 常用命令</title><link>https://xu-xp.com/posts/pytorch_geometrics_commons/</link><pubDate>Wed, 10 Aug 2022 00:00:00 +0000</pubDate><author>xiaopeng.xu@kaust.edu.sa (Xiaopeng Xu)</author><guid>https://xu-xp.com/posts/pytorch_geometrics_commons/</guid><description>&lt;p>Torch Geometric（PyG）是一个基于PyTorch的用于处理不规则数据（比如图）的库，或者说是一个用于在图等数据上快速实现表征学习的框架。由于速度和方便的优势，PyG 是当前最流行和广泛使用的GNN库。&lt;/p></description></item></channel></rss>