qemu-nbd https://forum.ff.co.za/index.php/ en Reclaiming space on qcow2 disk image https://forum.ff.co.za/index.php/documentation/reclaiming-space-qcow2-disk-image <span>Reclaiming space on qcow2 disk image</span> <span><span lang="" about="/index.php/user/jeff" typeof="schema:Person" property="schema:name" datatype="">Jeff</span></span> <span>Sat, 01/09/2018 - 10:20</span> <div class="layout layout--onecol"> <div class="layout__region layout__region--content"> <div class="body-padding field field--name-body field--type-text-with-summary field--label-hidden field--item"><nav aria-label="Book outline for Install QEMU-KVM on Debian 9" role="navigation"> <nav aria-label="Book outline for Install QEMU-KVM on Debian 9" role="navigation"> <p><strong>Table of Contents</strong></p> <ul> <li><a href="/documentation/install-qemu-kvm-debian-9">Install QEMU-KVM on Debian 9</a></p> <ul> <li><a href="/documentation/allow-users-run-virsh-and-manage-vms">Allow users to run virsh and manage VMs</a></li> <li><a href="/documentation/kvm-bridged-networking">KVM Bridged Networking</a></li> <li><a href="/documentation/kvm-virsh-console-access-debian-and-freebsd-vms">KVM Virsh Console Access for Debian and FreeBSD VMs</a></li> </ul> </li> </ul> </nav> <nav aria-label="Book outline for Elliptic Curve Crypto &amp; LetsEncrypt" role="navigation"> <ul></ul> </nav> <nav aria-label="Book outline for Managing qcow2 images" role="navigation"> <ul> <li> <ul> <li><a href="https://forum.ff.co.za/documentation/managing-qcow2-images" hreflang="en">Managing qcow2 images</a></p> <ul> <li><a href="/documentation/backup-virtual-machine">Backup virtual machine</a></li> <li><a href="/copy-sparse-file-across-network-using-dd-or-rsync" hreflang="en">Copy sparse file across network using dd or rsync</a></li> <li><a href="/documentation/how-mount-qcow2-image-qemu-nbd" hreflang="en">How to mount a qcow2 image with qemu-nbd</a></li> <li><em><u><strong><a href="/documentation/reclaiming-space-qcow2-disk-image" hreflang="en">Reclaiming space on qcow2 disk image</a></strong></u></em></li> <li><a href="/documentation/resize-qcow2-blockresize-running-vm">Resize qcow2 with blockresize on running VM</a></li> </ul> </li> </ul> </li> </ul> </nav> <ul> <li><a href="/documentation/migrate-vm-minimal-downtime">Migrate a VM with Minimal Downtime</a></li> </ul> <p> </p> </nav> </p> <p><em>Update 2018-11-11:</em></p> </p> <p>Source: <a href="https://dustymabe.com/2013/06/11/recover-space-from-vm-disk-images-by-using-discardfstrim/">dustymabe.com/2013/06/11/recover-space-from-vm-disk-images-by-using-discardfstrim/</a></p> <p>Recommended method is to use fstrim. See: <a href="https://pve.proxmox.com/wiki/Shrink_Qcow2_Disk_File">pve.proxmox.com/wiki/Shrink_Qcow2_Disk_Files</a></p> <p><em>"The recommended version is to pass TRIM commands (known from SSDs) from the VM to the backing storage. This has the advantage that it works automatically, does not need to write the whole free parts of all disks to zero and must only be setup once."</em></p> <p>The alternative manual method outlined below, is slow and painful - but seemingly necessary for the occasional apparent failure of fstrim, as experienced first-hand.</p> <h2>Convert the storage 'driver' from virtio-blck to virtio-scsi</h2> <p>Use VMM (Virtual Machine Manager) in a vncserver session on the hypervisor, over the VPN.</p> <p><strong>NB: Shut the VM down, first!</strong></p> <p>Remove the current virtio-blk disk image:</p> <p><img alt="" class="image-large colorbox" data-entity-type="file" data-entity-uuid="insert-large-9d1d45b4-2a30-4bd1-90b1-f57272129769" data-insert-class="image-large colorbox" data-insert-type="image" height="357" src="/sites/default/files/styles/large/public/2018-11/Screenshot%202018-11-11%20at%2012.36.42.png" width="480" /></p> <p>And add it back as virtio-scsi:</p> <p><img alt="" class="image-large colorbox" data-entity-type="file" data-entity-uuid="insert-large-b6cc7325-beac-4b6f-993f-2514f1fe91d1" data-insert-class="image-large colorbox" data-insert-type="image" height="357" src="/sites/default/files/styles/large/public/2018-11/Screenshot%202018-11-11%20at%2012.41.15.png" width="480" /></p> <p> </p> <h2>Then, to enable TRIM in the guest, first edit the VM's XML configuration file on the hypervisor:</h2> <p><code>virsh edit MACHINE_NAME</code></p> <p>And add <strong>discard='unmap' </strong>to the driver.</p> <pre> <code class="language-bash"> &lt;devices&gt; &lt;emulator&gt;/usr/bin/kvm&lt;/emulator&gt; &lt;disk type='file' device='disk'&gt; &lt;driver name='qemu' type='qcow2' discard='unmap'/&gt; &lt;source file='/var/lib/libvirt/images/new-ff.qcow2'/&gt; &lt;target dev='sda' bus='scsi'/&gt; &lt;address type='drive' controller='0' bus='0' target='0' unit='0'/&gt; &lt;/disk&gt;</code></pre><p> </p> <h2>Start the VM and TRIM</h2> <p><code>virsh start new-ff</code></p> <p>Then, SSH to the VM, and as root, run the following command:</p> <p>fstrim -v /</p> <p>Add to root's crontab to perform a trim weekly.</p> <p><code>crontab -e</code></p> <pre> <code class="language-bash"># Trim filesystem daily to recover space 0 2 * * 1 /sbin/fstrim -v / &gt;&gt; /home/jeff/fstrim.log</code></pre><hr /> <h1><strong>Alternative method:</strong></h1> <p>Shutdown and backup the VM's qcow2 disk image.</p> <p>Mount the qcow2 image using <a href="/documentation/how-mount-qcow2-image-qemu-nbd">this guide</a>.</p> <ol> <li>Zero-fill the drive (<code>dd if=/dev/zero of=/some/file</code> until you run out of space)</li> <li>delete /some/file</li> <li> </li> </ol> <p><code>qemu-img convert -O qcow2 original_image.qcow2 deduplicated_image.qcow2</code></p> <p>Or...</p> <p>In a terminal on the VM, run the dd if=zero commands until you run out of disk space. Before running this, be sure to stop any applications running on the guest otherwise errors may result.</p> <p style="margin: 0px 0px 0.5em; padding: 0px; border: 0px; font-style: inherit; font-variant: inherit; font-weight: inherit; font-stretch: inherit; line-height: inherit; font-family: inherit; font-size: 15px; vertical-align: baseline; box-sizing: inherit; word-wrap: break-word;"> </p> </div> <div class="field field--name-field-images field--type-image field--label-hidden field--items"> <div class="field--item"> <a href="https://forum.ff.co.za/sites/default/files/2018-11/Screenshot%202018-11-11%20at%2012.36.42.png" aria-controls="colorbox" aria-label="{&quot;alt&quot;:&quot;&quot;}" role="button" title="Reclaiming space on qcow2 disk image" data-colorbox-gallery="gallery-book-78-IHgZ2Vrbwko" class="colorbox" data-cbox-img-attrs="{&quot;alt&quot;:&quot;&quot;}"><img src="/sites/default/files/styles/thumbnail/public/2018-11/Screenshot%202018-11-11%20at%2012.36.42.png?itok=chc6MmN0" width="100" height="74" alt="" loading="lazy" typeof="foaf:Image" class="img-responsive" /> </a> </div> <div class="field--item"> <a href="https://forum.ff.co.za/sites/default/files/2018-11/Screenshot%202018-11-11%20at%2012.41.15.png" aria-controls="colorbox" aria-label="{&quot;alt&quot;:&quot;&quot;}" role="button" title="Reclaiming space on qcow2 disk image" data-colorbox-gallery="gallery-book-78-IHgZ2Vrbwko" class="colorbox" data-cbox-img-attrs="{&quot;alt&quot;:&quot;&quot;}"><img src="/sites/default/files/styles/thumbnail/public/2018-11/Screenshot%202018-11-11%20at%2012.41.15.png?itok=5dbLTRX6" width="100" height="74" alt="" loading="lazy" typeof="foaf:Image" class="img-responsive" /> </a> </div> </div> <div class="field field--name-field-category field--type-entity-reference field--label-above"> <div class="field--label">Category</div> <div class="field--item"><a href="/index.php/documentation" hreflang="en">Documentation</a></div> </div> <div class="fftags field field--name-field-tags field--type-entity-reference field--label-inline"> <div class="field--label">Tags</div> <div class="field--items"> <div class="field--item"><a href="/index.php/tags/qemu-nbd" hreflang="en">qemu-nbd</a></div> <div class="field--item"><a href="/index.php/tags/qcow2" hreflang="en">qcow2</a></div> <div class="field--item"><a href="/index.php/tags/dd" hreflang="en">dd</a></div> <div class="field--item"><a href="/index.php/tags/fstrim" hreflang="en">fstrim</a></div> </div> </div> </div> </div> Sat, 01 Sep 2018 08:20:52 +0000 Jeff 78 at https://forum.ff.co.za How to mount a qcow2 image with qemu-nbd https://forum.ff.co.za/index.php/documentation/how-mount-qcow2-image-qemu-nbd <span property="schema:name">How to mount a qcow2 image with qemu-nbd</span> <span rel="schema:author"><span lang="" about="/index.php/user/jeff" typeof="schema:Person" property="schema:name" datatype="">Jeff</span></span> <span property="schema:dateCreated" content="2018-09-01T08:05:10+00:00">Sat, 01/09/2018 - 10:05</span> <div class="layout layout--onecol"> <div class="layout__region layout__region--content"> <drupal-render-placeholder callback="flag.link_builder:build" arguments="0=node&amp;1=76&amp;2=bookmark" token="kPzuF-uOLyyJPY1iEvQa2Xs1wSL3GM-udNQUyhoLz1A"></drupal-render-placeholder><drupal-render-placeholder callback="flag.link_builder:build" arguments="0=node&amp;1=76&amp;2=like" token="JkZZnfJDXzp_gem0rRUfYTG9phWtQTBBXGGL2ggRo4c"></drupal-render-placeholder> </div> </div> Sat, 01 Sep 2018 08:05:10 +0000 Jeff 76 at https://forum.ff.co.za