Default page

Edit File: 060_forget.html

<!DOCTYPE html>
<html class="writer-html5" lang="en" >
<head>
 <meta charset="utf-8" /><meta name="generator" content="Docutils 0.17.1: http://docutils.sourceforge.net/" />

<body class="wy-body-for-nav"> 
 <div class="wy-grid-for-nav">
 <nav data-toggle="wy-nav-shift" class="wy-nav-side">
 <div class="wy-side-scroll">
 <div class="wy-side-nav-search" >
 <a href="index.html" class="icon icon-home"> restic
 <img src="_static/logo.png" class="logo" alt="Logo"/>
 </a>
 <div class="version">
 0.12.1

</div>
<div role="search">
 <form id="rtd-search-form" class="wy-form" action="search.html" method="get">
 <input type="text" name="q" placeholder="Search docs" />
 <input type="hidden" name="check_keywords" value="yes" />
 <input type="hidden" name="area" value="default" />
 </form>
</div>
 </div><div class="wy-menu wy-menu-vertical" data-spy="affix" role="navigation" aria-label="Navigation menu">
 <ul class="current">
<li class="toctree-l1"><a class="reference internal" href="010_introduction.html">Introduction</a></li>
<li class="toctree-l1"><a class="reference internal" href="020_installation.html">Installation</a></li>
<li class="toctree-l1"><a class="reference internal" href="030_preparing_a_new_repo.html">Preparing a new repository</a></li>
<li class="toctree-l1"><a class="reference internal" href="040_backup.html">Backing up</a></li>
<li class="toctree-l1"><a class="reference internal" href="045_working_with_repos.html">Working with repositories</a></li>
<li class="toctree-l1"><a class="reference internal" href="050_restore.html">Restoring from backup</a></li>
<li class="toctree-l1 current"><a class="current reference internal" href="#">Removing backup snapshots</a><ul>
<li class="toctree-l2"><a class="reference internal" href="#remove-a-single-snapshot">Remove a single snapshot</a></li>
<li class="toctree-l2"><a class="reference internal" href="#removing-snapshots-according-to-a-policy">Removing snapshots according to a policy</a></li>
<li class="toctree-l2"><a class="reference internal" href="#customize-pruning">Customize pruning</a></li>
</ul>
</li>
<li class="toctree-l1"><a class="reference internal" href="070_encryption.html">Encryption</a></li>
<li class="toctree-l1"><a class="reference internal" href="075_scripting.html">Scripting</a></li>
<li class="toctree-l1"><a class="reference internal" href="080_examples.html">Examples</a></li>
<li class="toctree-l1"><a class="reference internal" href="090_participating.html">Participating</a></li>
<li class="toctree-l1"><a class="reference internal" href="100_references.html">References</a></li>
<li class="toctree-l1"><a class="reference internal" href="110_talks.html">Talks</a></li>
<li class="toctree-l1"><a class="reference internal" href="faq.html">FAQ</a></li>
<li class="toctree-l1"><a class="reference internal" href="manual_rest.html">Manual</a></li>
<li class="toctree-l1"><a class="reference internal" href="developer_information.html">Developer Information</a></li>
</ul>

</div>
 </div>
 </nav>

<section data-toggle="wy-nav-shift" class="wy-nav-content-wrap"><nav class="wy-nav-top" aria-label="Mobile navigation menu" >
 
 <a href="index.html">restic</a>
 </nav>

<div class="wy-nav-content">
 <div class="rst-content">
 <div role="navigation" aria-label="Page navigation">
 <ul class="wy-breadcrumbs">
 <li><a href="index.html" class="icon icon-home"></a> »</li>
 <li>Removing backup snapshots</li>
 <li class="wy-breadcrumbs-aside">
 <a href="_sources/060_forget.rst.txt" rel="nofollow"> View page source</a>
 </li>
 </ul>
 <hr/>
</div>
 <div role="main" class="document" itemscope="itemscope" itemtype="http://schema.org/Article">
 <div itemprop="articleBody">
 
 <section id="removing-backup-snapshots">
<h1>Removing backup snapshots<a class="headerlink" href="#removing-backup-snapshots" title="Permalink to this headline">¶</a></h1>
All backup space is finite, so restic allows removing old snapshots.
This can be done either manually (by specifying a snapshot ID to remove)
or by using a policy that describes which snapshots to forget. For all
remove operations, two commands need to be called in sequence:
<code class="docutils literal notranslate">forget</code> to remove a snapshot and <code class="docutils literal notranslate">prune</code> to actually remove the
data that was referenced by the snapshot from the repository. This can
be automated with the <code class="docutils literal notranslate">--prune</code> option of the <code class="docutils literal notranslate">forget</code> command,
which runs <code class="docutils literal notranslate">prune</code> automatically if snapshots have been removed.
Pruning snapshots can be a time-consuming process, depending on the
amount of snapshots and data to process. During a prune operation, the
repository is locked and backups cannot be completed. Please plan your
pruning so that there’s time to complete it and it doesn’t interfere with
regular backup runs.
It is advisable to run <code class="docutils literal notranslate">restic check</code> after pruning, to make sure
you are alerted, should the internal data structures of the repository
be damaged.
<section id="remove-a-single-snapshot">
<h2>Remove a single snapshot<a class="headerlink" href="#remove-a-single-snapshot" title="Permalink to this headline">¶</a></h2>
The command <code class="docutils literal notranslate">snapshots</code> can be used to list all snapshots in a
repository like this:
<div class="highlight-console notranslate"><div class="highlight"><pre>$ restic -r /srv/restic-repo snapshots
enter password for repository:
ID Date Host Tags Directory
----------------------------------------------------------------------
40dc1520 2015-05-08 21:38:30 kasimir /home/user/work
79766175 2015-05-08 21:40:19 kasimir /home/user/work
bdbd3439 2015-05-08 21:45:17 luigi /home/art
590c8fc8 2015-05-08 21:47:38 kazik /srv
9f0bc19e 2015-05-08 21:46:11 luigi /srv
</pre></div>
</div>
In order to remove the snapshot of <code class="docutils literal notranslate">/home/art</code>, use the <code class="docutils literal notranslate">forget</code>
command and specify the snapshot ID on the command line:
<div class="highlight-console notranslate"><div class="highlight"><pre>$ restic -r /srv/restic-repo forget bdbd3439
enter password for repository:
removed snapshot bdbd3439
</pre></div>
</div>
Afterwards this snapshot is removed:
<div class="highlight-console notranslate"><div class="highlight"><pre>$ restic -r /srv/restic-repo snapshots
enter password for repository:
ID Date Host Tags Directory
----------------------------------------------------------------------
40dc1520 2015-05-08 21:38:30 kasimir /home/user/work
79766175 2015-05-08 21:40:19 kasimir /home/user/work
590c8fc8 2015-05-08 21:47:38 kazik /srv
9f0bc19e 2015-05-08 21:46:11 luigi /srv
</pre></div>
</div>
But the data that was referenced by files in this snapshot is still
stored in the repository. To cleanup unreferenced data, the <code class="docutils literal notranslate">prune</code>
command must be run:
<div class="highlight-console notranslate"><div class="highlight"><pre>$ restic -r /srv/restic-repo prune
enter password for repository:
repository 33002c5e opened successfully, password is correct
loading all snapshots...
loading indexes...
finding data that is still in use for 4 snapshots
[0:00] 100.00% 4 / 4 snapshots
searching used packs...
collecting packs for deletion and repacking
[0:00] 100.00% 5 / 5 packs processed

to repack: 69 blobs / 1.078 MiB
this removes 67 blobs / 1.047 MiB
to delete: 7 blobs / 25.726 KiB
total prune: 74 blobs / 1.072 MiB
remaining: 16 blobs / 38.003 KiB
unused size after prune: 0 B (0.00% of remaining size)

keep 1 snapshots:
ID Date Host Tags Directory
----------------------------------------------------------------------
4bba301e 2017-02-21 10:49:18 mopped /home/user/work

remove 1 snapshots:
ID Date Host Tags Directory
----------------------------------------------------------------------
8c02b94b 2017-02-21 10:48:33 mopped /home/user/work

1 snapshots have been removed, running prune
loading all snapshots...
loading indexes...
finding data that is still in use for 1 snapshots
[0:00] 100.00% 1 / 1 snapshots
searching used packs...
collecting packs for deletion and repacking
[0:00] 100.00% 5 / 5 packs processed

repacking packs
[0:00] 100.00% 2 / 2 packs repacked
rebuilding index
[0:00] 100.00% 3 / 3 packs processed
deleting obsolete index files
[0:00] 100.00% 3 / 3 files deleted
removing 3 old packs
[0:00] 100.00% 3 / 3 files deleted
done
</pre></div>
</div>
</section>
<section id="removing-snapshots-according-to-a-policy">
<h2>Removing snapshots according to a policy<a class="headerlink" href="#removing-snapshots-according-to-a-policy" title="Permalink to this headline">¶</a></h2>
Removing snapshots manually is tedious and error-prone, therefore restic
allows specifying which snapshots should be removed automatically
according to a policy. You can specify how many hourly, daily, weekly,
monthly and yearly snapshots to keep, any other snapshots are removed.
The most important command-line parameter here is <code class="docutils literal notranslate">--dry-run</code> which
instructs restic to not remove anything but print which snapshots would
be removed.
When <code class="docutils literal notranslate">forget</code> is run with a policy, restic loads the list of all
snapshots, then groups these by host name and list of directories. The grouping
options can be set with <code class="docutils literal notranslate">--group-by</code>, to only group snapshots by paths and
tags use <code class="docutils literal notranslate">--group-by paths,tags</code>. The policy is then applied to each group of
snapshots separately. This is a safety feature.
The <code class="docutils literal notranslate">forget</code> command accepts the following parameters:
<ul class="simple">
<li><code class="docutils literal notranslate">--keep-last n</code> never delete the <code class="docutils literal notranslate">n</code> last (most recent) snapshots</li>
<li><code class="docutils literal notranslate">--keep-hourly n</code> for the last <code class="docutils literal notranslate">n</code> hours in which a snapshot was
made, keep only the last snapshot for each hour.</li>
<li><code class="docutils literal notranslate">--keep-daily n</code> for the last <code class="docutils literal notranslate">n</code> days which have one or more
snapshots, only keep the last one for that day.</li>
<li><code class="docutils literal notranslate">--keep-weekly n</code> for the last <code class="docutils literal notranslate">n</code> weeks which have one or more
snapshots, only keep the last one for that week.</li>
<li><code class="docutils literal notranslate">--keep-monthly n</code> for the last <code class="docutils literal notranslate">n</code> months which have one or more
snapshots, only keep the last one for that month.</li>
<li><code class="docutils literal notranslate">--keep-yearly n</code> for the last <code class="docutils literal notranslate">n</code> years which have one or more
snapshots, only keep the last one for that year.</li>
<li><code class="docutils literal notranslate">--keep-tag</code> keep all snapshots which have all tags specified by
this option (can be specified multiple times).</li>
<li><code class="docutils literal notranslate">--keep-within duration</code> keep all snapshots which have been made within
the duration of the latest snapshot. <code class="docutils literal notranslate">duration</code> needs to be a number of
years, months, days, and hours, e.g. <code class="docutils literal notranslate">2y5m7d3h</code> will keep all snapshots
made in the two years, five months, seven days, and three hours before the
latest snapshot.</li>
<li><code class="docutils literal notranslate">--keep-within-hourly duration</code> keep all hourly snapshots made within
specified duration of the latest snapshot. The duration is specified in
the same way as for <code class="docutils literal notranslate">--keep-within</code> and the method for determining
hourly snapshots is the same as for <code class="docutils literal notranslate">--keep-hourly</code>.</li>
<li><code class="docutils literal notranslate">--keep-within-daily duration</code> keep all daily snapshots made within
specified duration of the latest snapshot.</li>
<li><code class="docutils literal notranslate">--keep-within-weekly duration</code> keep all weekly snapshots made within
specified duration of the latest snapshot.</li>
<li><code class="docutils literal notranslate">--keep-within-monthly duration</code> keep all monthly snapshots made within
specified duration of the latest snapshot.</li>
<li><code class="docutils literal notranslate">--keep-within-yearly duration</code> keep all yearly snapshots made within
specified duration of the latest snapshot.</li>
</ul>
<div class="admonition note">
Note
All calendar related <code class="docutils literal notranslate">--keep-*</code> options work on the natural time
boundaries and not relative to when you run the <code class="docutils literal notranslate">forget</code> command. Weeks
are Monday 00:00 -> Sunday 23:59, days 00:00 to 23:59, hours :00 to :59, etc.
</div>
<div class="admonition note">
Note
Specifying <code class="docutils literal notranslate">--keep-tag ''</code> will match untagged snapshots only.
</div>
Multiple policies will be ORed together so as to be as inclusive as possible
for keeping snapshots.
Additionally, you can restrict removing snapshots to those which have a
particular hostname with the <code class="docutils literal notranslate">--host</code> parameter, or tags with the
<code class="docutils literal notranslate">--tag</code> option. When multiple tags are specified, only the snapshots
which have all the tags are considered. For example, the following command
removes all but the latest snapshot of all snapshots that have the tag <code class="docutils literal notranslate">foo</code>:
<div class="highlight-console notranslate"><div class="highlight"><pre>$ restic forget --tag foo --keep-last 1
</pre></div>
</div>
This command removes all but the last snapshot of all snapshots that have
either the <code class="docutils literal notranslate">foo</code> or <code class="docutils literal notranslate">bar</code> tag set:
<div class="highlight-console notranslate"><div class="highlight"><pre>$ restic forget --tag foo --tag bar --keep-last 1
</pre></div>
</div>
To only keep the last snapshot of all snapshots with both the tag <code class="docutils literal notranslate">foo</code> and
<code class="docutils literal notranslate">bar</code> set use:
<div class="highlight-console notranslate"><div class="highlight"><pre>$ restic forget --tag foo,bar --keep-last 1
</pre></div>
</div>
To ensure only untagged snapshots are considered, specify the empty string ‘’ as
the tag.
<div class="highlight-console notranslate"><div class="highlight"><pre>$ restic forget --tag '' --keep-last 1
</pre></div>
</div>
All the <code class="docutils literal notranslate">--keep-*</code> options above only count
hours/days/weeks/months/years which have a snapshot, so those without a
snapshot are ignored.
For safety reasons, restic refuses to act on an “empty” policy. For example,
if one were to specify <code class="docutils literal notranslate">--keep-last 0</code> to forget all snapshots in the
repository, restic will respond that no snapshots will be removed. To delete
all snapshots, use <code class="docutils literal notranslate">--keep-last 1</code> and then finally remove the last
snapshot ID manually (by passing the ID to <code class="docutils literal notranslate">forget</code>).
All snapshots are evaluated against all matching <code class="docutils literal notranslate">--keep-*</code> counts. A
single snapshot on 2017-09-30 (Sat) will count as a daily, weekly and monthly.
Let’s explain this with an example: Suppose you have only made a backup
on each Sunday for 12 weeks:
<div class="highlight-console notranslate"><div class="highlight"><pre>$ restic snapshots
repository f00c6e2a opened successfully, password is correct
ID Time Host Tags Paths
---------------------------------------------------------------
0a1f9759 2019-09-01 11:00:00 mopped /home/user/work
46cfe4d5 2019-09-08 11:00:00 mopped /home/user/work
f6b1f037 2019-09-15 11:00:00 mopped /home/user/work
eb430a5d 2019-09-22 11:00:00 mopped /home/user/work
8cf1cb9a 2019-09-29 11:00:00 mopped /home/user/work
5d33b116 2019-10-06 11:00:00 mopped /home/user/work
b9553125 2019-10-13 11:00:00 mopped /home/user/work
e1a7b58b 2019-10-20 11:00:00 mopped /home/user/work
8f8018c0 2019-10-27 11:00:00 mopped /home/user/work
59403279 2019-11-03 11:00:00 mopped /home/user/work
dfee9fb4 2019-11-10 11:00:00 mopped /home/user/work
e1ae2f40 2019-11-17 11:00:00 mopped /home/user/work
---------------------------------------------------------------
12 snapshots
</pre></div>
</div>
Then <code class="docutils literal notranslate">forget --keep-daily 4</code> will keep the last four snapshots for the last
four Sundays, but remove the rest:
<div class="highlight-console notranslate"><div class="highlight"><pre>$ restic forget --keep-daily 4 --dry-run
repository f00c6e2a opened successfully, password is correct
Applying Policy: keep the last 4 daily snapshots
keep 4 snapshots:
ID Time Host Tags Reasons Paths
-------------------------------------------------------------------------------
8f8018c0 2019-10-27 11:00:00 mopped daily snapshot /home/user/work
59403279 2019-11-03 11:00:00 mopped daily snapshot /home/user/work
dfee9fb4 2019-11-10 11:00:00 mopped daily snapshot /home/user/work
e1ae2f40 2019-11-17 11:00:00 mopped daily snapshot /home/user/work
-------------------------------------------------------------------------------
4 snapshots

remove 8 snapshots:
ID Time Host Tags Paths
---------------------------------------------------------------
0a1f9759 2019-09-01 11:00:00 mopped /home/user/work
46cfe4d5 2019-09-08 11:00:00 mopped /home/user/work
f6b1f037 2019-09-15 11:00:00 mopped /home/user/work
eb430a5d 2019-09-22 11:00:00 mopped /home/user/work
8cf1cb9a 2019-09-29 11:00:00 mopped /home/user/work
5d33b116 2019-10-06 11:00:00 mopped /home/user/work
b9553125 2019-10-13 11:00:00 mopped /home/user/work
e1a7b58b 2019-10-20 11:00:00 mopped /home/user/work
---------------------------------------------------------------
8 snapshots
</pre></div>
</div>
The result of the <code class="docutils literal notranslate">forget --keep-daily</code> operation does not depend on when it
is run, it will only count the days for which a snapshot exists. This is a
safety feature: it prevents restic from removing snapshots when no new ones are
created. Otherwise, running <code class="docutils literal notranslate">forget --keep-daily 4</code> on a Friday (without any
snapshot Monday to Thursday) would remove all snapshots!
Another example: Suppose you make daily backups for 100 years. Then
<code class="docutils literal notranslate">forget --keep-daily 7 --keep-weekly 5 --keep-monthly 12 --keep-yearly 75</code>
will keep the most recent 7 daily snapshots, then 4 (remember, 7 dailies
already include a week!) last-day-of-the-weeks and 11 or 12
last-day-of-the-months (11 or 12 depends if the 5 weeklies cross a month).
And finally 75 last-day-of-the-year snapshots. All other snapshots are
removed.
You might want to maintain the same policy as for the example above, but have
irregular backups. For example, the 7 snapshots specified with <code class="docutils literal notranslate">--keep-daily 7</code>
might be spread over a longer period. If what you want is to keep daily snapshots
for a week, weekly for a month, monthly for a year and yearly for 75 years, you
could specify:
<code class="docutils literal notranslate">forget --keep-daily-within 7d --keep-weekly-within 1m --keep-monthly-within 1y
--keep-yearly-within 75y</code>
(Note that <cite>1w</cite> is not a recognized duration, so you will have to specify
<cite>7d</cite> instead)
</section>
<section id="customize-pruning">
<h2>Customize pruning<a class="headerlink" href="#customize-pruning" title="Permalink to this headline">¶</a></h2>
To understand the custom options, we first explain how the pruning process works:
<ol class="arabic">
<li>All snapshots and directories within snapshots are scanned to determine
which data is still in use.</li>
<li>For all files in the repository, restic finds out if the file is fully
used, partly used or completely unused.</li>
<li>Completely unused files are marked for deletion. Fully used files are kept.
A partially used file is either kept or marked for repacking depending on user
options.
Note that for repacking, restic must download the file from the repository
storage and re-upload the needed data in the repository. This can be very
time-consuming for remote repositories.
</li>
<li>After deciding what to do, <code class="docutils literal notranslate">prune</code> will actually perform the repack, modify
the index according to the changes and delete the obsolete files.</li>
</ol>
The <code class="docutils literal notranslate">prune</code> command accepts the following options:
<ul>
<li><code class="docutils literal notranslate">--max-unused limit</code> allow unused data up to the specified limit within the repository.
This allows restic to keep partly used files instead of repacking them.
The limit can be specified in several ways:
<blockquote>
<div><ul class="simple">
<li>As an absolute size (e.g. <code class="docutils literal notranslate">200M</code>). If you want to minimize the space
used by your repository, pass <code class="docutils literal notranslate">0</code> to this option.</li>
<li>As a size relative to the total repo size (e.g. <code class="docutils literal notranslate">10%</code>). This means that
after prune, at most <code class="docutils literal notranslate">10%</code> of the total data stored in the repo may be
unused data. If the repo after prune has as size of 500MB, then at most
50MB may be unused.</li>
<li>If the string <code class="docutils literal notranslate">unlimited</code> is passed, there is no limit for partly
unused files. This means that as long as some data is still used within
a file stored in the repo, restic will just leave it there. Use this if
you want to minimize the time and bandwidth used by the <code class="docutils literal notranslate">prune</code>
operation. Note that metadata will still be repacked.</li>
</ul>
</div></blockquote>
Restic tries to repack as little data as possible while still ensuring this
limit for unused data. The default value is 5%.
</li>
<li><code class="docutils literal notranslate">--max-repack-size size</code> if set limits the total size of files to repack.
As <code class="docutils literal notranslate">prune</code> first stores all repacked files and deletes the obsolete files at the end,
this option might be handy if you expect many files to be repacked and fear to run low
on storage.</li>
<li><code class="docutils literal notranslate">--repack-cacheable-only</code> if set to true only files which contain
metadata and would be stored in the cache are repacked. Other pack files are
not repacked if this option is set. This allows a very fast repacking
using only cached data. It can, however, imply that the unused data in
your repository exceeds the value given by <code class="docutils literal notranslate">--max-unused</code>.
The default value is false.</li>
<li><code class="docutils literal notranslate">--dry-run</code> only show what <code class="docutils literal notranslate">prune</code> would do.</li>
<li><code class="docutils literal notranslate">--verbose</code> increased verbosity shows additional statistics for <code class="docutils literal notranslate">prune</code>.</li>
</ul>
</section>
</section>

</div>
 </div>
 <footer><div class="rst-footer-buttons" role="navigation" aria-label="Footer">
 <a href="050_restore.html" class="btn btn-neutral float-left" title="Restoring from backup" accesskey="p" rel="prev"> Previous</a>
 <a href="070_encryption.html" class="btn btn-neutral float-right" title="Encryption" accesskey="n" rel="next">Next </a>
 </div>

<hr/>

Built with <a href="https://www.sphinx-doc.org/">Sphinx</a> using a
 <a href="https://github.com/readthedocs/sphinx_rtd_theme">theme</a>
 provided by <a href="https://readthedocs.org">Read the Docs</a>.

</footer>
 </div>
 </div>
 </section>
 </div>
 <script>
 jQuery(function () {
 SphinxRtdTheme.Navigation.enable(true);
 });
 </script>

</body>
</html>