- Append-Only Mode
- Other Questions
- What’s the difference between a SSH key, SSH password, SSH key passphrase and Borg repository passphrase?
- Which storage backend are you using?
- How can I migrate an existing repo including archives to or from BorgBase?
- Why is the repository size shown in the web interface different from
- How often should I run
- How do I fully remove my account?
- Is there a maintenance window?
- How do I get support?
If you get
Permission denied (public key),
Connection closed by remote host. Is borg working on the server? or get asked for a password (as opposed to Borg passphrase), it is almost always a problem with SSH keys. Double-check the following to debug further:
Have you already assigned a SSH key to the repo on BorgBase.com and is this the same key you are using locally? BorgBase will show the key’s SHA256 fingerprint in Account > SSH Keys. You can compare this to your local fingerprint like this:
$ ssh-keygen -lf ~/.ssh/id_ed25519
Is SSH using the key you assigned? By default only
~/.ssh/id_[rsa | ed25519 | ecdsa]are used. If you run Borg from a different user account, the SSH keys will differ too. When using a custom key name, you can add
~/.ssh/config. Or to only use this key with BorgBase:
Host *.repo.borgbase.com IdentityFile ~/.ssh/id_custom
- Is the SSH key you are using encrypted? Encrypted keys need to be unlocked (added to
ssh-agent) before using them with Vorta and you may get prompted for a password when using it with Borg directly. So not well suited for non-interactive use. For most use cases, using an encrypted key doesn’t increase security. It’s better to use one key per machine and not move it.
- Is SSH trying too many keys? The maximum number of keys (
MaxAuthTries) that can be tried per connection is 6. If you have more keys, specify the key to use, as shown above.
Are the key permissions OK? Private keys and important config files (if in use) need to have a permission of
0600. To change the permission:
$ chmod 0600 ~/.ssh/id_custom ~/.ssh/config
If you still get errors, try to connect to your repo using the
sshcommand with verbose logging enabled:
$ ssh -v firstname.lastname@example.org
This will print a list of keys being tried and potential problems. You won’t get a shell at the end, as BorgBase only supports access via
borg. Once you see
Remote: Key is restricted. or
PTY allocation request failed on channel 0 then the login step still worked.
If Borg happens to be busy on the client- or server side, it may not send data over the SSH connection for a while. In this case, some ISPs will terminate the connection after a period of inactivity, especially if a NAT is involved. You would then see an error like this:
Remote: packet_write_wait: Connection to xxx.xxx.xxx.xxx: Broken pipe Connection closed by remote host
Remote: client_loop: send disconnect: Broken pipe RemoteRepository: 2.61 kB bytes sent, 1.01 MB bytes received, 52 messages sent Connection closed by remote host
Which means the SSH connection has been terminated and Borg is unable to send data to the server-side process. Some possible solutions are:
Mostly helpful for long
borg prune or
borg check operations running server-side. Have the client send regular keep-alive packets while no data is sent by Borg. On the client machine, you can add the below configuration to
Host *.repo.borgbase.com ServerAliveInterval 10 ServerAliveCountMax 30
This configuration means that the client will send a null packet every 10 seconds to keep the connection alive. If it doesn’t get a response 30 times, the connection will be closed. BorgBase already has the appropriate
ClientAliveInterval configuration applied server-side.
Newer versions of Borgmatic can automatically retry a failed command. To benefit from this feature add this to your Borgmatic config:
storage: retries: 5 retry_wait: 5
You can also use your own retry mechnism in a shell script. Borg adds checkpoint archives every 30 minutes, so your progress is preserved if a retry is needed. Keep in mind that exit code
1 means Borg encountered warnings and you probably don’t want to retry in this case, but check the logs.
If you suspect routing issues or an unstable network connection (certain residential internet connections come with restricted upload speed), you can run the below network tests: a
mtr traceroute test to uncover packet loss or
iperf3 for excessive retransmits:
Traceroute to uncover routing issues and packet loss: (takes about 15 min to complete)
$ mtr -s 1000 -r -c 1000 xxxxx.repo.borgbase.com
Network performance test to uncover excessive retransmits and measure bandwidth: (contact support to enable this first)
$ iperf3 -c xxxxx.repo.borgbase.com
All our servers are connected with 1Gbit connections at a minimum and located in professional data centers. There is no throttline based on account type. We rarely get reports of slow backup speeds. If you do encounter slower-than-expected backups or slow upload speeds, you can follow the steps below to find the bottleneck.
First it helps to understand that Borg does more than just copy files. It will also chunk them into smaller pieces, calculate hashsums, compress and encrypt each part. See here for the full details.
So the upload speed is not always the main bottleneck. Depending on your setup, you should also watch out for CPU usage or disk IO. It’s usually difficult to improve uplink speed, but if you are CPU- or IO-limited, there are a few settings you can tune. Just be aware of the trade-offs. Maybe you are OK with a slower initial backup in order to have a smaller well-compressed backup in the future. So here some steps you can take to speed you backups:
- Avoid congested times of the day: Many users set their backups to run at midnight UTC, which can lead to backups being slower at that time. On the hour tends to be more busy too. To enjoy the best speeds, set your backups to run at random times.
- See the official FAQ on steps to speed up backups using different Borg flags.
- Make sure you choose an appropriate compression level for your data. In general
lz4(fast, but low compression)
zstd,3(medium compression) and
zstd,8(high compression) will work well.
- Ensure the local files cache is working correctly. By default Borg will compare
ctime,size,inodeand process the file if either changes. If you have e.g. a network file system with unstable inodes, try using
- On Linux: Use the BBR congestion algorithm, instead of the default cubic. Users have reported a 6x speedup after making this change. This feature needs to be enabled on the sending side and works best on low bandwith connections. Clients sending from data centers will likely not see an improvement, as reported by APNIC. Add the below lines to
/etc/sysctl.confand apply with
- Avoid excessive archive checking:
borg checkcan read all backup segments and confirm their consistency. For large repos this can take a long time. BorgBase already uses different techniques to avoid bitrot in the storage backend, so
borg checkis not strictly necessary for this purpose. In Borgmatic set
consistencysection. If you still need consistency checks, consider using the
repositoryoption to limit the check to the repository. Checking all archive metadata is done on the client and very time consuming. See the official Borg docs for details.
There are rare situations, where Borg repositories can be damaged by e.g. interrupted backups, incomplete migrations, hardware failures or other factors. Symptoms include:
Object with key XXX not found in repository
Cache is newer than repository - do you have multiple, independently updated repos with same ID?
Borg is generally good at fixing those issues and missing data will be added during the next backup run.
To fix errors related to missing data or files:
- First run
borg check $REPO_URL. This will first check the repository files and then one or more archives.
- If the first step reveals errors, you should make a local repo copy, unless the data is non-critical.
- Next run
borg check --repair $REPO_URLto ask Borg to fix any errors.
- Finally run
borg create ...to do a full backup run and add potentially missing data.
To fix errors related to the cache (from here):
- First try to delete the local cache
borg delete --cache-only $REPO_URL
- If the first step doesn’t resolve the issue, try moving the security folder in
~/.config/borg/security/$REPO_ID. Where the
REPO_IDcan be found using
borg config $REPO_URL id
- Run a repo check with
borg check $REPO_URL.
Also called “delayed deletion”. In this mode, Borg will never remove old segments and instead add a new transaction for any change in a transaction log. The result is that no data is ever deleted and operations (like archive prunes- or deletions) can be undone. This is useful if the client machine shouldn’t get full access to its own backups to e.g. prevent a hacker from deleting backups after taking over a client machine. For full details and instruction on how to roll back, see the official Borg docs.
The server-side Borg process doesn’t know about the high-level commands (
borg prune) you run. It only knows about adding chunks, removing chunks and so on. So with the current architecture, it’s not possible to reject e.g. a
borg delete command right away.
As a result, all high level operations still work, but old data isn’t deleted (delayed deletion). Effectively this means that while running backups with append-only SSH keys, no disk space will be recovered in your BorgBase repo with pruning. But you can run a prune with an all access ssh key when your free quota is running low, which will then clear pruned backups and free up disk space.
With append-only mode enabled, the repository will have a timestamped transaction log. This allows going back to previous states, even if prune- or delete-commands were issued by the backup client.
If you need to restore an older repo version, you can use SFTP mode to make the necessary edits or, preferably, download a backup copy of the whole repo before doing any edits.
There are situations where removing an archive doesn’t automatically delete the data associated with it. This can have multiple reasons:
- The same data segment is still associated with another archive. E.g. if you have two archives referencing the same files and you delete one of them, the data usage will stay the same.
- You are using an append-only key. In this case the server-side Borg process will ignore all requests to actually delete any data and transactions can be rolled back. See the whole append-only section for more.
- When using Borg
1.1.x, old data is cleaned up only during a write operation to the repository. So if you changed key permissions recently, there can be a delay before the data is cleaned up.
- When using Borg
1.2.xor higher, old data is only deleted when running
borg compactor running the Compact Repo action from the BorgBase Control Panel. Before that, no disk space is released.
When using append-only mode, old transactions and segments are never cleaned from the repo and the size can grow over time. The timing and conditions of when old segments are cleaned up depends on the Borg version in use:
- Borg 1.1.x: In this version, old segments are deleted implicitely when a write operation (e.g.
borg delete) is done with a full access key.
- Borg 1.2.x: This version added a new
borg compactcommand to explicitely clean up old segments at a suitable time. You can trigger this command from our control panel under More > Compact repo from the repo table. Or run
borg compactwith a full access key directly.
What’s the difference between a SSH key, SSH password, SSH key passphrase and Borg repository passphrase?
BorgBase uses SSH keys for authentication. This means to connect to a backup repository, you need to upload the public part of a SSH keypair and then use the private part to make a connection.
The SSH key and the Borg repository passphrase are totally separate. SSH keys are only used for authentication to log into the BorgBase repo and you can easily replace it in the control panel.
It’s possible to create an encrypted SSH key that’s protected with a passphrase. We don’t recommend this for backup purposes, since it doesn’t add much security and can interfere with automation. It’s preferred to generate a SSH key without passphrase and keep it on this one machine only. If the same Borg repository is used from another machine, generate a new key there.
In addition, SSH also supports login by password. This doesn’t work well with automation and isn’t supported by BorgBase. If you are prompted for a SSH password, there is an issue with your SSH key.
The Borg repository passphrase is used for encryption and is separate from SSH. It encrypts the backup data on your machine and we can’t help you recover or replace it. You should keep it somewhere safe off the source machine, so you always have access to the backup, even if something happens to the source machine.
To summarize the terms again:
- SSH key: The private part of a keypair used to authenticate with a SSH server. Used by BorgBase for authentication when connecting to a Borg repository.
- SSH password: Another way to authenticate with a SSH server. Not used by BorgBase.
- SSH key passphrase: To encrypt a SSH key.
- Borg repository passphrase: To encrypt the data in a Borg repository
We deploy a mix of hardware- and software RAID-6 backed storage servers with NVMe-backed write caches for performance and enterprise hard drives for reliablity. Replacement drives are ready on-site for quick failovers. For a list of the providers we work with, you can also see our GDPR page.
While this setup provides good durability against most kinds of hardware failures, BorgBase doesn’t offer geographical redundancy or redundancy across multiple data centers for single repos. It’s thus not recommended to use our service for archiving purposes, where there is no other copy of the data.
If you need geographic redundancy, you can add two repos in different regions and do parallel backup to both of them. This case is, e.g. well supported by Borgmatic, which allows adding a list of repositories. Here an example configuration:
location: repositories: - ssh://email@example.com/./repo # EU-based repository nr. 1 - ssh://firstname.lastname@example.org/./repo # US-based repository nr. 2
We offer free migration services for both, incoming and outgoing transfers. Currently this is done manually. To start a transfer follow these steps:
- For incoming transfers, create an empty repo in your BorgBase account.
- Make sure your old repo data is accessible from the internet somehow, e.g. via SSH, FTP or HTTP. For SSH we will provide you with a one-time public key to use.
- Contact our support and provide the BorgBase repo ID and the login details of the transfer source or target.
BorgBase always displays the actual disk usage, as measured on the file system. This includes some metadata and index files and slight variations from the space usage reported via
borg info under All Archives > Deduplicated Size are expected.
If you see larger variations, you are probably running your repo in append-only mode. This means
borg never really deletes old segments. So the actual disk usage will be higher than what
borg thinks over time. From the docs:
As data is only appended, and nothing removed, commands like prune or delete won’t free disk space, they merely tag data as deleted in a new transaction.
If you are OK to fully remove those old segments, then just write to the repo with a full-access key. This will clean up old segments:
Be aware that as soon as you write to the repo in non-append-only mode (e.g. prune, delete or create archives from an admin machine), it will remove the deleted objects permanently
borg check command can detect issues with your repository, but running it on large repos will make your backups much slower and cause higher load on our storage servers. It’s not really necessary to run it more often, since the underlying RAID controller already does weekly data scrubs to verify the data. As a result we strongly recommend running
borg check once a month only. When using Borgmatic, this can be done like this:
- Run this cron task every day to create a new backup:
borgmatic --create --prune
- And this once a month (some random day a few hours after the backup):
Our Ansible role also supports monthly checks, when
borgmatic_large_repo is set to
True (the default).
If we notice excessive repo checks on your account, we may contact you or interrupt those processes in order to guarantee the best speed for all users.
If you have found another backup service and prefer to remove your account, you can do so any time. Doing so will remove your account data permanently. If you ever choose to start using BorgBase again, you will have to open a new account. If you ever had a paid subscription, we will still keep some invoicing data, as required by law. To remove your account:
- Remove all your repositories and make sure the data is saved elsewhere. You can also transfer your whole archive via
rsync, as described here.
- Log into your account and navigate to Account > Profile. Then click Remove Account
Sometimes it’s necessary to restart services and servers to apply security- and maintenance updates. Usually such updates are not urgent, so we schedule them in a way to minimize any service disruption, while doing them. So the maintenance window for each region is as follows:
Usually there won’t be any downtime during this maintenance window. But if restarts or maintenance are required, it will be done during that time, unless more urgent action is needed.
If you have any questions or issues regarding your BorgBase account, simply email to email@example.com any time. You will generally receive a response within a few hours during working hours and within a day otherwise.
We are also happy to help out with questions around setting up backups using the various projects in the Borg eco system, for example Borgmatic, Vorta and others on a best-effort basis. If you have a specific question, you will usually receive a speedy response. For more general questions (e.g. “How do I restore my backup?”) or feedback on open source software, it’s better to first read the related documentation, search the project’s issue tracker or – for genuine bugs – open an issue on Github. Here some links for specific projects:
- Borg Backup: Documentation, Issue Tracker
- Borgmatic: Documentation, Issue Tracker
- Vorta: Documentation, Issue Tracker
If you require additional support or integration, we can also connect you with a relevant maintainer or our parent company’s Quick Support offering.