Syncthing
Syncthing is an open-source file synchronization client/server application, written in Go, implementing its own, equally free Block Exchange Protocol. All transit communications between syncthing nodes are encrypted using TLS, and all nodes are uniquely identified with cryptographic certificates.
Installation
Install the syncthing package.
Syncthing provides a #Web-GUI for control and monitoring. GUI wrappers like #Syncthing-GTK and #Syncthing Tray (provided in separate packages) also exist.
Running Syncthing
Starting Syncthing
Run the syncthing binary manually from a terminal. The multiple optional parameters are described in syncthing(1).
Autostarting Syncthing
Syncthing can either be installed as a systemd system-wide service or as a systemd user service to run automatically at startup.
System service
Running Syncthing as a system service ensures that it is running at startup even if the user has no active session, it is intended to be used on a server.
Enable and start the syncthing@myusername.service where myusername is the actual name of the Syncthing user.
useradd -r) then ensure that the user has a valid home directory otherwise the service will immediately fail. Syncthing attempts to put configuration files into $HOME/.config/syncthingUser service
Running Syncthing as a systemd user service ensures that Syncthing only starts after the user has logged into the system (e.g., via the graphical login screen, or ssh). Thus, the user service is intended to be used on a (multiuser) desktop computer. To use the user service, start/enable the user unit syncthing.service (i.e. with the --user flag).
Syncthing systemd user service can be started after a specific (optionally encrypted) device has been mounted and stopped when device has been unmounted. To create a user service dependent on a mount point, after device has been mounted, systemd mount name has to be identified by running systemctl list-units -t mount command. Afterwards, create a new service similar to the one below:
/home/$USER/.config/systemd/user/syncthing.service
[Unit] Description=Syncthing BindsTo=run-media-user-and-hash.mount [Service] ExecStart=/usr/bin/syncthing [Install] WantedBy=run-media-user-and-hash.mount
Syncthing-GTK
syncthing-gtk-python3AUR provides a GTK graphical user interface, desktop notifications and integration with the file managers Nautilus, Nemo and Caja. Syncthing can be launched by Syncthing-GTK: use the interface settings to run syncthing-gtk at startup, and to state whether to launch the syncthing daemon.
Web-GUI
Syncthing provides a web interface accessible by default on http://localhost:8384.
Syncthing Tray
syncthingtrayAUR complements the Web-GUI by providing a Qt-based system tray icon and desktop notifications. There exists a desktop environment neutral version and a plasmoid for Plasma 5. It also provides integration with systemd and the Dolphin file manager.
The packages also comes with the syncthingctl utility which allows to interact with Syncthing from the command line.
Configuration
After installation, Syncthing already has a proper start-up configuration. New servers and/or folders can be added by visiting the web interface. For detailed instructions on how to setup a simple network, read Syncthing's getting started.
After a successful first start, a default repository at ~/Sync is created. You can see this in the web admin interface. On the right is the list of nodes you have added. On the left is the list of repositories, which are folders you can choose to share with other nodes.
To add another node, click "Add Node" underneath the list of nodes. You will be prompted for their Node ID (which can be found on the other machine by clicking Edit > Show ID) as well as a short name and the address.
If you specify "dynamic" for the address, the syncthing announce server will be used to automatically exchange addresses between nodes. If you want to know more about Node IDs, including the cryptographic implications, you can read the appropriate Syncthing documentation page.
After saving the configuration, the syncthing server restarts automatically. Next, you can either change the configuration of the default node (click its name and then Edit), or create a new one to share data with. Simply tick the node you wish to share the data with, and they will have permission to access it.
Local network setup
In the typical case several machines share a LAN (Local Area Network) behind a NAT (Network Address Translation) router, it is advised for a versatile configuration to:
- Activate both local and global discovery on each node. This will allow discovery in all situations, including if some of the nodes are mobile devices like laptops or Android phones, and leave the LAN and connect to the internet from the outside. This way they will still be found with global discovery.
- Use a different listen address port for each machine, like
tcp://:22010,tcp://:22011,tcp://:22012and so forth. This will differentiate the nodes on the global discovery servers and avoid the "Connected to myself - should not happen" message on the other local devices whenever they leave the LAN.
- If running multiple instances for different users on the same machine, set a different port for each user's localAnnouncePort (IPv4 broadcasts) as to avoid Syncthing complaints and choose the same localAnnounceMCAddr (IPv6 multicasts) as to find other devices on the LAN without global discovery (see Options Element).
- If two instances on the same machine should find each other without global discovery, add
tcp://127.0.0.1:xxxxxas device's second address, e.g.,tcp://127.0.0.1:22001andtcp://127.0.0.1:22002(see Device Element).
- Enable if possible UPnP port forwarding or manually forward each port to the right machine on the LAN. When a new node is discovered, Syncthing tries to use its configured listening port, 22000 by default. If this port happens to be closed, it will seek another port locally: whenever NAT traversal is enabled in Syncthing, it will attempt to use UPnP to map a random external port to the internal listening port chosen, for example 22000. If UPnP is not supported or if this is not desirable, each port should be manually forwarded to the right machine on the LAN. Eventually, if no open port can be found on both sides, relaying will be used.
Using inotify
inotify (inode notify) is a Linux kernel subsystem that acts to extend filesystems to notice changes to the filesystem, and report those changes to applications. Syncthing supports inotify and the functionality can be enabled in the configuration menu for individual folders.
Participate in the infrastructure
One can participate in the Syncthing infrastructure by running a global discovery server or a relay server.
Running a relay
Syncthing has the ability to connect two devices via a relay when it is not possible to establish a direct connection between them. Relayed connections are end-to-end encrypted in the usual manner, so the relay has no insight into the connection other than the knowledge of the IP addresses and device IDs.
Anyone can run a relay server and it will automatically join the Syncthing relay pool and be available to all Syncthing's users. To run your own relay, install syncthing-relaysrv and Start/Enable syncthing-relaysrv.service. Rate limiting and other options can be configured via the command line. These options can be set in the ExecStart directive of the service drop-in file as follows:
/etc/systemd/system/syncthing-relaysrv.service.d/override.conf
[Service] ExecStart= ExecStart=/usr/bin/syncthing-relaysrv -global-rate 500000 -provided-by relayprovidername
-listen and -status-srv options if necessary. Running a discovery server
Global discovery is used by Syncthing to find peers on the internet. Any device announces itself at startup to the discovery server which stores the device ID, IP address, port and current time. Then on request, for a given device ID, it returns the information stored in JSON format, for instance.
As an example, the request https://discovery.syncthing.net/?device=ITZRNXE-YNROGBZ-HXTH5P7-VK5NYE5-QHRQGE2-7JQ6VNJ-KZUEDIU-5PPR5AM returns {"seen":"2020-02-29T14:56:08.34589801Z","addresses":["quic://212.121.228.172:22000","tcp://212.121.228.172:22000"]} .
A list of public of global discovery server is provided. In addition, anyone can run a discovery server, to run your own, install the syncthing-discosrvAUR package.
The discovery server requires certificates to run, which should ideally be placed in /var/discosrv. The user/group syncthing needs permissions to be able to read the certificate files. You need to edit the systemd unit file to correctly point to the certificates (and to undertake any other configuration change you may want, see list).
syncthing-discosrv.service
[Unit] Description=Syncthing discovery server After=network.target [Service] User=syncthing Group=syncthing ExecStart=/usr/bin/syncthing-discosrv -db-dsn /var/discosrv/discosrv.db -cert /var/discosrv/cert.pem -key /var/discosrv/key.pem Restart=on-failure SuccessExitStatus=2 PrivateDevices=true ProtectSystem=full ProtectHome=true NoNewPrivileges=true [Install] WantedBy=multi-user.target
To point the client to your discovery server, change the Global Discovery Servers variable under Settings to https://yourserver:8443/ (default port) or whatever port you have reconfigured to. The variable takes a comma-separated list of discovery servers. It is possible to include multiple ones, including the default one.
If you are using self-signed certificates, the client refuses to connect unless you append the discovery server ID to its domain. The ID is printed to stdout upon launching the discovery server. Amend the Global Discovery Servers entry to add the ID: https://yourserver.com:8443/?id=AAAAAAA-BBBBBBB-CCCCCCC-DDDDDDD-EEEEEEE-FFFFFFF-GGGGGGG-HHHHHHH.
Tips and tricks
Stop journal spam
Syncthing can be quite noisy even while it is not doing anything. The service ExecStart can be overridden to filter output directly without an extra script (adjust "grep" as needed):
/etc/systemd/system/[email protected]/nospam.conf
[Service] ExecStart= ExecStart=/bin/bash -c 'set -o pipefail; /usr/bin/syncthing -no-browser -no-restart -logflags=0 | grep -v "INFO: "'
Run in VirtualBox
It is possible to have Syncthing connect both locally and globally within a VirtualBox virtual machine (VM) while keeping its network adapter in the standard NAT mode (as opposed to bridged networking attached to the host computer's adapter).
To enable this mode, Syncthing should listen to a port in the VM different from the listening port already used by the host.
For example, if the default 22000 port is used by the host, one could use 22001 in the VM.
The listening port in the VM can be changed through Syncthing's Sync Protocol Listen Addresses to tcp://:22001 in the GUI Settings.
The 22001/TCP port of the host must be forwarded to the guest in this configuration. This can be done with the following command:
$ VBoxManage modifyvm myvmname --natpf1 "syncthing,tcp,,22001,,22001"
In this setup, relaying should not be necessary: local devices can connect to the VM on port 22001 while global devices are accessible as long as they have themselves an open port.
Running through a proxy
Syncthing can be run through a proxy to enable use behind a corporate firewall or tunneling via SSH. According to the using proxies documentation it is necessary to set the all_proxy environment variable, and it must indicate a socks5 proxy type.
- If the service is run from a script or from the command line, you must set the variables beforehand as follows:
export all_proxy="socks5://proxy_address:proxy_port" export no_proxy="127.0.0.1"
- If it is run as a service, you must define the variables in the service configuration as follows:
/etc/systemd/system/syncthing@myusername.service.d/override.conf
[Service] Environment="all_proxy=socks5://proxy_address:proxy_port" Environment="no_proxy=127.0.0.1"
You must then do a daemon-reload and restart the syncthing@myusername.service.
This file can be edited using systemd on the syncthing@myusername.service according to the systemd#Editing provided units section.
Syncthing FUSE
SyncthingFUSE is FUSE driver which allows to mount syncthing share without actualy syncing it to local storage.
When you open a file, the contents are served from a local cache, if possible. If the contents are not in the cache, then SyncthingFUSE asks peers for the contents and adds them to the cache. The local cache will not grow larger than a fixed size, though. If no peers are currently available for the file, then opening the file will fail.
Troubleshooting
Database issue
One may encounter database issue at some stage. To force a rescan of files and resync of database use the following command:
$ syncthing -reset-database
read-only file system error on /etc although run as root
In case Syncthing complains it is a read-only file system although the user (e.g. root on /etc) has write permissions, check the template unit's definition:
$ systemctl cat [email protected]
Within the [Service] part, there is a Hardening part and below that, there is a ProtectSystem directive which is set to full by default. See systemd.exec(5) § SANDBOXING for more information on this directive.
Create a drop-in file to override the value to something that suits your needs. If you are trying to sync a sub-folder of /etc, ProtectSystem=true should do the trick.
Others
See Debugging Syncthing.