NAME¶profile-sync-daemon - Symlinks and syncs browser profiles to RAM (tmpfs) thus reducing HDD/SSD calls and speeding-up browsers.
DESCRIPTION¶Profile-sync-daemon (psd) is a tiny pseudo-daemon designed to manage browser profile/profiles in tmpfs and to periodically sync back to the physical disc (HDD/SSD). This is accomplished by an innovative use of rsync to maintain synchronization between a tmpfs copy and media-bound backup of the browser profile/profiles. Additionally, psd features several crash-recovery features.
Design goals of psd:
- Completely transparent user experience.
- Reduced wear to physical discs (particularly SSDs).
Since the profile/profiles, browser cache*, etc. are relocated into tmpfs (RAM disk), the corresponding I/O associated with using the browser is also redirected from the physical disc to the RAM, thus reducing wear to the physical disc and improving browser responsiveness.
*Note that some browsers such as Chrome/Chromium, Firefox (since v21), Midori, and Rekonq actually keeps their cache directories separately from their browser profile directory. It is not within the scope of profile-sync-daemon to modify this behavior; users wishing to relocate this directory, may refer to the following url for several work-arounds: https://wiki.archlinux.org/index.php/Chromium_Tips_and_Tweaks#Cache_in_tmpfs
SETUP¶$XDG_CONFIG_HOME/psd/psd.conf (referred to hereafter as "the config file") contains all user managed settings.
NOTE: edits made to the config file while psd is active will be applied only after the service has been restarted.
- Optionally enable the use of overlayfs to improve sync speed and to use a smaller memory footprint. Do this in the USE_OVERLAYFS variable. The user will require no password sudo rights to /usr/bin/psd-overlay-helper to use this option and the kernel must support overlayfs version 22 or higher. See the FAQ below for additional details.
- Optionally define which browsers are to be managed in the BROWSERS array. If none are defined, the default is all detected browsers.
- Optionally disable the use of crash-recovery snapshots (not recommended). Do this in the USE_BACKUPS variable.
- Optionally define the number of crash-recovery snapshots to keep. Do this in the BACKUP_LIMIT variable.
NOTE: occasionally, updates/changes are made to the default config file (/usr/share/psd/psd.conf) upstream. The user copy ($XDG_CONFIG_HOME/psd/psd.conf) will need to be diffed against it.
PREVIEW MODE¶The preview option can be called to show users exactly what psd will do/is doing based on the entries in the config file. It will also provide useful information such as profile size, paths, and if any recovery snapshots have been created.
$ psd p Profile-sync-daemon on Arch Linux.
Systemd service is currently active. Systemd resync-timer is currently active. Overlayfs v23 is currently active.
browser/psname: chromium/chromium owner/group id: facade/100 sync target: /home/facade/.config/chromium tmpfs dir: /run/user/1000/facade-chromium profile size: 93M overlayfs size: 39M recovery dirs: 2 <- delete with the c option dir path/size: /home/facade/.config/chromium-backup-crashrecovery-20150117_171359 (92M) dir path/size: /home/facade/.config/chromium-backup-crashrecovery-20150119_112204 (93M) browser/psname: firefox/firefox owner/group id: facade/100 sync target: /home/facade/.mozilla/firefox/f8cv8bfu.default tmpfs dir: /run/user/1000/facade-firefox-f8cv8bfu.default profile size: 145M overlayfs size: 13M recovery dirs: none
CLEAN MODE¶The clean mode will delete ALL recovery snapshots that have accumulated. Only run it when sure these are no longer needed.
$ psd c Profile-sync-daemon on Arch Linux. Deleting 2 crashrecovery dirs for profile /home/facade/.config/chromium /home/facade/.config/chromium-backup-crashrecovery-20150117_171359 /home/facade/.config/chromium-backup-crashrecovery-20150119_112204
START AND STOP PSD¶With the release of the version 6.x series of psd, the only init system that is officially supported is systemd. Psd ships with a systemd user service to start or stop it (psd.service). Additionally, a provided resync-timer will run an hourly resync from tmpfs back to the disk. The resync-timer is started automatically with psd.service so there is no need to start the timer; only start psd.service.
$ systemctl --user [option] psd.serviceAvailable options: start stop enable disable
- Chromium (stable, beta, and dev)
- Firefox (stable, beta, and aurora)
- Firefox-trunk (this is an Ubuntu-only browser: http://www.webupd8.org/2011/05/install-firefox-nightly-from-ubuntu-ppa.html)
- Google Chrome (stable, beta, and dev)
- Heftig's version of Aurora (Arch Linux: https://bbs.archlinux.org/viewtopic.php?id=117157)
- Inox (https://bbs.archlinux.org/viewtopic.php?id=198763)
- Opera (legacy, stable, next, and developer)
NOTE ON SYMLINKED PROFILES¶Currently, psd does not support symlinked profiles and will refuse to sync if one is detected. For example, your firefox profile is ~/.mozilla/firefox/f8cv8bfu.default but you have moved that directory to /foo/bar/f8cv8bfu.default and replaced it with symlink:
$ ls -l ~/.mozilla/firefox lrwxrwxrwx 1 facade users 26 Oct 1 17:02 f8cv8bfu.default -> /foo/bar/f8cv8bfu.defaultRunning psd in preview mode will end in an error in forming you of this:
$ psd p Warning! /home/facade/.mozilla/firefox/f8cv8bfu.default appears to be a symlink but these are not supported. Please make the browser profile a live directory and try again. Exiting.A proper work around for firefox is to simply edit ~/.mozilla/firefox/profiles.ini defining the canonical path there. One also needs to adjust the IsRelative flag like so:
[Profile0] Name=default IsRelative=0 Path=/foo/bar/f8cv8bfu.defaultOther solutions may exist for others browser but documenting them all here is out of scope.
SUPPORTED DISTROS¶Since psd is just a bash script with a systemd service, it should run on any flavor of Linux running systemd. Around a dozen distros provide an official package or user-maintained option to install psd. One can also build psd from source. See the official website for available packages and installation instructions.
FAQ¶Q1: What is overlayfs mode?
A1: Overlayfs is a simple union file-system mainlined in the Linux kernel version 3.18.0. When used with psd, a reduced memory footprint and faster sync operations can be realized. The magic is in how the overlay mount only writes out data that has changed rather than the entire profile. The same recovery features psd uses in its default mode are also active when running in overlayfs mode.
See the example in the PREVIEW MODE section above which shows a system using overlayfs to illustrate the typical memory savings. Note the "overlayfs size" report compared to the total "profile size" report for each profile. Be aware that these numbers will change depending on just how much new data is written to the profile, but in common use cases, the overlayfs size will always be less than the profile size.
Q2: How do I enable overlayfs mode?
A2: First, be sure psd is not active or else any changes to the config file will be ignored until it is restarted. Overlayfs mode is enabled with the USE_OVERLAYFS= variable which should be set to "yes" in the config file. Psd will automatically detect the overlayfs version available to the kernel if it is configured to use one of them. It is recommended to run psd in preview mode to verify that the system can in fact use overlayfs (note that Debian 8 currently does not support an overlayfs capable kernel version).
Since version 6.05 of psd, users wanting to use this mode MUST have sudo rights without password prompt to /usr/bin/psd-overlay-helper or global sudo rights without password prompt. If the user does not have global rights, add the following line to /etc/sudoers after any other lines defining sudo access. It is recommended to use /usr/bin/visudo as root to set this up:
foo ALL=(ALL) NOPASSWD: /usr/bin/psd-overlay-helperQ3: Why do I have another browser profile directory "foo-back-ovfs" when I enable overlayfs?
A3: The way overlayfs works is to mount a read-only base copy (so-called lower dir) of the profile, and manage the new data on top of that. In order to avoid resyncing to the read-only file system, a copy is used instead. So using overlayfs is a trade off: faster initial sync times and less memory usage vs. disk space in the home dir.
Q4: I need more memory to accommodate my profile/profiles in /run/user/xxxx. How can I allocate more?
A4: The standard way of controlling the size of /run/user is the RuntimeDirectorySize directive in logind.conf (see the man page for logind.conf for more). By default, 10% of physical memory is used but one can increase it safely. Remember that tmpfs only consumes what is actually used; the number specified here is just a maximum allowed.
Q5: My system crashed for some reason and psd didn't sync back. What do I do?
A5: The "last good" backup of the browser profile/profiles should be happily on the filesystem. Upon restarting psd (on a reboot for example), a check is preformed to see if the symlink to the tmpfs copy of the profile is invalid. If it is invalid, psd will snapshot the "last good" backup before it rotates it back into place. This is more for a sanity check that psd did no harm and that any data loss was a function of something else.
Q6: Where can I find this snapshot?
A6: It depends on the browser. The snapshot will be located in the same directory as the browser profile and it will contain a date-time-stamp that corresponds to the time at which the recovery took place. For example, a chromium snapshot will be ~/.config/chromium-backup-crashrecovery-20130912_153310 -- of course, the date_time suffix will be different.
Q7: How can I restore the snapshot?
A7: Follow these steps:
- Stop psd.
- Move the "bad" copy of the profile to a backup (don't blindly delete anything).
- Copy the snapshot directory to the name that browser expects.
Example using chromium:
- systemctl --user stop psd.service
- mv ~/.config/chromium ~/.config/chromium-bad
- cp -a ~/.config/chromium-backup-crashrecovery-20130912_153310 ~/.config/chromium
At this point, launch chromium which will use the backup snapshot just copied into place. If all is well, it is safe to delete ~/.config/chromium-bad and the snapshot. Remember, to start psd, no browsers must be open (or psd will refuse to start).
Q8: Can psd delete the snapshots automatically?
A8: Yes, run psd with the "clean" switch to delete snapshots.
CONTRIBUTE¶Users wishing to contribute to this code, should fork and send a pull request. Source is freely available on the project page linked below.
BUGS¶Discover a bug? Please open an issue on the project page linked below.
- Currently, psd checks for running browsers before it starts/stops by their name. If a running process that happens to contain that name, it will falsely refuse to start until that process is not running. For an example, see: https://github.com/graysky2/profile-sync-daemon/issues/85
- Several cases of data loss have been reported when using eCryptFS and psd, therefore until this issue is flushed out, users of eCryptFS are encouraged not to use psd unless willing to help troubleshoot suspected browser corruption. See: https://github.com/graysky2/profile-sync-daemon/issues/158
- Project page: https://github.com/graysky2/profile-sync-daemon
- Wiki page: https://wiki.archlinux.org/index.php/Profile-sync-daemon
AUTHOR¶graysky (graysky AT archlinux DOT us)
|26 November 2016|