jump to navigation

Automatic Nightly Backups for Your Android Device to Your Computer November 1, 2012

Posted by GuySoft in Android, diy, linux, open source, Uncategorized, wifi, wireless.
Tags: , , , , ,
trackback
Android lifevest

Android backup will keep you safe

Hey all,

Friends keep coming up to me and complaining that they had to wipe all their phone because of some attempt to do something.
However, if you give me a brand-new phone and throw mine out of a high-building window, I won’t stop you, because I have a copy of my phone at home, from that same morning, and I don’t even have to think about it.

Want to feel free again? Not scared that this small thing you carry in your pocket with all your life gets ruined? Well, here is how you do it:

Things you will need

  1. A computer running Linux you can ssh to with Rsync installed, or windows running cygwin with rsync and ssh installed.
  2. Your device needs to be rooted – I am afraid thats a requirement for Titanium backup and all programs that back up app data
  3. Titanium backup – You can use the free version for backing up, but for a single-click restore you will need to pay (I bought it and recommended you do even if its the only thing you ever buy on the play store)
  4. rsync backup for Android – This will back up all your photos and data that isn’t part of your apps, if you don’t root but this is all you want to back up, this post might help you too!
  5. Llama – lets you set location profiles

What to do

  1. On titanium backup, open the “schedules” tab and set up a “Backup all user apps + system data” schedule and tick all the days you want your apps to backup on. Pick a time when you are likely be asleep and not using the phone, for this post lets assume its 6am.
    Titanium Backup, create the backup all apps and data sechedule

    Titanium Backup, create the backup all apps and data schedule

    Titanium Backup, Set the schedule

    Titanium Backup, Set the schedule

  2. On rsync backup for android,
    1. Tap the meny key and select “get binaries
    2. Tap the menu key and select “Generate keys“, this will create for you two files, one in /sdcard/dss_key and another in /sdcard/dss_key.pub (don’t hand your friends the dss_key, its a private key).

      Rsync for android's menu

      Rsync for android’s menu

    3. Tap the menu key and select “Add Profile” and call it “Backup All”, local file or directory should point to /sdcard (or you top-level of all your memory cards).
    4. Set usename to the username on the computer you are doing the ssh to.
    5. Server (IP or hostname) should point to the IP of your PC.
    6. Dropbear SSH private key should point to /sdcard/dss_key
    7. Remote file or directory should point to a folder on your PC (make sure you mkdir -p its path, and that you have writing permissions!).
    8. In Additional rsync options write: -a --exclude external_sd/.android_secure --progress
      Update – on my new Galaxy S4 I had to include:
      -a --exclude external_sd/.android_secure --exclude emulated/0 --exclude UsbDriveA --exclude UsbDriveB --exclude UsbDriveC --exclude UsbDriveD --exclude UsbDriveE --exclude UsbDriveF --progress
    9. You can check “Close log window after job if done” so you know in the morning it ran, check “delete remote files that are no longer in source directory. Leave “rsync in reverse” and “send notification using remove notifier” unchekced. Save the new profile.
    Rsync 4 android profile page 1

    Rsync 4 android profile page 1

    Rsync 4 android profile page 2

    Rsync 4 android profile page 2

  3.  On your computer,
    1. You should have ssh and rsync installed, on ubuntu/debian do apt-get install apt-get install rsync openssh-server
    2.  Paste the line from the file /sdcard/dss_key.pub on your phone to ~/.ssh/authorized_keys file on your PC ( Note: it begins with “ssh-dss” and ends with “android@bender”, this is a single line).
    3. At this point I suggest you go to rsync 4 android and tap on the “Backup all” profile you made, make sure that indeed files are getting backed up to your folder, as seen in the picture below. The first sync takes a while, afterwards only changes would be synced. If it does not work, go over the steps above, or comment here and we can try and troubleshoot.

    Rsync for android running

    Rsync for android running

  4. On Llama, (on first use accept the boring stuff) in the “Events” tab delete all the profiles, you don’t need them (if you do want to use Llama for other things feel free to keep them).
    1. Add. New event, name it ‘backup my phone” or something like that.
    2. Add in your new event a condition between 7:00 – 7:30 (or an hour later from when you set titanium to back up)
    3. Add a condition “WiFi Network Connected” and point it to your home wifi, that way it would only work if you are at home, if you have a problem with this condition you can try and set your own way of detecting you are at home
    4. Add a condition Battery Level and set it to “Battery raises above…” and 20%, this makes sure your phone won’t run out of battery at night if unplugged.
    5. Add action “Locale plugin”and select “rsync backup for Android” then pick “Backup All” you created before.

      Llama events setup tab

      Llama events setup tab

    Thats it! Your phone should back up all its apps, and all the information on the sd cards nightly.

    I would love to hear your thoughts and comments :-)

    Links for more information

Comments»

1. Mike - February 5, 2013

Works great, thanks. Is it possible to backup to an already-running (remote) rsync daemon (NICE’d accordingly etc.) via SSH rather than opening a new rsync process on the remote host (which seems to be how this method works)?

2. lapo - February 15, 2013

Using manual “–del” instead of “–delete-after” button it’s possible to use incremental file list (less memory and starts faster).

GuySoft - February 15, 2013

However del might be unsafe if your sync didn’t reach the end on that critical day when things go wrong. But it indeed would speed things

3. Jason - March 2, 2013

I keep getting the following error:
/data/data/eu.kowalczuk.rsync4android/files/ssh: Failed to open /data/.ssh/known_hosts

Host ‘192.168.1.10’ key accepted unconditionally.
(fingerprint md5 yadda yadda yadda)
Connection error!

Rsync seems to want me to have an established ssh connection prior to attempting an rsync. Do you know a remedy to this?

GuySoft - March 4, 2013

Hmm, no mention of that error on the net.
Are you sure you set correct permissions:
“chmod 700″ for .ssh and “chmod 600″ for authorized_keys ?

Jason - March 7, 2013

I got the connection working after fixing my ssh keys. But I cannot get away from the following message:

/data/data/eu.kowalczuk.rsync4android/files/ssh: Failed to open /data/.ssh/known_hosts

This doesn’t stop the rsync but is only a nuisance.

4. George - March 26, 2013

If I check the “Close log window after job runs” box, how can I take a look at the log file? Is this kept anywhere? Thanks.

5. Istvan - June 7, 2013

I cut/pasted dss_key.pub onto the target server ~/.ssh/authorized_keys, but all I get is
Host ‘hostname key accepted unconditionally’
(fingerprint …)
Connection error!

I tried it both letting rsync send the dss_key in an email, and cut/past the content of dss_key.pub.
The strange thing is that the two keys are different. When sending it in email, the key contains ‘rsync@android’ at the end of line, wherelse the dss_key.pub on the android device contains ‘app_140@localhost’.

But alas, whichever I cut/past into the target host authorized_keys, it does not work.

I use the domain name of the target, because I need to use it from outside my home network, but the port 22 works well, I tested it with connectbot ssh client.

Thanks for any help,
Istvan

GuySoft - June 7, 2013

Hey Istvan,
First, know the difference between a public key and a private key. You need the public key on your server, and the private key set on your phone.
When you cut and paste, make sure you do not introduce any characters (like line endings or spaces).
Also, BEFORE you try things as your domain name on the network, try and set this all locally, you are introducing places where things can go wrong. After you set it up locally, you can clone the solution to your server. Does the setting work locally for you?

6. Istvan - June 9, 2013

Thanks, GuySoft,
I double checked the keys and found I used the public and private keys the right way. But after your post, I tried it within my lan, only to find that it works like charm. Apparently, I had something wrong with the profile that I created for the wan acess.
So I deleted my old wan profile, duplicated the profile that works with lan, and changed the local host name to whatever.dyndns.org.
It works just the way it is meant to work now.
Very exciting. Now I have a perfect, predictable tool to get rid of the half-a-dozen vague, cloud based sync.
Very good work, thanks again.
Istvan

7. Shah Dil Awan - June 30, 2013

As my experience Using manual “–del” instead of “–delete-after” button it’s possible to use incremental file list

8. Pawan Nahar - September 4, 2013

Hi there!
I am using “ap-hotspot” package to create hotspot from my Ubuntu 12.04, where on eth0, my computer have IP of 9.x.x.x series and on wlan0 my computer has IP 192.168.150.1 . My phone which is connected to the hotspot has IP 192.168.150.9. I am able to SSH to both the IPs from my phone by user “pawan”.
After generating keys, dss_key.pub on phone ends with “u0_a257@localhost” and the one which the app asks to share ends with “rsync@android”. I added the public key ending with “rsync@android” to ~/.ssh/authorized_keys. I have created the profile as below:

Profile name: BackupAll
Local file: /sdcard1
Username: pawan
Server: 192.168.150.1 Port: 22
Dropbear SSH: /sdcard/dss_key
Remote file: /home/pawan/rsync
Additional rsync options: -a –exclude external_sd/.android_secure –progress
Only “Delete remote files …” is checked

After saving profile, when I tap on the profile, it shows a pop up as “Connection error!” and log shows:

Host ‘192.168.150.1’ key accepted unconditionally.
(fingerprint md5 d3:af:96:……………………..)
Connection error!

I have no clue whats wrong here. I have rsync installed on my computer. Please help :(

9. LaLa Bird - October 8, 2013

Kind of stupid question: how do I delete or edit an already existing profile in rsync for Android app, I did it wrong first time, works great second time but now I have two profiles and I want to get rid of one of them and also to exit the profile if something changes, please help.

10. Ari - December 3, 2013

What a great howto with detailed instructions! Many thanks for the author!

However, I was wondering if the Titanium backup should really do new backups every night as there is option to do “Backup new user+system apps & newer versions” ? Also I would like to inquire what value the author has in Titanium backup preference “Max backup history” ?

Thanks once more for this amazing howto which adds up great pieces of software and moulds them to one great backup process!

GuySoft - December 3, 2013

Hey,
You can change the frequency and what you would like to backup. I just picked the one that would give me the most complete backup.

Ari - December 5, 2013

I have now pondered the issue and came up with following process:

1) First the phone makes backups.

TitaniumBackup -> extSdCard/backup (5 backup versions kept)

1.50 AM Wed and Sun: Backup all user apps + system data

1.50 AM Mon, Tue, Thu, Fri, Sat: Backup new user+system apps & newer versions

2) Phone wakes up the computer with Llama / Locale plugin ‘WoL Wake On Lan Wan’ between 03.55 and 04.05 AM if connected to our WLAN.

(WoL, enabled from BIOS and with ethtool make sure the Wake on Lan is enabled, ethtool -s eth0 wol g)

3) Phone makes rsync to computer with the rsync plugin presented in the howto (between 4.10 and 5.10)

11. steviehs - January 8, 2014

It seems that Titanium Backup 6.1.5.2 kills Llama while backupping. Filtering of Llama was no success. So unfortunately after backup, Llama 1.2013.08.29.2128 is not up after backup to start rsync… all on Android 4.4

Any ideas how to fix this?

GuySoft - January 8, 2014

When you say “filtering off” you mean excluding Llama from the backup, correct?
Because that’s the only thing I can think of off the top of my head.
It seems like the reason titanum back is killing Llama is because its trying to back it up.

steviehs - January 8, 2014

Exactly…
BTW: when I start tb manually in batch mode, there is an option to leave out active applications, but I do not see this option in scheduled mode…

GuySoft - January 8, 2014

Sounds like it would be a good idea to contact the titanium team, and link them to this. We are paying customers after allnamd this is definitely a regression

12. Jay - January 20, 2014

Love it!

But the problem I’m having is trying to exclude thumbnails and cache stuff which is a waste of time. Going from the general rsync man page I came up with this addition to the options:

–exclude */.thumbnails/
–exclude *cache*
–exclude *CACHE*

but it fails with an error:

rsync: Failed to exec ssh: Permission denied (13)
rsync error: error in IPC code (code 14) at /home/sammael/src/jni/pipe.c(84) [sender=3.0.8]
rsync: connection unexpectedly closed (0 bytes received so far) [sender]
rsync error: error in rsync protocol data stream (code 12) at /home/sammael/src/jni/io.c(605) [sender=3.0.8]

If I merely remove those –exclude options everything works fine.

Please advise. Does your rsync handle –exclude?
Is there an alternate way to handle this?
Thanks.

GuySoft - February 15, 2014

Yes rsync does accept exclude.
I just moved to Galaxy S4 and use these settings, works like a charm:
-a --exclude external_sd/.android_secure --exclude emulated/0 --exclude UsbDriveA --exclude UsbDriveB --exclude UsbDriveC --exclude UsbDriveD --exclude UsbDriveE --exclude UsbDriveF --progress

Jay - February 16, 2014

Thanks for the reply, although you need to supply each directory single.

What I did find that worked is:

-vHrtD –chmod=Du+rwx,go-rwx,Fu+rw,go-rw –exclude=**/.thumbnails/*** –exclude=**/[Cc][Aa][Cc][Hh][Ee]***

It vastly reduces the noise that gets downloaded.

13. Ivar - January 28, 2014

I get the following error:

Host ‘host’ key accepted unconditionally.
(fingerprint md5 xy:xy:xy…)
Connection error!

Followed the steps to the letter and I believe I got the key right. Any tips?

GuySoft - January 28, 2014

I had a problem like that with a router that gave me problems. That router worked perfectly except for this. That’s the only time I saw this.

scholle - January 29, 2014

As Guysoft said: you have to check the “chmod 700″ for .ssh and “chmod 600″ for authorized_keys AND
check the directory/file rights for the user AND the path of the .ssh had to be in the home directory of the user (check the /etc/ssh/sshd.config for that too)!

Ivar - February 1, 2014

Oh well, this is embarrassing. There was no authorized_keys in the .ssh directory, so I made a directory named authorized_keys and put the dss_key.pub file there. When I tried using rsync on another server, I realized that authorized_keys is a file, not a directory and that I had not read the guide properly. I got it working now. Thank you both for helping.

14. Ivar - January 28, 2014

I am using an Asus RT-N66U router. I tried connecting the phone and the server to an older Topcom router, but I got the same error.

15. pepo - February 4, 2014

Hi,
1st: what if I need two phones to make backup with the same PC. How it would look like in authorized_keys file. Or what to do on the 2nd phone?
2nd question: What is the option if I just need to move the photos from DCIM to move to PC. (in that case I need not just copy, but really delete the source after copy)

Ivar - February 4, 2014

1. Each public key on their own line. If you only have the two public keys from your phones in the authorized_keys file, it will be two lines.

2. I believe what you are looking for is the –delete-source-files option

Ivar - February 4, 2014

–remove-source-files, not delete.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: