Search This Blog

Friday, September 3, 2010

Periodically download backups of Google App Engine Datastore

Recently I came across another interesting issue. I hope that this post will be helpful for all developers who develop their software as a service applications using Google App Engine.

After creating an application I wanted to schedule Google Datastore backups to be done every evening. Apparently it was not that strait forward. There is a tool called Gaebar ( but it works only for Django projects.

Google provides mechanism for downloading and uploading data:

however I couldn't find a solution that I could use to schedule downloads. 
Below is a solution for Windows.

I have created a batch file for downloading a datastore backup:
backupGAE.bat source:


echo %PASS% | "C:\Python25\Python.exe" "c:\Program Files\Google\google_appengine\" download_data --email=%EMAIL% --passin --application=%APPID% --url= --filename=%OUTPUT_FILE%

"If given, the tool accepts the Google account password in stdin instead of prompting for it interactively. This allows you to invoke the tool from a script without putting your password on the command line."

"|" symbol causes that the output of the first program is sent to the second program.

Then I wrote an application calls the backupGAE.bat file with required parameters:
-email address
-application id
-output file

Then compresses the downloaded file and secures it with password and deletes the original file.
Because I couldn't find any freeware python zip library that allows setting passwords on new archives I have used a 7-zip command line - works very well.

The application execution is scheduled on the server using Microsoft Scheduled Tasks and after is executed send an email status to a given email address.

No comments:

Post a Comment