Periodically download backups of Google App Engine Datastore

Recently I came across another interesting issue. I hope that this post will be helpful for all developers who develop their software as a service applications using Google App Engine.


After creating an application I wanted to schedule Google Datastore backups to be done every evening. Apparently it was not that strait forward. There is a tool called Gaebar (http://aralbalkan.com) but it works only for Django projects.


Google provides mechanism for downloading and uploading data:


http://code.google.com/appengine/docs/python/tools/uploadingdata.html


however I couldn't find a solution that I could use to schedule downloads. 
Below is a solution for Windows.


I have created a batch file for downloading a datastore backup:
backupGAE.bat source:


@ECHO OFF
SET EMAIL=%1
SET PASS=%2
SET APPID=%3
SET OUTPUT_FILE=%4


echo %PASS% | "C:\Python25\Python.exe" "c:\Program Files\Google\google_appengine\appcfg.py" download_data --email=%EMAIL% --passin --application=%APPID% --url=http://%APPID%.appspot.com/remote_api --filename=%OUTPUT_FILE%


Explanation:
--passin
"If given, the tool accepts the Google account password in stdin instead of prompting for it interactively. This allows you to invoke the tool from a script without putting your password on the command line."


"|" symbol causes that the output of the first program is sent to the second program.

Then I wrote an application calls the backupGAE.bat file with required parameters:
-email address
-password
-application id
-output file


Then compresses the downloaded file and secures it with password and deletes the original file.
Because I couldn't find any freeware python zip library that allows setting passwords on new archives I have used a 7-zip command line - works very well.


The application execution is scheduled on the server using Microsoft Scheduled Tasks and after is executed send an email status to a given email address.

A solution for "No service" problem in HTC Android Phones (Hero, Legend, Desire)

Since last few months I have been using a HTC Legend phone. There was no problems with the phone and then suddenly I started having the "No network" error. The phone was dropping down the network minimum 4 times a day and it was not possible to re-register to the network. The only solution was to restart the phone or turn on and off the airplane mode. The above did happen also if when the phone was on my desk untouched so I thought the 2G/3G switching that is mentioned by many people could not really be the reason.

I have send it twice to repair without any change. Luckily it is quite easy to restore the android phone, contacts are synchronised  with your gmail and the list of all installed apps can be found in the market so it was easy to install them back.

Funny think is that Meteor Ireland and HTC are not able to work out what causes this annoying problem. As I have seen on the Meteor Talk to Us,  Meteor claims that the majority of users are perfectly happy with their HTC Android devices. Should we ask - do they just not bother to complain or they really don't have the issue.

Because I really like the Legend, I started to think why it worked before and doesn't work now. The only difference were the installed apps.
I thought the culprit may be the Advanced Task Killer, so I uninstalled it, unfortunately the problem was still there.
I observed in the past, that I had the error after installing the Nimbuzz app, then I uninstalled it and it was gone. This time there was a Rebtel app on my phone. Both application probably monitor the network connection and probably hook to the network connection process, I didn't check it but I would think they work like this. The argument for the above was also the fact that I couldn't re-register to the network without restarting the phone. I thought it was because the network connection process is gone.
My theory is that the problem is caused by a number of applications that hook to the network connection process and cause its crash.

I have uninstalled both apps and since few days I am again a happy HTC Legend user.
Of course the above is just a theory that needs to be proved, but it solved my problem so I thought I publish it so it may help to solve the problem once and for all. At the same time I am not determined to dig deeper into the technical details.  I would thing that Meteor or HTC IT guys should take it from there.

Update:
Today - 06/10/2010 - after few weeks of using the phone I installed skype and the "no service" appeared again. This is a shame I can't use skype on my phone.

09/12/2010 - after installing Froyo, issue does not exist any more.
It is just worth mentioning that after the system update it is required to select Access Point otherwise there is no mobile internet.

Error 504 in WCF.NET 3.5

In my first post I wrote that I like XML processing with DataContract and CollectionDataContract.
I still think it is not bad however few days ago a came across an interesting issue. 
My function returns an object, which is serialized automatically. It appeared that if the XML was too big (in my case about 6 MB)  I received 504.  
The problem solved by serializing inside the function and returning a Stream.
If anyone has a better solution I will be glad to hear it.

Migrating from ExtJs 3.0.0 to 3.1.0

Hi again,
Today I migrated a front-end of our web application from ExtJs 3.0.0 to 3.1.0.  ExtJs team fixed lot of bugs, added some memory improvements and few new things (you can find them all at http://www.extjs.com/products/extjs/CHANGES_ext-3.1.0.html). By the way, great job ExtJs team, it is a pleasure to work with the library.
The application worked perfectly well with 3.0.0 but because of the above reasons we decided to upgrade.
Anyway all went quite smooth however I came across few small issues:
1. On one form I have two GridPanels both having own toolbars. After I run the program with 3.1.0 all buttons (including the ones that supposed to be on a bottom toolbar) appeared  on the top toolbar.
In 3.0.0 toolbar buttons had an internal id property, DOM id property was generated by the framework, it looks like this has been changed, my buttons had ids: 'edit', 'add', 'delete'  in both situations. Giving them unique ids solved the issue.

2. TabPanel resize event seems to be called in a different moment. In some situations I create widgets and then set their size in resize of the parent TabPanel.

resize : ( Ext.Component this, Number adjWidth, Number adjHeight, Number rawWidth, Number rawHeight 

It appeared that adjWidth parameter was unassigned. Using panel.getInnerWidth() solved the problem.


 I am not sure if this is my imagination or a fact, but the front-end seems to work faster with 3.1.0

Using unmanaged Delphi dlls in .NET

What to do if we have an old working Delphi code that is quite complex and well tested and at the same time and we need to use the same functionality in a new .NET application. Let it be for example a complex encryption/decryption functions.

Of course we can rewrite them in .NET, but why to waste time if we already have something ready and tested?
The easiest way is to create a Delphi dll, that can look like:

function EncodeString (ps: PChar): PChar; export; stdcall;
var
str: AnsiString;
begin
str := ps;
str := ExistingEncodeStringFunction(str);
Result := PChar(str);
end;

function DecodeString (ps: PChar): PChar; export; stdcall;
var
str: AnsiString;
begin
str := ps;
str := ExistingDecodeStringFuncion(str);
Result := PChar(str);
end;

exports
EncodeString, DecodeString;


Now we copy out Delphi dll into bin folder of our .NET project.
We add a class method with a DllImport decorator:

[DllImport("DelphiEncryption.dll", CharSet = CharSet.Ansi)]
static extern string DecodeString(String val);

We do the same for EncodeString, and now we are happy to use our existing Delphi code in .NET.

My experience with RESTful in Windows Comunication Foundation (WCF) and .NET 3.5

I always thought that it would be nice to share my programming experience with others. I finally made myself to write a blog. So here it is...

After working with Python and CherryPy I had a pleasure to write a RESTfull webservice using WCF in Visual Studio 2008 and .NET 3.5.

At the start I have switched off sessions and wrote a custom request authentication.

I liked XML processing with DataContract and CollectionDataContract. Unfortunately it appeared that they work nice only one way (at least if you are not a fan of using schema).

I have written deserialization manually.

Routing in WCF is done based on UriTemplate, a bit oldish comparing to the way CherryPy works, but C# is not a dynamic language as Python is, so we have to turn a blind eye.

Now one more "funny" think. Please look at the example below, it works perfectly.

[OperationContract]

[WebGet(UriTemplate = "/users")]

public string GetAllUsers()

{

return "GET";

}

[WebInvoke(UriTemplate = "/users", Method = "POST")]

[OperationContract]

public string AddNewUser(Stream stream)

{

return "POST";

}

but what happens if we add QueryString paramaters in GET, like below:

[OperationContract]

[WebGet(UriTemplate = "/users?test={test}")]

public string GetAllUsers(string test)

{

return "GET";

}

[WebInvoke(UriTemplate = "/users", Method = "POST")]

[OperationContract]

public string AddNewUser(Stream stream)

{

return "POST";

}

a GET method of our WebService doesn't work any more, we get 405 StatusCode. Looks like a dispatcher is trying to call POST method instead.

A workaround is to use /users/NEW but in such case is it really a REST support?. The interesting thing is that on msdn there is no similar example of using GET and POST on the same URI.

Personally I am going to avoid WCF for writing RESTful web services in the future. There are freeware solutions much better for this purpose and when we also consider a fact that e.g Python + CherryPy don't need Windows server and IIS a choice is even easier.