sourceA while ago I was writing
some sample applications (
source) for
Google App Engine. I noted the things that can be useful for other GAE programmers.
I used Google's
webapp framework, my code here is using it.
Please take a look at the
shell application, it can help you test simple code.
How to dynamically get application name and version?
This question was
asked before. You can use
os.getcwd() or
os.environ['PATH_TRANSLATED'].
>>> os.getcwd()
'/base/data/home/apps/shell/1.21'
>>> os.getcwd().split('/')[-2]
'shell'
>>> os.getcwd().split('/')[-1]
'1.21'
>>> os.environ['PATH_TRANSLATED']
'/base/data/home/apps/shell/1.21/shell.py'
>>> os.environ['PATH_TRANSLATED'].split('/')[-3]
'shell'
How to identify current host?
There's a very interesting file that should be unique for every server:
>>> open('/base/python_dist/search.config').read()
'datapath .\nsorttempdir .\ndisk /export/hdc3/borgletdata/dirs/prod-appengine.\
mpm_python_dist_v12.apphosting.77627982/bigfiledata/466024'
>>> open('/base/python_dist/search.config').read()
'datapath .\nsorttempdir .\ndisk /export/hdc3/borgletdata/dirs/prod-appengine.\
mpm_python_dist_v12.apphosting.77627739/bigfiledata/465336'
You can identify the machine on which the process is deployed by using hash based on this file. Something like that:
def get_server_id():
try:
fd = open('/base/python_dist/search.config')
data = fd.read()
fd.close()
except IOError:
return 'unknown'
return '%s' % data.__hash__()
Google doesn't inform you on how many machines your application is going to be deployed (this probably depends on the traffic your site generates). But you can add this
server_id to your site footer. Than you can do multiple wget's to know on how many unique machines your app is being deployed.
$ for i in `seq 20`; do
curl -s http://cometchat.appspot.com|\
grep server_id; \
done |sort -n|uniq -c
20 server_id: '7341146770217830363'
It seems that my app is deployed on only one server.
How to identify current process?
Yet again, how many processes with your app are deployed? This time a trick with global variable:
the_process_global = "something"
def get_process_id():
return '%s' % id(the_process_global)
Now I know that my application is deployed using two processes:
$ for i in `seq 20`; do
curl -s http://cometchat.appspot.com|\
grep _id;
done |sort -n|uniq -c
13 process_id: '12457625149327067176'
7 process_id: '3996238433791648184'
Are we on production or development server?
I use this snippet:
if os.environ.get('SERVER_SOFTWARE','').startswith('Devel'):
HOST='local'
elif os.environ.get('SERVER_SOFTWARE','').startswith('Goog'):
HOST='google'
else:
# logging.error('Unknown server. Production/development?')
HOST='unknown'
Captcha on GAE?
Joscha Feth wrote
tutorial about using reCaptcha on GAE.
Cookies?
Google suggests that
request and
response objects follow the
WebOb interfaces. This works for getting cookies from
request:
username = self.request.cookies.get('username', '')
Unfortunately you can't use WebOb method
response.set_cookie. But you can set cookies by hand:
self.response.headers.add_header(
'Set-Cookie',
'username=%s; expires=Fri, 31-Dec-2020 23:59:59 GMT' \
% username.encode())
You can find some
other hints on
google-app-engine discussions. I don't know if cookies work from
django-helper.
Debugging datastore access
I created very
simple datastore debugger. It appends some debugging info to the footer of generated page. To use it you must just change your classes to inherit from
debug.DebugMiddleware instead of
webapp.RequestHandler.
For example:
class List(debug.DebugMiddleware):
def get(self):
... blabla ...
Sample footer can look like that:
**** Request took: 830ms/170ms (real time/cpu time)
**** GQLs, datastore accessed 1 times.
98ms GQL app: ":self"
kind: "Image"
Order {
property: "modified"
direction: 2
}
args: (50,) {}
This GQL log was caused by the code:
ims = Image.all().order("-modified").fetch(50)
Yet another example of output footer:
**** Request took: 150ms/130ms (real time/cpu time)
**** GQLs, datastore accessed xx times.
219ms PUT ({'full':...
178ms PUT ({'full':...
6ms GET ([datastore_types.Key.from_path('Image', 350L, _app=u'srv')],) {}
2ms GET ([datastore_types.Key.from_path('Image', 349L, _app=u'srv')],) {}
2ms GET ([datastore_types.Key.from_path('Image', 348L, _app=u'srv')],) {}
This datastore debugger can be easily modified to be used as Django middleware.
Dynamic images uploading
This is the code I use. The template:
<form action="." method="post" enctype="multipart/form-data">
<label>File: </label><input name="file" type="file"><br />
<input type="submit">
</form>
Server side:
class Image(db.Model):
name = db.StringProperty()
content = db.BlobProperty()
class UploadImage(webapp.RequestHandler):
def post(self):
if 'file' not in self.request.POST:
self.error(400)
self.response.out.write("file not specified!")
return
if (self.request.POST.get('file', None) is None or
not self.request.POST.get('file', None).filename):
self.error(400)
self.response.out.write("file not specified!")
return
file_data = self.request.POST.get('file').file.read()
file_name = self.request.POST.get('file').filename
im = Image()
im.name = file_name
im.content = file_data
im.save()
self.response.out.write("image %r saved." % im.name)
How to get image size and type
Tj9991 found an implementation of function
getImageInfo that can extract image size without any external libraries. The usage is straightforward:
content_type, width, height = getImageInfo(im.content)
Dynamic images serving
There's
an article about this topic in the official docs. Here's my non-optimal code:
class ServeImage(webapp.RequestHandler):
def get(self, key):
im = db.get(db.Key(key))
if not im:
self.error(404)
return
content_type, width, height = getImageInfo(im.content)
self.response.headers.add_header("Expires", "Thu, 01 Dec 2014 16:00:00 GMT")
self.response.headers["Content-Type"] = content_type
self.response.out.write(im.content)
Image resizing
Google doesn't support image converting libraries like
PiL. You have to convert images using some external services. You need to upload your data somewhere far from GAE and then somehow get the resized image. Especially for this
I created a service (which is not the-most-stable way unfortunately). You can try
other people methods as well.
Is comet/http-push/long polling supported by GAE?
No, but keep reading. You could try to do normal
polling. For example by loading ajax data every second. But the GAE resources are limited, there are only 650k requests/day available. This limit is going to be reached with only 8 constant users for 24 hours (using ajax polling every second).
I created external service that allow you to use comet techniques from GAE.
You can also take a look at my
some sample applications that use my external services (
source).
