#.think.in
learn.create.enjoy

Geocoding and Maps without JavaScript

March 15, 2009 11:00 by tarn

Since playing with JQuery I've started thinking more about unobtrusive JavaScript and graceful degradation. Despite less than 5% of users browsing without JavaScript enabled, I found it very interesting disabling JavaScript on my Browser and checking out which sites still work and which ones fail. I found the Microsoft Live Search Maps has absolutely no clue your browsing the site without JavaScript, it just renders a loading spinning GIF.

Without JavaScript Enabled With JavaScript Enabled
image image

Google maps performs much better, providing a rich client side library with JavaScript enabled and a functional mapping service without it.

Without JavaScript Enabled With JavaScript Enabled
image image

There are also plenty of sites that lose some or all of their functionality, but let you know why. YouTube can't play videos, but you can browse them and it provides an message indicating why the movies don't work.

Hello, you either have JavaScript turned off or an old version of Adobe's Flash Player. Get the latest Flash player.

StackOverflow has graceful degradation implemented well. You can still ask and answer questions but there is a banner at the top of the page

Stack Overflow works best with JavaScript enabled

You then can't expand comments and you don't get the cool Ajax features that indicate when you've earned a badge or someone else has answered the question your currently answering, but its still pretty functional.

Geocoding without JavaScript

It turns out the good guys at Google have provided a REST API to do geocoding without using their fantastic client side JavaScript libraries. Its all documented in the Geocoding API Developer's Guide.

Google do state this in their terms for the REST geocoding service;

This service is designed for geocoding static (known) addresses using a REST interface, for placement of application content on a map. For dynamic geocoding of user-defined addresses (for example, within a user interface element), consult the documentation for the JavaScript Client Geocoder or the Maps API for Flash Client Geocoder.

I don't think this is a huge problem as a well designed website will only use this geocoding service for the 5% of users who don't have Javascript enabled. If JavaScript is enabled, the DOM can be updated using the rich JavaScript functionality.

I wrote a little script to wrap around the service, I wrote the script in Python as I'm hoping to use it on Google App Engine, but more about that later. To use this script you'll need to sign up to get a Google Maps API key and use it to set the GeoCoder.geoCodeKey property.

import urllib2
from xml.dom import minidom
class GeoCoder():
def __init__(self):
self.geoCodeUrl = 'http://maps.google.com/maps/geo?q=%(address)s&output=xml&oe=utf8&sensor=false&key=%(key)s'        
self.geoCodeKey = ''
def GetUrlKey(self,address):
return self.geoCodeUrl % {'key':self.geoCodeKey,'address':urllib2.quote(address)}
def GetKml(self, address):
response = urllib2.urlopen(self.GetUrlKey(address))
return response.read();
class KmlParser():
def GetText(self,node,nodeName):
return node.getElementsByTagName(nodeName)[0].childNodes[0].data
def GetCoordinates(self,node):
items = self.GetText(node, "coordinates").split(',')
return { 'latitude':items[0], 'longitude':items[1] }
def ParseAddress(self, node):
address = { 'address':self.GetText(node,'address'),
'country':self.GetText(node,'CountryName'),
'street':self.GetText(node,'ThoroughfareName'),
'postcode':self.GetText(node,'PostalCodeNumber'),
'coordinates': self.GetCoordinates(node) }
return address 
def Parse(self, xml):
results = []
dom = minidom.parseString(xml)
for placeNode in dom.getElementsByTagName('Placemark'):
results.append(self.ParseAddress(placeNode))
dom.unlink()
return results;
def GetAddresses(address):
return KmlParser().Parse(GeoCoder().GetKml(address))
if __name__ == '__main__':
import sys
if len(sys.argv) > 1:
for address in GetAddresses(' '.join(sys.argv[1:])):
print address
else:
print "Enter Address to Geocode, using the Google Geocoding API"
from xml.dom import minidom
class GeoCoder():
def __init__(self):
self.geoCodeUrl = 'http://maps.google.com/maps/geo?q=%(address)s&output=xml&oe=utf8&sensor=false&key=%(key)s'        
self.geoCodeKey = ''
def GetUrlKey(self,address):
return self.geoCodeUrl % {'key':self.geoCodeKey,'address':urllib2.quote(address)}
def GetKml(self, address):
response = urllib2.urlopen(self.GetUrlKey(address))
return response.read();
class KmlParser():
def GetText(self,node,nodeName):
return node.getElementsByTagName(nodeName)[0].childNodes[0].data
def GetCoordinates(self,node):
items = self.GetText(node, "coordinates").split(',')
return { 'latitude':items[0], 'longitude':items[1] }
def ParseAddress(self, node):
address = { 'address':self.GetText(node,'address'),
'country':self.GetText(node,'CountryName'),
'street':self.GetText(node,'ThoroughfareName'),
'postcode':self.GetText(node,'PostalCodeNumber'),
'coordinates': self.GetCoordinates(node) }
return address 
def Parse(self, xml):
results = []
dom = minidom.parseString(xml)
for placeNode in dom.getElementsByTagName('Placemark'):
results.append(self.ParseAddress(placeNode))
dom.unlink()
return results;
def GetAddresses(address):
return KmlParser().Parse(GeoCoder().GetKml(address))
if __name__ == '__main__':
import sys
if len(sys.argv) > 1:
for address in GetAddresses(' '.join(sys.argv[1:])):
print address
else:
print "Enter Address to Geocode, using the Google Geocoding API"

Which makes geocoding pretty easy from Python

Python 2.5.4 (r254:67916, Dec 23 2008, 15:10:54) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import geocoder
>>> addresses = geocoder.GetAddresses('main st, melbourne, victoria')
>>> for a in addresses: print geocoder.AddressToString(a)
...
Main St, Mordialloc VIC 3195, Australia (145.0858270,-38.0058450)
Main St, Lilydale VIC 3140, Australia (145.3507800,-37.7575460)
Main St, Mornington VIC 3931, Australia (145.0407850,-38.2231010)
Main St, Greensborough VIC 3088, Australia (145.1058060,-37.7025570)
Main St, Diamond Creek VIC 3089, Australia (145.1478460,-37.6744410)
Main St, Thomastown VIC 3074, Australia (144.9994220,-37.6787790)
Main St, Blackburn VIC 3130, Australia (145.1491980,-37.8258150)
Main St, Upwey VIC 3158, Australia (145.3366500,-37.9050420)
Main St, Croydon VIC 3136, Australia (145.2811170,-37.7961170)

Google also note this in the terms;

Geocoding is a time and resource intensive task. Whenever possible, pre-geocode known addresses (using the Geocoding Service described here or another geocoding service), and store your results in a temporary cache of your own design.

Even though the geocoding service is actually quite quick, it does make a lot of sense not to make a call to an external service if its not required. I suggest following this advice.

Maps without JavaScript

Normally some very clever JavaScript dynamically adds new map tile images to the DOM when sections of the map need to be rendered out, so the normal Google Maps service won't work without JavaScript. Luckily Google provide a REST Static Maps service which serves up map images. Its all documented in the Static Maps API Developer's Guide.

I wrote this Python script to get a map URL, again a Google Maps API key is required.     

key = 'MAPS_API_KEY'
url = 'http://maps.google.com/staticmap?'
colours = ['brown', 'green', 'purple', 'yellow', 'blue', 'gray', 'orange', 'red', 'white','black']
def GetImageUrl(params, markers):
if markers: 
marker = [','.join(p) for p in markers]
params['markers'] = '|'.join(marker)
if not params.has_key('key') : params['key'] = key 
return url + '&'.join(['='.join(p) for p in params.items()])     
if __name__ == '__main__':
# Generate map URL with 10 locations in Melbourne Australia
letters = 'ABCDEFGHI'
address = [['144.9994220','-37.6787790'],
['145.2811170','-37.7961170'],
['145.0407850','-38.2231010'],
['145.3507800','-37.7575460'],
['145.3366500','-37.9050420'],
['145.1491980','-37.8258150'],
['145.1058060','-37.7025570'],
['145.1478460','-37.6744410'],
['145.0858270','-38.0058450']]
markers = [ [e[1],e[0],colours[i]] for i, e in zip(range(len(address)), address)]
params = { 'size':'256x256', 'maptype':'roadmap', 'sensor':'false', 'markers': markers }
print GetImageUrl(params, markers)

Using this script and little more Python hacking I could quickly generate some HTML with a map and links to more detailed maps.

  A. Main St, Thomastown VIC 3074, Australia
  B. Main St, Croydon VIC 3136, Australia
  C. Main St, Mornington VIC 3931, Australia
  D. Main St, Lilydale VIC 3140, Australia
  E. Main St, Upwey VIC 3158, Australia
  F. Main St, Blackburn VIC 3130, Australia
  G. Main St, Greensborough VIC 3088, Australia
  H. Main St, Diamond Creek VIC 3089, Australia
  I. Main St, Mordialloc VIC 3195, Australia

 

 

Putting it all together

There is still some consideration to put this all together in website that supports server side mapping and geocoding without JavaScript and enhances the mapping with unobtrusive JavaScript.

Page responses need to return HTML that will work without JavaScript. If the browser executes the pages JavaScript, the script can attach an events and disable default behaviors to implement the richer client side functionality.  

On a page a user entered address can be geocoded, the requested HTML should contain a form that can post back to the server to do the geocoding. If client side JavaScript is not used the server can handle the post and use the geocoding service to respond appropriately, of course with JavaScript the addresses can be added to the DOM dynamically, probably with some slide or fade effect.

On pages that display maps, the static image URL can be sent in the requested HTML and used if the JavaScript Google Maps can't be loaded. As on Google Maps some additional controls can be added to allow some panning and zooming without JavaScript.

I'm going to see if I can get the all working on the Google App Engine, I'll let you know how I go.


Will you Django with me?

March 13, 2009 11:02 by tarn

I decided I'd have a look at Django to get a feel for a different web framework and to learn more about Python itself. Its no secret that I've been really interested in IronPython recently and I'm excited to give Python a go without the backing of the .NET framework (but with a complete standard library implementation, unlike IronPython).

For this post I basically just followed the installation guide and the first 4 tutorials in the excellent Django Documentation. Its not my intention to reproduce the tutorials in this post, I really just want to discuss things I found interesting while learning the framework.

Installing

After some fooling around I found the following versions of Python, Django and MySql were the easiest way to get started using Django on my Vista box. All of the versions below have windows installers so its pretty straight forward.

Python 2.5.4 (MySQL Python didn't have an installer for 2.6)

Django 1.0.2

MySql 5.0 (I had problems installing 5.1 on my Vista Box, so I reverted to 5.0 which just worked)

MySql Python Module

I deliberately didn't install the GUI tools for MySql as I wanted to get hardcore from the console. Djangos ORM can be used directly from the Python console and MySql has a complete command line interface.  

Getting Started

Django comes with a helper script to assist setting up and managing a site. To do things like create a new web project all you need to do is run following command:  

python django-admin.py startproject [sitename]

 

This creates the skeleton project structure and files.

.
..
manage.py
settings.py
urls.py
__init__.py

 

You can immediately validate everything is on the right track by running the development server. 

python manage.py runserver

 

I was pretty excited I could just create a new database from the same console window

c:\Working>mysql -u root -p
Enter password: *******
Welcome to the MySQL monitor.  Commands end with ; or \g.
Your MySQL connection id is 2
Server version: 5.0.77-community-nt MySQL Community Edition (GPL) 
Type 'help;' or '\h' for help. Type '\c' to clear the buffer. 
mysql> CREATE DATABASE demodb;
Query OK, 1 row affected (0.03 sec)

 

All I then needed to do was add my database credentials to the settings.py. Once this was setup I could use the syncdb command to build all the tables required by the framework (which includes an authorization module).

python manage.py syncdb

 

Which, without me doing anything else, creates the tables.

Creating table auth_permission
Creating table auth_group
Creating table auth_user
Creating table auth_message
Creating table django_content_type
Creating table django_session
Creating table django_site

 

Models

As with most MVC web application frameworks, there is a standard project structure convention. Django offers another command to quickly create an app, which is basically a small web application.

python manage.py startapp weblog

 

This creates a sub folder with the following files empty files

__init__.py
models.py
views.py

 

Then custom models can be added to the models python file. (Yes, I see that we probably would want more characters for a post, but I just want to get something going for now)

class Post(models.Model):
content = models.CharField(max_length=200)
pub_date = models.DateTimeField('date published')
class Comment(models.Model):
post = models.ForeignKey(Post)
comment = models.CharField(max_length=200)

 

To make the next bit of magic happen, we have to register this class in the main settings python file of this project.

INSTALLED_APPS = (
'django.contrib.auth',
'django.contrib.contenttypes',
'django.contrib.sessions',
'django.contrib.sites',
'DjangoSite.weblog',
)

 

Now the syncdb command can be used again to create the new tables for our model.

c:\Working\DjangoSite\DjangoSite>python manage.py syncdb
Creating table weblog_post
Creating table weblog_comment
Installing index for weblog.Comment model

We can now start checking out the ORM provided by Django in the interactive Python console.

>>> from DjangoSite.weblog.models import *
>>> import datetime
>>>
>>> post = Post(content = "Django Rocks!", pub_date=datetime.datetime.now())
>>> post.save()
>>>
>>> comment = Comment(post = post, comment="+1")
>>> comment.save()
>>>
>>> findPost = Post.objects.get(id=post.id)
>>> findPost.content
u'Django Rocks!'
>>>
>>> comments = findPost.comment_set.all()
>>> for c in comments:
...     print c.comment
...
+1
>>> findPost.delete()

 

That's pretty cool, and just 6 lines of code to create the models!

Administration

A few lines need to be uncommented in the urls python file, another line added to the installed apps list and the new table needs to added to the database to enable the default administration features. For the mean time this will serve pages to manage user accounts, later we'll be able to use the administration features to create and a web administration interface for any of our models.

The urlpatterns object is pretty cool. Its basically the URL routing and it uses regular expressions to map requests to handler scripts. More on this before this post is finished.

from django.conf.urls.defaults import *
# Uncomment the next two lines to enable the admin:
from django.contrib import admin
admin.autodiscover()
urlpatterns = patterns('',
(r'^admin/(.*)', admin.site.root) ,
)

 

Now if we run the server admin pages can be found from /admin directory of the site.

I won't go into to much detail about how its done but by simply registering a model, web pages are provided to manage that model (finding, adding, deleting and updating). Its all very clever, the framework does lots of introspection to display the correct html controls for each field type.

To separate the UI from the model, a class can be created which is used to guide how the admin framework renders the pages for the model, it is then used to map fields to properties on the model object. 

Views

Getting some weblog pages setup is amazingly straight forward. Its just a matter of adding a regular expression into the urlpatterns that matches the type of URL you want to handle and a function to handle it. The most basic handler method is what you'd expect, it takes a http request and returns a http response. 

from django.http import HttpResponse
def index(request):
return HttpResponse("This is the posts page!")

 

This can be expanded to actually use the weblog models. The code below gets the 5 most recent posts and creates a string with each post content separated with a <br /> tag.

from DjangoSite.weblog.models import Post, Comment
from django.http import HttpResponse
def index(request):
lastestPosts = Post.objects.all().order_by('-pub_date')[:5]
output = '<br />'.join([p.content for p in lastestPosts])
return HttpResponse(output)

 

Well that's pretty cool, but I don't think we'll be writing entire pages using standard Python. Besides, these look more like controllers than views, there must be more to it.

Templates

With templates, we can throw objects at a template to be rendered. The index method can now be separated from the rendering of HTML

from django.shortcuts import render_to_response
from DjangoSite.weblog.models import Post, Comment
from django.http import HttpResponse
def index(request):
latestPosts = Post.objects.all().order_by('-pub_date')[:5]
return render_to_response('weblog/index.html', {'posts': latestPosts})

 

And the template (weblog/index.html) looks something like this

{% if posts %}
<ul>
{% for post in posts %}
<li>{{ post.content }}</li>
{% endfor %}
</ul>
{% else %}
<p>There are no posts</p>
{% endif %}
 
 
It worth noting this isn't just Python embedded in HTML. I think this text from the Django documentation describes this design decision:
 
..you’ll want to bear in mind that the Django template system is not simply Python embedded into HTML. This is by design: the template system is meant to express presentation, not program logic
 
It makes sense to me.

Forms

A forms post data can be retrieved from a dictionary on the request object.

def index(request):
postTitle = resquest.POST["postTitle"]
...

 

I'm sure Django has support for mapping posts to model objects, but I haven't seen it yet.

Some more magic

It turns out there is heaps more support in Django for handling generic web problems. For example creating list and details pages is so common that Django has some generic functionality to help. You just need to provide a model and write some templates to describe how to render the models.

It appears there is heaps more of the magic throughout the framework some things I haven't yet looked into are:

  • Advanced form processing
  • Using the RSS framework
  • Using the cache framework
  • Using the comments framework

    Summary

    I've just stepped through all the basic steps to building a functional database driven web application. Overall I'm pretty impressed; it didn't take much time, code or leaps in faith to get it going. It appears to do generic things, like what I was doing, is even easier as the framework has generic support for it.

    I am glad I got into it from the ground up, I think I would have been a lot more skeptical had I written a simple weblog without writing a line of Python. I've come to learn that its not about the 90% things that are easy to do in a web application framework, its about the 10% of things you want to do but don't work out of the box. Of course I haven't had enough exposure with Django to say how deep the application framework actually goes. I personally feel that Django and Python are a fantastic language and framework to deal with the 10% of requirements that don't come out of the box.

    On anther note, I personally love coding from the console rather than an IDE, even with limited exposure I was getting pretty quick navigating my project, running the Django web development helpers and managing MySql. I personally think it's faster developing from a console with all your tools accessible without taking your hands of the keyboard.

    Its been lots of fun playing with the framework. I'd definitely like to build a site in Django some time, but I'm not sure when I'll find the time.

     

     


  • Tags:
    Categories:
    Comments (0)

    Executing IronPython in, err.. IronPython

    March 6, 2009 10:57 by tarn

    I just read the latest post from secretGeek, a brilliantly funny blog which often makes my day with posts like IT Industry Revolutionised By Labour Saving Device and Defensive Programming.

    The latest post, A 3 minute guide to embedding IronPython in a C# application, is a demo of a Windows application that can execute an IronPython script from a TextBox at runtime. Another TextBox control on the form is added to the IronPython script scope so the script can interact with it. This is both cool and powerful, the concept has been used in an IronPython powered spreadsheet from Revolver Systems.

    I thought it would be kind of fun to create an IronPython script that created a Windows Form which could execute a script. This means I could run the script and get window which I could then use to run the script again and get anther window, then..

    I posted the script in the comments and then decided it was pretty fun and it would be worth writing a post about myself. Below is the Form class script:

    import clr
    clr.AddReferenceByPartialName("System.Windows.Forms")
    clr.AddReference('IronPython')
    clr.AddReference('Microsoft.Scripting')
    from System.Windows.Forms import *
    from IronPython.Hosting import Python
    from Microsoft.Scripting import SourceCodeKind
    class MetaNote(Form):
    def __init__(self):
    self.button = Button()
    self.code = TextBox()
    self.text = TextBox()
    self.Width = 420
    self.Height = 400
    self.code.Multiline = True
    self.code.Height = 300
    self.code.Width = 300;
    self.button.Left = 310
    self.button.Text = "Execute"
    self.Controls.Add(self.button)
    self.Controls.Add(self.code)
    self.button.Click += self.run
    def run(self,sender,e):
    engine = Python.CreateEngine()
    source = engine.CreateScriptSourceFromString(self.code.Text, SourceCodeKind.Statements)
    scope = engine.CreateScope()
    scope.SetVariable("txt", self.text);
    source.Execute(scope)
    

     

    The only trick is, to start the application from the IronPython console (ipy.exe) you need to include this line at the end which creates the class and creates a standard application message loop.

    Application.Run(MetaNote())
    
     
    When the script is executed from within the application itself, you only need to create in instance of the class and call its Show method.
     
    MetaNote().Show()
    
     
    Then the fun..
     
    image 
     

    Tags: ,
    Categories:
    Comments (0)