Using Google App Engine as Proxy for Silverlight and Flash Cross-Domain Requests

When using Silverlight or Flash to fetch data from other domains one often runs into cross-domain access restrictions. For security reasons in order to access data from different domains, the remote server explicitly has to allow this by defining a crossdomain.xml (or, for Silverlight, clientaccesspolicy.xml is good as well). If this file cannot be found on the remote domain, the request is not executed.

This can be fustrating when querying against RSS feeds or JSON/XML web APIs that don’t define any of these files. The workaround for this issue is to use some sort of proxying service. In this article I’ll be showing how to use Google App Engine to create a simple proxy that will forward these requests for free – within a reasonable daily load.

Google App Engine Overview

The reason I’ve chose to implement the proxy using Google App Engine is because it has a free daily quota and getting started using it is really simple: all you need is a Google account and to download and install the Google App Engine SDK.

Google App Engine supports developing in both Java and Python. In my example I’ll be using Python. In order to use and deploy the code yourself as well, follow these steps:

Creating the Proxy In Python

Creating a simple proxy is pretty straightforward by using the urlfetch library:

try:
	response = urlfetch.fetch(url)
except (urlfetch.Error, apiproxy_errors.Error):
	# an error occured

In order to make the proxy a bit smarter, I’ll implement some caching using the App Engine’s memcache:

proxiedContent = memcache.get(memcacheKey)
proxiedContentInMemcache = True
if proxiedContent is None:
   # not in memcache: execute request
   # ...
   # add the result content to memcache for CACHE_TIME minutes
   memcache.add(memcacheKey,proxiedContent,CACHE_TIME)
else:
   # use the content from memcache

Using these snipplets, here’s how the main file of the proxy webapp will look like:

import datetime
import hashlib
import logging
import pickle
import urllib
import re
import time
import urllib
import wsgiref.handlers
 
from google.appengine.api import memcache
from google.appengine.api import urlfetch
from google.appengine.ext import db
from google.appengine.ext import webapp
from google.appengine.ext.webapp import template
from google.appengine.runtime import apiproxy_errors
 
CACHE_TIME = 1 # number of minutes to cache content for
 
URL_PREFIXES = ["http://www.google.com/finance"] # only allow URLs to be queried from certain domain(s)
 
def getMemcacheKey(url):
  url_hash = hashlib.sha256()
  url_hash.update(url)
  return "hash_" + url_hash.hexdigest()
 
class ProxyHandler(webapp.RequestHandler):
  def get(self):
    url = self.request.get('url')
    url = urllib.unquote(url)
    # only allow urls that start with prefixes defined in URL_PREFIXES to be used
    if not self.isUrlAllowed(url):
      self.response.out.write("The URL passed can not be proxied due to security reasons.")
      return
    memcacheKey = getMemcacheKey(url) 
 
    # Use memcache to store the request for CACHE_TIME
    proxiedContent = memcache.get(memcacheKey)
    proxiedContentInMemcache = True
    if proxiedContent is None:
      proxiedContentInMemcache = False
      try:
        response = urlfetch.fetch(url)
      except (urlfetch.Error, apiproxy_errors.Error):
        return self.error(404)
      proxiedContent = response.content
    if proxiedContent is None:
      return self.error(404)
 
    # Add the fetched content to memcache
    if (not proxiedContentInMemcache):
      memcache.add(memcacheKey,proxiedContent,CACHE_TIME)
    self.response.out.write(proxiedContent)
 
def isUrlAllowed(self, url):
  for urlPrefix in URL_PREFIXES:
    if url.startswith(urlPrefix):
      return True
  return False
 
app = webapp.WSGIApplication([
  ("/proxy", ProxyHandler),
], debug=True)
 
def main():
  wsgiref.handlers.CGIHandler().run(app)
 
if __name__ == "__main__":
  main()

The proxy can now be called by passing the URL to be proxied via the url parameter. So to proxy e.g. http://google.com, the following request should be made: http://myapplication.appspot.com/proxy?url=http://google.com.

Adding crossdomain.xml to the Application

Now that the proxy class is ready, all we need to do is wire it into the web application and include a crossdomain.xml static file which will allow requests from all hosts (it is advisable to change this to the URL the requests are actually made). Based on this, here is how the crossdomain.xml would look like:

<!DOCTYPE cross-domain-policy SYSTEM "http://www.macromedia.com/xml/dtds/cross-domain-policy.dtd"> 
<cross-domain-policy> 
  <allow-access-from domain="*" /> 
</cross-domain-policy>

And here is how the modified app.yaml descriptior will look like:

application: yourappname
version: proxyv1
runtime: python
api_version: 1

handlers:
- url: /crossdomain.xml
  static_files: crossdomain.xml
  upload: crossdomain.xml
- url: /.*
  script: proxy.py
  secure: optional

Download the Code

The code for the application can be downloaded here: Proxy Using Google App Engine.zip. After unzipping be sure to change the “yourappname” name in app.yaml in order to be able to deploy it on Google App Engine.

Update: Security Issues

Andrew noted in the comment that the original solution raised security concerns as it would have been easy for someone to hijack this proxy and use it for their purpose as no authentication or authorization is done. I’ve added a small fix where the proxy only forwards to URLs that start from a list of prefixes. This is probably a sufficient solution for common cases, however more sophisticated authorization methods may be needed in other cases.

12 Responses to Using Google App Engine as Proxy for Silverlight and Flash Cross-Domain Requests
  1. Geoff

    Cool, thanks for saving me some time. Was considering something similar to this for a couple WP7 projects

  2. Andrew Aylett

    This will work, but please consider the security implications of allowing unrestricted proxying of any site: if anyone malicious finds it, you’re probably going to find them breaking every rule in the GAE program policies on your behalf.

  3. Gergely Orosz

    Thanks for the comment, Andrew. Will be adding some detail on how to limit the proxy to a certain domain / adding some basic authentication / authorization.

  4. John

    The link to the zip file is broken.

  5. Gergely Orosz

    John: sorry about that. Fixed the broken link!

  6. Trackback: bramka
    internet... [...]Greg Does IT » Blog Archive » Using Google App Engine as Proxy ... darmowe-proxy.pl/index.php?q=aHR0cDovL2tvbmZsaWt0eS53cC5wbC9xdWVyeSwlMjJtaWNoYSVCMyUyMHN0YW5pdWwlMjIsc3p1a2FqLmh0bWw%3D
  7. Ajoutez un site à ce moteur de recherche... [...]Greg Does IT » Blog Archive » ... sukoga.com/index.php?page=user/viewcomments/images/6e6e61e2fec9c634951af674fcbc65fb/d3d3Lm1pc3Rlci1hdXRvLmJl
  8. Trackback: Proxy Seite
    Proxy Seite... [...]Greg Does IT » Blog Archive » Using Google App Engine as Pro... vpnandproxy.blog.com/author/ezelserie33
  9. Eswar

    Thanks….

  10. Trackback: domains
    domains... [...]Greg Does IT » Blog Archive » Using Google App Engine as Proxy f... zarden.com/?cat=1
  11. lemo

    Hi, I have to run with python 2.7 on the app engine for my site,
    and I can’t seem to find the right part to change in the script to make it work, any clue?
    Thanks a lot!

  12. lemo

    Okay now it works
    Had to remove the “isUrlAllowed” stuff for some reason…
    Not loading any picture or special characters though :\
    Should that happen?