Google Code offered in: 中文 - English - Português - Pусский - Español - 日本語
I'm sure your mind is positively buzzing with ideas for how to use Google App Engine, and a few of you might be interested in building an app that interacts with some of Google's other services via our Google Data AtomPub APIs. Quite of few of Google's products expose a GData API, (a few interesting examples are YouTube, Google Calendar, and Blogger--you can find a complete list here) and these APIs can be used to read and edit the user-specific data they expose.
In this article we'll use the Google Documents List Data API to walk through the process of requesting access from and retrieving data for a particular user. We'll use Google App Engine's webapp framework to generate the application pages, and the Users API to authenticate users with Google Accounts.
Some Google Data services require authorization from your users to read data, and all Google Data services require their authorization before your app can write to these services on their behalf. Google uses AuthSub to enable users to authorize your app to access specific services.
Using AuthSub, users type their password into a secure page at google.com, then are redirected back to your app. Your app receives a token allowing it to access the requested service until the user revokes the token through the Account Management page.
In this article, we'll walk through the process of setting up the login link for the user, obtaining a session token to use for multiple requests, and storing the token in the datastore so that it can be reused for returning users.
Google offers a GData Python client library that simplifies token management and requesting data from specific Google Data APIs. We recently released a version of this library that supports making requests from Google App Engine applications. In this article we'll use this library, but of course you're welcome to use whatever works best for your application. Download the gdata-python-client library.
To use it with your Google App Engine application, simply place the library source files in your application's directory, and import them as you usually would. The source directories you need to upload with your application code are src/gdata
and src/atom
. Then, be sure to call the gdata.alt.appengine.run_on_appengine
function on each instance of a gdata.service.GDataService
object. There's nothing more to it than that!
Applications use an API called AuthSub to obtain a user's permission for accessing protected Google Data feeds. The process is fairly simple--to request access from a user to a protected feed, your app will redirect the user to a secure page on google.com where the user can sign in to grant or deny access. Once doing so, the user is then redirected back to your app with the newly-granted token stored in the URL.
Your application needs to specify two things when using AuthSub: the common base URL for the feeds you want to access, and the redirect URL for your app, where the user will be sent after authorizing your application.
To generate the token request URL, we'll use the gdata.service
module included in the Google Data client library. This module contains a method GenerateAuthSubURL
which automatically generates the correct URL given the base feed URL and your website's return address. In the code snippet below, we use this method to generate a URL requesting access to a user's Google Document List feed.
In our app.yaml
file, we will define a URL mapping to create a separate URL for each step. Here's an example:
application: gdata-feedfetcher version: 1 runtime: python api_version: 1 handlers: - url: /step1.* script: step1.py - url: /step2.* script: step2.py ...
To illustrate this first step of using AuthSub in the app, we will create a step1.py script that looks something like this:
# Change the value of HOST_NAME to the name given to point to your app. HOST_NAME = 'gdata-feedfetcher.appspot.com' import wsgiref.handlers from google.appengine.ext import webapp import gdata.service class Fetcher(webapp.RequestHandler): def get(self): # Instantiate an instance of the GDataService() client = gdata.service.GDataService() authsub_next_url = ('http://%s/step1' '?token_scope=http://docs.google.com/feeds/' % HOST_NAME) # Generate the AuthSub URL and write a page that includes the link self.response.out.write("""<html><body> <a href="%s">Request token for the Google Documents Scope</a> </body></html>""" % client.GenerateAuthSubURL(authsub_next_url, 'http://docs.google.com/feeds/', secure=False, session=True)) def main(): application = webapp.WSGIApplication([('/.*', Fetcher),], debug=True) wsgiref.handlers.CGIHandler().run(application) if __name__ == '__main__': main()
In this example, the first URL passed to GenerateAuthSubURL
returns the user to our application, and the second is the Google Documents List feed base URL, which indicates which service our app is requesting authorization for. After you click the link and authorize your application, you will be taken back to the same original page but the single use AuthSub token will now be a URL parameter in the appspot URL.
Once we've generated an authorization request URL for a particular Google Data service, we'll need a way to use the token returned to our app to access the feed in question. Now, we need to retrieve the initial token returned to us for the Google Documents List API, and upgrade that token to a permanent session token. Remember that we told the service to redirect the user to the URL 'http://gdata-feedfetcher.appspot.com/?token_scope=http://docs.google.com/feeds/'. Let's extend our simple example above to do a few things.
Let's write the functionality that will handle the return request from the Google Data service the user signed in to. The Google Data service will request a URL that will look something like this:
http://gdata-feedfetcher.appspot.com/?token_scope=http://docs.google.com/feeds/&token=CKF50YzIHxCT85KMAg
Which is just our return URL appended with the initial authorization token for the service which grants our app access for our user. The code below first takes this URL and extracts the service and the token. Then, it requests an upgrade for the token for the document list service.
We use two new methods to achieve this. First, the ManageAuth()
method verifies the user is signed in to our application and we have received a token from the document list service, and calls the method UpgradeToken
.
UpgradeToken
calls out to Google's token management service for the long-lived session token. This uses the client library's existing function, UpgradeToSessionToken
Below is the code for step2.py
which illustrates upgrading to a session token.
Note: With Google App Engine, you must use the URLFetch API to request external URLs. In our Google Data Python client library, gdata.service
does not use the URLFetch API by default. We have to tell the service object to use URLFetch by calling gdata.alt.appengine.run_on_appengine
on the service object, like this: gdata.alt.appengine.run_on_appengine(self.client)
import wsgiref.handlers import urllib from google.appengine.ext import webapp from google.appengine.api import users import gdata.service import gdata.alt.appengine # Change the value of HOST_NAME to the name given to point to your app. HOST_NAME = 'gdata-feedfetcher.appspot.com' class Fetcher(webapp.RequestHandler): # Initialize some global variables we will use def __init__(self): # Stores the page's current user self.current_user = None # Stores the token_scope information self.token_scope = None # Stores the Google Data Client self.client = None # The one time use token value from the URL after the AuthSub redirect. self.token = None def get(self): # Write our pages title self.response.out.write("""<html><head><title> Google Data Feed Fetcher: read Google Data API Atom feeds</title>""") # Get the current user self.current_user = users.GetCurrentUser() self.response.out.write('<body>') # Allow the user to sign in or sign out if self.current_user: self.response.out.write('<a href="%s">Sign Out</a><br>' % ( users.CreateLogoutURL(self.request.uri))) else: self.response.out.write('<a href="%s">Sign In</a><br>' % ( users.CreateLoginURL(self.request.uri))) for param in self.request.query.split('&'): # Get the token scope variable we specified when generating the URL if param.startswith('token_scope'): self.token_scope = urllib.unquote_plus(param.split('=')[1]) # Google Data will return a token, get that elif param.startswith('token'): self.token = param.split('=')[1] # Manage our Authentication for the user self.ManageAuth() self.response.out.write('<div id="main">') self.response.out.write( '<div id="sidebar"><div id="scopes"><h4>Request a token</h4><ul>') self.response.out.write('<li><a href="%s">Google Documents</a></li>' % ( self.client.GenerateAuthSubURL( 'http://%s/step2?token_scope=http://docs.google.com/feeds/' % ( HOST_NAME), 'http://docs.google.com/feeds/', secure=False, session=True))) self.response.out.write('</ul></div><br/><div id="tokens">') def ManageAuth(self): self.client = gdata.service.GDataService() gdata.alt.appengine.run_on_appengine(self.client) if self.token and self.current_user: # Upgrade to a session token and store the session token. self.UpgradeToken() def UpgradeToken(self): self.client.SetAuthSubToken(self.token) # Sets the session token in self.client.auth_token self.client.UpgradeToSessionToken() def main(): application = webapp.WSGIApplication([('/.*', Fetcher),], debug=True) wsgiref.handlers.CGIHandler().run(application) if __name__ == '__main__': main()
When we upgrade the initial token using the gdata.service
method UpgradeToSessionToken()
this adds the session token to the valid tokens which will be used by the gdata.service
client. Below, we will take you through the steps to store this token in your application for use to fetch your user's feed in your application.
If we do not store the AuthSub session token, the user will need to perform the AuthSub authorization redirects each time that they use our application. To avoid the need to go through the authorization process every time, we can store the session token in our Google App Engine Datastore. The information we will store so that we can use it in the future to make requests is the user email, the session token, and the service scope for which this token is valid, in the form of a target URL.
We write a data model that extends the base class db.Model for this purpose.
from google.appengine.ext import db class StoredToken(db.Model): user_email = db.StringProperty(required=True) session_token = db.StringProperty(required=True) target_url = db.StringProperty(required=True)
Now, we will rewrite UpgradeToken
so that it also stores the session token information. We will rename this function UpgradeAndStoreToken
.
def UpgradeAndStoreToken(self): self.client.SetAuthSubToken(self.token) self.client.UpgradeToSessionToken() if self.current_user: # Create a new token object for the data store which associates the # session token with the requested URL and the current user. new_token = StoredToken(user_email=self.current_user.email(), session_token=self.client.GetAuthSubToken(), target_url=self.token_scope) new_token.put()
Don't forget to change the name of the function in ManageAuth()
after you make this change.
Now we've done everything necessary to allow our user to retrieve his or her document list feed from our application. The final step is to get the user feed from Google Docs and display it on our site!
First, let's write a function that will look up a Token for our user. For this function, we assume that the user has asked us to fetch his or her document list feed, and we've stored that URL in self.feed_url
def LookupToken(self): if self.feed_url and self.current_user: stored_tokens = StoredToken.gql('WHERE user_email = :1', self.current_user.email()) for token in stored_tokens: if self.feed_url.startswith(token.target_url): self.client.SetAuthSubToken(token.session_token) return
Now that we have our user's session token, let's fetch the feed!
def FetchFeed(self): # Attempt to fetch the feed. if not self.feed_url: self.response.out.write( 'No feed_url was specified for the app to fetch.<br/>') self.response.out.write('Here\'s an example query which will show the' ' XML for the feed listing your Google Documents <a ' 'href="http://%s/step4' '?feed_url=http://docs.google.com/feeds/documents/private/full">' 'http://%s/step4' '?feed_url=http://docs.google.com/feeds/documents/private/full' '</a>' % (HOST_NAME, HOST_NAME)) return if not self.client: self.client = gdata.service.GDataService() gdata.alt.appengine.run_on_appengine(self.client) try: response = self.client.Get(self.feed_url, converter=str) self.response.out.write(cgi.escape(response)) except gdata.service.RequestError, request_error: # If fetching fails, then tell the user that they need to login to # authorize this app by logging in at the following URL. if request_error[0]['status'] == 401: # Get the URL of the current page so that our AuthSub request will # send the user back to here. next = self.request.uri auth_sub_url = self.client.GenerateAuthSubURL(next, self.feed_url, secure=False, session=True) self.response.out.write('<a href="%s">' % (auth_sub_url)) self.response.out.write( 'Click here to authorize this application to view the feed</a>') else: self.response.out.write( 'Something else went wrong, here is the error object: %s ' % ( str(request_error[0])))
The above method uses the cgi
module, so be sure to add import cgi
to the list of imports at the beginning of your script.
Now that we have a method to fetch the target feed, we can rewrite the FeedFetcher
class' get
method to call this method after ManageAuth()
.
def get(self): # Write our pages title self.response.out.write("""<html><head><title> Google Data Feed Fetcher: read Google Data API Atom feeds</title>""") # Get the current user self.current_user = users.GetCurrentUser() self.response.out.write('<body>') # Allow the user to sign in or sign out if self.current_user: self.response.out.write('<a href="%s">Sign Out</a><br>' % ( users.CreateLogoutURL(self.request.uri))) else: self.response.out.write('<a href="%s">Sign In</a><br>' % ( users.CreateLoginURL(self.request.uri))) for param in self.request.query.split('&'): # Get the token scope variable we specified when generating the URL if param.startswith('token_scope'): self.token_scope = urllib.unquote_plus(param.split('=')[1]) # Google Data will return a token, get that elif param.startswith('token'): self.token = param.split('=')[1] # Find out what the target URL is that we should attempt to fetch. elif param.startswith('feed_url'): self.feed_url = urllib.unquote_plus(param.split('=')[1]) # If we received a token for a specific feed_url and not a more general # scope, then use the exact feed_url in this request as the scope for the # token. if self.token and self.feed_url and not self.token_scope: self.token_scope = self.feed_url # Manage our Authentication for the user self.ManageAuth() self.LookupToken() self.response.out.write('<div id="main">') self.FetchFeed() self.response.out.write('</div>') self.response.out.write( '<div id="sidebar"><div id="scopes"><h4>Request a token</h4><ul>') self.response.out.write('<li><a href="%s">Google Documents</a></li>' % ( self.client.GenerateAuthSubURL( 'http://gdata-feedfetcher.appspot.com/step3' + '?token_scope=http://docs.google.com/feeds/', 'http://docs.google.com/feeds/', secure=False, session=True))) self.response.out.write('</ul></div><br/><div id="tokens">')
You can see the final program at work by visiting: http://gdata-feedfetcher.appspot.com/. Also, view the complete source code, where we put all of this together at the Google App Engine sample code project on Google Code Hosting.
The AuthSub session tokens are long lived, but they can be revoked by the user or by your application. At some point, a session token stored in your data store may become revoked so your application should handle cleanup of tokens which can no longer be used. The status of a token can be tested by querying the token info URL. You can read more about AuthSub token management in the AuthSub documentation. This feature is left as an exercise to the reader, have fun :)
Using the Google Data Python client library, you can easily manage your user's Google Data feeds in your own Google App Engine application.
The Google Data Python client library includes support for almost all of the Google Data services. For further information, you can read the getting started guide for the library, visit the project to browse the source, and even ask questions on the gdata-python-client's Google group.
As always, for questions about Google App Engine, read our online documentation and visit our google group.