Archive for the ‘geo’ Topic

I really enjoy reading Shorpy, a website that features images scanned from the American National Archive. It’s an interesting, candid insight into American history and offers some great amateur photographs of scenes that wouldn’t normally be featured in the history books.

Sadly, however, my knowledge of American geography is terrible, so being able to put a place to a photograph geographically by description alone wasn’t easy.

So, using YQL, a little Javascript, and some HTML, I made a site that would solve this problem for me.

The site uses YQL’s ability to both read RSS feeds and — using a description of each photograph — discern any places mentioned. Since Shorpy uses a fairly standard format for describing each photograph it posts, I knew I had a good chance of coming up with a decent location for each.

First I needed to retrieve the description for each item in the RSS feed. The YQL query for this is simple: "SELECT description from rss where url='http://feeds.feedburner.com/shorpy?q=rss.xml''. (You could easily replace the 'SELECT description FROM ...' with 'SELECT * FROM ...' if you wanted to get more information about the RSS feed, but for the moment we’re only concerned with the description for each item.)

I used the handy JQuery YQL plugin to make this call:

	function getRSSDescription() {
	$.yql(
		"SELECT description from rss where url=#{rss}",
		{
			rss: "http://feeds.feedburner.com/shorpy?q=rss.xml"
		},
		function (data) {
			// Code to manipulate our results
		}
	)};

How simple was that?! I created a YQL statement and passed in an argument (our RSS feed location), and now I have a JSON object (the 'data' variable in the code) that consists of a results array of a description for each item in our RSS feed. I didn’t have to bother with explicitly calling an API, handling URL encoding/decoding, or parsing XML.

So what do we do with this description? Well, for each result in our data object I wanted to see whether it was possible to geoplace a location for the photograph. Since we’re handling a JSON object with a known, specific structure, iterating over the results is very easy.

	var resultsArray = data.query.results.item;
	var arLen = resultsArray.length;
	for (var i = 0, len = arLen; i < len; ++i ) {
		var locationString = resultsArray[i].description;
		// Code not shown here parses the year from the locationString 
                // using a simple regular expression, and we pass the original
		// HTML from the RSS feed so as to embed it in our map.
		getMapLocation(locationString, WINDOW_HTML, year);
	}	

After slightly pruning the resulting description to remove unnecessary embedded HTML, I made a second YQL query, this time to the YQL Placemaker table. This time I passed in the location description (returned from the query above) as an argument.

	function getMapLocation(locationString, WINDOW_HTML, year) {
	$.yql(
		"SELECT * FROM geo.placemaker(1) WHERE documentContent = '#{location}' AND documentType='text/html';",
		{
			location: locationString
		},
		function (data) {
			var latitude = JSON.stringify(data.query.results.matches.match.place.centroid.latitude);
			var longitude = JSON.stringify(data.query.results.matches.match.place.centroid.longitude);
			var JSONString = JSON.stringify(data.query.results.matches.match.place.centroid.latitude);
			// The function addPoint adds the resulting lat/lon to the Google Maps object, alongside
			// the original HTML from the RSS feed.
			addPoint(data.query.results.matches.match, unescape(WINDOW_HTML), year);
		}
);}

This time our YQL query has returned a returned a single result — we expressly asked for this by selecting from 'geo.placemaker(1)'. Again, as the result is a JSON object, it is easy to reference the necessary attributes using JavaScript (in this case, the latitude and longitude).

So now I have a geographical location for each photograph in Shorpy’s RSS feed, and all I’ve done is make two YQL queries!

At this point it’s possible to whatever you want with the data. I passed this to the Google Maps JavaScript library to plot on (see the addPoint() method in the source itself to see how this is done). But you could also write it out, store it, or do whatever you wanted with it.

While there are doubtless ways that JavaScript could be made more efficient — I am by no means a Front-end Developer — I find it amazing how simple YQL and Javascript make it to mix interesting datasets (like RSS) with dynamic services (like Placemaker and all) without any heavy lifting on the server-side. I highly recommend just reading through the datasources available as YQL tables to get an idea of what amazing things YQL is capable of.

Ryan Bateman
Java Engineer for Yahoo! Answers

As a demo for the Open Hack Day in Bengaluru (Bangalore), India, I thought I might play with Safari Extensions. And what better use than to put some geo magic into the browser?

Geo This! uses Yahoo Placemaker to either analyze the whole document or the highlighted text, and turns it into a map. See the following screencast to see it in action:

The developer source code is available on GitHub.

Christian Heilmann (@codepo8)
Yahoo! Senior Developer Evangelist

Today we announce the public release of Yahoo! PlaceFinder, a REST Web service that provides consistent multi-language global geocoding of named places.

PlaceFinder supports building-level address recognition in over 75 countries, and points of interest, airports, cities, and other place names (including administrative areas) for these countries and the rest of the world.

In addition to supporting free-form, single-line, and multi-line address inputs, PlaceFinder also supports queries that include cross streets, WOEID, and IATA airport codes. And for the first time from Yahoo! Geo Technologies, PlaceFinder also supports reverse geocoding (give it a longitude/latitude coordinate and get back the nearest address).

PlaceFinder replaces the Yahoo! Maps Geocoding API. But do not fear if you are using this service: The Yahoo! Maps Geocoding API will continue to be offered until the end of 2010. Information about migrating to PlaceFinder will be available in July.

It is important to note that no functionality available in the existing Geocoding API will be lost. PlaceFinder is all upside. And, don’t forget, there is always the developer Forum, where you can ask any questions that our documentation doesn’t address.

PlaceFinder gives you up to 32 response fields (as compared with up to 9 response fields for the existing Geocoding API). These elements and sub-elements are available:

• WOEID
• Neighborhood
• Time Zone
• Area Code
• Locale Code
• Bounding Box Coordinates

So, PlaceFinder not only provides the coordinate keys to visualize your place data on a map, it also provides a rich set of attribute data that enables a range of other location-based applications. Most fundamentally, it provides the Where On Earth ID (WOEID) that provides full and open access to a world of place discovery beyond the basic coordinates. WOEIDs are supported by other Yahoo! APIs, such as Yahoo! GeoPlanet, Flickr, and Weather.

We will be adding a PlaceFinder table to YQL in the near future, so developers can combine address geocoding with other location-aware services like Weather and Flickr in a single request.

Meanwhile, you can use the YQL Console to filter for and view existing Geo community tables.

We invite you to try out Yahoo! PlaceFinder. We think you’ll like what it can do for you.

Martin Barnes
Yahoo! Geo Technologies PlaceFinder

When we were at the Location Business Summit in Amsterdam last week, one of the things that fascinated me when showing off the GeoPlanet Explorer tool was that it found a lot of Points of Interest (museums, churches, sights) around the area of our hotel. I thought it might be fun to have an interface that does this as its main task, so let’s have a go at that.

Look around you is a mashup that finds your location and shows Wikipedia entries about the nearby surroundings. You can also navigate further and extend your area by clicking the names of neighbouring areas:

You can download the source code of the tool on Github.

In order to build the tool I used a few helpers:

• The W3C Geo API and a fallback to IP lookup provided by IP2location.
• Getting the nearest Wikipedia entries is done with the Geonames lookup API.
• Finding the nearest neighbours is powered by the Geoplanet API.

To make things easier and to “eat my own dogfood” I used the YQLGeo library we talked about earlier.

This not only provided me with the necessary functionality but also gave me a namespace (yqlgeo) to play with to prevent me polluting from the global namespace. Using YUI3 and YUI grids for the layout, I didn’t have to worry about browser inconsistencies either.

The HTML is simply a few DIVs (using the YUI grids for display next to another):

<div class="yui-g">
  <div class="yui-u first" id="map">
    <p class="info">Trying to determine your location...</p>
  </div>
  <div class="yui-u" id="sights"></div>
</div>
<div id="neighbours"></div>
<div id="children"></div>

For the JavaScript I embedded YUI, the Yahoo Maps API, and yqlgeo at the end of the code:

<script src="http://api.maps.yahoo.com/ajaxymap?v=3.8&appid={maps-id}'">
</script>
<script type="text/javascript" src="http://yui.yahooapis.com/3.1.1/build/yui/yui-min.js"></script>
<script src="yqlgeo.js"></script>
<script src="lookaround.js"></script>

This is all we need to make this work. So let’s get started, shall we?

YUI().use('node',function(Y){
  Y.one('body').addClass('js');
  yqlgeo.get('visitor',function(o){
    yqlgeo.getinfo(o);
  });

We tell YUI to use the node module to ease event handling and DOM manipulation. We add a class called js to the body of the document, which – together with the right CSS – effectively hides the point of interest information. Then we use YQLGeo to get the visitor’s location for us.

  yqlgeo.getinfo = function(o){
    var cur = o.place ? o.place : o;
    yqlgeo.rendermap('*', cur.centroid.latitude,cur.centroid.longitude,
                          cur.boundingBox.northEast.latitude,
                          cur.boundingBox.northEast.longitude,
                          cur.boundingBox.southWest.latitude,
                          cur.boundingBox.southWest.longitude;
    Y.one('#sights').set('innerHTML','Loading landmarks…');

Once we have the location, we call the rendermap() method with the coordinates and boundingBox measurements. To tell the end user what is going on, we show a loading message in the container that will later on show the sights around you.

    var url = 'http://ws.geonames.org/findNearbyWikipediaJSON?formatted=true'+
              '&lat=' + cur.centroid.latitude + '&lng='+
              cur.centroid.longitude+'&style=full&callback=yqlgeo.wiki';
    yqlgeo.get(url);

We assemble the REST URL pointing to the geonames API and give it the right latitude and longitude. As the callback method for the JSON-P call, we define yqlgeo.wiki() and call the get method to inject a script node into the document.

   Y.one('#neighbours').set('innerHTML','Loading neighbouring areas...');
    url = 'http://query.yahooapis.com/v1/public/yql?q=select%20*%20from'+
          '%20geo.places.neighbors%20where%20neighbor_woeid%3D'+
           cur.woeid+'&diagnostics=false&format=json&'+
          'callback=yqlgeo.neighbours';
    yqlgeo.get(url);
  };

We get the container for the neighbouring areas and give it a loading message (again so as to not keep our users in the dark as to what is happening). We call YQL with the correct query, give it the Where On Earth ID (WOEID) and define yqlgeo.neighbours() as the callback method before calling get().

  yqlgeo.get = function(url){
    Y.one('head').append('<script src="'+url+'"></script>');
  };

The get() method simply adds a new script node to the document with a given url as the src attribute. In effect, this means the REST APIs get called and return JSON to the defined callback methods.

  yqlgeo.rendermap = function(){
   var x = arguments;
   if(x[1]){
     yqlgeo.map = new YMap(Y.one('#map')._node);
     yqlgeo.map.addTypeControl();
     yqlgeo.map.addZoomLong();
     yqlgeo.map.addPanControl();
     yqlgeo.map.disableKeyControls();
     yqlgeo.map.setMapType(YAHOO_MAP_REG);
     var points = [];
     var point = new YGeoPoint(x[1],x[2]);
     points.push(point);
     var img = new YImage();
     img.src = '16x16.png';
     img.size = new YSize(16,16);
     var newMarker = new YMarker(point,img);
     yqlgeo.map.addOverlay(newMarker);
   }
   if(x[3] && x[4]){
     point = new YGeoPoint(x[3],x[4]);
     points.push(point);
   }
   if(x[5] && x[6]){
     point = new YGeoPoint(x[5],x[6]);
     points.push(point);
   }
   var zac = yqlgeo.map.getBestZoomAndCenter(points);
   var level = points.length > 1 ? zac.zoomLevel : zac.zoomLevel + 1;
   yqlgeo.map.drawZoomAndCenter(zac.YGeoPoint,level);
   yqlgeo.map.drawZoomAndCenter(points[0],level);
  };

The rendermap() method is a reusable way of showing a Yahoo map with a bunch of locations at the right zoom level. This version has been modified a bit, as I wanted a user icon at the centre of the map rather than another marker.

  yqlgeo.wiki = function(o){
    if(o.geonames){
      var out = '<ol>';
      for(var i=0;i<o.geonames.length;i++){
        var sight = o.geonames[i];
        out += '<li><h2>'+
               '<a href="http://' + sight.wikipediaUrl + '">'+
               sight.title+'</a></h2><p>';
        if(sight.thumbnailImg){
          out += '<img src="'+sight.thumbnailImg+'" alt="">';
        }
        out += sight.summary + '</p>'+
               '<p class="distance">'+sight.distance+' miles away</p>'+
               '<p class="url"><a href="http://' + sight.wikipediaUrl + '">'+
               'http://' + sight.wikipediaUrl + '</a></p>'+
               '</li>';
        var point = new YGeoPoint(sight.lat,sight.lng);
        var marker = new YMarker(point);
        marker.addLabel(i+1);
        marker.addAutoExpand(sight.title);
        yqlgeo.map.addOverlay(marker);
      }
      out += '</ol>';
    }
    Y.one('#sights').set('innerHTML',out);
  };

In the wiki() method we check if geonames returned us some points of interest and if it did we loop over them assembling the HTML string for displaying them as we go along. We also put new markers on the map for each sight that was found.

  yqlgeo.neighbours = function(o){
    if(!o.error && o.query.results && o.query.results.place){
      var out = '<ul><li>Around this area:<ul>';
      yqlgeo.neighbourdata = o.query.results.place;
      var all = o.query.results.place.length;
      for(var i=0;i<all;i++){
        var cur = o.query.results.place[i];
        out+='<li><a href="#n'+i+'">'+cur.name+'</a></li>';
      }
      out += '</ul></li></ul>';
      Y.one('#neighbours').set('innerHTML',out);
    } else {
      Y.one('#neighbours').remove();
    }
  };

The neighbours() method does the same, except we don’t plot on the map and instead we render out a list of locations with links with numbered anchors. We also cache the neighbourhood data in its own object for later use. If there are no neighbourhoods found, we remove their container from the document.

  Y.delegate('click', function(e) {
    e.preventDefault();
    var current = Y.one(e.target).get('href').replace(/.*#n/,'');
    if(yqlgeo.neighbourdata[current]){
      yqlgeo.getinfo(yqlgeo.neighbourdata[current]);
    }
  }, '#neighbours', 'a');

We use the event delegation functionality of YUI to add a click event to each of the links in the neighbourhood list. We then get the number of the link from the href attribute and call getinfo() with the data of the new neighbourhood.

  Y.delegate('click', function(e) {
    e.preventDefault();
    var dad = Y.one(e.target).ancestor('li');
    if(dad.hasClass('show')){
      dad.removeClass('show');
    } else {
      dad.addClass('show');
    }
  }, '#sights', 'h2 a');

});

For the sights list, we apply click handlers to each of the links inside the headings (not the Wikipedia ones in the description) and add or remove a class called show to or from the LI that contains the link that was clicked. This, together with the right CSS definition shows and hides the descriptions of the different places.

And that’s all there is to it! By using YUI for the functionality and YQL to get the data it is pretty easy to put together a neat little app like this.

Christian Heilmann (@codepo8)
Yahoo! Senior Developer Evangelist

Last week Yahoo!’s European Geo team was at the Location Business Summit in Amsterdam. Our main goal there was showing people what technology Yahoo! offers when it comes to finding things on this planet, and what you can do with the technology.

Overall, the experts at the summit agreed that location-based services are very good business and a marvellous idea, but also that there is quite a problem with accuracy.

To provide our users with a great experience when finding things in their surroundings, we need to pinpoint their physical (real) location. Without the users telling us where they are, this is normally done by checking their IP number on the Web — which can be terribly far off. For example, to see just how far your own IP is away from your real location, you can use this tool on a Firefox browser. You will see your IP location and get asked to share your location with the website. Allow this and you can see the distance. Click the following screenshot to try it out:

The workarounds are device services like the W3C geo location API used in the demo.

Aside from accuracy, attendees of the summit mentioned these other issues:

• Finding the place of a latitude/longitude (lat/lon) location
• Finding the place of an IP number
• Finding the places in a certain text
• Analyzing already existing Web content and finding places
• Finding things around you and geographical hierarchies

YQL Geo library

All these issues can be easily resolved, if you know how to use the tools. By using various geo services out there and Yahoo! Query Language (YQL) to mix and match them, we built a simple JavaScript library called YQL Geo. It solves most of these problems.

Finding a current user’s location

Using it, you can find the location of the current user with a simple JavaScript call:

yqlgeo.get('visitor',function(o){

  alert(o.place.name + ',' + o.place.country.content +
        ' (' + o.place.centroid.latitude + ',' +
               o.place.centroid.longitude + ')'
        );

});

The library automatically checks whether the visitor’s browser supports the more accurate W3C Geo API and asks them to share their location if it does. If there is no support, it falls back to the IP lookup.

Turning text into a location

You can also turn a text (name) into a location using the following code:

yqlgeo.get('paris,fr',function(o){

  alert(o.place.name+' ('+
    o.place.centroid.latitude+','+
    o.place.centroid.longitude+
  ')');

})

This wrapper call uses our Placemaker Service under the hood and automatically disambiguates for you. This means that Paris is Paris, France, and not Paris Hilton; London is London, England, and not Jack London.

Naming a latitude/longitude

You can get the name of a lat/lon location:

yqlgeo.get(33.748,-84.393,function(o){

  alert(o.place.name + ',' + o.place.country.content);

})

Analyzing a website content

And you can analyse the content of a website:

yqlgeo.get('http://icant.co.uk',function(o){

  var out = '';
  var all = o.place.length;
  for(var i=0;i<all;i++){
    out+=o.place[i].name+'\n';
  };
  alert(out);

});

You can try out each of these on the YQLGeo Demo Page. Also, you can peek under the hood (and learn all about the YQL statements) by checking the YQL Geo source code on GitHub.

GeoPlanet data

Once you have the location, the next big question is how to go on from there. To this end, we offer the GeoPlanet data and web service. Using this in-depth and validated data set, you can start at a certain location and find the children, siblings, parents, and neighbors in the geographical hierarchy.

To explore the detail and depth of the data set yourself, check out the GeoPlanet Explorer:

Where On Earth IDs, Concordance, and Geosetta

All in all, we have you covered for most of your geo needs — and by mixing the Yahoo! data with other sources like Geonames, you can get an even richer experience.

One remaining issue is that different geo data service providers have different ways of describing their data. Latitude and longitude are ambiguous: the same location could, for example, be the centre of a city or a landmark on top of it. This is why Yahoo!’s services use the Where on Earth ID to disambiguate. Other services have similar systems.

To use these different systems together you need a translation service, which is the newest offer we have. Our translation service is called Concordance. Using this service, you can convert between different services.

You can play with the system using the Geosetta demo:

In summary

The world of location-based services is not hard to master. The main issue we have right now is that all the building blocks for the solution are scattered across providers and the Web, and are hindered by licensing problems. If we put them together, we could give our visitors an amazing geo experience without having to jump through hoops to solve the issues.

Christian Heilmann (@codepo8)
Senior Developer Evangelist, Yahoo! Developer Network