By Chris Silver Smith
Google is combining Street View images, neural network analysis, and a reverse Turing Test via reCAPTCHA to analyze the exteriors of storefronts.
Welcome to the 21st century, where you may need to perform “optimization” on the real-world exterior of your stores to help ensure the best rankings in Google!
While at SMX Milan last month, colleague Luca Bove, in his presentation on the “Making Sense Of The Local Landscape” session, mentioned how Google potentially employs an algorithm with its local search systems in order to improve maps’ locational quality.
I was struck by how the data was very likely also used in validating business addresses for purposes of eliminating spam as well.
Google’s development of these capabilities is just one part of its data collection about the actual world around us that folds into the local search algorithms.
The Neural Network by rajasegar (CC BY-NC-ND 3.0)
There are some reasons why these sorts of developments are becoming more important to local businesses and local search marketers.
In the earlier days of the internet, a business tended to assume that anything about them on the internet was placed there by them and thus controlled by them, much as their presence in the yellow pages directories may have been handled in days past.
Of course, as the internet has matured, perhaps most businesses have come to realize that they do not solely control their online presence — numerous online directories, search engines, and other types of sites all compile data about entities and display it in various ways.
In the Web 2.0 world, many of us have learned and continue to learn how our online reputations and presence are something of a gestalt, formed of all the disparate pieces of information that are collected together and served up in multitudinous ways. (Indeed, this evolution resulted in the need for search marketing agencies and the more specialized niche of online reputation management or “ORM.”)
Even considering how the information age has evolved to find, collate and deliver up data about businesses and organizations, I’m not sure there has ever been a time when the physical reality of the world itself has been more intermingled with the virtual world.
More and more, things that happen in the physical world are having a direct effect upon your online presence, and so it’s becoming vital for local businesses to pay more attention to how these elements may affect their presence within local search.
Using Real-World Data To Pinpoint Locations
The location itself is one prime piece of the online presence, as we all realize. It’s sometimes a challenging datum.
As I described in my article from 2008, Top Causes Of Errors In Online Mapping Systems, online mapping systems have often been challenged with translating street addresses into the geocodes and pinpointing the locations on maps (though this situation has been steadily improving for many years).
In the past, there was a large element of estimation involved in mapping. As I described in 2008, there were instances when digital systems knew that an address was on a street, and on a particular side of it, but these systems had to interpolate in placing the addresses — spreading out all addresses equally along one side.
There were also instances when businesses’ addresses were clumped — such as at shopping centers and in tall office buildings.
As you may know, Google and some other systems incorporated building outlines for a great many cities into locational determinations. This sort of data likely helped further in determining actual organization address locations.
In addition, more and more types of street data pours into Google and other mapping systems, which map data sources from a variety of city, state and national government data sources.
Two different methods for pinpointing addresses include “rooftop” and “front door.” Using satellite images and/or building outlines, the centerpoints of buildings could be used, as well as front-door entrances, when computing location geocodes or when calculating driving directions.
Even considering the sophisticated mix of location determinations, there can be a lot of instances when geolocations are incorrectly calculated.
For businesses that like to “touch” their data a lot — either by uploading bulk files of multiple business locations in the case of large chain store companies, or small-to-medium businesses that simply take greater care in updating and customizing their online profiles — Google and other mapping providers will tend to have greater confidence in the geolocations associated.
In many cases, these companies will upload the precise geolocations of their outlets themselves, or they may hand-tweak them, such as with Google’s map pinpoint correction tool in Google Places interface.
But, for the countless businesses that are less “touchy-feely” with their online presence — and, this number remains in the many millions, I believe — mapping providers are less-confident with their generated location pinpoints.
A New Google Patent For Identifying Addresses
This may be one prime reason why Google developed the patent that was granted in July of this year entitled, “System and method of determining building numbers.”
In cases when you have longer facades of buildings with many doors on them, you begin to have a great chance of algorithmically determining door-front locations of addresses if you could have a system that would read the numbers off of the doors and doorframes.
On the face of it, this sort of system seems simple: Google’s mapping vehicles capture the fronts of many buildings, including storefronts, many of which include address numbers affixed to the building fronts.
Once these many image files are associated with geolocations of the street spots where they were captured, it’s a matter of running an OCR system through the images to capture any words, and most especially numbers.
Once you have this dataset, process it against the business/organization addresses in your database and see if the locations are close enough in proximity, or whether you need to adjust and update your business location.
Extracting the numbers from the images is a complex task, but Google research papers indicate it has developed a deep neural network system that can be trained to identify numbers of up to five digits long.
Google has apparently employed this system with a fairly high success rate and is automatically translating the numerals from images.
In practical application, this functionality is quite involved, and still prone to some percentage of errors or numbers that the neural net simply cannot identify. To combat this, Google apparently mixes in a human quality-check, according to its patent:
If an extracted value corresponds with the building number of the address of interest such as being substantially equal to the address of interest, the extracted value and the image portion are displayed to a human operator. The human operator confirms, by looking at the image portion, whether the image portion appears to be a building number that matches the extracted value. If so, the processor stores a value that associates that building number with the street level image.
How Is Google Incorporating Human Input?
Google has already incorporated such a system, perhaps in a couple of ways. First, it has been known for a while that Google has made use of a staff of humans to verify new business information by phoning these new businesses after they submit their listings to Google Places.
It’s quite conceivable that Google could display the scanned image number information to these individuals when they are performing their duties in validating new business listings, using some interface to display the numbers and asking the validators to click “yes” or “no” to determine if the address number images should associate with the physical location of businesses.
I’ve had reason to believe that these individuals were previously viewing Street View images, anyway, in order to help validate the businesses.
In cases where there is a suspect Street View image, such as if the validators don’t see buildings at the location, or if it seems to be of a cemetery or some such thing — then those listings get suspended. These building numbers could be used in the same way.
Indeed, if we look at the images that Google is sometimes delivering through its reCAPTCHA interfaces (a service Google offers for free to webmasters for the purpose of validating submission form content to reduce spam and harvesting of data by automated systems), then we see what are clearly building numbers being offered up in order to obtain the free labor to get back text validating their OCR system translations — use of the reCAPTCHA system is essentially a reverse Turing test. (Though Google has recently announced a change to its reCAPTCHA API, it will still be serving up these real-world images.)
Below are examples of Google’s reCAPTCHA interface and building numbers translated through it, according to Google’s research paper, “Multi-digit Number Recognition from Street View Imagery using Deep Convolutional Neural Networks”:
However, I’ve also seen Google leveraging its free captcha (now “reCAPTCHA”) systems for validating and improving quality on OCR translations of book text, and this is also an area where Google is obtaining human participation to improve address numbers and location associations.
In addition, Google’s reCAPTCHA service page indicates that it is also using these images for the purpose of reading the text on street signs. So, the data is further enhancing local search by verifying the street names and street locations from the Street View images (example below).
Now, Google already held a patent from further back that involved potentially identifying details from Street View images in order to help validate online business information in connection with building exterior information such as business name signs, hours of operation, street numbers and menus.
With the vague name, “System and method for the calibration of a scoring function,” this patent was granted in 2012. (Bill Slawski wrote of these developments back when the patent was granted and joked of posting a robots.txt disallow sign on his home to instruct search engines to not index his home images!)
I would say that the new patent, and the fact that Google’s reCAPTCHA images are apparently containing street address image numbers and street signs, indicate that it’s highly likely that Google is now coordinating this data in the way that the recent patent describes, and is likely folding this into the data quality of Google Maps/Local Search.
The ongoing development of these ideas indicates that it’s an important and prioritized area of the local team’s development efforts.
How Does This Impact Local Search Marketers?
Knowing that this real-word data is getting folded into Local/Maps and is potentially able to have an effect upon your online presence and local rankings, what should you do?
- Do Nothing: In most cases, local businesses need to do nothing. Most local businesses probably make an effort to ensure that the exterior of their shops make a good impression upon potential customers, and work to make sure that their signage is all accurate.
- Inaccurate Signage: If your signage is inaccurate, you need to update it! Various organizations such as the Better Business Bureau may ding you anyway if you are found to be misleading consumers in some way, so accurate representations on the exterior of your business place ought to be kept updated at all times.
- Business Name Change: Did you acquire a business and change the name, but not update the exterior sign? This sort of thing could now contribute to having outdated listings continuing to exist and rank in local search results, even if you set up a new business listing in Google and flagged the legacy listing for deletion. Or, did you attempt to add a new listing, only to have it go into a “pending” status that never resolves? Your business name is a key element and your exterior signage had better coordinate accurately with your business name in Google Local. Those of us in local search marketing have long harped upon auditing one’s citations online and correcting any that are out of sync, and this activity now extends to your offline, brick-and-mortar location information as well!
- Building Signage: Is the signage outside of your building confusing? This can happen with a great many strip shopping centers and businesses that are abutting each other in dense metro areas. Does it appear another business is located where yours is? Increase the user-friendliness of your exterior by trying to make it clear which businesses go with which signs.
- Signage Legibility: For that matter, is your signage easily legible? If your sign was painted up in a kooky font that is virtually illegible, or if your building number was painted on in a funky way by your favorite nephew, re-think it, and perhaps replace it.
- Hours of Operation Visibility: Are your hours of operation posted in large letters outside your business? If so, you may want to be sure they reflect the same info that you post online in directories and in local search engines. One of the illustrated embodiments shown with the earlier patent showed an hours of operation sign on a shop window — so, this might not be as farfetched as it sounds.
- Street Sign Visibility: Are the street signs in your area legible? I once spent an hour driving around the streets of a small town in Texas, searching in vain for a special sale — all because the words were completely worn off of all of the street signs! You may see computer technicians on investigative TV shows performing all sorts of cool image-clarifications on digital images, but I really doubt Google has much if any of this sorts of enhancement on their images. So, if your street signs in your area require psychic assistance to read, campaign to your local government to have them fixed and updated ASAP!
- Mail Store Location: For those companies using a mail store as their business location (a risky proposition for local search marketing), you may need to see if the Street View representation of the location makes it clear that your business couldn’t possibly be in this location. Increasingly, attempting to fool Google Local may work against you, as my fellow columnist Greg Gifford stated a few weeks ago.
I could take this even further to a more obsessive level and mention things like avoiding having your business look like a dump, or having consumer-unfriendly jokes such as your door sign always displaying a “CLOSED” notice (as Bernard Black’s bookshop sign did in the hilarious British sitcom, “Black Books”).
The eternally-closed door sign in the British sitcom, Black Books, behind actor Dylan Moran. Copyright of Channel 4.
(Your hours of operation were already a ranking component, particularly on mobile devices!)
But, these things really ought to go without saying! Give people a good experience when they attempt to find and visit your shop.
The real takeaway of all of this is that local store operators must always keep up their brick-and-mortar location’s real-world presence, in addition to feeding and watering their online presence.
Don’t let your exterior signage get out of whack — your exterior must be maintained both for the sake of your local search rankings as well as for making your business friendly and easily-approachable for the people who would be your customers.
The post Local Search Marketers: Optimize Your Physical Location For The Virtual World appeared first on Search Engine Land.