Getting Noticed by Search Engines

The Search Engine Dilemma

In the old days, marketers went looking for customers. We studied customers, figuring out where they lived, tracking their habits. Then we decided what message we would show them, as well as where we should show it, how and when. In those days, marketers ran the show.

But things have changed.

In the new economy, and especially online, customers are in charge. They look for us. And the number one way customers look online is with a search engine. A recent report by icrossing.com says more than half of adult Internet surfers use a search engine every time they are online. In studies conducted by Web usability expert Jakob Nielsen, more than 88 percent of Web page visits were found to begin at a search engine.

Getting noticed by search engines is crucial to online success. Not only do you want to appear in the search engine results pages (called SERPs), but you want to rank prominently, as high up on the page as possible. That way, the searchers who are looking for you can easily spot your site, click through and hopefully, convert – to a sale, a donation, a request for more information, a registration or whatever your goal is for the website.

Search engine optimization (SEO) is the process of optimizing a web page for search engines by carefully crafting the site’s content, structure and the behind-the-scenes coding. Of course, the intent of your website is to communicate clearly with human visitors. Search engine optimization is about making it easy for spiders to navigate and index your site – but without compromising content or clarity for human visitors.

Optimize = Measure + Modify

The first step in optimizing your site is defining business goals. What is your site attempting to accomplish? Once you’ve identified the purpose, whether it is sales, lead generation, event registration or customer information, the next step is to develop key performance indicators (KPIs).

KPIs are those behaviors you want your targeted site visitor to complete. They become the measuring stick. Examples of KPIs are leads generated, orders placed, average order amount or average order items, the ratio of orders placed to page views, etc. You might also need to track the most common entry and exit pages, as well as which search engines are sending traffic and what keywords searchers are using, or not using, to find your site. Equipped with the right KPIs, you then start measuring against them, looking for trends and analyzing changes.

Finally, the most critical stage is actually modifying pages based on what you’ve learned. This may involve editing or re-writing copy or developing new landing pages to specifically address the questions of particular groups of searchers. Optimizing your site is an ongoing process, as you gather more insight on customers and their needs, you’re able to adjust the site content and structure to improve search engine prominence.

How Search Engines Work

Spider-based search engines create listings by sending an automated spider (also called a crawler or robot or bot) out into the Web. The spider follows links from page to page and collects information from every page it can visit. Then, it returns all the data into a central index or catalog. Finally, when a searcher requests a word or phrase, search engine software sorts through all the indexed entries and creates results pages based on that particular request. The specific methods each search engine uses to spider, index and rank pages vary and are corporate trade secrets. In this piece, we will look at what marketers can do to make sites as welcoming as possible to search spiders. The goal is to get as many pages from your site listed in search engines results as possible and to get them listed accurately and prominently, so that when potential customers and clients come looking, your site will be easy to find.

Building Blocks of SEO

Before we get into the specifics, it’s helpful to understand some general principles for optimizing pages for search engines. The first principle is that human searchers (and therefore search engines and spiders) want fresh, relevant content. The second principle is that spiders are smart enough to go crawling around the Web on their own, but they aren’t geniuses. Spiders are actually fairly simple computer programs. Spiders can’t problem-solve, so sites and pages have to be built in ways that spiders can understand.

If you understand those two principles – that spiders like current content (just like humans) and that spiders need clear, direct instructions, most of what we call search engine optimization is fairly straightforward, common-sense advice.

With those two principles in mind, there are five building blocks for optimizing a page for search engines: keywords, pages, meta data, links and architecture.

Keywords. Human searchers generally use one- to three- word phrases when using a search engine. These keywords are the most basic element of SEO. Spiders use complex formulas to guess at the topic and focus of your pages. If you have identified keywords and then used those keywords to write copy, headlines and page titles, you make it easier for the spider to identify the focus of your page correctly.

Define keywords for each page of your site that are relevant and actually searched. Several online tools are available to help you research possible keywords. At these sites, you can enter keywords and receive suggestions for similar words and phrases and also get a report of how often those phrases are searched.

When defining keywords, look for words that succinctly describe your content. Keywords also have to be popular – meaning they have to be words and phrases that are actually searched and searched frequently. We found in working with an attorney who specialized in OHSA work that no one was searching for “OSHA attorney.” But searchers were looking for “OSHA regulations” and “OSHA safety.”

The competitiveness of the keyword is another issue to consider. For example, if your keyword is “socks,” you’ll end up competing with 8 million other pages. But switch to “women’s soccer socks,” the competition drops dramatically.

Once you’ve defined the keywords for your site, use them in the body copy of the page frequently but naturally. Be careful not to over do it, as the algorithms used to index pages are sophisticated and can sense attempts at fraud or manipulation. Also use keywords in headlines, in links on the page and in page titles. All of these techniques make it easier for the spider to determine the focus of the page.

Pages. Individual pages are the basic units of the Web. Websites aren’t ranked by search engines – individual pages are. Every page is spidered, indexed and served individually. That means each page should be given a unique, strong title, because when that page is shown on a SERP, the page title will appear as a bold heading. When writing page titles, think of them as headings, because that’s how they’ll end up in the search engines. Also remember that when a page is bookmarked by a user, that page’s title is saved to a bookmark list. If every page on your site has the same title, each entry in the bookmark list will appear the same. It’s easier for human searchers and spiders if every important content page of your website has its own, unique name and unique keywords that quickly summarize the content.

Another issue is the length of Web pages. Search engines often don’t use the whole page when looking for keywords. Most only consider the first 100 kilobytes or so. This has implications for long pages or for PDF files offered as downloads. If you have a 400-page PDF file, it’s possible that only the first few paragraphs of the first page will be considered for keywords. The whole document will be indexed, but only the first few paragraphs will be evaluated for keywords. Your searchers could be missing a lot of valuable, relevant content. In the case of a long page or a PDF file, you might want to offer an abstract or a table of contents with a brief summary of each chapter that incorporates the chapter’s keywords. On all pages, try to put your most relevant, important content towards the top of the page.

Pay attention to copy length on individual pages. Search engines seem to prefer longer copy because these tend to indicate content-rich pages. Most search engine experts recommend page lengths of around 250 words.

Meta Data. Meta data refers to information that summarizes or describes a particular set of data. On a web page, meta data is generally not apparent to human visitors, but it’s important to search engine spiders. When building a web page, you’re actually developing two types of content: content for real people who will visit the site and clues for search engine spiders that will then assist searchers looking for your content.

Spiders pay attention to some meta tags. The DESCRIPTION meta tag is important because the copy within this tag often appears on the results page. The KEYWORDS meta tag should contain keywords and phrases that are relevant to the page. Relevancy is important here, because a page can actually be penalized by a search engine if keywords are used in the meta tags that don’t appear in the human-readable body of the page.

Links. In-bound links to your site are important to search engines. They give clues to spiders about what your pages are about and are used by spiders to get an indication of the popularity of the site. Search engines consider the quality of the links to your site. Participating in reciprocal links (where website owners agree to link to each other on their own sites) and so-called link farms (sites created for the sole purpose of linking to other sites to build link popularity) can actually incur penalties from search engines. Conversely, search engines look at in-bound links from companies or organizations in your peer group or industry as a positive indication that your site is offering relevant or specialized content. Another important reason to develop in-bound links is that it’s a way to drive attention to your site. Your home page might be seen by only half of site visitors because they first arrive on pages other than the home page.

Architecture. Site architecture – how the whole site is structured and how individual pages are organized – should be as simple and clear as possible. On individual pages, keywords and important content should be placed prominently in the copy. Each page should be named individually, as we mentioned above, and a site map should be provided, which will help human users and spiders to navigate quickly. Internal links that lead from one section or page to another within the site are also helpful to users and spiders. Heading tags such as the <H1> and <H2> are another way to show spiders the important content. Use keywords in heading tags, but keep them short and to the point.

It’s not a coincidence that the almost all of the recommendations above are helpful not just for spiders, but for human visitors, too. That’s because spiders exist to help human searchers. Spiders crawl the web, indexing billions of pages into a catalog that can then be searched and served to human users. Spiders want what humans want: clarity, strong content and ease of navigation. Chances are, if your pages are satisfying the needs of your users, you’re already on the road to satisfying the needs of spiders.

Barriers

There are a handful of common mistakes that pose roadblocks for spiders. A few Web design techniques actually make it nearly impossible for spiders to read and index pages. That’s not to say these techniques can’t be used, it’s just that they have to be thoughtfully used in ways that accommodate spiders. Some things to avoid:

• Frames – Most spiders are unable to navigate frames. Content within frames is likely to be missed entirely by search engines, as they can’t jump from frame to frame.
• Forms and Drop-Down Menus -- These pose a similar problem to spiders as frames. Spiders can’t submit requests through a form or drop-down box, so the content behind the form or drop-down won’t be indexed. One solution is to create an alternate way for spiders to get through by offering an alphabetical list of the content or a linked site map. By offering both methods, you meet the needs of human visitors and spiders.
• JavaScript – JavaScript is programming language that can be embedded in Web pages and read by browsers to add interactive or dynamic functionality. When some spiders visit, they pretend to be browsers with JavaScript disabled. Some sites redirect browsers with JavaScript turned off to another page or site. We worked with a large e-commerce site with thousands of pages that wasn’t appearing in search engine results at all – despite its huge inventory of pages. The culprit was a JavaScript redirect. Everytime a spider arrived, it was redirected. Once that was fixed, the pages were soon indexed and its prominence in search engines, along with its online sales, soared. There are ways to effectively incorporate JavaScript in websites to take full advantage of its functionality, without discouraging search engines.
• Flash – Avoid welcome or splash home pages that offer little information or that are composed entirely of Flash animation. Flash is an animation software that makes it possible to add video or cartoon-like graphics to Web pages. But Flash is difficult or impossible for spiders to interpret. We often see home pages that heavily incorporate Flash, which is a problem for two reasons. First, if the whole home page is made of Flash, spiders won’t be able
• to get past the front door, effectively eliminating the site from search engines. Second, home pages should show the most important content of your site, as spiders and human visitors will often see the home page first. We’re not suggesting throwing out Flash entirely, as it is popular and can be used to great effect. On the Duo home page (www.duoconsulting.com), we’ve included a small amount of Flash, but left plenty of room for content. This way, we get the benefit of an appealing page, but without driving away search engine spiders.
• Bad HTML and Poor Spelling – Coding mistakes and spelling errors cause problems. Coding errors create obstacles for a spider, and a spelling error can keep your content from showing up in search engine results. On the other hand, attention to commonly mis-spelled words and tactfully incorporating those words onto a page can bring incremental traffic.
• Broken Links – Some search engines will penalize a page for having broken links, which are links that point to pages that no longer exist or have been moved. It’s another reason to stay on top of your site, checking and updating to insure accuracy.

Conclusion

Getting noticed by search engines is key to getting your business in front of customers online. Optimizing your site so that you have the best possible chance of appearing prominently in search engine results requires:

1. Knowing the business goals for your site;
2. Understanding what search engine spiders are looking for and how to accommodate them with researched keywords and carefully written and coded pages – without sacrificing clarity for human visitors;
3. Avoiding barriers that block spiders or prevent content from being indexed;
4. Finally, measuring and modifying pages based on what you’ve learned.

At Duo Consulting, our Web experts have helped businesses and organizations make significant improvements in their online sales and lead generation by using search engine optimization. To learn more about Duo and how to put our experience to work for you, see www.duoconsulting.com.

Online Tools for Keyword Research

• http://www.wordtracker.com/
• https://adwords.google.com/select/main?cmd=KeywordSandbox
• http://www.teoma.com/

Winning Keywords are:

• Relevant
• Popular
• Non-Competitive

Using Keywords to Optimize Web Pages

1. Establish the content focus of the page and define keywords.
2. Use keyword research tools (see above) to look for popular searches in your area of focus.
3. Search on candidate keywords to estimate how competitive they are. The more pages that are indexed, the more competitive the keyword.
4. Select those keywords that are popular (searched often) but less competitive.
5. Incorporate selected keywords into the page title, headlines and copy.