Introduction
Most of the people think that Search Engine is a website where we can search a term or a phrase and the search engine gives back the results. While this is absolutely true, the power of a Search Engine is not limited to just searching a few terms/phrases. Search Engines are a very powerful tool that can help us find virtually everything on the internet.
At present, there are three major search engines competing in the market, Google, Yahoo and Bing! In this article, let’s unleash the power of a search engine and see how effectively we can use it. But before going ahead with the details, let’s first brush-up our basics about a search engine.
What is a Search Engine?
A Web search engine is a tool designed to search for information on the World Wide Web. The search results are usually presented in a list and are commonly called hits. The information may consist of web pages, images, information and other types of files.
Web search engines work by storing information about many web pages, which they retrieve from the WWW itself. These pages are retrieved by a Web crawler (sometimes also known as a spider) — an automated Web browser which follows every link it sees. Exclusions can be made by the use of robots.txt. The contents of each page are then analyzed to determine how it should be indexed (for example, words are extracted from the titles, headings, or special fields called meta tags).
Now that we know what a search engine is, let’s begin by understanding how to effectively use search terms or phrases. Badly given search terms may result in unexpected search results. Hence it is very important to know – What to search for?
Building Effective Search Words/Phrases
When we want to search a particular article, we first think of what is the focused area of that article. For example, if I want to search for “anonymous methods in C#”, then my search phrase should contain the highly occurring words from that article. These words should be spelled correctly and should be short and to-the-point. Let’s see some of the key points for making searches more effective:
- Spell words correctly
- Remember to leave a space between each word in your search query
- Use most effective words – i.e. words that you expect to occur frequently in that web site. The choice of words makes a lot of difference to the search engines. Remember that every word matters to the search engine!
- Use OR and NOT keywords to combine or exclude words. The support for all or some of these keywords differs from search engine to search engine. Also, note that the words OR and NOT should be capitalized
- Use fewer descriptive words, or try words that have a different but similar meaning. This may result in different search results and the one you are expecting
- Search for exact phrases by placing the search words within quotation marks
- Do not use long search phrases, since search engines limit the number of words that could be searched
- Click on the category to see category-specific search results such as web, images, people, maps, etc.
Here are few key points to be noted about most of the search engines:
- Search engines aren't case sensitive.
- Common words such as a, the, an, as, etc. are ignored by the search engine. These are called STOP words. If you want these words to be included, then enclose them in double quotation marks.
- If you are searching for a date, then make sure you use standard date formats. Any custom date format that you are using might not be known to the search engine. Also as far as possible, use month names instead of their integer equivalents.
- There is no need to use the word “AND” in your search query. By default, all the searches are “AND” searches .i.e. they will use your search words using AND. For example: “green trees” means “green AND trees”.
Apart from these, you can use binary operators and wildcards to fine tune your search results:
- Phrase Search (“”): As discussed earlier, use double quotation marks for searching exact phrases.
- Search within a website (site:[site_name]): The search of a query can be limited to a particular website by including the term “site:[site_name]” for example: C# site:sandeep-aparajit.blogspot.com. This search query will search for C# in the website http://sandeep-aparajit.blogspot.com.
- Search for a title (intitle:[search_words]): If you want to search for pages with a specific title, then you can use the “intitle” tag for searching the query. For example: intitle:Sandeep, will search all the pages which has title containing the word “sandeep”.
- Search exact term (+): By attaching a + immediately before a word (remember, don't add a space after the +), you can get the search results for the precisely typed word. This will specifically ignore any synonyms.
- Terms to exclude (-):Attaching a minus sign immediately before a word indicates that you do not want pages that contain this word to appear in your results. This is typically used to exclude a word from your search result. The (-) sign works similar to the NOT keyword we discussed above.
- Fill in the blanks (*): The * represents a wildcard and indicates that the search engine should consider * as a set of unknown words.
Apart from these generic tips, there are some search engine specific tips that you can find on their respective help pages. By following these common rules, you can make an effective search.
Searching for Files
It is often required to search for files on the internet. And at the same time, it is very difficult to visit all the websites and check if the required file is present in that website or not. For example, if I want to search for PowerPoint presentations on C#, I can search the presentations, i.e. a file having extension .ppt or .pptx using the “filetype” tag as shown below:
Search Query: C# filetype:ppt
This will give me search results that directly point to the PowerPoint presentation which are related to C#. This is supported by Google, Yahoo and Bing.
Finding Vulnerabilities Using Search Engine
This might sound weird at first sight, but this is absolutely true. Hackers or malicious attackers make use of search engines to find web based vulnerabilities. A powerful search engine is actually a helpful tool for hackers for finding various flaws and vulnerabilities. Let’s understand this using an example. TSWEB is a tool used to acquire a Remote Desktop Connection via Internet. Many of the companies expose their computer systems via TSWEB for flexible operation and controlling of the system. You need to expose a URL for letting an internet user access your computer system via TSWEB. The TSWEB tool is specifically used by system administrators to control the systems remotely. But when this exposed URL gets indexed by a search engine, it actually gets exposed to Hackers. Hackers search for TSWEB enabled systems using the search engine and then these systems are attacked. Hence it is a best practice to hide such URLs from hackers, using robots.txt. Thus it becomes important to make the website Search Engine Optimized for efficient searching and security reasons.
Search Engine As Calculator
Yes! Your search engine is your handy online calculator. Search engines are made intelligent enough to search for mathematical calculations and at the same time performing those calculations and giving out the result. So if you search for 5+2 in search engine, then it will give the result as 7 and will also search for websites containing the expression 5+2. Most of the arithmetic calculations are performed by search engines.
Custom Search Engine for Your Website
Often we require a search engine which will search for keywords ONLY in our site or in a list of sites. This is typically required for products site, where a huge list of products are display on the web and the user wants to search for a particular product on your site. This can be achieved by creating a custom search engine. Both Google and Bing provide a facility for creating a custom search engine that suits your needs. Bing also provides programmatic interface to the search engine where it sends the search result in the form of XML or JSON. Bing APIs are exposed for achieving this.
Why Not Earn Money from Search Engines?
Yes, search engines provide a facility to embed advertisements into the result set of your custom search engine. When users click on these advertisements, you get paid. Well this is not the primary use of a search engine, but definitely one of the most used options for earning. ;)
Search Engine Optimization
Search engine optimization (SEO) is the process of improving the volume or quality of traffic to a web site from search engines via "natural" ("organic" or "algorithmic") search results. By following the SEO practices you increase the probability of your site appearing at higher positions in the search result. It’s important to note that SEO practices provide the potential for higher content coverage and ranking, but do not guarantee it. SEO discussion is out of scope for this topic. You can read my blog on SEO tips for search engines.
This is only a part of the Search Engine capabilities that we have discussed here. The Search Engine is a huge beast! Kudos to Google, Yahoo and Bing for providing such powerful search engines!
History
- First version of this article released on June 17, 2009
