Posted on May 24th, 2011

When it comes to adding integrated site search to a web application there are plenty of options. One of those options is to leverage an existing search provider like Bing and elektromotory-menice.cz via an API. This provides a quick way to get a site search solution up and running in your web application, provided that Bing has indexed your site of course! Lets pull out our keyboard tray and build us some Bing integration code, starting with some setup.

Initial Setup

1. Apply for a Bing API 2.0 AppID (Windows Live account required).
2. Read up on the API Basics. It is important to note the What you must do and What you cannot do sections at the end of this document.
3. Use the Bing Webmaster Tools to submit your site and get Bing to index your site pages (if you haven't already).

Bing API Summary

We are going to use the Bing Json API to run a web search query and get the results back in Json. The url for the Bing Json API is:

http://api.search.live.net/json.aspx

It takes the following parameters:

Appid - Your unique AppID.
sources - The type(s) of content to search. We will be using the web source to search web pages.
query - The query string to search for.

To conduct a search of content on a single site you can pass in the following as part of the query variable:

site:iwantmymvc.com

The full url structure to hit the API with a request for pages on the iwantmymvc.com site with the term twitter would look like so:

http://api.search.live.net/json.aspx?Appid=YOUR_KEY&sources=web&query=site:iwantmymvc.com%20twitter

Note that you would need to replace YOUR_KEY with your AppID from Bing.

Bing API Code and Json Deserialization

If we begin with an empty MVC 3 web application, our first step is to add some appSettings to the Web.Config. We will add keys for the API url, our API key, and the site portion of the query. Assuming that the Web.Config file is the stock file that comes with a new MVC 3 web application, the appSettings section will look like so:

With our settings in place we can move to writing some classes to handle the communication to the Bing Json API and to represent the result objects. We will take advantage of the JavaScriptSerializer class in the .NET Framework (versions 3.5 and 4.0) to handle deserializing a Json string into C# objects. Our application needs some classes that represent the result data structure from the Bing API, so lets create those. We will create a folder under the Models folder called BingApi and add a class file named BingApiSearchResponse.cs to that folder. This way we can leverage a namespace structure to organize our Bing specific code and be free to name our classes with names that match the property names in the Json string returned to us. While not required, it will make it more user friendly when reading the code and wrapping your mind around how it maps to the Json result string.

An example of the Json result string from a query request for site:iwantmymvc.com twitter:

{
    "SearchResponse": {
        "Version": "2.2",
        "Query": {
            "SearchTerms": "site:iwantmymvc.com twitter"
        },
        "Web": {
            "Total": 2,
            "Offset": 0,
            "Results": [{
                "Title": "Building dynamic content templates using Razor - I Want My MVC",
                "Description": "Long description here...",
                "Url": "http:\/\/iwantmymvc.com\/2011-03-20-dynamic-content-templates-using-razor",
                "CacheUrl": "http:\/\/cc.bingj.com\/cache.aspx?q=twitter&d=4878726567300861&w=4717360c,f7ae1ce",
                "DisplayUrl": "iwantmymvc.com\/2011-03-20-dynamic-content-templates-using-razor",
                "DateTime": "2011-05-19T23:58:00Z"
            }, {
                "Title": "Build a MVC web site HtmlHelper library and deliver it with NuGet ...",
                "Description": "Long description here...",
                "Url": "http:\/\/iwantmymvc.com\/mvc-web-site-htmlhelper-library-and-nuget",
                "CacheUrl": "http:\/\/cc.bingj.com\/cache.aspx?q=twitter&d=4587347404456266&w=16e33228,94f2897a",
                "DisplayUrl": "iwantmymvc.com\/mvc-web-site-htmlhelper-library-and-nuget",
                "DateTime": "2011-05-19T20:19:00Z"
            }]
        }
    }
}

To get JavaScriptSerializer to work its magic we want to create and object graph that mirrors the Json structure. Lets take a look at the classes that will represent the Bing Json API results and then we will discuss how they work together. Note that I am a big proponent of putting each class definition in its own file, but delivering content via a web post tends to bend your practices a bit to provide a better content consumption experience for your target audience. That being said, here is our object graph all in a single class file named BingApiSearchResponse.cs:

using System;
using System.Collections.Generic;

namespace Website.Models.BingApi
{
    public class BingApiSearchResponse
    {
        public SearchResponse SearchResponse { get; set; }
    }

    public class SearchResponse
    {
        public float Version { get; set; }
        public Query Query { get; set; }
        public WebResponseType Web { get; set; }
    }

    public class Query
    {
        public string SearchTerms { get; set; }
    }

    public class WebResponseType
    {
        public int Total { get; set; }
        public int Offset { get; set; }
        public List Results { get; set; }

        public WebResponseType()
        {
            this.Results = new List();
        }
    }

    public class WebResult
    {
        public string Title { get; set; }
        public string Description { get; set; }
        public string Url { get; set; }
        public string CacheUrl { get; set; }
        public string DisplayUrl { get; set; }
        public DateTime DateTime { get; set; }
    }
}

The BingApiSearchResponse is our top level class. This is the class we will feed into the JavaScriptSerializer.Deserialize method. The rest of the classes represent property object types within that class and the other classes. All of the properties have names that match those in the Json string. Note that these are not case sensitive. So if you have some Json that you are working with that doesn't follow the same naming conventions that you do (say, camel case on public properties instead of pascal case) you are free to name your class properties in your own convention. However, you are stuck matching the characters in the property names. If there was a Json property named search_response then your class property would have to contain that underscore. If you wanted to work around this you could look into using the DataContractJsonSerializer class instead of the JavaScriptSerializer class. This would allow you to decorate your class properties with the DataMember attribute and specify the name from the Json string to map the data.

With our response object classes written we can turn our attention to creating a client class to call the Bing API. Within the Models/BingApi folder we will add a class file named BingSiteSearchClient.cs. This class will have a constructor that takes in 3 strings for our settings as well as an empty constructor that automatically injects the settings from the Web.Config. This will allow us to call the empty constructor from our controller actions, but still allow us to write unit tests against the code without requiring a Web.Config file. The class will also have a method named RunSearch that will take in a query string and return a SearchResult object. For the scope of this article we will not be digging into pagination of the search results. Note that it can be done fairly easy since the search results from the Bing API return the total count and the offset. Lets take a look at the code and then review what it does.

using System.Configuration;
using System.IO;
using System.Net;
using System.Web.Script.Serialization;

namespace Website.Models.BingApi
{
    public class BingSiteSearchClient
    {
        private string bingJsonApiUrl;
        private string bingApiKey;
        private string bingSiteQueryPiece;

        public BingSiteSearchClient() : this(
            ConfigurationManager.AppSettings["BingJsonApiUrl"], 
            ConfigurationManager.AppSettings["BingApiKey"],
            ConfigurationManager.AppSettings["BingSiteQueryPiece"])
        { }

        public BingSiteSearchClient(string bingJsonApiUrl, string bingApiKey, string bingSiteQueryPiece)
        {
            this.bingJsonApiUrl = bingJsonApiUrl;
            this.bingApiKey = bingApiKey;
            this.bingSiteQueryPiece = bingSiteQueryPiece;
        }

        public SearchResponse RunSearch(string query)
        {
            var url = string.Format("{0}?Appid={1}&sources=web&query={2} {3}",
                this.bingJsonApiUrl, this.bingApiKey, this.bingSiteQueryPiece, query);
            var result = string.Empty;
            var webRequest = WebRequest.Create(url);
            webRequest.Timeout = 2000;
            using (var response = webRequest.GetResponse() as HttpWebResponse)
            {
                if (response.StatusCode == HttpStatusCode.OK)
                {
                    var receiveStream = response.GetResponseStream();
                    if (receiveStream != null)
                    {
                        var stream = new StreamReader(receiveStream);
                        result = stream.ReadToEnd();
                    }
                }
            }
            if (string.IsNullOrEmpty(result))
                return null;
            var javaScriptSerializer = new JavaScriptSerializer();
            var apiResponse = javaScriptSerializer.Deserialize(result);
            return apiResponse.SearchResponse;
        }
    }
}

We define 3 private fields to store our settings and have our 2 constructors to handle setting those field values. The RunSearch method takes in a query string. This will be the query string entered by a user in a search box on the site. We are expecting this to contain the terms the user wants to search for. The RunSearch method will take our settings and this search term and append them all together to form the url request to the Bing Json API. From here it uses the WebRequest class to call out to the API and read the response stream into a local string variable. This is the same approach used in my previous post, Write a Twitter user timeline controller action in MVC 3. Once we have received the response text we create a new JavaScriptSerializer object and call the Deserialize method to convert the Json string into our class object. Finally, we return the SearchResponse object from our BingApiSearchResponse.SearchResponse property.

Controller Actions and Views

Continuing on the assumption that we are working with an empty MVC 3 web project, we will create a HomeController that will have action methods named Index for our home page and SiteSearch for our search results page. The HomeController code:

using System.Web.Mvc;
using Website.Models.BingApi;

namespace Website.Controllers
{
    public class HomeController : Controller
    {
        public ActionResult Index()
        {
            return View();
        }

        public ActionResult SiteSearch(string query)
        {
            var bingSiteSearchClient = new BingSiteSearchClient();
            var model = bingSiteSearchClient.RunSearch(query);
            return View(model);
        }
    }
}

The SiteSearch method instantiates our BingSiteSearchClient object, calls the RunSearch method, and sends the returned SearchResponse object in as our model to the View.

The Views/Home/Index.cshtml file has some simple markup to give us our bearings.

@{
    ViewBag.Title = "Home";
}

Home

The Views/Home/SiteSearch.cshtml file is a strongly typed view that has markup to render some summary info about our search as well as the search results.

@model Website.Models.BingApi.SearchResponse

@{
    ViewBag.Title = "Site Search Results";
}

Site Search Results

    Found @Model.Web.Total results for the query @Model.Query.SearchTerms.


@foreach(var item in Model.Web.Results) 
{ 
    

        @item.Title
        
@item.Description
        
@item.DisplayUrl
    
 
}

The last piece of the puzzle is adding the search box to our site shell. While we are at it we will add a link back to our home page for giggles. We can update the stock /Views/Shared/_Layout.cshtml file to look like so:

    
    
    
    
    



    
    @RenderBody()

We use the Html.BeginForm helper to render the markup for the form and send it to our SiteSearch action method in the HomeController. From here we can do an F5 and test out our search.

The resulting structure of our solution tree (based on starting with an empty MVC 3 project) looks like so:

Where Do We Go From Here?

With the basic logic in place we are up and running with an integrated site search solution. The next step you would want to take is to address the What you must do and What you cannot do points in the API Basics documentation, including adding attribution (like a Powered by Bing message and image) and writing some caching logic to adhere to the 7 queries per second per IP address requirement. With those details addressed you could move on to add pagination of the results, update the View to use AJAX to return the results with another controller action and a partial view and add an extension method to help render the query string without the site:yoursiteurl chunk. Don't forget to step away from the logical role for a bit and have some creative fun styling your search results with some CSS!

Note About Cross Site Scripting
We have not added any extra cross site scripting checks to our code in this example. Out of the box, the ASP.NET framework will handle throwing a System.Web.HttpRequestValidationException about A potentially dangerous Request.Form value was detected from the client.. if we try and pass in some script text like in our search box. If we try to hack our way around it by passing in the script in the url (/Home/SiteSearch?query=\x3cscript\x3e%20alert(\x27hi\x27)%20\x3c/script\x3e), a vulnerability Jon Galloway covers in his post Preventing Javascript Encoding XSS attacks in ASP.NET MVC, we can see that we are not affected with our current code because we are rendering out the query string value as returned by the Bing API rather than the query string our MVC 3 application received. That being said, I am not sure that we would want to rely upon an external API response data to ensure that we are protected. Make sure you plan out some XSS testing and refactoring before you go to production with your site search solution!