Mental Pivot

Notes and observations from a lifelong pursuit of learning.

Insights and interesting reads delivered straight to your inbox.
Sign up for the free Mental Pivot Newsletter.

Using DuckDuckGo for Site Search on Your Blog

Last week I looked into the limited search options for bloggers like myself using Ghost CMS. The Ghost CMS search options are quite limited. After experimenting with some of the JavaScript search plugins, I ended up going with a completely different solution: I opted for a simple DuckDuckGo (DDG) search box.1

You can see the implementation at the bottom of the page or by visiting this page.


First, I want to set expectations about DuckDuckGo search. The biggest advantage to using DDG is that it is free to use, easy to integrate, privacy-focused (doesn’t collect or sell personal data), and the results are excellent. There is no need to keep any libraries up-to-date or deal with any browser compatibility issues that you might encounter with a JavaScript solution.

The biggest downside to using DDG for site search is that you cannot truly integrate DDG into your website. This, in part, is due to the fact that DDG does not have the rights to syndicate their search results (DDG aggregates its results from over 400 sources).

DDG does offer an “Instant Answer API,” but this search API offers a limited set of results, not a comprehensive full-text search of the entire DDG index.

Due to these constraints, when a user initiates a DDG search, the user is directed to the DDG website. The results are displayed on the DDG site and the user must then click on a search result to be returned back to the original website. DDG does provide limited options for customizing the look and feel of the search results, but the user experience is not ideal.

Ultimately you need to weigh the utility, results quality and user experience when deciding if this approach is right for you.


Basic Search Box Integration

Adding a DuckDuckGo search box to your website is quite simple. There are two ways to do it.

 

1. Add a DuckDuckGo iFrame to your site.

This is the official way to integrate search onto your site. You can find the instruction on how to do so by visiting this page:

https://duckduckgo.com/search_box

 

2. Add a DuckDuckGo HTML form to your site.

Given the disadvantages of using an iFrame, this is my preferred approach. Simply create an HTML form that takes the keyword input and initiates a search on DuckDuckGo.

At a bare minimum, you’ll want something like this:

<form method="get" id="search" action="https://duckduckgo.com/">
<input type="hidden" name="sites" value="mentalpivot.com"/>
<input class="search" type="text" name="q" maxlength="300" placeholder="Search"/>
<input type="submit" value="Search" style="visibility: hidden;" /></form>

The second line uses a search parameter to limit the search to the website defined input tag’s value parameter.

The final line is for the search button which is hidden in my example, but can be exposed if desired.


Search Box Customization

DuckDuckGo provides a number of URL parameters that can be used to customize your search results.

The full list of URL parameters can be found here:

https://duckduckgo.com/params

If you are using an HTML form to integrate, it is quite simple to add query parameters to the form via input tags. Each input tag will be appended to the resulting URL when the user initiates the search.

For example, URL parameter list (see link above) lets you set the color for the header, URLs, and background. The parameter name for background color is “k7.” If you wanted to make the background color bright red, you can pass the value “r.”

To do so, you would modify the HTML form in the following way:

<form method="get" id="search" action="https://duckduckgo.com/">
<input type="hidden" name="sites" value="mentalpivot.com"/>
<input type="hidden" name="k7” value=“r”/>
<input class="search" type="text" name="q" maxlength="300" placeholder="Search"/>
<input type="submit" value="Search" style="visibility: hidden;" /></form>

Line 3 contains the input tag with the URL parameter name and value.

Note that DDG requests that you do not remove/disable the following parameters: ko (header/branding) and k1 (advertising).

In addition to the DDG URL parameter customizations, you can use CSS and JavaScript to customize the look and behavior of the search box itself. I’ve used CSS to style the box and JS to add a rudimentary input clearing function to mine.


Search Sources Used by DuckDuckGo

While experimenting with DuckDuckGo, I noticed that the indexing “freshness” for Mental Pivot was over a week old. In other words, when I’d search for keywords that were in my very most recent posts (from the past several days), I wouldn’t see any results.

Perplexed by this, I decided to delve into how DDG aggregates its results. If you look at DDG’s official word on their search sources, you’ll notice that the majority of their “traditional” search results come from Microsoft’s Bing.

Like Google, Bing offers a dashboard called Bing Webmaster Tools. Among the many features of Bing Webmaster Tools is the ability to facilitate the indexing of your site. For instance, you can submit new URLs via Bing Webmaster Tools and they will be indexed immediately by the search engine. Moreover, once you submit your URLs to Bing, this will also update DDG’s search results.

Naturally, you don’t have to do this. You can just rely on the periodic crawls from Bing and DDG to keep your site content fresh in the respective search indices. It is nice, however, to have the option to ensure maximum currency if desired.


Additional Resources

There weren’t many useful articles on the topic of integrating the DuckDuckGo search box into a website or blog, but here are the best ones I found:

Lastly, I had never seriously considered using DDG search for my site until I noticed that the popular Daring Fireball uses it. If it’s good enough for John Gruber, it’s probably good enough for the rest of us.


1 Unfortunately the problems with Ghost JavaScript plugins were too numerous to overcome.

A few issues that prevented me from using a Ghost JS search plugin:

  • Results not as robust as Google or DuckDuckGo (no contextual excerpts).
  • Ran into numerous date formatting issues. Presumably due to issues between Ghost Content API and the JS plugin expected values.
  • Browser compatibility issues (not interested in testing various browser/OS configs).
  • Ghost Content API key is exposed in the HTML (since these are client-side solutions. Maybe isn’t a huge deal since the Content API is read-only, but I’d rather not take any chances on unknown security bugs.
  • Substring matching was substandard. For instance I couldn’t find all results for keyword “economics” despite having a number of articles with that exact term in the full text, terms like “microeconomics”, and category tags by that name.
  • Client-side search requires all content to be downloaded and searched on the client’s computer. Encountered periodic issues of search not working properly b/c of this.
  • Didn’t want to deal with keeping a JS file up-to-date if self hosting.

If I were to recommend a JS plugin, I would recommend the SearchinGhost plugin. This was the easiest, fastest and best of the bunch I tried out.



Get weekly email updates and additional content: Sign up for the free Mental Pivot Newsletter.