Author: Ward

We recently incorporated RSS feeds into Kotoba. I noted while implementing my solution that there are some tutorials [1, 2] out there that provide quick-and-dirty solutions to getting RSS feeds into a Rails application. While the tutorials are a good start, they do neglect the fact these implementations will drive a lot of (unnecessary) traffic between your site and the site with the RSS feed.

What can you do? What else! The time honored solution for all things that ail software: cache it.

First, let us create our RSS controller, or:

require 'kotoba_rss/feed'

class RssController < ApplicationController

  def parse_feed
    feed_url                = params['rss_url']                      || 'http://word.wardosworld.com/?feed=rss2&cat=6'
    maximum_number_of_items = params['maximum_number_of_items'].to_i || 5
    title                   = params['title']                        || t('rss.rss_feed')
    details                 = params['details']                      || t('rss.read_more')
    details_url             = params['details_url']                  || 'http://word.wardosworld.com/?cat=6'
    feed(feed_url, maximum_number_of_items, title, details, details_url)
  end

  def feed(feed_url, maximum_number_of_items, title, details, details_url)
    kotoba_rss_feed = Kotoba_Rss::Feed.new
    result = kotoba_rss_feed.get(feed_url)
    return render( 
      :partial => '/rss/feed', 
      :layout => 'popup',
      :locals => { 
        :result => result, 
        :maximum_number_of_items => maximum_number_of_items, 
        :title => title,
        :details => details,
        :details_url => details_url
      }
    )
  end
end

The next step is to create a class that will fetch RSS feeds for the rest of our application caching as appropriate, lib/kotoba_rss/feed.rb:

require "rss"
require 'rss/2.0'
require 'open-uri'

class Kotoba_Rss::Feed
  
  def initialize
    @cache = Kotoba_Rss::Cache.instance
  end
  
  def get(url)
    result = nil
    if @cache.has_expired?(url)
      open(url) do |http|
        response = http.read
        result   = RSS::Parser.parse(response, false)
      end
      @cache.add(url,result)
    else
      result = @cache.get(url)
    end
    return result
  end
  
end

Finally, we want to be able to cache our RSS feed results between calls, lib/kotoba_rss/cache.rb:

# Keep a cache of fetched RSS feeds.  This is to ensure
# we do not hit an RSS feed too often; an issue when we
# have many page fetches.  
#
# This is also a very useful thing when we are fetching
# from sites that might be throttling requests from specific
# users and, or domains (e.g. twitter.com).
class Kotoba_Rss::Cache
  include Singleton
  
  ENTRY_EXPIRES_IN_SECONDS = 300 # 5 minutes
  
  def initialize
    @cache = Hash.new
  end
  
  def add(url, result)
    entity = Kotoba_Rss::Cache::Entry.new(Time.now,result)
    @cache[url] = entity
  end
  
  def get(url)
    @cache[url].feed_result
  end
    
  def has_expired?(url)
    if @cache.has_key?(url)
      entry        = @cache[url]
      current_time = Time.now
      time_expired = current_time - entry.feed_time
      if time_expired > ENTRY_EXPIRES_IN_SECONDS
        return true
      else
        return false
      end
    else
      # if we do not have the URL yet then 
      # consider it expired
      return true
    end
  end

end

In the same file, lib/kotoba_rss/cache.rb, I also include:

class Kotoba_Rss::Cache::Entry
  attr_reader :feed_time, :feed_result
  
  def initialize(time, result)
    @feed_time   = time
    @feed_result = result
  end
  
end

I consider Kotoba_Rss::Cache::Entry an inner class that is only really useful for Kotaba_Rss::Cache, thus a great candidate for being included in its parent’s file; however, if you prefer you can certainly put it in its own file. In truth, Kotaba_Rss::Cache can also be included in lib/kotoba_rss/feed.rb which makes an even better design (note to self to refactor afterwards).

To use we have in our ApplicationHelper

  def link_to_blog_popup
    link_to(
         image_gif('blog',t('meta.blog')), 
         { 
           :action => 'parse_feed', 
           :controller => 'rss', 
           :rss_url => 'http://word.wardosworld.com/?feed=rss2&cat=6', 
           :maximum_number_of_items => 10, 
           :title => t('meta.blog'),
           :details => t('meta.blog_read_more'),
           :details_url => 'http://word.wardosworld.com/?cat=6'
         } , 
         :class => 'popup',
         :onclick => "return hs.htmlExpand(this, { objectType: 'ajax', contentId: 'popup', wrapperClassName: 'highslide-no-border', dimmingOpacity: 0.75, align: 'center'} )"
     ) 
  end

Note that we are using I18n t() method for localization.

In summary, we have a good separation of responsibility between all our moving parts. We are using a controller the way it is intended, with our presentation properly off-loaded to views. In our library we have a means of making a service that invisibly caches its results.

In my experience, caches should almost always be hidden from the caller; otherwise, you begin to degrade cohesion and venture into the land of tight coupling. Additionally, it makes the cache design much harder with multiple point of entry.

That is it. Whenever we have held a result in our cache longer than ENTRY_EXPIRES_IN_SECONDS our RSS feed will get new result that it stores in its cache. This will ensure that as our site grows (and it will!) we will not be hitting external services with every page fetch; something that would yield little value to our users (slower fetches) and make us bad clients of other sites’ feeds.

Testing.  Testing.  Testing. There any number of ways to qualify testing in software.  Unit testing.  Functional testing.  Systems testing. Integration testing.  Usability testing.  Load testing.  Et cetera.  And what testing you do do, or do not, is somewhat predicated upon the type of software you are developing.

One thing is for certain, though; no matter the domain the interface is always key.   Kotoba is no different.  We have been doing a lot of exploratory technology development with only the bare-bones paid to sustainable (read automated) testing.  While we have it, we can always have more.

One area where automated testing can be a real chore is at the user-level.  This is more than just calling back into our application using URLs as is often done at the functional level; we really want to, as best as possible, mimic the user experience as if we are the user. And while we could do this manually, it is replete with tedium and errors. So how do we automate this step in our quality assurance?  Que Selenium; a power tool that uses JavaScript to test our application in any browser that supports JavaScript. It even has its own “IDE” that allows you to record events that then can be saved as Selenium tests. More to boot, Selenium is supported by a number of languages and frameworks; albeit, we are mostly interested in Selenium on Rails.

Over the weekend I spent time understanding how best to integrate Selenium into Rails. Out of the box Selenium is ready to go but for one thing: user login. Kotoba uses hashed passwords with a temporally sensitive salt to help ensure user passwords are secure. Unfortunately, this makes our life a little more difficult in part because Y(a)ML test fixtures do not go through ActiveRecord, but create SQL statements directly. Which means that all our model instance methods that help create our secure password are bypassed.

More specifically, whenever we run Selenium setup while our user accounts are created, they are done without passwords. Consequently, we need a way to attach passwords once the accounts are created. Unfortunately, Selenium provides no known callback mechanism to provide model/application initialization. While I explored implementing this callback mechanism, it become apparent that doing so would be not much than a hack without some concentrated engineering. Additionally, we must remember that Selenium tests our application using JavaScript. That means that while we can include Ruby in our RSelenium tests, these are only run the first time the test is read in by Selenium while Selenium generates JavaScripts calls that it then uses to run the tests. To wit, the next time is run these ruby calls are not made.

Imagine I have something like:

1. Setup app; clean fixtures: Javascript
2. Setup passwords: Ruby
3. Login: Javascript

On the first pass we clean our database, create our fixtures, set our user passwords and create our login test.

However, the moment we actually run our test we do everything but set our user passwords. The result? Our test breaks.

Our solution is to create a controller that includes initialization methods. To ensure that these controllers and methods are never called in anything other than test using a before_filter.

class InitializeKotobaController < ApplicationController

  before_filter :verify_environment

  def a_method
  end

private

  def verify_environment
    unless ['test'].include?(RAILS_ENV)
      raise "This controller cannot be used in this Rails environment [#{RAILS_ENV}]"
    end
  end
end

From our Selenium test, we can call open, or:

open('/initialize_kotoba/a_method?param1=a&param2=b')

Note how we are able to pass parameters into our controller allowing us to make it as versatile as we need it for our tests.

While you can use partials in our tests, this makes our tests a little less accessible to the average tester. Ideally, we want to be able to hand over our integration testing to someone who is more focussed on the user experience than the inner-workings of our programming environment. While arguably not a concern with Kotoba, it is nevertheless a philosophical one I prefer to strive on all my projects.

The solution is to include your own DSL (domain specific language). This is not as difficult as it sounds, nor nearly as cutting-edge as some blogs might make it seem. In reality, many DSLs s are really nothing more than a API that is catered to better mimic the business domain's lingua. The below is no different.

# Include our own Selenium DSL functions
config.after_initialize do
  if SeleniumOnRailsConfig.get(:environments).include? RAILS_ENV
    class SeleniumOnRails::RSelenese
      class_eval do
        include Selenium::DSL
      end
    end
  end
end

Then we add a file at $RAILS_ROOT/lib/selenium/dsl.rb

module Selenium::DSL

  def login(user, password)
    # include our RSelenium tests here
    open('/')
    assert_title('Kotoba « Accounts « Login')
    assert_text('login_legend',I18n.t('login.title'))
    type('email', user.email)
    type('password', password)
  end

end

Now we can just call

login(user,password)

from our RSelenium tests to automatically login as a specific user.

Want to quickly create icons quickly that all follow a similar size, coloring, and text?  Then check out http://www.rss-icon.com/icon.php

We used this on http://kotoba.wardosworld.com to quickly add customized icons for our blog and twitter.

Basically, you can enter in the text, size of the icon, colors of background and text to generate an image.  The image, when downloaded will typically include the suffix, .php.html.  Just change this to .gif and you are done!

For those wanting a quick translation of the UI, use the below cheat sheet.

文字タイプ: Font

文字サイズ: Font Size

アイコンの幅: Icon Width (in pixels)

アイコンの高さ: Icon Height (in pixels)

中心のボーダーの位置: Position of middle border (in pixels)

外側のボーダーの太さ: Outside border width (in pixels)

内側のボーダーの太さ: Inside border width (in pixels)

左ボックス-内容: Left box’s content

左ボックス-文字色: Left box’s font color

左ボックス-背景色: Left box’s background color

左ボックス-間隔: Left box’s padding (in pixels)

右ボックス-内容: Right box’s content

右ボックス-文字色: Right box’s font color

右ボックス-背景色: Right box’s background color

右ボックス-間隔: Right box’s padding (in pixels)

Once you filled things in click on アイコンを作成！

The image that appears can be saved to your computer.  Remember, you will rename so that the suffix is .gif.

Sometimes it is the little things that can make you smile.  Case is point is Kotoba‘s new favicon.  

I wanted something that was clean and easily recognizable in the toolbar.  While options included something abstract, I felt that a monogram might best represent the intent of the site.  While “K” might be arguably more universal, I ultimately decided that the 「言」という文字 is both identifiable to many using the site and abstract enough to act as a “logo”.

In order to create the image, I went through the process of  getting various Japanese-supporting applications installed just to create a very simple image.  This process culminated in the installation of OpenOffice.org’s OpenOffice and  Gimp.   

There is a wealth of information about installing Japanese versions of these pieces of software:

• OpenOffice.org 1.1.4日本語版 for X11 on Mac OS X
• LeopardのX11.appでかんなとkinput2を使う
• Inside Leopard: X11.appと日本語環境の問題
• Beginning OS X 10.5/運用編

In the end, installing the latest versions of both OpenOffice and Gimp is sufficient to get out-of-the-box compatibility with Japanese display and input.  While OpenOffice supports Japanese input directly into the application, the same cannot be said for Gimp.   Nevertheless, it is relatively painless to use OpenOffice to create images with Japanese text that can then be imported directly into Gimp.  

Once we had a means of creating an image, the final step is to get it into a format that is universally used by web browsers.  I lucked out when I found the very  wonderful website, http://www.favicon.jp/, which will take any image and create a version in .ico format.  Once this file is generated, it is straight-forward to added to your site you can point to it in your HEAD tag.

And that is how we get something small but that gives us a large, warm fuzzy in our tummy.  😀