Speaking at IT Underground Security Conference and Workshop in Prague

it_underground_prague_125x125_enTomorrow I will be speaking at IT Underground Security Conference and Workshop in Prague, Czech Republic. That is three day long event that starts today.

I will give a talk on “Websecurity through conventions and best practices” which will be focused on securing web applications from the most common attacks with examples from Ruby on Rails web development framework.

Expect to see materials from the conf here in few days.

Rails 2.2.2, Ajax and respond_to

As I wrote some time ago in the article about Rails, Ajax and jQuery, sometimes there are problems with Rails not interpreting correctly content type headers of ajax requests. It’s because not all web browsers send that header in the same way. 

What I proposed was to sort the request.accepts array (array containing content type headers sent by browser) so that xml content type would be the first element. That would then trigger format.xml in our respond_to block.

However that approach does not work in Rails 2.2.2, because now the request.accepts array is frozen and it cannot be modified. I spent some time googling for the solution, but with no effect. So I dived into the API and Rails’ source code and came up with pretty nice and simple solution to the problem.

class ApplicationController < ActionController::Base
  before_filter :xhr_to_xml
 
  def xhr_to_xml
    request.format = :xml if request.xhr?
  end
end

This piece of code is an equivalent of the snippet I proposed in the article I referred to at the beginning. Now all ajax request will trigger format.xml in respond_to blocks.

Materials from AIESEC Conference in Częstochowa, Poland

Last month the netguru team has been invited to Corporate Responsibility and Information Technology Conference organized by AIESEC (international student organization) which took place in Częstochowa, Poland. We went there and gave some talks on various topics and I think some of you may find materials from this conference quite interesting.

First, we led an unofficial discussion on “The (fading) differences between desktop and web applications”. We introduced the topic to our hearers and then let them express themselves, it was quite successful.

On Day 2, we organized Ruby on Rails workshops which consisted of “Introduction to RoR” talk and one-hour live-coding session during which a simple twitter like application has been developed. You can find the results on our Github account.

We finished Day 2 with a talk on Enterprise 2.0 and on Day 3 we talked about “Developing a web-development company (startup)”, which was quite different from what people heard on earlier talks gave by the representatives of big corporations. I have no materials to share on this, though. Sorry.

Rails, Ajax and jQuery

The more ajaxified application, the more fun it is to use. But it is also more painful do develop. What is written below is my approach to pairing Rails and Ajax. It’s a mix of tips I found over the net on blogs and forums. I use jQuery for JavaScript, but I don’t use jRails or any JS/Ajax helper methods provided by Rails. Note that all Javascript/HTML code presented here can be used even if you dont use Rails or Ruby as your web development platform. Let’s begin.

Rails is RESTful

Thanks to Rails’ RESTfulness the only thing to take care of server side is setting proper response in controllers’ actons.

class PostsController < ActionController::Base
  def index
    @posts = Post.find :all
    respond_to do |format|
      format.html
      format.xml { render :xml => @posts.to_xml }
    end
  end
end

Rails decides which format block to call basing on routes defined in routes.rb file (map.connect ‘:controller/:action/:id.:format’) and accept headers sent with request by the client.

In most cases we want Ajax requests to trigger format.xml blocks in our controllers’ actions, so we need to set proper accept headers. Let’s do it just once with application-wide setting.

// All ajax requests will trigger the format.xml block
// of +respond_to do |format|+ declarations
$.ajaxSetup({
  'beforeSend': function(xhr) {xhr.setRequestHeader("Accept", "text/xml")}
});

Browsers’ quirks

There is something worth noting here, a problem I had once with IE and Safari. The code above may work differently in various browsers. Browser set text/html accept header by default. Here IE and Safari will append text/xml to it so you’ll get something like ‘text/html; text/xml’, while Firefox will replace text/html with text/xml and you’ll get ‘text/xml’ only. This is very important because Rails will take the first format it detects in accept header and trigger respective block in controller’s action, which will be html for IE and Safari. Here’s a fix for this that shifts application/xml (if it is present) to the beggining of accept headers array.

class ApplicationController &lt; ActionController::Base
  before_filter :correct_safari_and_ie_accept_headers
 
  def correct_safari_and_ie_accept_headers
    request.accepts.sort!{ |x, y| y.to_s == 'application/xml' ? 1 : -1 } if request.xhr?
  end
end

Ajaxify your links

Here’s a quick way to ajaxify your existing links. Add this JavaScript to your application.js file.

jQuery(document).ready(function() {
  // All A tags with class 'get', 'post', 'put' or 'delete' will perform an ajax call
  jQuery('a.get').livequery('click', function() {
    var link = jQuery(this);
    $.get(link.attr('href'), function(data) {
      if (link.attr('ajaxtarget'))
        jQuery(link.attr('ajaxtarget')).html(data);
    });
    return false;
  }).attr("rel", "nofollow");
 
  jQuery('a.post').livequery('click', function() {
    var link = jQuery(this);
    $.post(jQuery(this).attr('href'), "_method=post", function(data) {
      if (link.attr('ajaxtarget'))
        jQuery(link.attr('ajaxtarget')).html(data);
    });
    return false;
  }).attr("rel", "nofollow");
 
  jQuery('a.put').livequery('click', function() {
    var link = jQuery(this);
    $.post(jQuery(this).attr('href'), "_method=put", function(data) {
      if (link.attr('ajaxtarget'))
        jQuery(link.attr('ajaxtarget')).html(data);
    });
    return false;
  }).attr("rel", "nofollow");
 
  jQuery('a.delete').livequery('click', function() {
    var link = jQuery(this);
    $.post(jQuery(this).attr('href'), "_method=delete", function(data) {
      if (link.attr('ajaxtarget'))
        jQuery(link.attr('ajaxtarget')).html(data);
    });
    return false;
  }).attr("rel", "nofollow");
 
  jQuery('a.get, a.post, a.put, a.delete').removeAttr('onclick');
});

Just add a CSS class .get, .post, .delete, or .put to a link to make turn it into an ajax-link. I recommend you use LiveQuery plugin which will automatically bind click events to new links that appear on the page (loaded with Ajax call for-example). You can optionally set ajaxtarget attibute of the link. It expects a selector of a container in which you want to place the response.

link_to 'my cool article', article_path(@article), :class => 'get', :ajaxtarget => '#article_container'

Ajaxify your forms

For this you’d need jQuery Form Plugin.

  jQuery('form.ajax').livequery('submit', function() {
    jQuery(this).ajaxSubmit();
    return false;
  });

Now all your forms that have “ajax” class will be submitted via Ajax.

<form class="ajax">
  ...
</form>

CSRF and authenticity token

Rails has built-in protection from cross-site request forgery attacks. It relies on an authenticity token which Rails look for when dealing with POST, PUT or DELETE requests, so this token needs to be sent by the browser together with the request. The token is automatically added as a hidden field to any form you create with form_for method, it is also attached to links that have :method param set to :post, :put or :delete. In fact the token is added dynamically by Javascript code placed in link’s onclick attribute. However in one of code snippets above we stripped that onclick attribute from links to prevent the page reload after we click the link. Now we need to attack that token ourselves. First we will alter our application layout:

<head>
  <% if protect_against_forgery? %>
    <script type='text/javascript'>
    //<![CDATA[
      window._auth_token_name = "#{request_forgery_protection_token}";
      window._auth_token = "#{form_authenticity_token}";
    //]]>
    </script>
  <% end %>
</head>

Now we need to ensure that the token is sent together with ajax requests.

jQuery(document).ready(function() {
  // All non-GET requests will add the authenticity token
  // if not already present in the data packet
  jQuery("body").bind("ajaxSend", function(elm, xhr, s) {
    if (s.type == "GET") return;
    if (s.data && s.data.match(new RegExp("\\b" + window._auth_token_name + "="))) return;
    if (s.data) {
      s.data = s.data + "&";
    } else {
      s.data = "";
      // if there was no data, $ didn't set the content-type
      xhr.setRequestHeader("Content-Type", s.contentType);
    }
    s.data = s.data + encodeURIComponent(window._auth_token_name)
                    + "=" + encodeURIComponent(window._auth_token);
  });
});

We’re done, we have our ajax requests protected from CSRF attacks.

Modifing page after Ajax calls

Standard way to do page modification after Ajax call is to use Javascript code that inserts content returned by the call somewhere on the page. The other method is to put the modifying code in views that are returned by the server and just execute it in the browser. For this I’d recommend another jQuery plugin – Taconite. As the author says: “The jQuery Taconite Plugin allows you to easily make multiple DOM updates using the results of a single AJAX call. It processes an XML command document that contain instructions for updating the DOM”. Thanks to this you can for example easily use flash messages in your Ajax views.

Let this be a part of your usual layout:

<div id="flash_notice" class="flash"<%= ' style="display: none"' unless flash[:notice] %>><%= flash[:notice]  %></div>

Now let this be your taconite layout you’d use when returning views for Ajax requests:

<taconite>
  <hide select="#flash_notice" />
  <% if flash[:notice] %>
    <replaceContent select="#flash_notice">
      <%= flash[:notice] %>
    </replaceContent>
    <fadeIn select="#flash_notice" arg1="slow" />
  <% end %>
  <%= yield %>
</taconite>

This will display flash notice messages with fade-in effect after Ajax requests. Similarly you can update other elements of the page.

What’s in your toolbox?

I would love to hear from you on how you deal with Ajax in your web applications. What libraries/plugins do you use?

UploadPack – easy and flexible way to upload files with CakePHP

I have seen and worked with many scripts and plugins (not only written in PHP) that tries to deal with file-upload functionality on server side, thanks to this I think I have some view on how good file-upload plugin should work. The thing was to put it in CakePHP clothes and code it.

Here are the requirements I set:

  • save procedure of record with attached file should be no different to usual record
  • it should be possible to do some additional processing of uploaded file at the time of saving it (for example thumbnail generation)
  • easy access to uploded file, it’s URL and alternatives (different thumnails) from view level
  • everything should require no or minimum configuration to work, but still remain flexible if the whole application needs it
  • natural integration with CakePHP framework

Here’s the effect of some planning and coding – UploadPack. Right now it contains:

  • UploadBehavior – deals with saving files to disk and post-save processing
  • UploadHelper – provides nice access to files (URLs, thumbnails) from view level

Everything works quite gracefully, I think. You only need to add one field to model’s database table (which will hold file name) and attach behavior to model. The rest is done automatically. Everything is documented on repository’s page.

It’s still an early version – 0.1, but there is much it can do right now. The work is underway to provide new features. Take a look at it.

If you have any views or ideas, please leave a comment. I’d appreciate it.

Uploading to multiple S3 buckets with Paperclip and Rails

When your application uses many static files (photos for example), you should consider placing these files on different hosts to improve the speed at which they are downloaded by web browsers.

Most web browsers have a limit on how many simultaneous connections can be made to a single named host. And this limit is usually two. It means that if you let’s say display many photos on a single page, user’s browser can only download two at a time. The broadband internet connection will be no help here. The browser will not fully use it, because it will keep opening and closing connections.

With little server and Rails configuration you pretend to be serving your files from different hosts and trick the browser (read more). This is easy if you keep all the files on your own server, but when you use Paperclip to upload files to Amazon’s S3 servers it gets more trickier. In fact Paperclip doesn’t support uploading to different hosts (buckets).

First, notice that these addresses means the same for Amazon’s S3:

  • http://s3.amazonaws.com/bucket_name/filename.ext
  • http://bucket_name.s3.amazonaws.com/filename.ext

So all we need to do is to make Paperclip upload to different buckets and return attachment’s url of the latter type. I wrote an extension that accomplishes this and included it in PaperclipExtended. All you need to do is download both Paperclip and PaperclipExtended and change your model definition.

class User &lt; ActiveRecord::Base
    has_attached_file :avatar,
                      :storage =&gt; :s3,
                      :s3_credentials =&gt; "#{RAILS_ROOT}/config/s3.yml",
                      :path =&gt; "avatars/:id/:style_:extension",
                      :bucket =&gt; lambda do |attachment|
                        i = attachment.instance.id % 4
                        "bucket_#{i}"
                      end
end

This will place each avatar in one of four buckets: bucket_0, bucket_1, bucket_2 or bucket_3. The exact bucket is chosen at runtime and in this case it’s based on models id.

Getting attachment’s url:

puts User.find(1).avatar.url(:original)
# =&gt; http://bucket_1.s3.amazonaws.com/avatars/1/original.jpg

If you have a page on your website that displays many uploaded photos, placing them in different buckets should make noticeable difference to user – they will load faster.

SwfUpload and Flash 10


SwfUpload is a graceful combination of flash and javascript that provides a mechanism for uploading files which is far more user-friendly than standard <input type=”file” />. It allows to upload many files at once and displays nice-looking progress bar.

Not everything works fine in every environment, though. Some problems are caused by the differences between some versions of flash player or even by bugs in flash players. Many of them can be solved, I wrote a post about it some time ago. But now when the Flash 10 came out, we have new obstacles to overcome.

Flash 10 is more restrictive about security issues that it’s predecessors. Now you cannot trigger opening file-select window from outside of the flash and it affects SwfUpload which used to do it from Javascript.

Thankfully, developers reacted quickly and are now working on new version (2.2) of the library. Alpha version is already out there and it deals with the mentioned problem. Since it is only alpha I recommend upgrading to stable version as soon as it comes out.

Here are inctructions how to reconfigure your app to work with SwfUpload 2.2.

First, download the most recent version of the library from SVN repository:

svn co http://swfupload.googlecode.com/svn/swfupload/trunk/core swfupload

You’ll only need two files from this: swfupload/swfupload.js and swfupload/Flash/swfupload.swf. These are the new version of files you should already have in your application. Replace old ones with these.

The new SwfUpload creates “Select files” button on it’s own, you only need to provide a placeholder for it.

<div id="swfuploadButtonPlaceHolder"></div>

The last thing is to update some Javascript.

swfu = new SWFUpload({
...
button_placeholder_id: "swfuploadButtonPlaceHolder",
button_image_url: "../images/swfupload/button.jpg",
button_width: "216",
button_height: "25"
...});

“button_image_url” accepts a path relative to the swf file. The image it points to, should look somewhat like this:

One image containing three states of a button (normal, on mouse over, on mouse click). You can have text instead of image if you want:

swfu = new SWFUpload({
...
button_text: '<span class="theFont">Upload</span>',
button_text_style: ".theFont { font-size: 16; }"
...});

That’s it. You can see working example I based on here.

If you find it useful, please share.

NamedScopeBehavior upgrade

I have just added a small functionality to my NamedScopeBehavior.

Now you can use named scopes from multiple models at once in a single find query. Assuming that given models are associated through belongsTo or hasOne association.

Quick example:

// model definitions
class User extends AppModel {
  var $actsAs = array(
    'NamedScope' => array(
      'activated' => array('User.activated' => 1)
    )
  );
  var $belongsTo = array(
    'Group'
  );
}
 
class Group extends AppModel {
  var $actsAs = array(
    'NamedScope' => array(
      'admins' => array('Group.name' => 'admins')
    )
  );
}
 
// in controller
$this->User->find('all', array('scope' => array('User.activated', 'Group.admins')));

New version can be found in repository.

Named scope in CakePHP

Some time ago I wrote how to improve Cake controllers’ callback system basing on a solution from Ruby on Rails. Today I am going to port another cool feature of Rails to CakePHP – named scope.

Check this Rails example first if you don’t know what the named scope is:

# model definition
class User < ActiveRecord::Base
  # only activated users
  named_scope :activated, :conditions => "activated_at is not null"
 
  # only currently online users
  named_scope :online, :conditions => "date_add(last_activity, interval 5 minute) < now()"
end

This is very simple example. In fact, named scope functionality can be used for much more than this. However my Cake version I present below is limited to defining conditions.

So we defined some scopes and gave them names. Now we can apply them to searching.

# let's find all activated users
users = User.activated.find(:all)
 
# let's find all activated users who are currently online and has more than 10 points
users = User.activated.online.find(:all, :conditions => "points > 10")

I bet you can already see advantages of such notation. Often used conditions (for example: often you want to limit search results to activated users only) are placed in named_scope definition, the others (probably more situation-specific ones) go to usual :conditions option. Improved readability, less writing, fewer chances to make mistakes.

Let me present you NamedScopeBehavior which will give you similat functionality in CakePHP. What it does is converting scope definitions to conditions parameter of find() method and that is done transparently in beforeFind callback method.

Download named_scope.php file from repository and place it it app/models/behaviors folder of your Cake app. Then attach this behavior to a model and define named scopes.

class User extends AppModel {
  var $actsAs = array(
    'NamedScope' => array(
      'activated' => array('User.activated in not null'),
      'online' => array('date_add(User.last_activity, interval 5 minute) > now()')
    )
  );
}

Then you can use it.

// in your controller
$this->User->find('all', array('scope' => 'activated'));
$this->User->find('all',
  array('conditions' => 'points > 10', 'scope' => array('activated', 'online')))

Pagination also works fine with scopes.

// in your controller
$paginate = array(
  'User' => array(
    'order' => 'created ASC',
    'limit' => 20,
    'scope' => array('online', 'activated')
  )
);

Happy developing!