Category Archives: Libraries

Uploading to multiple S3 buckets with Paperclip and Rails

Posted on by
Reply

When your application uses many static files (photos for example), you should consider placing these files on different hosts to improve the speed at which they are downloaded by web browsers.

Most web browsers have a limit on how many simultaneous connections can be made to a single named host. And this limit is usually two. It means that if you let’s say display many photos on a single page, user’s browser can only download two at a time. The broadband internet connection will be no help here. The browser will not fully use it, because it will keep opening and closing connections.

With little server and Rails configuration you pretend to be serving your files from different hosts and trick the browser (read more). This is easy if you keep all the files on your own server, but when you use Paperclip to upload files to Amazon’s S3 servers it gets more trickier. In fact Paperclip doesn’t support uploading to different hosts (buckets).

First, notice that these addresses means the same for Amazon’s S3:

  • http://s3.amazonaws.com/bucket_name/filename.ext
  • http://bucket_name.s3.amazonaws.com/filename.ext

So all we need to do is to make Paperclip upload to different buckets and return attachment’s url of the latter type. I wrote an extension that accomplishes this and included it in PaperclipExtended. All you need to do is download both Paperclip and PaperclipExtended and change your model definition.

class User < ActiveRecord::Base
    has_attached_file :avatar,
                      :storage => :s3,
                      :s3_credentials => "#{RAILS_ROOT}/config/s3.yml",
                      :path => "avatars/:id/:style_:extension",
                      :bucket => lambda do |attachment|
                        i = attachment.instance.id % 4
                        "bucket_#{i}"
                      end
end

This will place each avatar in one of four buckets: bucket_0, bucket_1, bucket_2 or bucket_3. The exact bucket is chosen at runtime and in this case it’s based on models id.

Getting attachment’s url:

puts User.find(1).avatar.url(:original)
# => http://bucket_1.s3.amazonaws.com/avatars/1/original.jpg

If you have a page on your website that displays many uploaded photos, placing them in different buckets should make noticeable difference to user – they will load faster.

SwfUpload and Flash 10

Posted on by
4


SwfUpload is a graceful combination of flash and javascript that provides a mechanism for uploading files which is far more user-friendly than standard <input type=”file” />. It allows to upload many files at once and displays nice-looking progress bar.

Not everything works fine in every environment, though. Some problems are caused by the differences between some versions of flash player or even by bugs in flash players. Many of them can be solved, I wrote a post about it some time ago. But now when the Flash 10 came out, we have new obstacles to overcome.

Flash 10 is more restrictive about security issues that it’s predecessors. Now you cannot trigger opening file-select window from outside of the flash and it affects SwfUpload which used to do it from Javascript.

Thankfully, developers reacted quickly and are now working on new version (2.2) of the library. Alpha version is already out there and it deals with the mentioned problem. Since it is only alpha I recommend upgrading to stable version as soon as it comes out.

Here are inctructions how to reconfigure your app to work with SwfUpload 2.2.

First, download the most recent version of the library from SVN repository:

svn co http://swfupload.googlecode.com/svn/swfupload/trunk/core swfupload

You’ll only need two files from this: swfupload/swfupload.js and swfupload/Flash/swfupload.swf. These are the new version of files you should already have in your application. Replace old ones with these.

The new SwfUpload creates “Select files” button on it’s own, you only need to provide a placeholder for it.

<div id="swfuploadButtonPlaceHolder"></div>

The last thing is to update some Javascript.

swfu = new SWFUpload({
...
button_placeholder_id: "swfuploadButtonPlaceHolder",
button_image_url: "../images/swfupload/button.jpg",
button_width: "216",
button_height: "25"
...});

“button_image_url” accepts a path relative to the swf file. The image it points to, should look somewhat like this:

One image containing three states of a button (normal, on mouse over, on mouse click). You can have text instead of image if you want:

swfu = new SWFUpload({
...
button_text: '<span class="theFont">Upload</span>',
button_text_style: ".theFont { font-size: 16; }"
...});

That’s it. You can see working example I based on here.

If you find it useful, please share.

Custom thumbnail generation with Paperclip

Posted on by
1

Paperclip is a great plugin for Ruby on Rails which eases the pain of image upload and resize process. The usage is very simple, results are what you want in most cases. Not in all cases though.

Your role is only to define the sizes of thumbnails that will be generated from original image. Thumnailing in is fact done by calling ImageMagick’s convert command with -scale argument, for example convert -scale ’640×480>’.

You can alter how exactly the image is scaled by appending modifiers to desired thumbnail size, like ‘>’ in ’640×480>, however it won’t give you unlimited power over thumbnailing process.

Consider following example. I have a small image (120×120).

Scaling it to smaller size gives me the result I want. But when the result image needs to be bigger than original, the original image is enlarged in a way that you’ll see single pixels.

What I want to achieve is the enlargment done not by scaling the original image, but by adding a border around it.

This cannot be achieved by adding any of modifiers to desired thumbnail size, so why not to pass additional arguments to convert command? Yes, that would be a solution.

And here’s my solution – PaperclipExtended. It’s a plugin that modifies original Paperclip, so that now it accepts additional (optional) parameter :commands when defining the thumbnails sizes.

class User &lt; ActiveRecord::Base
  has_attached_file :avatar, :styles =&gt; { :medium =&gt; "300x300&gt;", :small =&gt; "100x100&gt;" },
    :commands =&gt; { :medium =&gt; "-background white -gravity center -extent 300x300 +repage" }
end

During thumbnail generation Paperclip will now append given commands to convert command. Convert command for medium style will be now:
convert -scale ’300×300>’ -background white -gravity center -extent 300×300 +repage
Convert command for small style will remain unchanged.

Passing such commands will help me with the enlargment I mentioned above. For what else could be done check the documentation of ImageMagick’s command line options.

PaperclipExtended is not a replacement for Paperclip, just an extension. It works with version 2.1.2 of Paperclip plugin (the current one and the only one I tested). Compatibility with future version is not guaranteed.

I put a plugin on GitHub.
http://github.com/netguru/paperclip-extended

How to install:

script/plugin install git://github.com/netguru/paperclip-extended.git

Configuration is the same as of original Paperclip plugin, but you can now add an optional :commands parameter.

Hope it’ll be helpful!

If you have any questions, post them in comments to this post.

Problematic SwfUpload

Posted on by
3

SwfUpload is a nice and easy to use javascript/flash library that resolves the problem of uploading many files at once to the server.

There are however some issues that sometimes make the experience of using the library a painful one. The problem come not from the SwfUpload itself, but rather from flash plugins installed on the browsers. Especially older versions.

One of the problem is that some of the plugins (Adobe’s Flash < 9.0 if I remember correctly) doesn't pass cookies during the upload, which results in lost session. Detailed description and possible solutions can be found around the web, for example here.

The other problem I came accross was that an user got logged out every time he uploaded a file with IE6. It was caused by User Agent mismatch when restoring session, so User Agent check had to be disabled for upload requests.

When it comes to using SwfUpload on Mac some new issues appear. One is a subdomain problem that causes a 404 error.

Couple of days ago I was hit by an error that took me all day to cope with. It was none of the above, the request seemed to be all correct, input data passed to an action (CakePHP application) was complete. The action itself didn’t work though. After quite painful debugging I found out that there was a port number appended to host name in $_SERVER['HTTP_HOST']. For an usual configuration it should not make any difference, however here it did and an absence of database connection was that difference.

The application gets deployed to more than one server and database it connects to depends on the server. The code that determines database hostname relied on the $_SERVER['HTTP_HOST'] variable and didn’t care about port numbers. As it appeared it should have!

I didn’t see such a problem described anywhere else, so I decided to share. Hopefully this can help someone few hours of work.