Reggie's picayune

Carrierwave is a really nice gem to use when you want to upload and save a file within a ruby object. But how do you test it, if you don’t use anything more that test/unit or minitest?

Most of the articles describing testing Carrierwave rely on rspec or capybara, but I don’t like adding test gems unless I have to. I prefer to use the test objects provided within the ruby standard library. So test/unit and minitest.

Fortunately, it is fairly easy to manage without anything complicated. All you have to realise is that when a form is submitted to a controller, the uploaded file is presented within params via a ActionDispatch::Http::UploadedFile object.

So for example, I have a model SupportingDocument, which has an attribute ‘document’ that holds the CarrierWave::Uploader associated with uploaded files.

To functionally test the supporting_documents controller actions I added a couple of methods to my test SupportingDocumentsControllerTest, that create an UploadedFile object:

I was then able to use this object within a test:

Using Rails Engines is a really nice way to introduce yourself to creating gems. It provides an environment to create and test code that is very similar to that used when creating a full rails app. There are couple of things to be aware of, such as that you run tests from the root (rake test), but run the test server and console (via rails s and rails c respectively) at test/dummy, but otherwise any rails developer should be able to get up to speed with engines reasonably quickly.

However, the more gems I write, the more I’m learning that keeping dependencies to a minimum is a really good idea. For example, do you really need a specialist HTTP gem like curb or typhoeus or can you manage with net/http. In a rails app, the simplicity and features that excellent gems like curb and typhoeus provide, often make them compelling. In a gem, the extra dependencies they add can be an issue. For example, what if a host rails app is already using a version of typhoeus, and that version clashes with the one your code uses.

A key rule with a gem is that you cannot predict the environment in which it is run, and therefore you should put some effort into reducing the gem’s dependencies.

Using engines to create gems, by definition, adds a dependency on rails, and all the baggage that comes with rails. If the gem is only ever going to be a rails plugin, and it adds its own controllers and views to the host app, dealing with this ‘baggage’ is well worth the effort. However, that is often not the case. With a little effort you can minimise the dependencies. For example, my most popular gem (array_logic) has no dependencies beyond the ruby standard library.

So as I have been creating more gems, I’ve started to identify strategies that allow me to minimise dependencies. For example, indulgence needed ActiveRecord, but by using standalone-migrations, I was able to create a nice test environment using ActiveRecord, without any other dependencies.

In fact, it is fairly easy to create a working test environment. I’m a great fan of test/unit and minitest, and as both are now part of ruby standard library, using them again reduces my gems’ dependencies.

What I’ve been missing is an easy way of getting to an equivalent of the rails development console. Today, I’ve had yet another ‘why didn’t I think of that before‘ moment. There is an easy way to create a console. I’ve added a console.sh file to my latest gem project (webdav_file_store) with this content:

The -r option of irb is doing all the work. It is loading the file that opens my WebdabFileStore namespace Module, along with the requirements it loads. This only works in a bash shell, and you have to make the file executable, but once that is done, all I do is run ./console.sh and I am in a console session with all my WebdavFileStore objects loaded.

So another technique I can add to my strategies to minimise my gem dependencies.

Update:

See this article on Ernie Miller’s blog for an alternative approach. That is, to create a rake task that opens the console with the gem loaded. The comments are well worth a look through. I think Ernie’s is probably a better technique to the one I describe above.

I’ve been installing passenger on my development PC so that I can debug an issue that is only apparent in production (it’s due to the app being hosted in a sub-uri). I don’t tend to use passenger much myself, as in most of the environments I’ve worked in there has either already been a server set up, or there is a sys-admin who needs to manage the server set up. However, doing this myself has been really useful and I’ve learnt a lot from doing it. (Another “why haven’t I worked out all this before” moment).

The main problem I had was that I kept having to point RackBaseURI at the public folder in my app, which left ‘/public’ in the app’s url. So instead of ‘/mysite’, I was getting ‘/mysite/public’.

As is so often the case once you know the solution, the answer was very simple. I had overlooked the significance of a single line in the passenger documentation:

That sets up a symlink to the public folder of your app. It also means that RackBaseURI is not pointing at the real folder where the code is, but to a symlink in your web root. Now I realise the significance of this line in the docs, it is obvious to me that this a better way to arrange things.

With the the symlink in place everything started working properly.

One thing that playing with this has taught me is that the passenger documentation is very good. A couple of times google searches led me to outdated methods that no longer worked with the version of passenger I was using. Each time the solution was found within the passenger docs.

The problem is that the apache guide is a very long document. However, my experience was that it was much more fruitful to search the passenger documention page for keywords, than to google for the solution.

I’ve been digging into active_support to see how config_accessor was working. This is the code I found:

What it is doing is fairly straight forward. It is creating setters and getters for attributes of a config object. The setter definition being held in the variable writer and the getter in reader.

The thing that really piqued my interest, was the construction of reader, writer and line. In particular, what line was doing.

After a little reading around I learnt that line is used to ensure the exception backtrace points to the place where the method is being defined. So for reader, the back trace will point to the place where the method was defined, and not to where class_eval was called.

To help me get my head around this, I wrote this:

This dynamically creates two methods via module_eval: LinePlay.raise_here, and LinePlay.raise_there. I called module_eval within make_into_method, to separate the definition method from the eval method. It then calls the two dynamically defined methods, rescues the exceptions and outputs the backtraces.

This is the output:

For raise_here, the line number where that method is defined, is passed to module_eval. If you look at the resulting backtrace, this shows line 4 as the source of the error. That is, the line where the method causing the error, was defined.

For raise_there, __FILE__ and line number are not passed to module_eval. In this case the backtrace points to where class_eval was called and not where it was defined.

So using this technique generates much more useful error reporting. A neat trick I think.

However, have you spotted the error in the original code? The variable line for reader is overwritten on the next line when writer is assigned. That means backtraces for both reader and writer will point at the writer definition. This error has since been corrected. See this commit.

A Rails app often has to connect to other services via an api key. For example, if you want users to be able to log in via their google credentials. One problem this raises is how to secure these api keys.

The solution I’ve used in my latest app, was to encrypt the key in the database. With this solution, the key is:

• instance specific;
• can be made easy to access through the app, but difficult out side the app;
• is not stored in the app code;
• doesn’t rely on having to load extra files to production environments.

I decided to base my solution on a Setting model I had used in another app. This would give me a way of storing the setting data, and retrieving it.

However, each setting value is stored in plain text in the database, so the setting would not only be accessible via the app, but also directly from the database. The value needed to be encrypted. I decided that I would not want to encrypt all settings, so I wanted encryption to be an optional setting.

The end result was mournful settings.

With this in place I was able to store my api keys and secrets encrypted in the database and retrieve them like this (example from devise initializer):

I used seeding to set up the setting objects with invalid values, and exposed the setting model via active_admin with this register:

In this way api keys can be accessed via administrators and modified as needed.

Of course, this is only as secure as the active admin interface, but that’s fine for this project. If I was being paranoid, I’d not expose the setting model via a web interface, and instead create and edit settings via the console.

An update
Recently I needed to secure some api keys without putting them into the database, and I used figaro. Definitely worth a look at if you need to secure api keys without having to exposed them via an admin interface.

I’m currently working on a questionnaire system which has two main objects: Question and Questionnaire. I had been associating them using has_and_belongs_to_many, but a new requirement means that I now need to be able to define a question order within each questionnaire.

act_as_list gives me much of the functionality I want. However, I could not just add that to the Question model, because I may need a question to be at position 1 in one questionnaire, and position 4 within another.

The solution was to refactor the has_and_belongs_to_many association to use has_many :through. To do this I had to create a joining class. I called this new class QuestionnairesQuestion. Using a plural on the first word of the class name allowed me to use the has_and_belongs_to_many join table, so I wouldn’t lose the existing relationships.

The required migration took a little work. First I had to add in an id and make it a primary key (the existing table only had question_id and questionnaire_id fields). Then I also needed to generate some initial values for the new fields I was adding. This was my solution:

I then needed to add the acts_as_list declaration to QuestionnairresQuestion with a scope option so that sorting was per questionnaire. These are the associations that worked for me:

This worked, but I did have an issue with determining which question was top within each scope. The problem was that act_as_list assumes the top item will have the position ‘1’. However, I’d made the position match the id, which meant that at best only one item would have a position of ‘1’. To fix this I created this rake task:

Running that got the associations working. There is probably a better way of doing this, but this will do for now.

My next problem was how to access the act_as_list functionality.

If questions were the act_as_list object, you could do things like this

However, as questions are used on multiple questionnaires and they need to be independently sortable within each questionnaire, it is the through table model QuestionnairesQuestion that acts_as_list. To change position the change must be made in the context of the questionnaire.

My solution was to add these methods to Questionnaire:

pass_method_to_questionnaires_question in combination with method_missing, allows you to pass to a questionnaire the acts_as_list method together with the question it needs to effect. The equivalent move_higher call then becomes:

You can now also do:

I’m working on an application that needs to send notifications via email, and I want to be able to test it in the development environment. In the past, I’ve either been able to set up a small Windows mail server (hMailServer is great for this) or just use my standard email account. However, I’m a little restricted in my current environment, and working in a virtual machine. What I really need is a local-only mail solution that will work on my Xubuntu VM.

In the past I’ve shied away from setting up a mail server on Linux. I failed miserably to set up sendmail in the past. Today I decided to try postfix, and found it very easy to set up a system to work the way I wanted it to.

Being lazy, I just used the Ubuntu software centre to install postfix. The installation process popped up a config screen, and I selected the local only option. Looking at the postfix documentation, I came up with this configuration (etc/postfix/main.cf):

I restarted postfix (‘sudo postfix reload’) and was then able to test (see this)

That worked fine, but I wasn’t sure where the email was going. I needed a client.

I first tried the default email client, which was Thunderbird. However, the configuration seemed to expect an internet mail server and I gave up on that idea pretty quickly.

Claws to the rescue. This mail client is easy to install via the software centre. The main issue was getting the basic settings right. These worked for me:

Claws mail client account preferences

Hey presto, “get mail” retrieved the message I’d created while testing at the command line. With the configuration above, I was even able to send more messages to myself.

Looking at this, it became obvious where the emails were being stored: /var/mail/rob. A quick look there and I found a text file containing all my emails.

Next step was to create an anonymous alias that I could use in my rails code. To do this I updated my etc/aliases file by adding this to the end:

At first this had no effect. I then found this article by Elliot Smith, which told me what I was missing:

That command regenerates your alias database file. As soon as I did that, I found I was able to send emails to developer@localhost.

Last but not least I needed to set up my rails app to work with this configuration. I added this to /config/environments/development.rb:

And added this to the end of that file

Overriding the mail recipient causes all emails leaving your rails app, to be redirected to the given address. With that in place, I could play with things like devise sign up confirmation emails without anything leaving my virtual environment. Just what I wanted.

It was all so simple, I don’t really know why I haven’t done this before.

Update: Catching all emails
I’ve needed to update my settings so that I can send an email to any email address, and it will end up in the rob@localhost mailbox. To do this, I followed the instructions given here.

I created /etc/postfix/virtual :

I then added this to the bottom of etc/postfix/main.cf :

I then ran these two commands to create the mapping and restart postfix:

Now all emails sent to localhost:25 end up in the rob@localhost mailbox.

I was chatting with a colleague who has asked me to build an app, and I was struggling to get to the root of what he wanted. After the chat, I had a little think about the problem, and realised that a lot of what was required could be dealt with by adding a disclaimer. That is, redirecting a user to a page that described things the user needed to be aware of before they got to their destination page, and to which they would have to acknowledge. The result was my latest gem ‘disclaimer‘.

I wanted it to behave like a before_filter, so the easiest way to do that was to use a before_filter under the bonnet. I also wanted to allow for the use of multiple disclaimer documents, so different destinations could have different disclaimers. So the system takes the name of the disclaimer to be displayed. The result allows you to do this in a controller:

The before_filter hook in was the bit that took the most head scratching, and the result was adding some methods to ActionController::Base.

I used an engine to provide the functionality, and that allowed me to easily add controllers to build disclaimer documents.

For my current requirement, I needed to break the document down into segments, so I added a model and controllers to do this. I also wanted the segments to be displayed via a JQuery accordion. I think in the long term, I’ll remove this, and just have a simple document controller. Developers will then be able to over-ride the simple document as they need.

I think I’ll also want to add some configuration options to the disclaimer, but I’m leaving adding them until I need them and therefore have a better idea of how they should behave.

One thing I will do soon, will be to add a default behaviour: so if you just add ‘disclaimer’ to a controller, it will display the first or a default disclaimer. If the latter, I’ll need to add a way of configuring this. I also need to add an elegant way of catching when a disclaimer call points to a document that doesn’t exist.