## Enabling PostCSS with Rails 6

PostCSS is a tool for transforming CSS with JavaScript. To enable it in a Rails application, I needed to make the following changes to my app. Please note that my app does not use turbolinks.

/app/views/layouts/application.html.erb

Update so that the style sheet directive in the header uses stylesheet_pack_tag. That is, replace:


&lt;%= stylesheet_link_tag 'application', media: 'all' %>



With:


&lt;%= stylesheet_pack_tag 'application', media: 'all' %>



Sorry – opening brackets aren’t rendering nicely using the code include I’m using in this blog. Both declarations should start with a less than chevron.

/package.json


{
"name": "my_app",
"private": true,
"dependencies": {
"@rails/actioncable": "^6.0.0-alpha",
"@rails/activestorage": "^6.0.0-alpha",
"@rails/ujs": "^6.0.0-alpha",
"@rails/webpacker": "^4.0.7",
"polyfill-nodelist-foreach": "^1.0.1",
"postcss-browser-reporter": "^0.6.0",
"postcss-import": "^12.0.1",
"postcss-inline-svg": "^4.1.0",
"postcss-preset-env": "^6.7.0",
"postcss-reporter": "^6.0.1",
"postcss-svgo": "^4.0.2"
},
"version": "0.1.0",
"devDependencies": {
"webpack-dev-server": "^3.9.0"
}
}



/postcss.config.js

Add the following content to a file called

postcss.config.js
in the root of the app:


module.exports = () => ({
plugins: [
require("postcss-import"),
require("postcss-preset-env")({
autoprefixer: {},
features: {
"focus-within": true,
"nesting-rules": true,
"color-mod-function": {
unresolved: "warn"
},
"custom-properties": {
preserve: false,
warnings: true
}
}
}),
require("postcss-browser-reporter"),
require("postcss-reporter")
]
});



I believe that the rails webpacker stack already contains the PostCSS packages, but you may have to install them via yarn. I didn’t but then I have been trying to get this to work for a couple of days and may have installed them via another mechanism.

With the the above changes in place, I was then able to load PostCSS files. Note that it was simplest to do this via the javascript folder as they needed to be accessed relative to packs application.js file.

So as an example, I created the following file:

/app/javascript/packs/test.css


html,
body {
background: lightyellow;
}



Then to enable it I had to modify /app/javascript/packs/application.js by adding the following line:


import './test.css'



When I ran my app, the background turned yellow as expected.

One thing to note is that once these changes have been put in place, the sass stylesheets in /app/stylesheets are no longer loaded by the app.

Posted in Uncategorized | Comments Off on Enabling PostCSS with Rails 6

## Private methods in Ruby

I think the way private methods are defined, is one of Ruby’s few weaknesses. The main problem being that they separate the private methods from the methods that use them. For example, I prefer this:

def one
internal_method(1)
end

def two
internal_method(2)
end

def internal_method(n)
n
end

def three
other_internal_method(3)
end

def four
other_internal_method(4)
end

def other_internal_method(n)
n
end

To:

def one
internal_method(1)
end

def two
internal_method(2)
end

def three
other_internal_method(3)
end

def four
other_internal_method(4)
end

private

def internal_method(n)
n
end

def other_internal_method(n)
n
end

Because the former keeps the internal_method code close to the methods that use it.

This issue is minimised if we keep classes small, but that’s not always as easy as it sounds.

I’m also not particularly convinced about most arguments for making methods private. It takes a much more far sighted developer than me, to be sure that a method that I currently only use internally, won’t be useful externally later.

I’d also be more convinced of the necessity to privatize methods if accessing private method was particularly hard, but it is not: send(:private_method)

It is also worth considering what is being written. If the code is part of a shared library (a gem for example), there is a more convincing argument for privatizing methods. However, if the Ruby code is written as part of a monolithic application (a Rails app for example), then I think the argument is far less convincing.

Surely the only good reason for using a private method, is where the logic is only valid within the context of the current object, and using it outside that context is likely to result in errors. That is, external use will break things.

For example where I do use private methods is in Rails controllers, where public methods could accidentally be exposed by a routing error causing security vulnerabilities and unexpected usage. If only the action methods are public, this vulnerability is greatly reduced.

In summary, I’d say that a method should only be private if using it externally breaks something. If calling it externally doesn’t break anything, why restrict its use unnecessarily.

Posted in Ruby | Comments Off on Private methods in Ruby

## Downgrading the ElasticSearch version in SemaphoreCI

In our current project, we are using Elasticsearch for rapid search (it is the best search engine I’ve used to date), and are using Semaphore CI for continuous integration (Semaphore is new to me, but so far it looks very good). However, since the start of the project and now, the current Elasticsearch version has changed, and our code doesn’t match the latest version. Semaphore has Elasticsearch available, but in a new (and for us incompatible) version to the one used in our project.

Medium term, we need to update the version of Elasticsearch we are using on our project, but for now we need Semaphore to work with our code. To achieve that we need to install an older version of Elasticsearch in our Semaphore environment.

So my first step was a google, but that didn’t give me an obvious answer (there were a couple of scripts, but I wasn’t sure where to put them or how to use them).

I found the solution by using Semaphore support live chat. The response was very quick and I had a work solution within minutes. Another plus point for Semaphore.

The solution was to use the following commands:

These needed to be added to our Semaphore Project’s Build Settings, in the Setup section:

With that code in place we were able to set the Elasticsearch version to 2.4.1 in our Semaphore project and the problem was solved.

An update
On using the script in anger for a few days we found we were sometimes getting build failures.

Purging configuration files for elasticsearch (5.2.2) ...
userdel: user elasticsearch is currently used by process 2662
dpkg: error processing package elasticsearch (--purge):
subprocess installed post-removal script returned error exit status 8
Errors were encountered while processing:
elasticsearch

The process to purge the existing Elasticsearch files was occurring before the process had stopped. To fix this we created a modified version of the script with a 1 second pause prior to the file purge.

service elasticsearch stop
sleep 1
apt-get purge -y elasticsearch

This seems to have fixed the problem.

Posted in Ruby | Tagged , | Comments Off on Downgrading the ElasticSearch version in SemaphoreCI

## The How, When and Why of Ruby

One common question asked when people start using Ruby, is which books should I read. For me there are three essential Ruby books. However, I don’t think anyone should just read all three at once, and certainly not if they are just starting with Ruby. Here I wish to list those books, and why they should not be read together. The three books are:

For me, there is no better starting point for a Ruby developer than Dave Thomas’ Programming Ruby. It was the second Ruby book I read (I started with “Agile Web Development with Rails”), and it was the first to really bring home the elegance of Ruby, and how I could use it. Then and now, most of my Ruby development is done in a Rails environment, but I know that Rails stands on the shoulders of the giant that is Ruby. So even if all you want to do is develop Rails applications, I’d strongly recommend that you start by reading Programming Ruby. I’m confident you’ll be a better Ruby developer for it.

For me, Programming Ruby is all about how to program Ruby. What the syntax is, what to put where to get it to work, and how the core objects work together. And for this, I’ve yet to read a better book.

I’d been developing Ruby apps for a few years when I came across Russ Olsen’s fabulous Design Patterns in Ruby. It had a fundamental effect on the way I write my Ruby code. This wasn’t a book that taught me how to program, rather it made me think about the choices I make in coding in certain ways. This book is all about when to use certain coding patterns. When to split code into separate objects, when to use other objects to manage processes, and when to refactor code.

Design Patterns in Ruby will improve the way you write Ruby code, but I am confident that to get the best from it, you must first have a good appreciation of how Ruby works. For that, you need to have used Ruby in anger. It’s not a book for a new Ruby developer.

The last of the three books in my trilogy, is Pat Shuaghnessy’s Ruby Under a Microscope. I have only just finished reading this book in the last couple of days. It is easily the best Ruby book I’ve read since reading Design Patterns in Ruby. It is a fascinating investigation of how Ruby works under the bonnet. It teaches why Ruby works the way it does.

For me, Ruby Under a Microscope is THE book to read once you have learnt the lessons from reading the other two books, and used those to write Ruby code, and hone your skills.

So if you’ve got this far, I hope you will understand why I think these three books shouldn’t be read at the same time. They represent three stages of moving from a beginner to a seasoned Ruby developer, and it’s not until you’ve master the lessons from one, that you should move onto the next in my opinion. That takes time and effort, but it is well worth it. They are three excellent books. Just don’t rush to read all three at once.

Posted in Ruby | 2 Comments

## In praise of the super user: The problem with the admin/user paradigm

I am coming to the end of my longest ruby contract so far – building web applications for Warwickshire County Council. It’s been a wonderfully productive couple of years, and I’ve gained a lot from working with the team at WCC. However, I think my most long lasting lesson learnt may come from a mistake.

I am entering the last week of my contract and the main focus is now hand over. A lot of this has already happened, and I’ve detected a common thread. I am being repeatedly asked the same question: “What can users do as admin, and what should developers do as admin?” This morning I have woken up with the realisation that the reason I’m being asked this, is because I’ve made a fundamental mistake with my app designs: I’ve overlooked the need for the super user.

If I’d defined super users in my apps, I wouldn’t now be asked this question.

I think I’ve fallen into a trap of thinking the default access levels for a web app should be user and admin. I’ve only gone beyond this access model paradigm if the project specification has stated that a more complicated model is required. I can understand why I’ve fallen into this way of thinking. Just have a look at a google search for “ruby admin user” and see how many of the results point to examples using just admin and user. For another example, have a look at the Devise README; its examples use just user and admin.

I don’t think it’s the fault of the people writing these examples. It is much easier to write example code using the simplest models, and admin/user is the simplest access model. The mistake is then thinking that the simplest model should also be the default model. Just because a model is useful as an example, doesn’t make it the best model to put into production!

So why do I need a super user? I think the easiest way to answer that is to ask “why do I need an admin user?”. The answer is, I want to make it easy to modify the way the application behaves without having to go to the bother of writing more code – to be more accurate – without having to write more code after I’ve moved on to building my next great application. I hate having to go back to an old code base to modify it. I’d much rather spend my time building swanky new applications.

So I try to build my apps with enough in-built flexibility, that their behaviour can be changed via tweaks at the admin level. Of course there is a limit to this, but in my experience, it’s not that difficult to predict the main ways that an app may need to be modified in future, and to build admin functions accordingly.

There will always be some behaviour changes that either are not predicted, or too complicated to build into the original app. In which case, building the next iteration of the app is the only option. But we have to live with that.

OK – so that’s explained why I have admin users. Why super users?

If you look at the way a web app needs to change through its life there are two types of changes:

1. Changes anyone could make, without breaking the app
2. Changes that could break the app, and need to be done by someone who understands the consequences.

Type 1 changes could be made by super users, and type 2 changes by admins.

Some examples super user changes:

• Assign users to groups
• Modify header and label texts
• Correction of erroneous user data entry

• Modification to whitelist/blacklist regex parameters
• Changing oauth id/key pairs
• Modification of data at the database table level

There is some cross over. There may be a good reason why you wouldn’t want a super user in a particular app, to add users. But these examples demonstrate the point.

For an app you manage as admin, the more you can allow super users to manage themselves, the less work you’ll have to do. And you’re not the only one who’ll be pleased about that. The super users will feel empowered too and enjoy using your app more if they feel they have some control of it. It’s win, win!

However, before we spend too much time slapping each other on the back, there is another key point to highlight: admins and super users need different interfaces!

Have a look at the last two examples for each type:

Super user
Correction of erroneous user data entry
Modification of data at the database table level

It is too easy to think that the way to give super users the ability to correct erroneous data, is to give them database table level access to the data. If you do that, a super user will break your app at some time. Instead, the super user has to be given an interface where they can safely modify data without breaking the app. That usually means a modified version of the origin data input form.

That means that most web applications need both an admin and a super user portal.

And don’t forget, not every user is a super user. The default model should be three levels of access:

User
The people who need to be identified, and do the day to day input and access of data.
Super user
The people responsible for the management of the app. This is often the owner of the app – the customer who asked you to build the app – and the people within their team who they delegate to be responsible for the app.
The app developers and the technical team tasked with supporting the app

I believe the main mistake I’ve made at WCC, is to provide apps with a single admin portal that both admins and super users use, and then relying on my direct instructions to super users as to what they should and should not do within the admin portal, to prevent accidents. The handover process has demonstrated to me that this is a poor way of managing things, as it only lasts as long as super users remember the instructions.

Posted in Blog, Ruby | Comments Off on In praise of the super user: The problem with the admin/user paradigm

## The £1.01 coin

The Monster Raving Loony Party in the Kenilworth area have released their Manickfesto described here:

One of the proposals is to introduce a 99p coin to make shopping easier in 99p stores.

However, I think they are missing a trick. They should also introduce a £1.01 coin for change.

Not only would this make getting change in 99p stores easier, it will make it easier to use 99p coins with other coins to make more expensive purchases: 99p + £1.01 + 99p = £1.99

Permission is hereby granted, free of charge, to any person
(you know who I’m talking about Nick Green) thinking
of creating a £1.01 coin to use for change in 99p stores (the
“Idea”), to deal in the Idea without restriction, including
without limitation the rights to use, copy, modify, merge, publish,
distribute, sublicense, and/or sell copies of the Idea, and to
permit persons to whom the Idea is furnished to do so, subject to
the following conditions:

The above copyright notice and this permission notice shall be
included in all copies or substantial portions of the Idea.

THE IDEA IS PROVIDED “AS IS”, WITHOUT WARRANTY OF ANY KIND,
EXPRESS OR IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF
MERCHANTABILITY, FITNESS FOR A PARTICULAR PURPOSE AND
NONINFRINGEMENT. IN NO EVENT SHALL THE AUTHORS OR COPYRIGHT HOLDERS BE
LIABLE FOR ANY CLAIM, DAMAGES OR OTHER LIABILITY, WHETHER IN AN ACTION
OF CONTRACT, TORT OR OTHERWISE, ARISING FROM, OUT OF OR IN CONNECTION
WITH THE IDEA OR THE USE OR OTHER DEALINGS IN THE IDEA.

Posted in Brilliant Ideas | 1 Comment

## Using Jasmine with Rails 4.1

I had a few problem getting fixtures to work within a Jasmine environment. I was getting the error “ReferenceError: loadFixtures is not defined”

The fixture was an html pages at spec/javascripts/fixtures/form.html, and this was my initial test code:

describe("Rollback prompt", function(){

beforeEach(function(){
});

describe('check test environment', function(){

it('should always pass', function(){
expect(true).toBe(true);
});

});

});

I’d copied this format from a rails 3 project where I’d used the jasmine gem successfully.

To get this to work with Rails 4.1, I had to:

## Use jasmine-rails gems

I added this to my Gemfile (replacing gem ‘jasmine’)

group :development, :test do
# JavaScript test environment
gem 'jasmine-rails'
gem 'jasmine-jquery-rails'
end

And added a spec_helper at spec/javascripts/helpers/spec_helper.js:

//= require jquery
//= require jasmine-jquery

## Mount fixtures separately

I mounted my fixtures via config/initializers/jasmine.rb:

# Map fixtures directory for Jasmine suite
if defined?(JasmineRails)
JasmineFixtureServer = Proc.new do |env|
Rack::Directory.new('spec/javascripts/fixtures').call(env)
end
end

And then updated my config/routes.rb:

if defined?(JasmineRails)
mount JasmineRails::Engine => '/specs'
mount JasmineFixtureServer => '/spec/javascripts/fixtures'
end

After that my test passed successfully and I was ready to start building by JavaScript functions.

## Solution source

These pages were key to me finding this solution:

Posted in JavaScript, Ruby | Comments Off on Using Jasmine with Rails 4.1

## The lows and highs of SAML

I’ve been working on a number of OAUTH based authentication systems recently as part of the work I’m doing for Warwickshire County Council on the Alpha Identify project. This has led me to also look at SAML. I’ve been building a simple SAML demo app to help me understand how it works, and in expectation that I will need to start building SAML systems in the next month or two. I’ve been using Onelogin’s ruby-saml as a guide to what is needs, and Lawrence Pit’s ruby-saml-idp project was a great help in getting started.

In working through ruby-saml, it seems clear to me that it is a porting from another language. This works, and means it is based on proven techniques. However, I think starting over and building some of the tools needed to produce SAML solutions, in a more ruby way, would make the code cleaner and more easy to extend. So I have started building saml_tools as a ruby toolbox for working with SAML objects.

Another major driver behind saml_tools stems from my belief that the starting point for building many SAML solutions will be the SAML documents themselves. That is, the developer will know the SAML document that will be sent to request authentication, and the document that will be sent in response to that request. They will then need to build the tools that will create, send, receive and process these documents. So I wanted a system that will make it easy to use an existing SAML document as a template for generating more documents. For me, the obvious solution was to use erb templates. It works well for Rails; why not SAML too. The resulting template is a lot easier to ‘read’ and understand, than a document that is solely built in multiple Nokogiri or REXML insertions.

However, the process has not been plain sailing and has involved a lot of head scratching, particularly regarding the signing of SAML responses. To quote this blog: “using a certificate to sign a request is the root of all evil.“, and putting the assertion signature inside the assertion itself seems perversely complex to me. I think the best demonstration of the lows and highs I went through in trying to work it out are three email I sent to a college during my ordeal:

Hi,

Thought I’d let off steam a little:

Unravelling SAML is A PAIN IN THE ARSE!

To test something, you have to pull something from over there, and manipulate it based on something somewhere else, and then compare it with the element here!

For example, the signature is in the stuff being signed. So you have to extract the signature, before you can check if it matches the stuff it was in.

Who designed this system!

Rob – working through it slowly but surely.

Which was followed fairly quickly with:

I think I’ve come up with a really simple example of why inserting the signature into the data being signed is fundamentally flawed.

Imagine my data is all held in a foo tag, I create a signature that I store in a bar tag:

<foo>
<bar>
<\bar>
<\foo>

Before I can check the signature I need to remove bar. However, what do I leave behind? For example, should foo now look like this:

<foo>
<\foo>

or this:

<foo><\foo>

Now that may seem a trivial difference, but the problem is that the signed data is wrapped up with base64 encoding. And those two examples output different results when encoded!!!!

At that point I was ready to give up. I’d written a sequence of steps that appeared to be working, but when I put them together to check part of a signature from a real SAML request document, it failed. I could not see what was wrong and I was starting to go round in circles. Fortunately, being the pig headed buffoon I am, I persevered and finally:

The solution to my problems was to read up on XML security. Reading this was my eureka moment:

http://users.dcc.uchile.cl/~pcamacho/tutorial/web/xmlsec/xmlsec.html

The way SAML signs responses isn’t so much a SAML thing, as an XML thing. It’s one of the “standard” ways of signing XML documents. I still think it is mad to put the signature inside the section of XML being signed – especially as the standard gives you other options.

I think it must come from when you sign a whole document. In that case, where else can you put the signature but inside the document? Otherwise you risk breaking the integrity of the document, or separating the signature from the document being signed.

Also, two things I struggled to get my head round: canonicalization, and the format of white space around the removed signature; turned out to be intimately related. Canonicalization is the fix for the white space problem; it formats any XML document or fragment, into a standard format that is shared across all implementations of XML. So it ensures that the space betweens two elements, that is left when a signature is removed, is always of the same format.

So in two days I’ve gone from being ready to give up on SAML, to now having a much better understanding of how it all fits together.

With my new found knowledge, I was able to build a SAML response from scratch, and when I tested that against my code, it validated correctly. It turned out that the problem was not the code but the document I was testing it against. That’s why I could not see a fault in the code!

However, the key point is this is yet another example of where failure leads to much better understanding. If my code had just worked, I would not have had to dig so much deeper into the underlying systems, and I would not have learnt as much as I did about how SAML works.

Posted in Ruby | Tagged , , , | Comments Off on The lows and highs of SAML

## Using Exceptions to help separate functionality

I have recently had to split a class into two and found raising exceptions via a new error class was key to the success of this process.

I’ve been working on Geminabox. The goal was to add the facility for Geminabox to act as a proxy for RubyGems.

At an early stage of the development, there was a requirement to split the Geminabox class into two. The problem was that this class was both handling the storage of local gems, and the HTTP requests for this service. This made it difficult to hook into Geminabox to store a new local gem, without also loading up the Sinatra component that was handling the HTTP requests.

A key to understanding the problem is to have a look at the original Geminabox#handle_incoming_gem method. This method calls further Geminabox instance methods to handle the storage of the incoming gem.

The solution was to create a new class that would handle the gem storage, and for Geminabox#handle_incoming_gem to pass the new gem to that class. The new class was called GemStore.

In the original Geminabox class, the error handling was combined into a single method that also controlled the output of an HTML error report via HTTP. That method being Geminabox#error_response. This method could be called by a number of Geminabox instance methods active in the gem storage process. The raising of errors needed to be separated from the mechanism for displaying error messages.

The result was a new error class GemStoreError. Now GemStore could raise a GemStoreError when a problem was identified. Then any process using GemStore could easily identify these errors and handle them appropriately.

With GemStore and GemStoreError in place Geminabox#handle_incoming_gem could be refactored, and other processes could use GemStore to store gems within Geminabox.

Posted in Ruby | Comments Off on Using Exceptions to help separate functionality

## Intercepting Ruby exceptions

The way some Ruby applications catch exceptions makes it more difficult to debug the underlying issue. That is because a common pattern for handling an exception is this:

begin
do_something_that_fails
rescue SomeError
raise MyAppError.new "Something went wrong"
end

So where is the problem? The problem is that by capturing the error in this way, the originals exception information is lost. That is, the original exception usually contains a message and back-trace that make debugging the error easier. With the pattern above, the output is only the MyAppError message, and the back trace to this code.

Recently, this issue has led to me not trapping errors as much as I used to, because letting the underlying exception bubble up through my apps has often resulted in more meaningful and useful errors. However, I’ve just found nesty, and I think it is the solution to this issue. That is, it allows exceptions to be caught, flagged and handled within an app, whilst preserving the original error message and back-trace.

I’ve been playing with nesty and the result is three simple ruby programs that I believe demonstrate the value of the nesty approach to handling exceptions from underlying code.

I’ve created three files that contain the code shown below. Each one has File.new(‘no_such_file’) called on line 8 so that there is some consistency between the errors raised by each example. (Unfortunately the empty lines used to achieve this don’t display output in this blog – so please assume that File.new(‘no_such_file’) is always on line 8 )

Version one: No Error handling

File.new('no_such_file')

The error output when this run is:

wrong_file.rb:8:in initialize': No such file or directory - no_such_file (Errno::ENOENT)
from wrong_file.rb:8:in new'
from wrong_file.rb:8:in <main>'

So a nice meaningful error that tells me what the underlying issue was. However, it would be nice if I could add some information about what I was trying to do when this error occurred. To do that I need to trap the error and handle it.

Version two: Trap the error and raise an app specific exception

class MyError < StandardError

end

begin
File.new('no_such_file')
rescue
raise MyError.new "Unable to access file"
end

This outputs this error message:

wrong_file_rescued.rb:10:in rescue in <main>': Unable to access file (MyError)
from wrong_file_rescued.rb:7:in 
<main>'

This contains error information that is specific to my app, but a lot of information is lost. Even the line number where the error occurred is lost. Line 7 is the begin statement.

require 'nesty'

class MyError < StandardError
include Nesty::NestedError
end

begin
File.new('no_such_file')
rescue
raise MyError.new "Unable to access file"
end

And this outputs:

wrong_file_nesty.rb:10:in rescue in <main>': Unable to access file (MyError)
from wrong_file_nesty.rb:7:in <main>'
from wrong_file_nesty.rb:8:in initialize': No such file or directory - no_such_file (Errno::ENOENT)
from wrong_file_nesty.rb:8:in new'
from wrong_file_nesty.rb:8:in <main>'

Now I have the best of both worlds: both my app specific error details and those of the underlying exception.

I found nesty via the authors blog: Skorks, and I’d thoroughly recommend you have a look at that (and the rest of that excellent blog).

Posted in Ruby | Tagged | Comments Off on Intercepting Ruby exceptions