Sails JS tasks configuration: defining a task to compile TS

The main part of the configuration happens in the “/tasks/register/compileAssets.js” file, where we define the compilation pipeline for our application. This compilation pipeline is then executed as a step in higher level pipelines, for instance where the whole build process is defined for a specific environment (“/tasks/register/build.js” for dev, “/tasks/register/buildProd.js” for prod).

Every step in those pipelines is defined as “{taskName}:{taskArgument}”, where the taskName is a Grunt module, defined in a file with the same name within the “/tasks/config/” folder, and the taskArgument defines what configuration we want to refer from it.

We can use all the pre-installed Grunt modules coming with Sails or we can install new ones from NPM. All of them can be referred in our compileAssets pipeline.


Example: adding a new task to compile TS

As first step we need to install the “typescript” NPM dependency, to use the TS compiler, and the “grunt-exec” Grunt module, in order to execute it within our pipeline.

Then we can create a “compileTs” configuration for “grunt-exec” to invoke, where we call the TS compiler. We put it into a new “/tasks/config/exec.js”:

module.exports = function (grunt) {

    grunt.config.set('exec', {
        compileTs: {
            cmd: 'tsc'
        }
    });

    grunt.loadNpmTasks('grunt-exec');
};

As we can see our command is just “tsc”, which will run the TS compiler with no additional configuration. This is because, at start, it will search for a “tsconfig.json” file in the application’s root folder, and is much easier to have our configuration there.

Here is an example of a “tsconfig.json” file, where we configure our compiler to get all the “.ts” files within our “apps” folder and to compile them into the “.tmp/public/js/apps” folder. Given that the “.tmp/public” folder is the output of our build (all the “copy” tasks just move all the necessary assets to there) we don’t need to add a “copy” task to copy them in the right place after the compilation.

{
    "compilerOptions": {
        "target": "es6",
        "module": "commonjs",
        "moduleResolution": "node",
        "sourceMap": false,
        "emitDecoratorMetadata": true,
        "experimentalDecorators": true,
        "removeComments": true,
        "noImplicitAny": false,
        "noEmitOnError": true,
        "outDir": ".tmp/public/js/apps",
        "rootDir": "apps"
    },
    "include": [
        "apps/**/*.ts"
    ]
}

Now that we created our task we just need to insert it into the compileAssets pipeline:

module.exports = function(grunt) {
    grunt.registerTask('compileAssets', [
        'clean:dev',
        'jst:dev',
        'less:dev',
        'copy:dependencies',
        'copy:dev',
        'exec:compileTs',
        'coffee:dev'
    ]);
};
Advertisements

Creating a Signup/Login authentication with Sails and Passport

In this example we want to implement a simple signup/login authentication system using Sails JS and Passport, with no ORM support. In order to crate this system we need to add the following dependencies:

  • bcrypt
  • passport
  • passport-local
  • mysql

The example is based on this post: http://iliketomatoes.com/implement-passport-js-authentication-with-sails-js-0-10-2/


Authentication Controller

As first thing we create an “authController” to handle signup, login and logout actions. Here we’ll use BCrypt to generate a secure hash for our passwords and Passport to handle the authentication. We’ll also use MySql to directly access the database without using an ORM layer. Apart for that we’ll keep logging in and out our application with the standard Sails’ methods req.logIn() and req.logout().

var passport = require('passport');
var mysql = require('mysql');
var bcrypt = require('bcrypt');

module.exports = {
    _config: {
        actions: false,
        shortcuts: false,
        rest: false
    },

    signup: function (req, res) {
        bcrypt.genSalt(10, function (err, salt) {
            bcrypt.hash(req.body.password, salt, function (err, hash) {
                if (!err) {
                    var user = {
                        email: req.body.email,
                        password: hash
                    };
                    var connection = mysql.createConnection(sails.config.connections.mysql);
                    connection.query('insert into users set ?', user, function (err, result) {
                        connection.end();
                        if (!err) {
                            user.id = result.insertId;
                            req.logIn(user, function (err) {
                                if (!err) {
                                    return res.redirect("/");
                                } else {
                                    sails.log.error(err);
                                    return res.redirect("/signup");
                                }
                            });
                        } else {
                            sails.log.error(err);
                            return res.redirect("/signup");
                        }
                    });
                } else {
                    sails.log.error(err);
                    return res.redirect("/signup");
                }
            });
        });
    },

    login: function (req, res) {
        passport.authenticate('local', function (err, user, info) {
            if (!err && !!user) {
                req.logIn(user, function (err) {
                    if (!err) {
                        return res.redirect("/");
                    } else {
                        sails.log.error(err);
                        return res.redirect("/login");
                    }
                });
            } else {
                sails.log.error({
                    err: err,
                    user: user,
                    info: info
                });
                return res.redirect("/login");
            }
        })(req, res);
    },

    logout: function (req, res) {
        req.logout();
        return res.redirect('/');
    }
};

Here is a description of what we are doing in each method:

  • signup
    We call the BCrypt’s genSalt() and hash() methods to generate a secure hash for our password. Then we connect to the database and store the user’s name and hash. In case of success we call the req.logIn() Sails method to login with the newly created user, otherwise we redirect back to the signup page.
  • login
    We call the Passport.authenticate() method, that checks the authentication and then invoke a callback where we can handle the result. In case an existing user has been matched by passport we call the req.logIn() Sails method to login with the matched user, otherwise we redirect back to the login page.
  • logout
    We simply call the req.logout() Sails method.

Home Controller

We create a simple controller whose actions can’t be accessed without authentication to test our system:

module.exports = {
    index: function (req, res) {
        return res.view();
    }
};

Login and Signup views

“/login” view:

<h1>Login</h1>

    
    
    

“/signup” view:

<h1>Signup</h1>

    
    
    

“/home/index” view:

<h1>Home</h1>

Routes Configuration

module.exports.routes = {
  '/': {
    controller: 'home',
    action: 'index'
  },
  'get /signup': {
    view: 'signup'
  },
  'post /signup': 'authController.signup',

  'get /login': {
    view: 'login'
  },
  'post /login': 'authController.login',

  '/logout': 'authController.logout'
};

Adding the Passport middleware to our pipeline

In order to have Passport included in the http pages processing pipeline we need to add it in the right position. We do this by editing the “/config/http.js” file:

module.exports.http = {
  middleware: {
    passportInit: require('passport').initialize(),
    passportSession: require('passport').session(),

    order: [
      'startRequestTimer',
      'cookieParser',
      'session',
      'passportInit',
      'passportSession',
      'myRequestLogger',
      'bodyParser',
      'handleBodyParserError',
      'compress',
      'methodOverride',
      'poweredBy',
      'router',
      'www',
      'favicon',
      '404',
      '500'
    ]
  },
};

We place the passportInit and passportSession middle-wares after the session element.


Configuring the Passport local authentication strategy

Now that Passport has been included in the pipeline we need to configure its behaviour to fit with the usage we made of it in our authentication controller. We do this by creating a new “/config/passport.js” file:

var passport = require('passport');
var LocalStrategy = require('passport-local').Strategy;
var bcrypt = require('bcrypt');
var mysql = require('mysql');

passport.serializeUser(function (user, done) {
    return done(null, user.id);
});

passport.deserializeUser(function (id, done) {
    var connection = mysql.createConnection(sails.config.connections.mysql)
    connection.query('select * from users where id = ? limit 1', [id], function (err, result) {
        connection.end();
        if (!err) {
            var user = result.length == 1 ? result[0] : null;
            return done(null, user);
        } else {
            return done(err);
        }
    });
});

passport.use(new LocalStrategy({
    usernameField: 'email',
    passwordField: 'password'
}, function (email, password, done) {
    var connection = mysql.createConnection(sails.config.connections.mysql)
    connection.query('select * from users where email = ? limit 1', [email], function (err, result) {
        connection.end();
        if (!err) {
            var user = result.length == 1 ? result[0] : null;
            if (!!user) {
                bcrypt.compare(password, user.password, function (err, res) {
                    if (!err) {
                        if (!!res) {
                            var returnUser = {
                                email: user.email,
                                id: user.id
                            };
                            return done(null, returnUser);
                        } else {
                            return done(null, null, { message: 'Invalid password' });
                        }
                    } else {
                        return done(err);
                    }
                });
            } else {
                return done(null, null, { message: 'Invalid email' });
            }
        } else {
            return done(err);
        }
    });
}));

Here is a description of what we are doing in each method:

  • serializeUser
    Here we map an user object into its serialised key, so we simply return its id
  • deserializeUser
    Here we map an user’s serialised key into its user object. To do that we read from the database using its id as key.
  • use
    Here is where we place out local authentication strategy. In the first parameter we specify what user object’s fields correspond to the user’s username and password. In the second parameter we define a function that, having received the username and password, checks them against our users table and returns whether a matching user is present or not. We use the BCrypt.compare() method to check our password against the hash we stored at signup time. The 3 values we return through the done() method here are the “err”, “user”, and “info” parameters we were receiving from the passport.authenticate() callback in the authentication controller.

Defining the Authentication policy

This is the policy we’ll apply to our home controller, so that it won’t be possible to access our pages without being authenticated. We create a new “/api/policies/isAuthenticated.js” file:

module.exports = function (req, res, next) {
    if (req.isAuthenticated()) {
        return next();
    }
    else {
        return res.redirect('/login');
    }
};

Now we bind the new policy to every route (hence the “*”), apart for those defined in the AuthController, by editing the “/config/policies.js” file:

module.exports.policies = {
  '*': 'isAuthenticated',
  'authController': true
};

Session integration

A simple session management can be added with the “express-mysql-session” module, in the same way as explained in a previous post.

Implementing a DB migration system (Sails JS)

DB migrations are a very useful way to keep track of the changes to the database and to have them automatically applied as soon as the application starts. There are many implementations of this, a famous one is Flyway, but in this example we want to write our own to work with Sails JS.

As first step we write a dbUpdateService service with the logic. The checkAndUpdate method is able to scan a db_scrpts fonder, detect all the scripts in the format “v{major}_{minor}.sql” and run them against the db depending on its version. The DB version is detected from a config table and is updated after having applied every script.

var mysql = require("mysql");
var fs = require("fs");
var co = require("co");

module.exports = {
    checkAndUpdate: function () {
        var connection = mysql.createConnection(sails.config.connections.mysql);
        connection.query("select `value` from config where `key` = 'version'", function (err, result) {
            connection.end();
            if (!err || err.code == "ER_NO_SUCH_TABLE") {
                var currentVersion = !!result && result.length > 0 ? result[0].value : "0";
                sails.log.info("Database version detected as", currentVersion);
                fs.readdir("db_scripts", function (err, files) {
                    if (!err) {
                        var versions = files.map(function (file) {
                            return file.replace("v", "").replace(".sql", "").replace("_", ".")
                        }).filter(function (version) {
                            return compareVersions(version, currentVersion) == 1;
                        }).sort(compareVersions);
                        if (versions.length > 0) {
                            sails.log.info("Database updates detected:", versions);
                            co(function* () {
                                for (var n = 0; n < versions.length; n++) {
                                    var version = versions[n];
                                    try {
                                        yield updateDb(version);
                                        sails.log.info("Database updated to version", version);
                                    }
                                    catch (err) {
                                        sails.log.error(err);
                                    }
                                }
                            });
                        } else {
                            sails.log.info("Database is up to date");
                        }
                    } else {
                        sails.log.error(err);
                    }
                });
            } else {
                sails.log.error(err);
            }
        });
    }
};

function updateDb(version) {
    return new Promise(function (resolve, reject) {
        fs.readFile("db_scripts/pawnsAndTiles_v" + version.replace(".", "_") + ".sql", "utf8", function (err, data) {
            if (!err) {
                var baseConfig = sails.config.connections.mysql;
                var multipleStatementsConfig = Object.assign({}, baseConfig, { multipleStatements: true })
                var connection = mysql.createConnection(multipleStatementsConfig);
                connection.query(data, function (err, result) {
                    connection.end();
                    if (!err) {
                        updateVersion(version)
                            .then(function () {
                                resolve();
                            })
                            .catch(function (err) {
                                reject(err);
                            });
                    } else {
                        reject(err);
                    }
                });
            } else {
                reject(err);
            }
        });
    });
};

function updateVersion(version) {
    return new Promise(function (resolve, reject) {
        var connection = mysql.createConnection(sails.config.connections.mysql);
        connection.query("update config set `value` = ? where `key` = 'version'", [version], function (err, result) {
            connection.end();
            if (!err) {
                resolve();
            } else {
                reject(err);
            }
        });
    });
};

function compareVersions(version1, version2) {
    var tokens1 = version1.split(".").map(function (token) { return parseInt(token) });
    var tokens2 = version2.split(".").map(function (token) { return parseInt(token) });
    for (var n = 0; n < Math.max(tokens1.length, tokens2.length); n++) {
        if (tokens1.length == n) {
            return -1;
        } else if (tokens2.length == n) {
            return 1;
        } else if (tokens1[n] != tokens2[n]) {
            return tokens1[n] < tokens2[n] ? -1 : 1;
        }
    }
    return 0;
};

Now we need to call our service at bootstrap, so that it will check and update the DB as first thing before actually running the application. To do this in Sails JS we can edit the “config\bootstrap.js” file:

var dbUpdateService = require('api/services/infra/dbUpdateService.js');

module.exports.bootstrap = function(cb) {
    dbUpdateService.checkAndUpdate();
    cb();
};

DB Connection and initialisation

Given that we aren’t using any ORM (Sails comes with Waterline) we manage the DB connection directly with mysql. In this case we added the connection configuration to the “config\connections.js” file:

mysql: {
  adapter: 'sails-mysql',
  host: '127.0.0.1',
  port: 3306,
  user: 'root',
  password: 'root',
  database: ’test-node-sails'
}

We also need to initialise the database with our config table. The dbUpdateService is able to detect if the config table is missing or empty and in that case it assumes that the version is 0, so we can do that in our first script, where we will create the table and set our DB version to 1.0. We can add a script “v1_0.sql” with this content:

CREATE TABLE `config` (
  `key` varchar(50) NOT NULL DEFAULT '',
  `value` varchar(255) NOT NULL DEFAULT '',
  PRIMARY KEY (`key`)
);

INSERT INTO `config` (`key`, `value`)
VALUES ('version','1.0');

A good free IDE to work with mySql is SequelPro.

Node JS – REST calls with ES6 Promises + Co JS Module

Here is an example of factory that creates an httpService to communicate with an external API with REST. We are using the npm module “request” to perform the HTTP calls. We are using as well ECMAScript 6 classes and promises, as long as the npm module “co”, that allows us to take advantage on the ES6 generators to await for async elaborations that returns a promise.

https://www.npmjs.com/package/co

"use strict";

var request = require("request-promise");
var co = require("co");

class HttpService {
     constructor(baseUri) {
          this.baseUri = baseUri;
          Object.seal(this);
     }

     call(options) {
          var self = this;

          console.log("Calling", self.baseUri + options.uri + "..");

          return new Promise(function (resolve, reject) {
               co(function* () {
                    try {
                         var data = yield request({
                              uri: self.baseUri + options.uri,
                              method: options.method,
                              headers: options.headers,
                              form: options.postData,
                              json: true
                         });
                         console.log(data);
                         resolve(data);
                    } catch (err) {
                         if (err.name == "StatusCodeError") {
                              console.log("ERROR: returned status code was", err.message);
                              reject(err.statusCode);
                         } else {
                              console.log("ERROR:", err.message);
                              reject(500);
                         }
                    }
               });
          });
     }
}

module.exports = {
     make: function (baseUri) {
          return new HttpService(baseUri);
     }
};

Node JS – Services + Globals (Sails JS)

We can define our services inside the “api/services” folder. Each service defined here will be globally accessible by its file name (without extension).

module.exports = {
     myServiceMethod: function (args) {
          return ...;
     }
};

However this approach has some limitation: all the services have to stay on the same level (sub-folders are not supported). So is better to disable this behaviour and to directly go through the standard Node JS require() mechanism.

To disable it we can edit the “config/globals.js” file in this way:

module.exports.globals = {
     ...

     services: false
}

Then we can refer the service we need from any controller or other service in this way (please note that we are using the npm module “app-root-path” to get the root path):

var appRoot = require("app-root-path");
var myService = require(appRoot + "/api/services/myService");

module.exports = {
     ...
}

This is the standard way in Node JS to require a module given its path.


Having a service globally available

Since we disabled the automatic behaviour we now have to explicitly require every service we need. However it could be useful to have some service automatically available without needing to request it everywhere. To have this we can add them to the global object by editing the “config/bootstrap.js” file:

global.appRoot = require('app-root-path');
global.myUtilService = require(global.appRoot + "/api/services/utils/myUtilService");

module.exports.bootstrap = function(cb) {
     ...
};

In this way we can refer the service through global.myUtilService (or even just myUtilService)from everywhere.


Specifying the current path to Node JS (NODE_PATH)

To avoid using external modules like “app-root-path” we can launch Node JS with the information of the current path we want it to consider when we require for modules. In this way we can use relative paths with no problems.

In order to do this we can specify the path with an environment variable before starting the app (in any of the ways we can start it), here is an example:

NODE_PATH=. npm start

In order to have VS Code doing the same thing when we debug the application we can edit its launch configuration (under “.vscode/launch.json”) to add “NODE_PATH”: “.” to the “env” node of the “launch” configuration.

Node JS – Authorisations (Sails JS)

In Sails JS we can define authorisations (among other things) with the policies. Policies are operations (usually checks) that are executed before any call to a controller. Since a policy is agnostic about what controller will be associated to and what other policies will be associated to the same controller, it behaves as a chain’s ring: if the result of a policy is positive it will just call the next ring, that could be another policy or the controller, depending on the configuration.

We define policies under the “api/policies” folder, here is an example of authentication policy reply with an “unauthorised” message whether the user isn’t logged in (we check this by searching for the user’s object in the session).

module.exports = function (req, res, next) {
     if (req.session.userObj) {
          return next();
     }

     return res.json("unauthorized");
};

To apply the policy to our controllers we have to edit the “config/policies.js” file in this way:

module.exports.policies = {
     ...

     "*": “<myPolicyFileNameWithoutExtension>",

     loginController: {
          "index": true
     }
}

With this configuration we are telling Sails JS that we want to apply our policy to all the controllers/actions, apart the loginController/index actions, that will be free to access.

In case our controller is inside a subfolder we can refer to it in this way:

“myFolder/myController”: {
     “myAction”: ...
}

Node JS – Session stored in MySQL (Sails JS)

In order to have the session stored in MySQL we can use the npm module “express-mysql-session”, that works directly with the Express JS framework that lies under Sails JS.

To configure our session we have to edit file “config/session.js” file in this way:

var MySQLSessionStore = require('express-mysql-session');

module.exports.session = {
     ...
     
     store: new MySQLSessionStore({
          host: 'localhost',
          port: 3306,
          user: 'root',
          password: 'root',
          database: 'test-node-sails'
     })
}

We can save data into the session from any controller through req.session:

req.session.user = { … };

Node JS – Controllers (Sails JS)

We can auto-generate a new controller by typing “sails generate controller []” from a terminal. For instance we can type:

sails generate controller test action1 action2 action3

to generate a “TestController.js” file like this:

module.exports = {
     action1: function (req, res) {
          return res.json({
               todo: 'not implemented yet'
          });
     },

     action2: function (req, res) {
          return res.json({
               todo: 'not implemented yet'
          });
     },

     action3: function (req, res) {
          return res.json({
               todo: 'not implemented yet'
          });
     }
};

The controller is generated inside the “controllers” folder. Is however possible to have subfolders, whose controllers will be automatically bound (and referred) with the corresponding subpath.

A controller’s behaviour is quite usual, we can return the default view (or a different one if we specify it with a parameter) by writing:

action2: function (req, res) {
     return res.view();
}

The default view is normally located under “views//.ejs”, so in this case will be under “views/test/action2.ejs”. Here is an example about what it can contain:

<h1>
     action 2
</h1>
<h2>
     this is raw html <%-'<button>button</button'%>
</h2>

By default every view in injected inside the general layout defined in the “views/layout.ejs” file, so we don’t need to define things such the page’s header or title. The place where our view will be rendered inside the layout is specified by “”.


Custom routing + viewModels

Is of course possible to define custom routes by editing the “config/routes.js” file. For instance we can add a new parametrised root that calls the “action3” action of our controller in this way:

'/test/:value/action3': {
     controller: 'test',
     action: 'action3'
}

Then inside our action we can read the parameter and pass it to the view through the viewModel in this way:

action3: function (req, res) {
     var value = req.param('value');

     return res.view({
          receivedValue: value
     });
}

Finally here is how to render something from the viewModel in the view:

<h1>
     action 3
</h1>
<h2>
     the received value is <%=receivedValue%>
</h2>
<h3>
     <%if(receivedValue == 'bingo') {%>
          <p>bingo!</p>
     <%} else {%>
          <p>not bingo..</p>
     <%}%>
</h3>