Grunt Deploying

Preface: This is the kind of guide we wish we had when we were starting to learn Grunt (or when we run into problems). It's lengthy, so if you want to skip that, there's a handy gist that goes along with it so you can share in our process.

This guide is separated into 3 parts. This is Part 3 and covers our deployment process. Part 1 covers our local development process and Part 2 covers our testing process.


grunt build

Before we get to deploying code, there's some optimization tasks that need to happen. These tasks are all handled by this build task which will produce us a dist folder than contains our entire, production-ready code.

We mentioned previously how we had a grunt serve:dist task that allowed us to check our production code locally. Turns out, that task uses this task for its files. So while this isn't a task we (usually) run directly, it is central to our deployment process.

Here's how we've configured it:

grunt.registerTask('build', [
  'clean:dist',
  'configureProxies:dist',
  'wiredep',
  'ngAnnotate',
  'injector',
  'concurrent:dist',
  'useminPrepare',
  'ngtemplates:dist',
  'autoprefixer',
  'concat',
  'copy:dist',
  'cssmin',
  'uglify',
  'filerev',
  'usemin',
  'htmlmin'
]);

clean:dist

We looked at our clean:server task earlier. Below is what we use for production:

clean: {
  dist: {
    files: [{
      dot: true,
      src: [
        '.tmp',
        '<%= yeoman.dist %>'
      ]
    }]
  }
}

Fairly straightforward again. We're clearing out any temporary files and our soon-to-be-generated dist folder.

That dot property? That's part of Grunt's file operator options which "Allow patterns to match filenames starting with a period, even if the pattern does not explicitly have a period in that spot."

configureProxies:dist

Since our deployments involve running end-to-end tests, we set up our proxy server within our build task to encourage those tests run against our compiled code. We covered how we configure proxies with grunt in our first post. To reiterate, grunt-connect-proxy allows us to test our code in a near to production environment; conveniently we can (without running a virtual machine) by proxying our API calls to an actual staging server.

Since it's not very different, the task is found in our gist at line 151.

ngAnnotate

Minifying Angular JavaScript can get pretty messy. In order for the injection system to work you have to use the array injection and pass in strings and ... it's just repetitive work that should be automated. That's why there's grunt-ng-annotate! The ngAnnotate task runs through your JavaScript files looking for injections that it can modify to make them minification-ready.

Our configuration for it is pretty simple.

// ngAnnotate adds, removes, and rebuilds angularjs dependency injection annotations
ngAnnotate: {
  options: {
    singleQuotes: true
  },
  app: {
    files: [
      {
        expand: true,
        src: ['<%= yeoman.app %>/**/*.js', '!<%= yeoman.app %>/**/*.test.js'],
        dest: '.tmp'
      }
    ]
  }
}

We use single quotes in our code and luckily there's even an option for that to match our own! Our file configuration is similar to others, we're just ignoring our test files and dumping the results into our .tmp directory.

Sidenote: The version of Grunt we're using (0.4.0) doesn't yet allow for piping of results into other tasks. That's where our .tmp directory comes into play. In order to continue with the result of a task, we need a place to store those results. We don't want to overwrite our existing files, so we toss them into our .tmp folder and do our work there. We've heard Gulp's piping can speed up performance of your tasks if that's your thing.

concurrent:dist

In our first post we showed how we used concurrent but didn't do much with it? For deployments, we now have more tasks to run concurrently (we could probably stretch it to more too)!

concurrent: {
  dist: [
    'sass:dist',
    'imagemin',
    'svgmin'
  ]
}

sass:dist

There are no changes here from our previously mentioned sass:server task. However, we're keeping it separate just in case we do decide to change it in the future.

sass: {
  dist: {
    files: {
      '.tmp/styles/main.css': '<%= yeoman.app %>/styles/main.scss'
    }
  }
}

imagemin

Image optimization is one of those funny tasks that needs to happen but is cumbersome to do -- that is, unless you automate it! We do that with grunt-contrib-imagemin. We don't have any specific settings, so we just tell it to look for all of the images it can find and optimize those.

imagemin: {
  dist: {
    files: [{
      expand: true,
      cwd: '<%= yeoman.app %>/images',
      src: '{,*/}*.{png,jpg,jpeg,gif}',
      dest: '<%= yeoman.dist %>/images'
    }]
  }
}

Sidenote: According to the documentation, this task can optimize SVG files too. It looks like SVG support wasn't in place when we initially started our product. The next task could be combined with this one!

svgmin

We run our SVG optimization as a separate task with grunt-svgmin (since it wasn't originally supported with imagemin). It's simple again:

svgmin: {
  dist: {
    files: [{
      expand: true,
      cwd: '<%= yeoman.app %>/images',
      src: '{,*/}*.svg',
      dest: '<%= yeoman.dist %>/images'
    }]
  }
}

useminPrepare

Once our concurrent tasks are done, we move onto the first of our two usemin tasks, which is useminPrepare. useminPrepare is a task that looks through your HTML files for any usemin tasks that need to run and then -- in memory -- automatically generates the needed Grunt configuration options for usemin tasks to use later.

Ours is configured, once again, quite simply to look for source files and where to output them when processed.

// Reads HTML for usemin blocks to enable smart builds that automatically
// concat, minify and revision files. Creates configurations in memory so
// additional tasks can operate on them
useminPrepare: {
  html: '<%= yeoman.app %>/**/*.html',
  options: {
    dest: '<%= yeoman.dist %>'
  }
}

ngtemplates:dist

We've already seen how we run this task for local development. Here's what we use to create our distribution files.

ngtemplates:  {
  options: {
    module: 'SourceClear',

      return '/' + templateString;
    }
  },
  dist: {
    cwd: '<%= yeoman.app %>/',
    src: [
      '**/*.html',
      '!index.html',
      '!404.html'
    ],
    dest: '.tmp/scripts/templates.js',
    options: {
      usemin: '/scripts/app.js' // <~~ This came from the <!-- build:js --> block
    }
  }
}

The biggest difference between this task and our local task is the options property. grunt-angular-templates supports our usemin task, so we tell it in which usemin-generated javascript file to append our templates.

copy:dist

copy: {
  dist: {
    files: [{
      expand: true,
      dot: true,
      cwd: '<%= yeoman.app %>',
      dest: '<%= yeoman.dist %>',
      src: [
        '*.{ico,png,txt}',
        'index.html',
        '404.html',
        'images/{,*/}*.{webp}',
        'fonts/**/*'
      ]
    }, {
      expand: true,
      cwd: '.tmp/images',
      dest: '<%= yeoman.dist %>/images',
      src: ['generated/*']
    }, {
      expand: true,
      cwd: './bower_components/components-font-awesome/fonts',
      src: '*',
      dest: '<%= yeoman.dist %>/fonts'
    }]
  }
}

This one's a bit complicated, but we can break it down easily!

For our first set of files to copy, we're taking all of our non-template html files, all of our fonts, a few root directory items (like our icon files), and any images that haven't been optimized yet (the webp filetype, for example). If you're using a .htaccess file, you'll want to leave the dot: true set so Grunt will pick it up as a file.

For our second set, we're copying from our .tmp/images/generated directory all of our optimized images and tossing them over into our distribution directory.

And lastly, if you have a Bower component that's not being imported otherwise, you'll want to copy it over at this step too.

the usemin plugins: concat, cssmin, uglify, filerev

Grunt-usemin knows how to use these plugins out of the box, all you need to do is install them. We don't even have to define configurations for them.

tasks that didn't change

We've already looked at wiredep, injector, autoprefixer from our local development server.

htmlmin

Finally, the last step, is to minify our final HTML files.

htmlmin: {
  dist: {
    options: {
      collapseWhitespace: true,
      conservativeCollapse: true,
      collapseBooleanAttributes: true,
      removeCommentsFromCDATA: true,
      removeOptionalTags: true
    },
    files: [{
      expand: true,
      cwd: '<%= yeoman.dist %>',
      src: ['*.html', '/scripts/**/*.html', '/views/**/*.html'],
      dest: '<%= yeoman.dist %>'
    }]
  }
}

With this, we're basically just changing our index.html and 404.html files. We have the same destination as the source because this is the final step and we won't need to pipe the files anywhere else.

We used to leave our template files out of our JS, but with the ngtemplates above, that's no longer the case. You can see the above setting as an example for how to use multiple paths as the source though!


grunt deploy

And finally the good stuff! Where we actually get to push code!

We haven't set up a continuous integration system (yet), but we're also not manually moving our files either! We have a staging and a production server and each pulls code from their respective branches in our front end git repository. This task gets our code built and placed into the proper branch!

We went through the grunt build task above, so let's just touch on the deploy steps below.

/**
 * Use:
 * grunt deploy:target --force
 * @param  {string} target  matches what follows deploy:*
 */
grunt.registerTask('deploy', 'Test, build, then git deploy', function (target) {

  // returns true if included but not set to anything in particular
  var force = grunt.option('force');

  /*
    Don't allow forced deploys to prod... for now.
   */
  if(force) {
    return grunt.task.run(['build', 'buildcontrol:staging']);
  }

  if(target === 'prod') {

    grunt.task.run([
      'test:suite',
      'build',
      'buildcontrol:' + target
    ]);

  } else {

    grunt.task.run([
      'test:suite',
      'build',
      'buildcontrol:' + target
    ]);

  }

});

Using the --force

While it's probably not the best idea in the world, we do maintain it as an option to force a deployment to our staging server. However, we do not want a forced deployment to our production server and don't allow for it. We return here to exit out of the function and not do anything else.

grunt deploy:prod

We placed an if statement here expecting the possibility of running different tasks for production (perhaps if we didn't want to uglify our staging code). However, as you can see, they're the same!

We test our code, build it, and then run it through buildcontrol.

grunt buildcontrol

Our final step in deployment is saving our built code. Which for us means committing and pushing it to a specific branch of our project. There's a great plugin, grunt-buildcontrol, that lets us do just that. Our config:

buildcontrol: {
  options: {
    dir: 'dist',
    commit: true,
    push: true,
    message: 'Built %sourceName% from commit %sourceCommit% on branch %sourceBranch%'
  },
  staging: {
    options: {
      remote: '[email protected]:owner/project.git',
      branch: 'staging'
    }
  },
  release: {
    options: {
      remote: '[email protected]:owner/project.git',
      branch: 'release'
    }
  }
}

In the options we set the directory we want to deploy from, that we want to commit and push it, and we provide a simple commit message. The details are basically straight from grunt-buildcontrol's usage example.

Sidenote: We ran into some issues with this when we didn't clean our dist directory. Since we started cleaning it, we've no longer run into any issues.

Extra content

Useful requires

There's two requires that save us tons of time. The first is load-grunt-tasks, which -- you guessed it -- loads all of our grunt tasks for us so we don't have to manually (we're all about that automation) specify each dependency. The second is time-grunt, a task that runs along side our others, timing them. This is greatly helpful when we need to address performance issues with our tasks.

// Load grunt tasks automatically
require('load-grunt-tasks')(grunt);

// Time how long tasks take. Can help when optimizing build times
require('time-grunt')(grunt);

Gruntfile.js organization

Maintaining all of this stuff in one file is a pain. An option to improve that (and one we're likely moving toward) is to set up something similar to how sailsjs Gruntfile.js is organized. You can see how they're separating concerns and pulling in configuration and tasks from separate files. Conveniently they have a readme on their task import. Nice!

Wrapping it all up.

Developer workflows -- like products -- are always a work-in-progress, there's always something to be made faster, simpler, more convenient. We hope this guide gives you a jumping off point for further exploring application build tools.

We've created this gist that contains our running, completed Gruntfile.js, a package.json that includes the dependencies it needs, and a file with all the example commands you need to have a blast using it.

If you've made it this far, thanks for reading! Have a comment or question? Let us know in the comments or hit us up on Twitter!

Mark Curphey, Vice President, Strategy Mark Curphey is the Vice President of Strategy at CA Veracode. Mark is the founder and CEO of SourceClear, a software composition analysis solution designed for DevSecOps, which was acquired by CA Technologies in 2018. In 2001, he founded the Open Web Application Security Project (OWASP), a non-profit organization known for its Top 10 list of Most Critical Web Application Security Risks. Mark moved to the U.S. in 2000 to join Internet Security Systems (acquired by IBM), and later held roles including director of information security at Charles Schwab, vice president of professional services at Foundstone (acquired by McAfee), and principal group program manager, developer division, at Microsoft. Born in the UK, Mark received his B.Eng, Mechanical Engineering from the University of Brighton, and his Masters in Information Security from Royal Holloway, University of London. In his spare time, he enjoys traveling, and cycling.

Love to learn about Application Security?

Get all the latest news, tips and articles delivered right to your inbox.

 

 

 

contact menu