Category Archives: Server Administration

Build your Vagrant box using Chef

What is Chef

The goal of Chef is to give you the ability to have an immutable infrastructure. This could be for your stand alone project or for an entire business. The purpose of Chef is to be able to create cookbooks that build your servers automatically. These cookbooks enable you to build your infrastructure in a logical and object oriented way. You don’t have to come from a server administration background to create them either. Chef was built using Ruby scripts. So as long as you have programmed in the past and know a little bit about Ruby you should be all set. For server knowledge all you really have to have is a basic understanding of how servers work. You will also have to have an understanding of what components you need. After that you can browse around for cookbooks and find what you are looking to install. After you build your initial cookbook or download an existing one you will start seeing how simple it can be. I would suggest that you start out with an existing cookbook. You will start finding out that the majority if not all cookbooks you need were already built by the community. Most developers give you full access to their cookbooks Github page. If for some reason you cannot find an existing cookbook you should be able to reference others for how to do it. When I setup my Chef infrastructure all cookbooks were already created. I simply had to modify a couple of them to fit my needs.

Why use Chef

Chef has many use cases. The main one of course is the ability to have an immutable infrastructure. These days with virtualization being so hot it’s important to be able to create a new instance of your infrastructure without having to thinking about. Traditionally before Chef server administrators would write their own shell scripts. After creating these shell script’s they would run them manually after installing the fresh OS. Not only was it time consuming to run these scripts on each server but they also had to be updated all the time. For example if a new version of the application came out. So they tended to get out dated faster. So a tool came out called Puppet. The purpose of Puppet was to take all of these shell scripts and to put them into a tool. This tool would provide some infrastructure so it was easier to manage the scripts. It also allowed you to create generic scripts so when an update came out the process was easier to upgrade. Puppet allows you to define a series of scripts to run. It was definitely a step up from running all of those shell scripts manually. It was a better way to manage these scripts and keep them organized.

After Puppet was out for a while Chef came out. Chef took a lot of the ideas from Puppet and made them better and even easier to use. Chef is the tool that allows a developer to create the infrastructure. It takes infrastructure out of the hands of server administrator’s and into the hands of developers. The reason why Chef is so friendly and powerful is because it was written in Ruby. Ruby is a very elegant language and is very human friendly. Also it’s an object oriented language so it’s very easy to abstract your cookbooks.

Another advantage to Chef is it has great enterprise support. If you sign up for Chef enterprise you can configure which cookbooks run and when. Then you can run a simple knife command and it will interact with the Chef enterprise server. It will look for your cookbooks defined per server and run them in the order you put them in. You can set a different order per server. You can also set it to run different cookbooks per server. Chef enterprise is actually free up to five nodes. If you are at all interested in it it’s a great place to start. So you can create up to five different configurations for free.

How do you use Chef with Vagrant

Vagrant ships with Chef Solo integrated. It’s very easy to get Vagrant started with Chef. The first thing you need to do is get Vagrant installed and setup. You can follow the following instructions on how to do that. After you get that installed you need to setup your Vagrant file with all of your chef configurations.

The following shows some of the configuration needed to get it up and running. The first line is the operating system you want to use. For my case it was ubuntu 14.04 but you can use whatever you like. This even includes Redhat. The next line is even more important. This will tell it to use the latest omnibus version. This line tells Vagrant to install chef on the server before anything else. When you put in :latest in it will take whatever the latest version of Chef available at the given time.

 

  config.vm.box = "chef/ubuntu-14.04"
  config.omnibus.chef_version = :latest

This next set of code is the configuration for Chef to use. These configuration settings are what Chef Enterprise allows you to do through their nice GUI interface. In Vagrant of chef solo you have to define these in the Vagrant file. In turn for Chef Enterprise you define them using their tool.

  config.vm.provision "chef_solo" do |chef|
    chef.cookbooks_path = "cookbooks" # path to your cookbooks
    chef.data_bags_path = "data_bags" # path to your data_bags
    chef_gem_path    = "/opt/chef/embedded/lib/ruby/gems/1.9.1"
    chef.binary_env  = "GEM_PATH=#{chef_gem_path} GEM_HOME=#{chef_gem_path}"
    chef.binary_path = "/opt/chef/bin"

    chef.add_recipe "apt"
    chef.add_recipe "build-essential"
    chef.add_recipe "oh-my-zsh"

    chef.json = {
      oh_my_zsh: {
        users: [
          {
            login: 'vagrant',
            theme: 'af-magic',
            plugins: [
              'git',
              'vi-mode',
              'rails',
              'ruby',
              'bundler',
              'virtualenv',
            ]
          },
        ],
      },
    }
  end

You can also run Chef solo by itself. If you use it standalone without Vagrant you have to tie everything together. I am not familiar with that and cannot speak to how that would work. I hope that this overview helps you learn a little bit more about Chef. I hope that it at least sparks your interest enough to play around with it. Vagrant is a great place to start if you do not have a server to work with.

For a working sample visit my vagrant-chef repo.

Why use Phusion Passenger for your Rails server

Phusion Passenger is one of three big players in the rails server game. The other two players are Unicorn and Puma. No matter which one you choose if you can configure it correctly and get through the setup process all of them work. All three of them are considered viable and able to handle the job. In my opinion though I feel that Passenger is the best out of the three.

Support

The first reason why I really like Passenger is because they have really great documentation. If you are unable to resolve your issue after reading through their documentation you didn’t look close enough. For example this is Passengers documentation for their nginx module. Yeah it has a ton of configuration options that you can just stick into your nginx configuration file and go. Also if you are wondering why something is not working you will be able to find the answer easily either by going through the documentation yourself or going to Google. If you Google for an issue using Unicorn you will probably end up finding it but it could take you a while. When I consider choosing a technology to use support is the first place I look at like am I able to get support and is the tool so easy to use that I most likely won’t need that support. If the answer is yes to both of them to me that is a good place to start. If the answer is yes to documentation but not to suppose I think that is an okay starting point. Sometimes you will not be able to find both.

Popularity

This one is a little controversial as the saying goes just because it’s popular doesn’t mean it’s great. That saying does hold true in some cases but when it comes to technology it usually doesn’t mean that it usually means a good sign. I have noticed that people in technology usually don’t stick with something too long if it’s not satisfying their needs. Passenger has great support including the rails core team. It also has support from some major companies including Basecamp, The New York Times, AirBnb, and Apple to name a few of them. If you would like to see a bigger list check out builtwith. Now the argument could be made that all of these companies have no clue what they are doing using Passenger but they all have very high levels of traffic and are well known as reputable companies. There is a reason why and passenger is a tool that helps them get there.

 Ease of Use

To be totally honest I only have experience in running Passenger and Unicorn. I have never had the experience setting up Puma so I will only be comparing Unicorn and passenger here. Although I have looked at Puma’s documentation and it doesn’t look so bad. Unicorn seems to be pretty easy to install on a single instance rails server. You can get your rails application up and running in a short amount of time. There are a lot of Unicorn scripts out there to get you started and up and running. The problem with Unicorn is when you have to do anything outside of running one application on the server. Dealing with multiple environments on one machine is possible (I think) but it seems like it wasn’t made for that. It also feels like Unicorn isn’t fully polished yet. Small things kept coming up like shutting down a Unicorn process without using the kill command. Like that example it just feels like commands that are suppose to work don’t. Now to be honest I am no server administrator so the problem may be obvious to someone else. So if you are a experience Unicorn professional these arguments could easily be debunked. The problem is most rails developers are not and therefore that argument really doesn’t matter. A lot of companies are rolling with developers being the server administrators also so the easier the tool is the better.

Install passenger is also a very simple process. The reason why it’s so simple is it ships with an installer that installs both passenger and nginx. You also install passenger and apache if that is your goal. In my case it was nginx. How cool is it that you get both nginx and passenger with one install. If you already installed nginx it’s recommended that you remove it and then run the passenger install. Also once it’s installed and started passenger handles everything else. This holds true even when you have multiple environments running on the same server. You don’t have to do any extra configuration to set this up either. You basically create another nginx file point it to the directory and go. Of course if you are adding another application on you will have to restart nginx but not passenger. Also passenger comes with a couple of pretty cool tools to monitor memory and performance. You simply type in the following command to get the memory usage.


rvmsudo passenger-memory-stats

Yes it’s that easy to monitor your rails processes. Unicorn you have to manually so a ps -ef | grep ‘unicorn’ to see the process. I suppose there are some tools like this for Unicorn but they don’t see as obvious to use. This tool is built into passenger so you can run it from anywhere you have an application running.

At the end of the day you have to use a tool that works best for your situation. Is passenger that tool probably but if you are an experience server administrator tools like Unicorn may be better. We all have our reason for choosing our toolset but you should have justification for those reason. At the end do whatever makes your life easier. In my case Passenger made my life easier as a non server administrator.

Here are some resources I used to determine the content in this post

  • https://github.com/phusion/passenger/wiki/Unicorn-vs-Phusion-Passenger
  • https://www.engineyard.com/articles/rails-server
  • https://www.digitalocean.com/community/articles/a-comparison-of-rack-web-servers-for-ruby-web-applications

Setting up multiple SSH profiles with Github

 

I recently had to setup multiple SSH profiles for Github. The reason why I had this requirement as my work required me to have a Github account outside of my personal account. The machine I am using is used for both work and personal development. I needed both SSH keys to work. I had never ran into this before as I never had two Github accounts. Since you are not able to use the same ssh key between multiple accounts in Github I had to come up with a way for both to work together. This makes sense for security purposes that the key should only be identified with one account. The process to set this up was pretty simple overall but that is after you understand what you need to do. Understanding exactly what is needed can be tricky at least from the documentation I found online.

The first thing you need to do is create a SSH key pair for both your personal account and work account. They should be using different email addresses.


$ ssh-keygen -t rsa -C "your_personal_email@youremail.com" # follow prompts use defaults if you want

$ ssh-keygen -t rsa -C "your_work_email@youremail.com" # specify new name for key

After that is done you should have a couple of keys. For example you could have id_rsa and id_rsa_work or whatever you called them. Add the proper SSH keys to your Github accounts following Generating SSH Keys – Github. The documentation there is really well done so no further explanation is needed.

So now you should be able to use each SSH account by adding it. Everything will work at this point but you will have to keep adding the ssh-add when you switch between accounts. If you don’t switch often this may be fine but it’s a really pain if you switch often. Also you tend to forget overtime why it’s broken all the sudden.

ssh-add ~/.ssh/id_rsa

The next step is you need to create an SSH config to handle the multiple accounts.

sudo vim ~/.ssh/config # add in the hosts

# Inside the file add the following

Host github.com-personal
    HostName github.com
    User git
    IdentityFile ~/.ssh/id_rsa
Host github.com-work
    HostName: github.com
    User git
    IdentityFile ~/.ssh/id_rsa_work

Save and close it out and that part should be all set. The next part is the most critical. If you have an existing repo go and modify your SSH string to add in the -work or -personal. This is whatever you named your host. Here is an example for my personal project.

git@github.com:[host-name]:[repo-name]/[repo-name]

If you are adding a new clone or new remote continue to use the same host name you set in your SSH profile. After you run the process it all makes sense it looks at the ssh config to determine which key to use.

Vagrant

I started using Vagrant just over a year ago. When I first started using it I didn’t know much about it. To use it you really don’t need to know much about it either. The concept of vagrant is you create your development environment in a virtual box and then use vagrant’s technology to connect to it. This connect is like a regular SSH connection but also has various other features. Vagrant basically supplies a way to make the virtual box act as a server. I recently had to create my own vagrant box from start to finish and had an interesting experience. Overall the experience was good and it went pretty smooth. For not knowing what I was doing at first I was able to get everything installed and running within five hours.  That’s not to bad when you add in download time and reading about how to do things. If you need to know how to install vagrant I would follow the documentation as it was well done. While doing the instal through I found some advantages and disadvantages to using vagrant.

I see a lot of advantage to vagrant. The first of course is you can create a vagrant box and then deploy it to many machines. So one developer on your team spends the time creating the box and the rest of the team benefits by not having to create it themselves. They simply can download the .box file and vagrant up. It’s that easy. This is a great time saver when you have a team. Even if you are by yourself it can be a time saver if you ever switch machines. Another advantage is everybody on the team is literally using the same environment. This is everything from OS version down to package version. This can be important especially if a major version is the difference. You can also start with a predefined box using Vagrant Cloud. For example you could start with a fresh Ubuntu box or you could search for specific package you need installed. This can save you time because you don’t have to do all of the installation process yourself. Even thought there is a lot of good there is some downside to using Vagrant.

First you have to be running a virtual machine using either VirtualBox or VMWare. This can slow down your machine and it takes extra resources. So if you have a machine that is not very powerful this can be a huge burden. Another disadvantage is everything has to go through that server connection. So when you access localhost:3000 it has to call the virtual server and run through that. This will add a little bit of overhead compared to just running it on your local machine. This can be a little frustrating to deal with if you are use to developing on your local machine.

To sum it up I think Vagrant is a great tool and has really advanced over the last year. Vagrant Cloud looks super neat and is a great idea. To share these environments should help developers for years to come. The biggest pain for a developer is to setup their environment. Vagrant makes that whole process a whole lot easier.

Ubuntu Server Not Defaulting Properly

I run an Ubuntu server and had it all setup for some time now. I never really had an issue like today come up until now. I was baffled into what could have went wrong. I didn’t even touch it but for some reason it just broke. I noticed that I was unable to get to my site using localhost like I always use to be able to do. After doing some research I found a solution that fixed it for my server.

When I initially setup my server I followed this guide and everything worked out great. Anybody looking to setup a LAMP stack server I would recommend reading this first. Even if you think you know what your doing this guide will make sure you don’t miss anything. Okay back to how I fixed my specific issue. I noticed that it wasn’t defaulting to any project. After thinking about it for some time it came to mind maybe the default project somehow got enabled again. So the fix turned out to be very simple. I just had to disable the default project again and it works great again.


sudo a2dissite default