Monthly Archives: January 2014

Automating simple tasks

Leave a reply

When I first started my development career I was nervous about scripting things. I felt that if I wasn’t using some sort of GUI tool I had a higher potential for error. So I was afraid to automate my tasks even the most simple ones. I thought the risk wasn’t worth the reward. I thought if it only takes me a minute each time to do that task why would I spend more than that time creating an automated script. As I gain more experience in my career my view has changed on this. The turning point was only a couple of days ago. Over the holiday break I read The Passionate Programmer: Creating a Remarkable Career in Software Development (Pragmatic Life) and it really opened my eyes that everything we do over and over again should be automated. Not only tasks that take hours but even simple tasks that could take only a short period of time. So I started automating some of my work tasks. These are tasks that I repeat over and over again.

The first task that I automated was creating branches for Github. I found myself doing something similar to this over and over again.


git remote update; git co -b [ticketId]_description upstream/[branch]; git push [repo] [branch]

This process would only take me a little bit of time. I had never thought about automating it because the time it took wasn’t very long. After reading the book I decided to give it a try to see if it made it easier or faster. It turns out that I now save over half the time per branch creation. It may not seem like a lot of time at first but when you do it multiple times per day it adds up.

After I automated that process I thought what other tasks do I do over and over again that could potentially be automated. At my job we create pull requests for each branch when it’s ready to be merged. We use Github so the process that I was following was to go up to Github and use the GUI provided. Well it turns out that Github has an API that is very easy to use. So I created a automated script for creating pull request’s. Now instead of it taking me a minute or more to create a simple pull request it takes around 10 seconds. Again the time savings adds up. For example most days I will create up to three or more pull requests per day. You can see how the time saved really adds up.

I wasn’t satisfied with that level of automated. So I decided to take it up another level. Each pull request we create we then need to turn around and go down to our ticket tracking system and put that pull request link in there. We use PivotalTracker for ticket tracking. Well that seems like a tedious task especially since I am already using the Github API to create the pull request. So I added that to my automated process. So now to make a pull request and update the ticket status I can do so in around 10 seconds. This allowed me to cut the whole process pull request to updating the ticket to be under 15 seconds. What use to take me more than a minute now takes under a half of a second.

The point of this post is that automating even the simplest of tasks is important. It can make a huge difference. My thought process now when I get a new task is how can I automate the tedious part of the task. That way I can spend more time dealing with complex issues that are not able to be automated. If you are new to automation I would start with something simple. Don’t try to automate your entire process through one script. Possibly create multiple smaller scripts so you can do it over time.

Ticket tracking tools

Leave a reply

Ticket tracking is important in any software development project. Whether your project is small or large it’s important to know the story. It’s also nice to have a history so you can go back and know what you were thinking a year ago. Over the course of my career I have used many different ticket tracking systems. To name a few I have used Redmine, Bugzilla, Fogbugz, Github Issues and Pivotal Tracker. While using these tools I have noticed that they all can do the job just in different ways. I’ve started to wonder why is there so many ticket tracking systems out there and why isn’t one dominating the market. The simple answer is I don’t know because all of these tools can do the job each in their own way. I suppose the same question is raised with operating systems why isn’t everybody using a Mac or Window both can do the job right? I suppose with anything in life we all have our own preferences.

Redmine and Bugzilla require you to setup your own server (if you want the free edition). There is some Redmine or Bugzilla hosting out there. Both are able to do basic ticket tracking. Bugzilla uses a standard interface and doesn’t really do anything extra. The interface seems to be a little bit dated to what we expect of sites today. The same holds true for Redmine it’s a pretty basic interface to. Although the interface is modern it doesn’t look as flashy as other sites. One neat feature of redmine is that you to connect to a git or svn repository. Then you can view your source code and check out diff’s between commits. Other than that both do the same sort of create a ticket resolve a ticket flow. Of course with either solution you can modify the source to create your own flow.

The other tools I listed are all product based tools. They all come with a cost other than Github issues which is free for any open source projects. For the service based tools I have used I have found that they do not really add any more value than the open source ones. They might actually make your environment more limiting because you are not able to really customize them at all. The one benefit from a product based tool through as you are able to get the latest updates without having to manually add them yourself. Also you do not have to maintain or even have your own server setup to run them. Also their interfaces tend to be a little more flashy and a little more eye appealing.

At the end of the day the best tool depends on your situation. If your working on an open source project the best tool might be Github issues. That way everybody can use the same credentials they use to get into the repository. There would be no need to setup each developer with another account to another system.

If you are working on a private project my favorite tool would be Redmine. It’s a very stable project and does everything if not more than any of the product based tools. Also with the ability to customize the code to your needs you are able to do more with it. The product tools will be able to get the job done but at the same time come with a cost. That cost to me isn’t worth it when there are some great open source tools out there. Also with todays hosting server setup time is incredibly fast. Also if the project is using some type of source control probably git pull in updates is very easy also.

Of course this is only my opinion. I encourage you to try them out for yourself and pick the tool that best matches your needs.

Ruby to automate Git and Github tasks

Leave a reply

I am trying to automate my Github process more. I started by creating a script that will take some plain text and put it into a branch name format. The format is one that I defined. From there I thought it would be nice to automate sending up a pull request. That seems to take the most time. So I decided to create a Ruby script for that to. I put those scripts inside my Github repository so that anyone could benefit from them. All of this is using the Github API so you can reference their documentation for further details.

https://github.com/wallerjake/github

Fix ZSH af-magic theme

Leave a reply

There is a bug with the ZSH al-magic theme. To fix this you need to go into themes/af-magic.zsh-theme and update this line.


-PROMPT='$(virtualenv_prompt_info)$my_gray%n@%m%{$reset_color%}%' +RPROMPT='$(virtualenv_prompt_info)$my_gray%n@%m{$reset_color%}%'

 

That should fix your theme. The update is in a pull request and should be updated sometime in the near future hopefully.

Thinking Sphinx on rails projects

Leave a reply

I have been working with Thinking Sphinx for the past year and have made some observations. When I first started using it I thought it was a pretty neat tool. The concept is basically you define a set of relationships through a sphinx index. Then sphinx will build a query around them and cache that query for reuse. The query will be optimized so it will run very fast. This all works great when your talking about simple relationships. Which basically means a has_one or a has_many without a through relationship inside of the model. The problem with Sphinx is when you get into more complex datasets.

For example when you are creating a complex scope. Say you already created that scope using rails active record. You know that the scope works and you want sphinx to use that same scope. The issue is that sphinx makes you create the index in their format. So you are unable to reuse that same scope. It’s not a huge deal but it’s duplicating code just for indexing purposes. So if a future developer comes in and just updates the rails active record scope the sphinx indexing will be off. It might not even be noticeable until a user does a specific search. This can lead to bugs very easily.

Thinking sphinx also has a concept of indexes vs has. An index is a text object that will be able to be searched. A has relationship is made for numeric values so they can be sorted as a numeric value instead of a text value. So if you want one of these has relationships to be searchable you will need to define it as an index and has relationship. Another form a duplication where it should be smart enough to know if it’s indexed or has a has relationship it should be both sortable and searchable.

The last point as once you get a lot of data indexing is a pain. With a small amount of data indexing takes only around a minute. As you keep adding data the indexing process gets slower and slower. Eventually it takes a good half and hour to finish. It’s not a huge deal in a production environment that you can run indexing over night. The problem is when you are unable to have a slow time. If your site is getting hits all times of the day and even during the night you cannot slow it down for indexing. So you are kind of trapped in a way. Also indexing is a pain for developers. If they pull down a backup of the large dataset they have to wait that full indexing time before they can continue the development process. If they are working on a feature that they need to use the index for. The rest of the site will work just not the part that is using the sphinx index.

When you look at the big picture sphinx could be a great tool depending on the data set you have. For example if you have a complex dataset that requires a lot of joins sphinx probably isn’t the answer. It might be more pain to setup than it’s worth. If you have a simple table structure sphinx might be a great choice. It just all depends on how much data you have and how complex your data structure is. I would recommend exploring it but keeping in mind the more complex the structure the more time it takes to setup. Also the more data the longer indexing takes. So you have to weight that into if you are going to use this tool or not.