I have used both fixtures and factories in writing tests. I actually use both currently I use fixtures for my personal project and factories at work. I feel that their can be benefits to both but if I had to go with one I would go with fixtures. It seems to me that you can do everything and more with fixtures. Also your test suite stays a little more organized.

The first question one may ask is what is the difference between a fixture and a factory. Well a  fixture is a set of data that is static in nature. This data will get loaded for each test run. A fixture is created using a static file within your testing structure. This is different for each language you use it in. For example in Rails it will be a yaml file while in PHP it’s just an array. This could also depend on the framework your using. If you want the table to start out empty you can create an empty fixture file. Having the static file allows you to setup specific test scenarios. Then you can write your tests around those scenarios and you know the data will always be the same. So if a test shall fail you know it’s not because of a dataset change. In contrast factories create the object and the data at the same time. The database is not stocked with any preloaded data. If you need data for your specific test case you need to call the factory and have it create the data for you. Instead of creating a static file you need to create a factory file. This file will contain the name of the factory and then any custom data you need. Using a factory can have some benefits as you will get random data so your code is being tested against unknown data. Which may be reasonable as you have outside users using your system passing in random sets of data to.

So why do I feel that fixtures are better than factories if fixtures take longer to setup. I find it that factories tend to get messy as you have data objects being created all over your test suite. No factory is identical so you tend to have to work though a lot of different data scenarios. Also if you don’t setup your factory to pass in the proper data you could get false positives. For example maybe your interface validates a field to be one of three values. But you forget to setup your factory with one of those values. Now your tests are failing but the interface and application are running just fine. I suppose you could make the argument you can make the same mistake in a fixture but I feel you have to think about it more as you are filling this data in. Another reason why I don’t think factories are better is they are slower running. For example if you create your fixture files they run once per test run. But they run fast because they don’t have to use active record and create the object. Factories do create an object while creating the data. This takes extra time especially if you have setup a lot of data in your scenario. Another issue I have with using factories is you have to setup your data scenario on each test. You can keep it dry in a way by calling sub classes but again a lot of times you end up just doing factory.new because it’s a one off type of setup. So you tend to duplicate setting up factories a lot especially in a larger development team.

I feel that using either factories or fixtures will be fine. But I feel that fixtures gives you a slight edge for keeping your test suite organized. You are able to write tests against predictable scenarios. Also you are not having to repeat the same datasets over and over again. I feel that this is easier to avoid using fixtures rather than factories.

When I started out my development career I thought the idea of a framework was stupid. If I already know the language then what was the point in learning a framework. It’s just another thing I have to know about. At the time my mindset was I can do anything the framework can do. Yeah I might have to spend the time coding it up but at least I understood it at the end. Also a lot of the core functionality you need when you are creating a site is built into the language itself. For example PHP has all of the date functions one would need. At the time PHP wasn’t really not an object oriented language either so there wasn’t frameworks built for it when I started my career. When PHP was enhanced to support objects they kind of came over time. At that time I was in the mindset that I didn’t want to learn a custom framework. Since I already was doing everything myself. Then it all changed when I worked at a company who started working with a framework. After struggling for a little bit while first learning it I am glad I spent the time.

The main benefit I see from a framework is consistency. If you start with a framework and follow the frameworks pattern’s your code base will be consistent. It makes adding new developers onto the team way easier also. You are able to point them to documentation already created. Also that documentation is always up to date. If you are rolling your own framework most of the time documentation is the last step or in most cases doesn’t get done. Not only will you have consistency but you will not trap yourself into a bad pattern. If you choose a popular framework you know it has been tested on a wide variety of applications. If it can handle all of those applications you know it can handle yours. Unless you are trying to do something way out there.

Another benefit from using a framework is you get to use the communities code. Most frameworks at least the good ones have some form of extensions. They are called something different with each framework. For example in the yiiframework they are called extensions while in ruby they are called gems. These extensions allow you to easily pull in code and modify it to your needs. You save yourself a ton of time by not having to write that datatable helper or the audit helper. A lot of this is already done. Why not use something you know is working for others? You can then focus your time on the important things. No need to reinvent the wheel.

There can be some drawbacks to using a framework. One thing we face these days is frameworks are becoming to heavy. They have a ton of code that projects may not use. Also they kind of force you into a certain way of doing things. You can always code your way out of it but it might take you extra time. Another drawback is if you fall behind on major versions. If you fall behind upgrading can be a big pain. If you don’t upgrade to the latest version you could have security issues or extension compatibility issues.

At the end of the day you want to get your website up and running super fast. When you have a framework to start with you can have a simple application up and running in minutes. Also you don’t want to spend you time writing common code that is already been done a million times by other developers. Instead you want to focus on creating the customized aspects of your website or product. Even though it does take a little investment learning the framework it’s well worth the time. After you learn one framework you will see the pattern throughout other frameworks will be very similar.