Preparing for the build

The internals to a .deb are fairly straightforward. First you need a directory called “DEBIAN” that contains a few files that detail and control the processing during installation; this directory and all files contained in it MUST be named exactly as you see them below. Secondly, you need to include the full target directory structure for any files that you want to copy over during installation. You can have as many directory structures as needed for your package.

Example:

+ setup
    + DEBIAN
        - control
        - debian-binary
        - postinst
        - postrm
        - preinst
        - prerm
    + etc
        + apache2
            + sites-availalble
                - mysite

Explanation:

• setup: This is just the top level directory that holds the contents you want to package up – call it whatever you want
• DEBIAN: This is the directory that holds the instructions for deploying the package. This must be in all caps.
• DEBIAN/control: This is the control file for the package. It contains meta data and dependancy information – more on this later.
• DEBIAN/debian-binary: This file just contains the version number for the package manager you are using.
• DEBIAN/postinst: This is a shell script that gets executed after installation
• DEBIAN/postrm: This is a shell script that gets executed after removal
• DEBIAN/preinst: This is a shell script that gets executed before installation
• DEBIAN/prerm: This is a shell script that gets executed before removal
• etc/apache2/sites-available: This is the directory structure that matches the target system. This structure gives the package the path on where to copy over the “mysite” file
• mysite: this is the apache virtual host configuration file that i want to copy to the server

Configuration

DEBIAN/control

We need to add our configuration data to the control file.

Also, since our application uses some PHP features for CuRL and memcache, I need to make sure that the target system has the appropriate packages installed.

Full details on the control file can be found here, but for our needs we can just use the following:

Package: serversetup
Version: 1.0
Section: devel
Priority: optional
Architecture: amd64
Depends: memcached, php5-curl, php5-memcache
Installed-Size: 4
Maintainer: My Name <my@email.com>
Description: Setup package for MySite

Explanation:

• Package: This is the name of your package.
• Version: This is the version number of your package. Make sure to increment for subsequent releases.
• Section: This is the section where the package is stored in the Debian FTP sites.
• Priority: This is the priority for installation of the package. Setting this helps apt sort the package properly when installing multiple items.
• Architecture: This is chip architecture you are packaging for.
• Depends: This is the list of packages that need to be installed for your package to install/run properly
• Installed-Size: This is the amount of space that will be taken up by installing the package.
• Maintainer: This is the name and email of the person who maintains the package
• Description: This is a brief summary of the package contents

DEBIAN/debian-binary

This file just contains the version number for the package manager. So just put the text: “2.0″ followed by a line break.

mysite

This file contains the apache virtual host configuration.

<VirtualHost *>
    ServerName mysite.com
    DocumentRoot /var/www/vhosts/mysite/pub
</VirtualHost>

Scripts

This package is pretty simple, so we really only need to worry about running a few commands after the package is installed. For our purposes here, the postinst script will fit our needs.

#!/bin/bash

# add document root
mkdir -p /var/www/vhosts/mysite/pub
echo "<h1>Empty Document Root</h1>" > /var/ww/vhosts/mysite/pub/index.html

# enable vhost
a2ensite mysite
/etc/init.d/apache2 restart

Building the Package

Now that we have our control file defined and our scripts written, we can move on to bundling all these files together into an actual debian package. To do this, we just run a single command:

dpkg -b /path/to/setup serversetup_1.0_amd64.deb

Obviously you need to adjust the first parameter to match the location of your setup directory, and you may adjust the name of your .deb file to meet your needs. Typically you will want to name it something in line with the pattern “package_version_architecture.deb”.

Full detains on naming conventions can be found here.

Installing the Package

One of the great things about debian packages is the auto solving of dependancy issues. The thing with manually installing a local package via dpkg is that it does not resolve the dependancies for you, it just errors out and lists what you need to resolve yourself.

If you want to have the dependancies resolved for you, use gdebi instead to install packages. All dependencies specified in your control file will be handled just like installing a remote package via apt-get.

sudo gdebi /path/to/serversetup_1.0_amd64.deb

Taking it Further

By using this method it is now possible for all developers in my group to create their own virtual machine and get up and running with minimal effort. My setup package details all the needed packages that we use for development to make sure that the virtual machine is running everything we need it too. I also bundled some php extensions that we use for charting and debugging so setting these items up is a minimal task.

I’m currently looking at using debian packages to deploy large scale php application updates to a cluster of servers. In theory we should be able to set up an apt repository and allow each individual server pull down and install updates with the built in package manager.

Have any comments or ideas on how to make this better? Please share!

I saw on slashdot today that Nokia is going to be releaseing QT under the LGPL license starting with version 4.5 (site). Back in the day, when I was toying with leaving the web industry and going more into traditional software developement, QT caught my eye. The thing that stopped me from diving into QT at that time, was the $5000 license fee for anything non open source.

Now that it is being released as LGPL only, there are no more entry fees. Guess I will be looking at QT again

• ansi
• mysql323
• mysql40
• postgresql
• oracle
• mssql
• db2
• maxdb
• no_key_options
• no_table_options
• no_field_options

So, for my purposes I can execute the following command:

mysqldump my_database --compatible=mysql323 > sql.dmp

The Spreadsheet

One of my previous employers had a consulting firm put together a really crappy web based time tracking application. The system pulled project/defect ids from a different system and automatically made them available to the user in the “interface” so that we could enter in how many hours we worked on a specific task. Management wanted us to update this information daily, but it took so long to manually track and enter that data, that I ended up creating my own method behind the scenes and using it.

I built an excel spreadsheet that had multiple sheets and some “advanced” (well, advanced for me) features like automatic data collection based on project and fancy pie charts. It allowed me to monitor how much time I spent on non-project related tasks, and it had the added benefit of allowing me to easily import that data into the company time tracking application. Using the spreadsheet instead of the crappy web interface actually saved me about 1-2 hours a week.

Something that always bothered me about this method was the fact that I had to have a spreadsheet open all day, and since I never used excel except for that, it seemed like wasted cpu and memory – not to mention that in order to update my status I had to leave my dev environment which took away even more time from actual coding.

The Gnome Applet

My current employer sometimes needs me to report my time, so I have gone on a time tracking tool quest again. At first I tried to use Hamster, which is a Gnome applet for tracking your time on generic “activities”. I initially set this up with each activity being a trac id + description, and then just kept switching the active activity as my day went on. At the end of the day, I could look up the activity total and manually update trac as needed. This seemed to work very well, and I used it for a number of weeks.

While this method was very flexible, it still lacked some features and seemed to add unneeded maintenance to my already busy day. I stopped using it after a while and just started estimating my time instead of actually tracking it, since I was not gaining much automation.

[ad#tg-tshirts]

The Eclipse Plugin

Since most of my development happens inside of Eclipse (PDT or Zend Studio etc), I wanted to find an integrated solution that was flexible and easy to use. I ended up finding Mylyn and I was very intrigued. This plugin has built-in support for some common project/defect management systems like bugzilla and trac and allows the user to link up your eclipse task list with the external tool. With this setup I can add the task list view to my layout, and just click on the task that I am working on to start/stop the timer. If I need to switch to a different task, I can just select the new task from my view and move on – no need to leave the development environment or spend time switching to a different application to update my status. Any task status updates made in trac, automatically propagate down to my task list, and I can even push some updates back to trac. Mylyn even remembers what files were open while working on a task, and will “reset” your workspace to go back to the state it was in the last time you left it.

Mylyn seems very cool, but it lacks basic reporting features. While it is nice to see the total amount of time spent on a task, I need to know how much time I spend on that task today, or this week – not just the aggregate. Another annoyance is that even though *most* of my day is spent in eclipse, I still need to track time spent in meetings/research/planning against a task, and with Mylyn, if it didn’t happen in eclipse, it did not happen at all.

The Eclipse Plugin – Redux

After a little more searching, I found Tasktop which is a product built on top of Mylyn, but adds in nifty features like a dashboard for quick reporting, report creation and export (very handy), and the ability to adjust times or add new sub tasks as needed. They have a cool screen cast of the reporting features here which really shows how flexible and powerful the tool is. You can also turn off the “inactivity” feature of Tasktop to allow the timer to keep running even when eclipse is not the focused window.

I have only been using Tasktop for 2 days now, but I am really impressed with how easy it is for me to track my time on multiple tasks – it really seems to address the shortcomings of Mylyn. I look forward to the end of the week when I can generate a report of my time and submit the portion of my hours that are customer billable – all with little effort on my part. I’m sure that after using the tool a few more days I might find some hangups, but for now, I am in time tracking heaven.

How About You?

What do you use to track your time? I would love to learn about other tools that might help me be more productive.

Throughout my career I’ve had the opportunity to work for a few shops that claimed to be ‘Agile’ but were really just using the term poorly and inflicting increasing amounts of stress on the developers by trying to manage a non-agile process/project in an agile manner.

Now it is possible that you do not know what Agile Development is, so let me pause and summarize a wikipedia article on the subject.

Agile software development refers to a group of software development methodologies that promotes development iterations, open collaboration, and process adaptability throughout the life-cycle of the project. It chooses to do things in small increments, with minimal planning, rather than plan at length. This helps to minimize the overall risk, and allows the project to adapt to changes more quickly. There is also an emphasis on stakeholder involvement. Meaning at the end of each iteration, the stakeholder is consulted about the product and comments are noted.

From this article I get the following main points:

• Emphasis is placed on producing working software as a measure of progress instead of project planning and documentation
• Development time for each iteration or release is typically between 2-4 weeks
• Strong reliance on unit tests. In fact the unit test is usually written before the actual development. Once the unit test passes, the development is considered complete
• Constant communication with the customer is vital to ensure that the small changes being made are in line with the customers requirements

The article even goes on to give some recommendations on what types of cultures work with agile methods:

Agile:

• Low criticality
• Senior developers
• Requirements change very often
• Small number of developers
• Culture that thrives on chaos

Plan-driven:

• High criticality
• Junior developers
• Requirements don’t change too often
• Large number of developers
• Culture that demands order

So in order for Agile Development to work, you need to have a small group of really good developers that WANT to work within these bounds. The problems that I continually see is that businesses see the benefits that come from using agile methodologies, so they make a decision to go that route without understanding the basic requirements that must be met to be successful. You must have a small team, or break the team up into smaller sub-teams. You must have talented programmers – trying to do this with junior developers will result in chaos. You must adhere to the process – use test driven development, and continually strive for technical excellence. You must empower your developers to do their jobs without running into roadblocks and trust them to do the right thing.

My current employer claims that we are an agile environment but we adhere to only one rule – short turn around on projects. In all reality we are more of a waterfall environment in an agile time frame. We do not have a unit testing framework, and many times our code is released without being tested by QA. I think a coworker summed it perfectly when he said “We do agile development without any of the safety nets”.

So this brings me to my question. Does anyone do Agile Development correctly? Is it possible to take an existing team and code base and make it into a successful agile environment?