isnot.tv

I've been keeping an eye on this question on Stack Overflow.

Interesting to see the distribution of free apps (Eclipse, Emacs, Vim, etc) alongside commercial apps (Komodo, Textmate, etc).

Loving the sunflower packing, such an awesome way of presenting quantity without the need for supporting legends, etc

A bloody awesome collection of photos staring album covers and people with too much time on their hands.

Click for many, many more.

Here’s a great article/talk on a topic I’m very passionate about. The value in the tools used to generate products & results.

In the talk Jason Fried quotes:

Whenever you make something, you create valuable by-products that you can sell.

When you cosider that’s how Basecamp came to be, and therefore Ruby on Rails, you couldn’t help but see the truth in the statement.

It’s really a great read, especially if you seem to be writing more tools & scripts to build your apps vs actually building them :D

You can find the article here.

Over the past week I’ve really begun to notice the personality that lives within the applications we use.

It really kicked something off in my mind when I took part in the 37Signals Basecamp survey.

If you didn’t participate you would have missed a very interesting question which asked what body type Basecamp would have if it were a real person.

Being a multiple choice question, the possible options were Athletic/Toned, Average, Slightly plump and Obese.

While physical attributes allow developers and managers to understand several characteristics like bloat, speed and, well, sex appeal, it was interesting that they didn’t ask, or delve into the area of emotional or more personal characteristics like the applications personality.

Having worked in creative agencies for many years where copy has such a strong influence on the success and impact of a campaign, I can see the importance of identifying the personality of an application. In some ways it should drive the form and function of the application.

In a little experiment I went through a few apps that I use on a daily basis to try and categorise the applications personality and physical attributes. Here are my thoughts.

Adobe Creative Suite

I recently tweeted about this, and Adobe is what spurred on the blog post. Doesn’t it feel like Adobe Creative Suite is that greedy, attention seeking child who you love, but just want to slap for being so smug? It’s always making mistakes, constantly begging for resources and updates etc. But it has a great life ahead, so much potential, just needs to go on a bit of a diet and be more considerate to others.

Coda

Is that new developer who joined your team. He’s awesome, snappy dresser, gets work done really fast and is proficient in many things.
Although, he’s really stuck in his ways. He learned how to code in one way and expects everyone else to follow suit.

LaunchBar

That 30 somethings fellow who’s spent a majority of his life in the military.
He’s smart, factual and to the point.
No messing about, simply gets the job done with as little personality or communication as possible.

Photo Booth

That friend that everyone has who’s a bit of a dead beat during the day, but is the life of the party after a beer or two.

Outlook

The suit who was really efficient when they were younger but has almost lost their passion and is simply exhausted. They know how to get things done, but is constantly competing with the new, younger suits who are driven and know how to get things done without fuss.

I went through my entire applications directory and began to see two very obvious patterns.

Firstly; and yes, I may be a little sexist in saying so; all the apps I use seem to be guys?

I sat with @coreytsd and tried to think of applications we use that are female, and we thought of a few that we considered to be feminine, but weren’t as definitive as the apps mentioned above.

Additionally, I was surprised to find that very few Open Source applications had a personality like the ones discussed above. Most of them have character, but since they’re not managed in the same fashion as closed source applications they lack the creative direction which defines personality in an application.

I’d be interested to hear others thoughts on this matter, obviously the way people perceive personality and character differ with each individual, which would make this a very interesting experiment to pursue.

I recently came across the need to aggregate server logs and data from many hosts (currently about 30+ web servers) and stumbled upon the reporting application Splunk.

It's a little different to your normal log apps since it isn't really a 'reporting' app per se, but rather a search app that all allows you to generate reports and charts based on search queries. So far I've found it to be pretty sexy, it's easy to setup and get running, although the documentation isn't that straight forward and whilst they've got a great library of screencasts and introduction videos; I can't seem to watch them since the video player they use is buggy as hell. So learning the app and it's quirks isn't something you'd want to do in a short timeline.

I'll no doubt be posting a few articles over the coming months as I become more savvy with the app, but for now, I'll just summarise my install and setup process and the basics of the interface. So here goes.

An interesting thing about Splunk is it's free for those who need to analyse less than 500mb of data a day, after that you need to pay for an enterprise license which is based on the data processing requirements, I've exceeded the 500mb limit and will soon be buying a license, more on that in a later post.

Getting it installed

You can set Splunk up in a few different ways, one method, is to install Splunk on the server that you want to report on. You simply install Splunk and show it the location of your logs and it's all done.

That setup doesn't work for me though, I need to report on 30+ servers, so I use their 'Forwarder' to distribute the logs across the cluster. That's the setup we'll be running through today.

So, setting up a forwarder means we install Splunk on two or more machines. One machine will be the web app; that's the web interface we log into, and the server that'll house the Splunk DB, etc; the other machines will have the Splunk forwarders, which regularly post (through some form of voodoo) the local logs to the Splunk server for reporting and aggregation.

Teeny bit of info before we dive in, the below scripts have been tested on 32-bit CentOS, on MediaTemple and AWS servers.

The setup is as follows (all dones via the terminal).
We'll first start by getting the Splunk web server up and running, so make sure this server has plenty of HDD space and memory, since analysing the logs is an intensive task.

# The Splunk version may have changed since
# authoring of this post, so you may want to
# check for the current download at:
# http://www.Splunk.com/download?r=SP-CAAADFV
wget 'http://www.Splunk.com/index.php/download_track?file=3.4.9/linux/Splunk-3.4.9-57762.i386.rpm&ac=&wget=true&name=wget&typed=releases'
rpm -i Splunk-3.4.9-57762.i386.rpm

Once installed we need to make some modifications to our bash profile adding a few paths etc

echo "export SPLUNK_HOME=/opt/Splunk" >> ~/.bash_profile
echo "export PATH=/opt/Splunk/bin:$PATH" >> ~/.bash_profile
source ~/.bash_profile

That's pretty much it for the install, now onto the setup. Splunk comes with a pretty handy command line tool to administer the app, so first up, we'll start the baby up, set some ports and add some logs

# get Splunk to start on startup
# note: you'll see the license at this point, and you'll need to agree to the terms
Splunk enable boot-start

Once you've accepted the terms paste in the following commands

# start Splunk
/etc/init.d/Splunk start
 
# change the ports
# you may want to change these to something more
# suitable to your needs & setup
splunk enable listen 9002
splunk set web-port 9000
splunk set Splunkd-port 9001
 
# after changing the ports we need to restart Splunk
service Splunk restart

Ok, so that's Splunk setup, now we can log into the web interface. Open up a web browser and hit http://XXX.XXX.XXX.XXX:9000 (obviously using your own IP address and web-port if you changed it from 9000).

Nothing fancy just yet since we don't have any data, so lets add a monitor.
Monitors keep an eye on folders, files or ports for data to log, the simplest way of getting started is to add the /var/log/ directory.

# add the log directory
sudo splunk add monitor /var/log/

Now if we look at the Splunk interface we should start to see some data come through, depending on what your server does, it may be enough data to play with, or not. Either way, we'll move on to setting up the forwarders.

Again, we'll need to install Splunk but the setup is slighly different than before.

# The Splunk version may have changed since
# authoring of this post, so you may want to
# check for the current download at:
# http://www.Splunk.com/download?r=SP-CAAADFV
wget 'http://www.Splunk.com/index.php/download_track?file=3.4.9/linux/Splunk-3.4.9-57762.i386.rpm&ac=&wget=true&name=wget&typed=releases'
rpm -i Splunk-3.4.9-57762.i386.rpm
 
# setup the paths
echo "export SPLUNK_HOME=/opt/Splunk" >> ~/.bash_profile
echo "export PATH=/opt/Splunk/bin:$PATH" >> ~/.bash_profile
source ~/.bash_profile
 
# get splunk to start on startup
splunk enable boot-start

So splunk is installed on the 2nd server, now just some configuration and we're done.

One thing we need here though is the IP of the first Splunk server we setup.

# start splunk
/etc/init.d/splunk start
 
# change the ports
splunk set web-port 9000
splunk set splunkd-port 9001
 
# setup the forward server
# NOTE: MAKE SURE YOU USE YOUR OWN IP ADDRESS HERE
splunk enable app SplunkLightForwarder
splunk add forward-server XXX.XXX.XXX.XXX:9002
 
# restart splunk
service splunk restart
 
# add the log directory
sudo splunk add monitor /var/log/

And that's pretty much it, checkout the web app now and you should see more sources and hosts pop up as the server obtains data from the other servers.

Bam, that's pretty much it. The next post will be about the basics of reporting and searching the content, but it's pretty simple to get the ball rolling.

Let me know if you have any issues with it, or how it works with other platforms. If you're familiar with Splunk i've got a tonne of questions to ask, so get in touch via twitter @d2kagw.

Enjoy!