Road to validation.

Overseeing the current Wayne State University homepage is like running with a tiny pebble in your shoe. You feel it every step but your going so fast you cannot stop to take it out.

Working with a site so large it can become discouraging but taking one step at a time one can accomplish anything. Yesterday we took our first step, moving the site to a new server in an environment that we are comfortable and flexible in.

The old version of the site was sitting on an NT box, a requirement of the company who built it over 4 years ago. When I started I had a few goals, one was to get the site off that server and get it validated and rewritten in POSH. Well that day has come and we managed to sprinkle in a few other goodies along the way, here are some highlights:

It took some work to get here but we made it. There is still some tweaking to go but all in all its setup. Now its time to concentrate on the content, we are going to be doing a page by page overhaul and adding greater functionality and layout to each child page.

In addition to the content we will also be shrinking the file size and HTTP requests further and further down to the base minimum. Right now we are ~90k depending on the panel that loads and 16 HTTP requests with an empty cache, we hope to cut the size down by a third and get rid of 2-4 requests.

Stay tuned for more progress.


Save the Environment – Ditch the home server

Jungle DiskMost technical people I know have at least one server in their house to share files and or keep backups of their important stuff. Mine is on 24/7, it takes quite a bit of time to manage and I always worry if my data is really secure. It happens to sit in my basement in the same house my main computer, if there were a disaster all the data would be gone with everything else.

AmazonS3 is by far the best thing to happen to small time data management, well even large scale data management. And a software application JungleDisk that seamlessly connects your computer, Windows, OSX, Linux directly to the storage service.

A server running 24/7 uses a lot of energy. Lets say on average a mid range computer without monitor uses 152 watts of power. That is almost twice the amount of energy a TV needs. So doing a rough estimate of the math,
((Watts * Hours * Days) / 1000 kwh) * 1.4 kWh Rate = Cost per Year.
((152 * 24 * 365)/1000) * 0.14 = $186.41 per Year.
Now that is a lot of money not including the hardware and the time it takes to maintain the software and configuration.

I have JungleDisk setup two ways, a bucket for “files” which is my documents that I need to get to often and I use it like a Documents folder. And another “backup” bucket which I use for automatic backup each night of my whole system.

I have ~20 gigs stored on the drives and it only costs me ~2.50 a month for the storage, so $30.00 per Year. 1/3rd the cost, no maintenance time, and no initial cost for the hardware or setup. Not to mention your data is now in a secure 24/7 location and your helping to save the environment.

Since Amazon has so much storage it can efficiently handle the distribution across its systems. It can use a greater percentage of each of the drives producing a smaller cost (wattage wise) per gigabyte stored. This cost savings would not be able to be achieved if everyone had a house with a personal file server using 152 watts of power and only using 30% or so of the actual space on the drive.

I encourage everyone to try it out. Amazon has no setup fee or no minimum amount to spend per month, in fact my first few months only cost me $0.02 each. In the spirit of saving the environment I am announcing that I am turning off my home file server as of today. No more 24/7 usage and a definite relief on me and the environment.

For other ways to save the environment, check out today’s major initiative named Blog Action Day.


Battling massive amounts of email

Sending email from a web site using PHP’s mail() function is a pretty routine task. Contact forms, friend requests, alerts, errors… Servers hosting one or two sites the time to send an email is not even noticeable, but on a server that hosts 200+ sites the time creeps up the two to three second range.

In a shared server environment like we have at Wayne State University, we have found there is no easy way to keep track of the emails being sent through php’s mail() function. It would take an analysis of the mail server logs or a friendly sysadmin of another server to alert you if there is a bot taking advantage of a mail() script on your site.

To combat this we are developing an extension for PHPSimpl’s Mail class to instead of send out an email directly to inserts the message into a database and every five minutes a cron runs to send out all emails in the table and records them in a sent table.

This accomplishes:

  • Making pages load faster, a DB insert is faster than a mail() function.
  • Frees up server resources to host pages instead of doing sendmail’s.
  • Determines really how many emails are being sent from the web server.
  • Keeping track of what sites are sending mail from what pages and by who.
  • The ability to flag emails before they are sent with keywords like “Viagra” and “Enlargement”.

We are currently just testing the system and hope to include it right into PHPSimpl as a main class but it still needs some stress testing. So far the tests have all came back positive, we have discovered a few of our own forms that have been taken advantage of and we resolved some potential issues.

We also over estimated the amount of emails being sent out, we thought with 200 or so sites it would be ~500-700 emails a day, well it turns out once we fixed a few of the forms we are only sending out ~100 emails a day, not bad.

The graph at the top of the post is the last 5 days and how many emails have been sent out. Today the 6th there has only been 4 emails so far at the time i took the screenshot, 8am.

On the 4th we saw a spike in bot related emails, after some investigation and a few recaptcha additions we just about halved the number of emails the next day.

Next week we hope to have the system fully functional and have a better analysis and report of our findings.


New Host! New Home!

Friday NightMy new setup is quite comfortable. Hosting by Joyent running on a Shared Accelerator using Mongrel proxyied through Apache to run a Rails application known as SimpleLog with a green inspired simple style theme created by Myself using a new CSS framework called Blueprint.

From the time my DNS change went through it took only about an hour to get the whole thing up and running. Although while typing this I am still waiting for a Mongrel port, running Mongrel by hand works just fine.

The setup was good timing because Andrea is working late and I am still trying to wrap my head around all the upcoming decisions needed to be made about I am now the Interim Web Manager in the Marketing Department. With the new position comes some new goals and I am currently putting them together and hope to bring my experience with high load shared server environments in some upcoming articles.

Eventually all my old archived articles from my blogger account will get moved over, I just need the motivation to do it.

Accelerated By Joyent

Working with Blueprint: A CSS Framework

SimpleLog base installSo I decided to change my blogging software, from no software (Blogger) to SimpleLog. I setup everything up in my dev environment and it was by far the easiest setup I have seen so far. Took about 10 minutes including setting up the virtual hosts.

While getting super interested in my blog again I was looking for some nice examples of css blogs to get inspired I stumbled upon Blueprint, which is a css framework so I decided to give it a whirl.

First impressions it is super lightweight and non-intrusive, just two lines that have to be added. to the head which includes the style sheets. My only gripe so far was that they just include one grid option which is 14 columns, its not a big deal all that I think it needs is a few other background images to show the options of columns from the grid. Like 7 columns, 3×8×3 and so forth. in progressSo I hit up Photoshop and came up with a basic color scheme and outline for the new blog. Took about a day of going back and forth on how much I wanted to add to the site but being inspired by rails iterative approach to programming I decided to stick with the simple and expand from there.

Back to Blueprint, with my design it ended up being very easy to implement in blueprint. It doesn’t have anything nested tho which I was really interested in since it looked to handle nesting very gracefully. I tried as hard as I could to make this design fit into a 14 column grid but I made the design decision to keep the width at 780 instead of Blueprints default of 960. But not to fear we are just working with css here all I did was just override two of their default widths in my screen.css to keep the integrity of their files in case I decided to upgrade them in the future it will be fully degradable.

Their reset.css worked great, I am use to using Eric Meyers reset.css and then creating everything from there. But Blueprint had an unexpected typography.css which setup quite a bit of the default typography. Which was a huge time saver, because Eric’s just leaves you out in the cold to setup everything on your own which is totally cool if your project warrants it but sometimes there are elements that get introduced and have no control over.

SimpleLog + Blueprint BaseSo setting up a new template in SimpleLog was easy and I started editing the site.rhtml layout to use the naming conventions per Blueprint. Just a few class changes and I removed some default SimpleLog items that I didn’t include in my final design and boom! We have an outline!

Fast forward through time, 26 css declarations and 3 images later and my homepage was almost done. Time in total was about 2 hours to copy and setup an entire new template with SimpleLog and Blueprint. I was amazed, usually I hate when things try to be “smart” and assume they know what options and action you are going to preform but Blueprint didn’t get in my way at all. Since everything for the base elements was already setup I just used my ID’s and Classes to extend the base and life was good.

Although I was just concentrating on the homepage today flipping through all the other pages they didn’t look that bad by default. SimpleLog uses a lot of the same css names and conventions throughout their pages meshed with Blueprints base styles and my overrides the only things left to do were the specifics of the page.

Initial design for

Last but not least, my current host is not very Rails friendly so I am looking for a new host for my blog. While doing that I will continue to update here and work on the new version on my dev server. So far:

SimpleLog: A+

  • Ton of features and rock solid. Easy install and great documentation. I have not looked into any extensions or plugins yet but there are a few areas where I can see a few more helper functions. But for getting a blog up quick and painless SimpleLog is the way to go.
  • SimpleLog Website

Blueprint: A-

  • For just being at 0.3 release it is a huge time saver, its not just the resets for cross browser compatibility but it also includes a good typography and print styles which I have always been looking to create. I hate going through and trying to think of all the HTML elements a client could potentially use and make sure they are set accordingly. Blueprint defiantly has a future in my web development life and releasing it to the community will only make it better.
  • Blueprint CSS Framework Website