A Day Late

For all 2 (if not less) of you who check this web site here’s a little update. Ciara’s busy with work and school and I’ve been keeping myself busy with work, the gym and trying to finish our basement. The basement has been a fun little project and was an excellent excuse to buy some great power tools (Christmas came early this year, at least for me).

I’ve been busy with work this past month but things might slow down just a little in October. When I’m not at work, it seems like I’m either at the gym, or working on the basement. Over the past year I’ve been able to improve my fitness level quite a bit. I’ve even become one of those people that I thought I’d never become…one who likes running. I’m now to the point where I actually miss running on the days I don’t have time to do it. Now, if I would start eating healthier I might be in really great shape :).

As for our future plans, we’re still waiting to hear on a few things. Well…one major thing in particular. Hopefully we’ll hear something in the near future (next month?) but it seems like I’ve been saying that for 6 months now. In the meantime we have plenty of things to keep us busy. When we hear something, I’ll be sure to let everyone know.

I updated our photo gallery software and also uploaded a few new photos. I also locked down the gallery a little and to see most of the photos, you’ll need a login. For those who’d like to see all the photos, go ahead and register a username. It’s not very hard and I’ll give you access to the rest of the photo gallery.

That’s it for now. I’ll try and be a little more diligent about updating the site and uploading photos. I created a login for Ciara to update the site, so maybe we’ll see a post from her in the near future.

I’ve been working on a problem at work for almost 7 hours and I’m no closer to finding a solution. I’d give up if I wasn’t sure someone else has had the same problem. Here’s what’s going on. On our server we use coldfusion and use application security throughout our site. When someone logs in to our site we set various cookies and session variables. On pages that require security we check for the proper credentials and then load the page. If the proper credentials are not found the user is redirected to the login page. However, that approach only provides security for files coldfusion recognizes (CFM and CFC files). PDF, word document files are available for anyone to see.

This hasn’t really been a problem until recently. We allow all employees a certain amount of web space to post files and other documents in their own web folders. On numerous occasions we’ve told them NOT to upload files with information that should remain confidential. Apparently that’s not working because they keep doing it. The problem is, most of the files they upload are word documents, pdf’s, etc which cannot be put under our coldfusion application security.

I figured a good solution would be to implement directory security on those folders and require basic authentication. This would authorization to access any file. I want to create a single username on the server with access to that folder. When a user logs in to our site, throught the coldfusion mechanism, we also do a transparent login using this special username and password using Basic Authentication. Problem is I can find NO way to do this. I thought I could manipulate the authentication-info header but that doesn’t seem to be working. I’m having a hard time believing it’s not possible to have application security AND directory security. If only I could use mod_rewrite I wouldn’t have to worry about directory security, but sadly this is a Microsoft IIS server. If I haven’t figured this out in the next half hour I might have to beat my head against my desk.

The other day at work I was flipping through a copy of the latest MX Developers Journal. I started reading an article from the Chief Editor, Charles Brown entitled “Where Are We Going?” The first sentence reads: “As a Web developer, trainer, and conference speaker, the question I am asked most frequently is, “Where is this industry going?” My answer is one simple word: Flash!!!” If that is true, someone please kill me now!

The longer I work with web related technologies the more impressed I am with sites that deliver content and functionality in a clean and simple form. I realize that’s possible with flash but its much more cumbersome than using html and css. I suppose there are plenty of people who would argue otherwise, feel free. As someone who has worked with flash and html quite a bit, I’ll choose html any day.

It’s been a while since I posted an entry. I expected work to pick up a little when fall semester started but I’ve been busier than I anticipated. There’s a lot I’d like to accomplish before the end of the year and it seems the list keeps getting longer (or at least not any shorter) and the time to accomplish it all is disappearing.

On top of all that I’m seriously considering something that I never thought I would…going back to school. I’ve started working on my graduate school application and it’s quite possible that this time next year, I’ll be a student again. Feelings are mixed. I’m still planning to continue working full-time while going to school (I know…obviously I learned nothing from my undergrad experience).

The next few years should be interesting. Life sure hasn’t developed like I thought it would. I’m not saying there’s anything I’d choose to do differently, but I never would have guessed this is where i’d be. It should be exciting to see how the next 5 years turn out.

I’ve spent a little time recently doing some simple benchmarking on our MySQL servers. MySQL comes with a nice little benchmarking suite in the sql-bench directory. When I started I was expecting to get a general idea of how our servers were performing under their current configuration. I didn’t expect to discover configuration errors, but discover them I did. I learned our “open files” limit was set too high. I checked the operating system kernel variable and it indicated the operating system would handle 4096 open files / process. However, as soon mysql would exceed 1024 open files it would fail. I set the mysql file limit to 1024 and the benchmark tests completed without any trouble.

So what did I learn from the benchmarking? It’s time to buy some new servers. When I started this job I assumed our servers wer top of the line, in terms of performance. What the benchmarking indicated was the mysql server I have running on this little dual processor pentium 3 machine, in my basement, was outperforming our UNIX servers by quite a bit. I checked into our server specs and discovered they’re only dual 400mhz processor machines (NOTE the benchmarking suite does not utilize dual processors, since the process is serialized). That explains a lot. We’ve started to run into some serious performance issues recently and it appears our processor load is getting to be too much to handle. Fortunately I was able to spread the load across both servers (originally one of them was entirely a test server), which has alleviated our load problems temporarily. That should keep us going until I can replace the servers.

Anyway, if you have some time, play around with the benchmarking suite. It was interesting to see how things changed by modifying a few variables here and there. It was particularly interesting to see the effect on performance by simply changing the tables from myisam to innodb. The sql-bench suite will give you a good idea of general performance but if you want to get serious about optimization I hear the MySQL super-smack program is the way to go. It’s completely flexible and allows you to create custom bechmark tests that will simulate requests and loads your production server will likely be handling.

I’m back from Vegas. The trip was much too short. I had a good time. Mike and I stopped by the Bellagio one night and I saw the gardens and fountain show. I have to admit the fountains were more impressive than I expected. It really was an amazing thing to see. If you’re in Vegas, you should really check it out (at night of course).

School starts on Monday which means it’ll be back to work as usual. The traffic around Provo is already getting crazy. I forget how much nicer this place is over the summer when it’s not so crowded. My development team was able to accomplish most of the things we wanted to over the summer. There’s a still a few things I would’ve liked to complete before school started but isn’t that how it should be? All the developers I work with will be cutting back to 20 hours per week which means it will be less to manage but projects will now take twice as long to complete (basically).

I’ve always liked this time of year (I like it MUCH better now that I’m not in school though). It still feels like a good time to take a new look at things and start over. I still have the urge to go buy new “school clothes” (probably left over from my elementary school days). Anyway, hopefully this year will be a good one.

We’ve been having some trouble with our MySQL server running on the HPUX machine. It’s been crashing multiple times for the last few days. When it crashes it becomes completely unresponsive. No clients can connect and any clients that were connected don’t respond. I tried keeping a few windows open to monitor the processlist and extended-status but they didn’t help a whole lot when the server crashed (all values seemed normal when the crashes occurred). I was even unable to shutdown the server. I tried running the script to stop the server but the server wouldn’t respond to any kill commands. Basically the only thing I could do was kill the process with a “kill -9″ (NEVER A GOOD THING).

I’ve been watching the logs and I haven’t found anything out of the ordinary. Nothing significant in the error log. Nothing in the binary log and nothing in the query log file. I assumed it was one particular query that was killing the server but the last queries in the log files never matched. I checked the system log files and couldn’t find anything suspicious except for one thing. The server was restarted a few days ago for unknown reasons. I put in a ticket to engineering inquiring why the server had been restarted (assuming they’d made some changes). I got a response back indicating the server had crashed. I suspected the database might have some corruption problems due to the kill -9’s but it wasn’t until I learned the server had crashed that I thought corruptions could be the cause of the problem.

I decided to take the server down and run a myisamchk. It found quite a few errors. I ran it again with the –recover option. Just to be safe I then ran a CHECK TABLE [table name] EXTENDED on all the tables (MyISAM and Innodb) to verify they all had an “OK” status. They did. Hopefully this solved the problem. We’ll have to see how things go tomorrow.

I learned the benefit of using the “straight_join” keyword while working on a complex query at work today. For some reason the query would only complete if I was running it on the mysql client on the mysql server. All remote connections would simply die. Actually they didn’t die, they would hang. I watched the processlist as the query was running. The status indicated “copying to tmp table”. Eventually the query would disappear from the processlist but no results or information would return and the client appeared to still be waiting for a response. I tried increasing the tmp_table_size variable but that didn’t help. I suspect it has something to do with the tmp directory (possibly not enough space available or something like that).

Anyway, I was able to work around the problem by rewriting the query using the straight_join keyword. Apparently MySQL isn’t necessarilly good at choosing the join order in complex queries. By specifying a complex query as a straight_join the query executes the joins in the order they’re specified. By placing the table I assumed to be the least common denominator first and specifying straight_join I was able to improve the query performance by a few minutes. The new query also completed successfully on the remote clients. Now…if I could just figure out why the first query hung.