A Day Late

I spent the last hour or so trying to figure out why my Subversion server wasn’t handling diff requests properly. I first noticed it when I was examining a piece of source code and noticed a piece of code I added a while back was missing. Fortunately I’ve been moving all our source code over to a Subversion repository so I pulled up my subversion client and examined the log for the source code file. I noticed a recent revision committed to the repository and suspected that’s where the code was removed. When I tried to display the differences committed in that revision the following error was returned: “GET of ‘/svn/repository/!svn/bc/284/trunk/temp/application.cfm’: 301 Moved Permanently (http://localhost)“.

A quick google search for subversion and “Moved Permanently” returned a link to the Subversion FAQ’s but the answer didn’t seem to help much. First, the question dealt with a commit action which uses PROPFIND as opposed to a diff action which uses a GET request. I still read through the response and verified that the server was in compliance with the suggested configuration. After trying a few different searches I came across this which basically says if your Location tag in your Apache config file maps to /svn then you can’t have a .svn working directory in your document root. Sure enough renaming the .svn directory in the document root allowed me to successfully perform the diff statement but that wasn’t an exceptable solution. Since I want the code for the entire website in the repository, the document root must have a working directory named .svn and renaming it everytime I want to compare differences in a file didn’t seem like a good work around. I was debating changing the Location from /svn to /subvn or something like that but now that we’ve been using the repository for quite a while a URL change wouldn’t be fun.

I spent the next half hour searching the Subversion documentation and website trying to discover where it says you can’t have a .svn working directory if your repository location starts with /svn. I came up empty handed so I tried working the problem a different way. Clearly Apache was handling the GET request for /svn different than it was handling a PROPFIND request for /svn (since commits were working fine). To figure out why it was returning the .svn directory I created a directory in the document root called ‘.test’ and placed a basic index.html file inside of it. I opened my browser and pointed it to ‘http://localhost/test/’ and the index file popped up. However, I noticed the URL had been slightly modified to http://localhost/.test/. It took about 2 seconds for the answer to pop into my head…mod_speling. Mod Speling is an Apache module I installed a while back to help with a case sensitivity problem we were having. We’re moving our site from a windows machine (case insensitive) to a linux machine (case sensitive). To avoid breaking any links that weren’t using the proper case I installed mod_speling which tries to determine what an invalid request is looking for. For example, if a file was named index.html and a request came in for Index.html the Apache server would return a file not found error. With mod_speling the module looks at the invalid requests, sees there is a file called index.html and redirects the request to that file.

So, for my situation mod_speling was intercepting the GET request for /svn/… and was trying to redirect it to /.svn/… To fix the problem I turned off mod_speling on the Apache virtual server handling the subversion repository and the diff worked fine. Fortunately this virtual server is a test server and didn’t really need mod_speling. I was able to keep mod_speling running on the production server where it was really needed. If I ran into the situation where I absolutely needed mod_speling running the same virtual server as Subversion, I’m not sure what the solution would be. I suspect you’d have to turn mod_speling off on the first level of the document root folder entirely. I don’t see any way where you could exclude only the svn directory. I believe it’s a problem with the mod_speling module. Shouldn’t it check the Location tags to see if it’s a valid location before trying to redirect the request? That seems logical to me. If anyone comes up with a better solution, let me know.

As a little disclaimer, I know I’m going to get some complaints about my first post in 5 months being about something technical. I’m sure the complaints will come from the same friends that have complained about lack of updates over the past few months. I’m sorry, I’ll try and post something more interesting soon, but since there was no reference to this problem on the web, and I’m sure someone else will come across it sooner or later, I thought I’d better post it.

I’ve been working on a problem at work for almost 7 hours and I’m no closer to finding a solution. I’d give up if I wasn’t sure someone else has had the same problem. Here’s what’s going on. On our server we use coldfusion and use application security throughout our site. When someone logs in to our site we set various cookies and session variables. On pages that require security we check for the proper credentials and then load the page. If the proper credentials are not found the user is redirected to the login page. However, that approach only provides security for files coldfusion recognizes (CFM and CFC files). PDF, word document files are available for anyone to see.

This hasn’t really been a problem until recently. We allow all employees a certain amount of web space to post files and other documents in their own web folders. On numerous occasions we’ve told them NOT to upload files with information that should remain confidential. Apparently that’s not working because they keep doing it. The problem is, most of the files they upload are word documents, pdf’s, etc which cannot be put under our coldfusion application security.

I figured a good solution would be to implement directory security on those folders and require basic authentication. This would authorization to access any file. I want to create a single username on the server with access to that folder. When a user logs in to our site, throught the coldfusion mechanism, we also do a transparent login using this special username and password using Basic Authentication. Problem is I can find NO way to do this. I thought I could manipulate the authentication-info header but that doesn’t seem to be working. I’m having a hard time believing it’s not possible to have application security AND directory security. If only I could use mod_rewrite I wouldn’t have to worry about directory security, but sadly this is a Microsoft IIS server. If I haven’t figured this out in the next half hour I might have to beat my head against my desk.

The other day at work I was flipping through a copy of the latest MX Developers Journal. I started reading an article from the Chief Editor, Charles Brown entitled “Where Are We Going?” The first sentence reads: “As a Web developer, trainer, and conference speaker, the question I am asked most frequently is, “Where is this industry going?” My answer is one simple word: Flash!!!” If that is true, someone please kill me now!

The longer I work with web related technologies the more impressed I am with sites that deliver content and functionality in a clean and simple form. I realize that’s possible with flash but its much more cumbersome than using html and css. I suppose there are plenty of people who would argue otherwise, feel free. As someone who has worked with flash and html quite a bit, I’ll choose html any day.

I learned the benefit of using the “straight_join” keyword while working on a complex query at work today. For some reason the query would only complete if I was running it on the mysql client on the mysql server. All remote connections would simply die. Actually they didn’t die, they would hang. I watched the processlist as the query was running. The status indicated “copying to tmp table”. Eventually the query would disappear from the processlist but no results or information would return and the client appeared to still be waiting for a response. I tried increasing the tmp_table_size variable but that didn’t help. I suspect it has something to do with the tmp directory (possibly not enough space available or something like that).

Anyway, I was able to work around the problem by rewriting the query using the straight_join keyword. Apparently MySQL isn’t necessarilly good at choosing the join order in complex queries. By specifying a complex query as a straight_join the query executes the joins in the order they’re specified. By placing the table I assumed to be the least common denominator first and specifying straight_join I was able to improve the query performance by a few minutes. The new query also completed successfully on the remote clients. Now…if I could just figure out why the first query hung.

src="http://www.w3.org/Icons/valid-xhtml10"
alt="Valid XHTML 1.0!" style="margin:5px" height="31" width="88" border="0" align="right" />
src="http://jigsaw.w3.org/css-validator/images/vcss"
alt="Valid CSS!" align="right"/>
I spent a little time running this page through some web standards validators last night. I was curious to know how much work it would take to validate it with XHTML 1.0 and CSS. Surprisingly it didn’t take much. A half hour after I started I was done. I also decided to run it through a 508 standards validator (accessibility), it passed on the first run. For those that are interested, I added the validator links to the bottom of the right menu.

Web services are really cool. I’ve received a few requests for access to our MySQL database from a few different groups recently. For example, one of the student clubs recently contacted me and told me they wanted to pull a list of their members and use it dynamically on their web site. Problem is they host their site on their own server and I’m not willing to punch holes in the firewall or create MySQL accounts on the server. They left frustrated and I felt guilty that I couldn’t help them out. I kept thinking about it and finally the idea hit me, web services. If this isn’t a perfect example for a web service, I don’t know what is. I’d never developed a web service on Coldfusion before but every thing I’d read indicated it was very simple. So I broke out the developer manual today and started playing around with it. About 30 minutes later it was all done (well, the basics. I plan to add more functionality). Initially I created a plain coldfusion component (CFC) and created a function called “getMembers” (with access=remote) that would return a list of club members. Inside the function I dropped in a simple cfquery tag to pull the list of club members and then return the query. Here’s what the code looks like:

<cfcomponent hint=”Web Service functions for the clubs”>

<cffunction access=”remote” name=”getMembers” returntype=”query” hint=”Returns a list of club members”>

<cfargument name=”user” type=”string” required=”yes” hint=”A valid username to access this resource”>

<cfargument name=”pass” type=”string” required=”yes” hint=”A valid password to access this resource”>



<!— SET THE DEFAULT RETURN VARIABLE TO A NULL STRING —>

<cfset variables.getMembers = “”>



<!— CHECK THE USERNAME AND PASSWORD —>

<cfif arguments.user EQ “SomeUser” AND arguments.pass EQ “SomePassword”>



<!— USERNAME AND PASSWORD ARE VALID, RETRIEVE INFORMATION FROM THE DATABASE

AND SET IT TO THE RETURN VARIABLE —>

<cfquery datasource=”ClubDatasource” name=”variables.getMembers”>

SQL STATEMENT TO RETRIEVE MEMBER LIST GOES HERE
</cfquery>



<cfelse>

<!— USERNAME AND/OR PASSWORD WERE INVALID, THROW AN ERROR —>

<cfthrow detail=”INVALID username AND/OR password” message=”INVALID username AND/OR password”>



</cfif>



<!— RETURN THE RESULT —>

<cfreturn variables.getMembers>

</cffunction>

</cfcomponent>

I wrote a quick test.cfm page to call the web service to see if it worked and it did. The Coldfusion code to invoke such a web service is:

<cfinvoke

webservice=”https://WEBSERVERURL/xml/clubs.cfc?wsdl”

method=”getMembers”

user=”SomeUser”

pass=”SomePassword”

returnvariable=”variables.query”

>



<cfoutput query=”variables.query”>

#clubId# #Name#<br/>

</cfoutput>

That only solved half the problem. I knew how to call and use the web service using Coldfusion, but I knew the club was using PHP on their server. The next step was figuring out how to call a web service using PHP. PHP doesn’t have any built in web service functions (yet), but you can download the NUSOAP library which makes it remarkably easy to use web services in PHP. Here’s the quick PHP script I wrote:

<?php

require_once(’nusoap.php’);

$wsdl=new soapclient(’https://WEBSERVERURL/xml/clubs.cfc?WSDL’);

echo
$wsdl->call(’getMembers’,array(’user’=>’SomeUser’,'pass’=>’SomePassword’));

echo ‘Response: <xmp>’.$wsdl->response.’</xmp>’;

?>

I loaded the page and it worked like a charm. There in front of me was a nice, XML formatted member list from our clubs database. Obviously I still have some work to do, like checking the username and password against our authentication database and such, but for the most part the hard part is done. This little project has got me thinking about what else we can use XML and web services for. I think my next step project will be generating an RSS feed for our news release site.

UPDATE* Coldfusion exhibits some strange caching behavior with web services. After I made my web service and tested it, I added a few more arguments to it. I changed my test invoking method and it said it couldn’t find a matching method with the parameters I passed. Remarkably enough php still worked fine (using the new arguments) but the test.cfm file did not. I did some searching and came across this message thread. It appears the only way to get rid of the cached version is to login to the Coldfusion Administrator, click on web services, delete the appropriate entry and reload the cfm page. So basically you have to do this anytime you add a web service function/method or change the arguments of a function/method. It’s a little annoying.

While looking around in web services in the Coldfusion Administrator I also noticed you can assign a username and password to each web service directly instead of imbedding them in the function like I did. If you want to keep authentication simple that’s a good way to go.

Each year I can attend one conference. Last year I attended the Macromedia MAX conference in Salt Lake City (its close proximity had a lot to do with that decision). I need to decide which conference I’m going to attend this year, but I’m having some trouble deciding. I thought about the MySQL Users Conference, but since it ends tomorrow it’s not an option. I’m considering attending the Macromedia MAX Conference again (in New Orleans this year) and it seems to be the logical choice.

My development team spends about 90% of their time developing on ColdFusion using Macromedia products. Some might ask why that’s the case. It has a lot to do with the environment I work in. My development team is made up entirely of students, which usually means they don’t have advanced technical/programming skills when they start (that’s why they’re in school). Also, they’re not long term employees. Generally if they stick around more than 2 years I consider myself lucky. The combination of high turnaround and unskilled applicants means we need to use something that’s easy and quick to learn. Coldfusion fits that need fairly well. More so than Java, PHP and other development languages (that I might prefer more).

Anyway, back to the original subject, what conference should I attend? Like I said, the Macromedia conference is the only one I’m considering at the moment but I’m open to any other recommendations.

I received an email the other day at work notifying me that our home page didn’t look very good on Internet Explorer for Macintosh. I pulled it up and sure enough it was just a jumble of garbage. This confused me because I use a Mac and I was sure I’d looked at it on IE, although I knew it was quite a while ago (I usually only use Mozilla or Safari).

Anyway…I noticed when I commented out a few javascript functions that dynamically manipulated a table the page looked fine (from what I’ve read, javascript support is awful on IE for Mac). That would seem to indicate the problem was with the javascript, however the exact same javascript was being used on other pages and worked just fine. I decided to rebuild our homepage starting from scratch, adding things in piece by piece. I discovered a few tables and other tags that weren’t properly closed. When I was finished the page was working on IE for Mac. However…I decided to reload the page a few times just to be sure and was disgusted by what happened. On average if I loaded the page 10 times, 9 times it looked just fine and 1 time it was a jumble of garbage. So, it seems to work 90% of the time. From a computing perspective I’m confused. IE is receiving the exact same code each time but it seems to randomly interpret it differently. Odd.