A couple days ago I met with a potential client who posed an interesting challenge. He wanted to upload very large files (upwards of 6GB-7GB) to his server using PHP. I told him it was possible, but I don’t recommend it. There’s a few reasons why.

One of the biggest problems I have with most web applications is file uploads. As a developer, I rarely need to upload anything larger than, say, 100MB - this covers a lot of code files, libraries and so forth. Generally, in an OOP world, if a file is larger than a couple megabytes you’re probably doing something wrong anyways.

But, I’m a developer. I work with people on a daily basis that aren’t developers. Hi-res photos, PSDs, even digital media files such as AVIs & MOVs, are something that should be considered when building an application. A complicated PSD, for instance, can easily grow upwards of two gigabytes.

And therein lies the problem.

Let’s face it, PHP, for all its glory, is quite simply, not the best language for uploading files. This is a three pronged problem. The first is PHP, which I’ll elaborate on. The second is HTTP. HTTP is just not suitable for receiving large files; don’t get me wrong, it’s possible, but I question the wisdom. The third is actually client based - it takes time to upload a file, and most people I know (especially non-web savvy folk) get irritated when it doesn’t work immediately.

Let’s take, for instance, what most people do to get around the upload problem by overriding the PHP default limit. Formerly, you could set these variables using ini_set() and ini_restore(), but some of the overrides are no longer available through these methods. You can use a php.ini file, set the options directly but chances are you do something like this, via an .htaccess file.

Make no mistake, this is bad, and a potential security hole. Why? Because someone could easily setup a denial of service attack and bring your servers to its knees.

Addressing the Problem

Okay, so now that I’ve said that uploading files as big as we’re talking is a bad idea, what’s the solution?

First, address the HTTP problem. We’ve got several options, but something I found a few years ago is very promising. Take the challenge outside PHP.

Meet Tramline (http://infrae.com/products/tramline), an upload and download accelerator that plugs into Apache. Using mod_python, it provides a direct means to bypass PHP altogether and hook into Apache. A bit more security tweaking and it is an acceptable solution. There’s more alternatives too. Curl, WebDAV, Java, Streams, Python and Perl to name a few.

At this point, it is fair to say that one should choose the right tools for the right job. This is also true with the client upload problem. It is completely unacceptable to use a file control, have the user hit submit and expect them to wait there for five minutes or more. That said, there are alternatives. SWFUpload, YUI Uploader and various other Javascript methods as well, and with a bit of tweaking I’m sure they could be modified to utilize a service.

6GB-7GB? Easy. Just don’t stick to your PHP guns.

I’m interested to hear how other folks have handled large file uploads.

Short answer: Yes.

Recently, I had the displeasure of having to design wireframes for my application. Now, it was a very hard lesson for me to learn several years ago that I just wasn’t a designer, and the notion of even trying to dive into photoshop again made me somewhat sick to my stomach. So rather than getting into layers and gradients, I instead busted out my trusty code editor and mocked up the wireframes as how I envisioned the application being used, to the point where they were no longer wireframes at all, but a mashup of logic, aesthetics and mockups.

I kicked, screamed, dragged my feet, but eventually I did finished them. Dashboard, login, views, listings, I did it all. Actually, it was a much better exercise than I anticipated and something I’ll have to encourage other developers to do.

It’s a good exercise because it helped me think out some features I wanted to do. All of a sudden I’m staring at a blank page going “Well, that won’t work” and it’s incredibly important that I make it do so. If I hadn’t made a mockup, I would not have addressed that fact till I actually started development, or worse - built an architecture around a theory.

Some of these challenges with logic and engineering can easily be thought out. Others will creep up on you. So what if you’re not a designer. Don’t focus on the actual interface itself, focus on the presentation of the data. All the data flow diagrams and specification documents aren’t going to help if you can’t figure out how to present two objects with different metadata in the same table - which was, incidentally, my problem.

After you’ve designed a wireframe or - if you’re a little more bold - a mockup, hand it off to a designer; have them take it to the next level, get all mad scientist on it and give it some better colors than #FF0000 and #EEE. Now they have an even better idea of what kind of interface your user will need.

So in closing, don’t treat spec docs, data flow diagrams or mockups as the be all end all to your application. All of these should be used, complimenting one another; it’ll save you considerable headache in some scenarios where you scratch your head and go:

Uh, I didn’t think of that.

Inspired by: How do I write my UI specs?

I am going to write a real entry soon, I promise.

http://www.456bereastreet.com/archive/200708/microformats_book_review/
This is a great read; I’ve used Microformats in my applications and I think they are invaluable. The comments are especially useful in getting other people’s opinions.

http://ajaxian.com/archives/yui-compressor-the-latest-minification-tool
Compress those JS files! Have to try this out.

http://ajaxian.com/archives/builtwith-what-tech
Interesting. Seems a little devious to investigate what technologies are being used, but still interesting all the same.

http://www.techcrunch.com/2007/08/14/34-more-ways-to-build-your-own-social-network/
I would never use one of these tools, but it’s a good resource to have.

Who Owns the Concept if No One Signs the Papers?
Really long link, sorry.

http://yuiblog.com/blog/2007/08/13/rte-notes/
YUI looks great for simple rich text editing, haven’t tried it out yet since the YUI release. This gives me a good idea of what to expect.