So I’ve been wanking up a new upload system for Fakku — basically one that lets Jacob et all to just upload a zip file and attach metadata all through a webform (inb4 uploading large files over HTTP is silly) and it automatically unpacks the archive and uploads the data to the appropriate servers. It’s a pretty straightforward thing, except for fucking progress bars.

When I read through the project specs the progress bar raised a red flag. There simply isn’t any standardized way to query the current status of an upload. As far as I see, there’s only two ways to do it –

1. Use flash to do the upload.
2. Have the webserver provide a JSON interface to query.

The first is fucking stupid — I hate flash.

The second is much more reasonable. The idea is that the browser provides a random key in part of the POST data to identify the upload uniquely. The web server then provides a JSON endpoint to query the status of uploads by the client-specified key. Naturally, this is a pretty straightforward thing and there are implementations available for various servers. Unfortunately, Fakku is running lighttpd-1.4.x which has no such implementation. There’s a mod_uploadprogress for 1.5, but 1.5 is a worthless broken piece of shit.

Anyway, the reason 1.4 doesn’t have an upload progress module is apparently Lighttpd buffers requests before sending them to a backend. This seems like a stupid reason to me. Why the hell would that prevent the web server from providing transfer information to the client? The backend has absolutely nothing to do with the transfer of POST data.

Before writing a full-blown Lighttpd module to implement this, I took a look at how Lighttpd actually handles the buffering of uploaded data. I have to say, it’s pretty hacky. It stores the data in /var/tmp/lighttpd-upload-* (with names generated via something like tempnam). It splits large files into many 1MB buffers.

So I hacked together a PHP script which provides two JSON methods — one to acquire the name of the most recently modified (I wish POSIX let me query file creation time) Lighttpd upload buffer, and another to look up the current size of a buffer based on it’s name. This results in a shitty interface which allows you to basically track how much data you’ve uploaded, except the transfer isn’t linked to the client so it’s got a race condition by design. And you can’t determine the full size of the upload — you only get amount uploaded so far (so you couldn’t implement a progress bar or ETA).

But I’m willing to bet it wouldn’t be impossible to write a Lighttpd module which provided an interface to query this shit. Dunno if I’ll get around to hammering it out though; Lighttpd modules are a pain in the dick to code — there’s close to no documentation aside from other existing modules.

Bleh.

4 comments

I haven’t been posting lately because I left my laptop in my new house. I can’t go into my new house because I’ve developed a deathly allergic reaction to mold (or something — potentially a systemic fungal infection) after two weeks of living there. After signing a 25-month lease and subletting to 4 other people.

Needless to say, I’m drinking from an enormous shit cocktail and on all kinds of crazy meds (actually, just steroids, anti-histamines and anti-fungals).

Meanwhile I’m installing CouchDB to putter around with because it looks interesting. Thinking about writing a Task/Bug tracking system that doesn’t suck absolute balls. Realistically, I should just throw up a sqlite3 database (I don’t see the point of running a full-blown RDBMS like PostgreSQL when there are only three people in the office and probably like 50 tasks/week). CouchDB is probably overkill too, but <3 Erlang.

5 comments

I’m so bored right now. I think Jacob is going to have me do some more work on Fakku (or some other projects) but he hasn’t gotten back to me with any details or anything yet so I’m kind of sitting on my hands. Because of that I don’t really have anything much to do. I guess I should ACTUALLY GET THAT FUCKING NETBSD MACHINE TO RECOGNIZE THE ATHEROS CARD LOL.

Or learn pkgsrc or something jesus I’m so fucking lazy.

Anyway so I was rolling around 4scrape like I usually do and lol there’s so much fucking drama on /wg/. Seriously — just do a search for 4scrape and you’ll find lots of awesome drama. I guess the rest of the boardsinternets aren’t much different though.

BRB NETBSD.

UPDATE: pkgsrc is still a steaming pile of shit on NetBSD. I was hoping they had improved it some since FreeBSD picked it up, but ugh. Not only that, but this machine is absolute shit. There’s almost no point in installing anything outside of syscons (or whatever the fuck the OS uses for the console) or svgalib (for mplayer). I might look into slapping some gross Linux shit on it instead or something because thus far both FreeBSD and NetBSD have given me nothing but shit on the hardware.

inb4vista.

3 comments

ahahahaha. I know, I fail for a Monty Python reference, but I just had to try it.

WolframAlpha just won the game.

2 comments

Okay, so one of you assholes suggested installing NetBSD. Obviously FreeBSD didn’t work and Theo is a faggot so NetBSD is the only real choice. So I went off and grabbed the 5.0 disc image with the goal of doing a network install.

NetBSD hides their PXE binary; it’s sitting in i386/installation/misc/pxeboot_ia32.bin. So I slapped that into my tftp directory.

The first attempt to boot ended miserably — the PXE binary was loaded, but it ended with “boot failed” before loading the kernel. A little investigation showed that this was a problem with my dhcpd.conf — I was using

option root-path 10.0.0.1:/usr/diskless/base

But for some reason NetBSD doesn’t like the host in there. Switched that over to

option root-path /usr/diskless/base

and the kernel started to boot fine.

The next problem was that it couldn’t find init. Apparently NetBSD does some weird stuff — you basically have to hand-extract the distributions. Running

for f in base etc ; do
    tar -xzvpf binary/sets/$f.tgz
done

(or whatever, I normally use that demon spawn csh so I’m paraphrasing) and voila, I had a perfectly set-up diskless boot.

Unfortunately for me, NetBSD does some weird shit with their installer. Unlike FreeBSD which has a userland installer, NetBSD’s sysinst is some kind of demented kernel module or some crazy shit like that. So you actually have to boot a special kernel to invoke it. Was beating my head on the table for fucking ages before I had the mind to ask on an IRC channel.

Anyway, to do a netinstall, you have to boot the INSTALL_FLOPPY kernel. I did this the easy-modo way by cp binary/kernel/netbsd-INSTALL_FLOPPY.gz netbsd but I imagine the right way to dick around with it is to play with boot.cfg. Whatever, I don’t have time for that kind of shit.

At this point, the conditioner in my air compressor blew up and showered my machines with refrigerant. Awesome. Thankfully once the right kernel is booted it’s pretty straightforward (though I like FreeBSD’s sysinstall better than sysinst, but whatever). Gonna have to pick up XMonad and shit to watch porn on later; gonna start getting drunk now that I’ve gotten the hard shit out of the way.

10 comments