Rui CarmoThere's a good chance there won't be much news coverage in the next few days, since I've been sucked through the programming language time machine and find myself using Perl to, well, make reports, since the time-honored GUI tools I have - i.e., Excel - just can't cope with the massive amount of data I'm sifting through.
I've already gone to the point of setting up a Makefile-driven harness for the whole thing (querying, collating, reporting and plotting), plus the Subversion-enabled version of CVSTrac to keep myself sane. Before you ask, using a normal reporting engine won't cut it - not with the data I have, or the time I don't have available for learning it. It's much faster to hack my own.
And boy, does it feel good. Even the Perl bits, which (despite the usual hassle of having to re-learn several different conventions for scoping variables all over again) have already taken me into the Numerical Analysis field and other suchlike niceties.
When you spend most of your time immersed in Outlook, Excel and Word doing planning and specs work, coding is like a breath of fresh air - not the kind of thing I want to go back to for a livelihood, but by now the difference from my usual line of work is enough for me.
The output is the really fun bit, though, and since the kind of queries I'm running (across several million records) take quite some time, I spit out partial data snapshots left and right to run through Gnuplot for a quick look.
Not having AquaTerm handy is a bit of a pain, though, since I'm working on Windows - the company standard, which chokes on the 300MB datasets I generate - and Xubuntu LTS, which has so far survived enough invocations of apt for me to keep using it.
I think this is the first Ubuntu install that has given me no significant problems, but I must confess I'm somewhat afraid to jinx it by writing this.
So I'm running ploticus under Cygwin to re-acquaint myself with its idiosyncrasies and render the summary graphs (since Excel can't handle them at all), and Gnuplot via X11. Not the ideal solution, but my world doesn't revolve around a Mac (oh, the shock! the horror!).
Anyway, getting down to the actual point of this post: Since I can't seem to wind down (I know I must, but I'm still thinking of ways to slice through that data), I've been looking for more Graphing and Visualization tools, and found SciGraphica - which sort of runs under Ubuntu, and might just be able to do everything I need.
The Xubuntu default GTK look it's using is hideously clunky and X11 forwarding via my VPN seems to have a case of the hiccups tonight, but it's is full of promise, and I might just try it in earnest once I've gotten some of the more prosaic stuff out of the way.
Update: I'm now taking a look at R instead, which runs natively on the Mac as well. Many thanks to Mike Malecki for the heads-up.