(soft|hard)ware
Silence
2008-04-29
People who've visited me at home know that I'm a huge
fan of total silence. I've taken considerable steps
to end up with an office which is totally
quiet (and consumes virtually no power as a nice
side-effect). An obvious tip: turn off what you don't
constantly need. Better still: make stuff turn off by
itself after a certain time. Slightly less obvious
but very effective: use 2.5" USB-powered external
HD's where possible (on my NSLU2-based server, and
attached to my MacBook Pro for Time Machine backups).
Disk capacity is no longer an issue.
As I said, the result is perfect - and music is fun again, even at a very low volume.
Which makes this post hilarious ;)
As I said, the result is perfect - and music is fun again, even at a very low volume.
Which makes this post hilarious ;)
Too heavy for Spotlight?
2008-02-12
RAID + backup
2008-02-11
I've solved the problem of having robust storage both
on- and off-line: RAID 1 over 3 disks with a
write-intent bitmap.
Let me explain. I've been moving more and more information to disk lately, scanning in books and de-duplicating all cdrom/dvd backups (we had all our vinyl and negatives digitized a while back). Takes only 250 Gb so far, not counting mp3's/m4v's that is (which have less uniqueness value to me). See this NYT article - where Brewster Kahle summarizes it well: "Paper is no longer the master copy; the digital version is".
But with bits, particularly if some of 'em are in compressed files, data integrity is a huge issue. Which is where RAID makes sense. So I got two big SATA disks, hooked them up to trusty old "teevie", a 6 year old beige box stashed away well out of sight and hearing. With RAID 1 mirroring, everything gets written to both disks, so if either one fails: no sweat.
But RAID does not guard against fire or theft or "rm -r". It's a redundancy solution, not a backup mechanism. I want to keep an extra copy around somewhere else, just in case. Remote storage is still more expensive than yet another disk, and then you need encryption to prevent unauthorized access. Hmm, I prefer simple, as in: avoid adding complexity.
So now I'm setting up RAID to mirror over three disks. The idea being that you can run RAID 1 just fine in "degraded" mode as long as there are still 2 working disks in the array. Once in a while I will plug in the third disk, let the system automatically bring it in sync with the rest, and then take it out and move it off-site again.
But the story does not end here. Bringing a disc up to date is the same as adding a new disk: RAID will do a full copy, taking many hours when the disks are half a terabyte each. Which is where the "write-intent bitmap" comes in: it tracks the blocks which have not been fully synced to all disks yet. Whenever a block is known to be in sync everywhere, its bit is cleared. What this means is that I can now put all three disks in, let them do their thing, and after a while the bitmap will be all clear. Once I pull out the third disk, bits will start accumulating as changes are written to the other two. Later, when putting the third disk back on-line, the system will automatically save only the changed blocks. No need to remember any commands, start anything, just put it on-line. Quick and easy!
If a disk fails: buy a new one, replace it, done. Every few months, I'll briefly insert the third disk and then safely store it off-site again. If I were to ever mess up really badly (e.g. "rm -r"), I can revert via the third one: mark the two main disks as failed, and put the third one in for recovery to its older version.
Methinks it's the perfect setup for my needs. Total cost under €300 for the disks plus cheapo drive bays. Welcome to the digitalization decade.
Let me explain. I've been moving more and more information to disk lately, scanning in books and de-duplicating all cdrom/dvd backups (we had all our vinyl and negatives digitized a while back). Takes only 250 Gb so far, not counting mp3's/m4v's that is (which have less uniqueness value to me). See this NYT article - where Brewster Kahle summarizes it well: "Paper is no longer the master copy; the digital version is".
But with bits, particularly if some of 'em are in compressed files, data integrity is a huge issue. Which is where RAID makes sense. So I got two big SATA disks, hooked them up to trusty old "teevie", a 6 year old beige box stashed away well out of sight and hearing. With RAID 1 mirroring, everything gets written to both disks, so if either one fails: no sweat.
But RAID does not guard against fire or theft or "rm -r". It's a redundancy solution, not a backup mechanism. I want to keep an extra copy around somewhere else, just in case. Remote storage is still more expensive than yet another disk, and then you need encryption to prevent unauthorized access. Hmm, I prefer simple, as in: avoid adding complexity.
So now I'm setting up RAID to mirror over three disks. The idea being that you can run RAID 1 just fine in "degraded" mode as long as there are still 2 working disks in the array. Once in a while I will plug in the third disk, let the system automatically bring it in sync with the rest, and then take it out and move it off-site again.
But the story does not end here. Bringing a disc up to date is the same as adding a new disk: RAID will do a full copy, taking many hours when the disks are half a terabyte each. Which is where the "write-intent bitmap" comes in: it tracks the blocks which have not been fully synced to all disks yet. Whenever a block is known to be in sync everywhere, its bit is cleared. What this means is that I can now put all three disks in, let them do their thing, and after a while the bitmap will be all clear. Once I pull out the third disk, bits will start accumulating as changes are written to the other two. Later, when putting the third disk back on-line, the system will automatically save only the changed blocks. No need to remember any commands, start anything, just put it on-line. Quick and easy!
If a disk fails: buy a new one, replace it, done. Every few months, I'll briefly insert the third disk and then safely store it off-site again. If I were to ever mess up really badly (e.g. "rm -r"), I can revert via the third one: mark the two main disks as failed, and put the third one in for recovery to its older version.
Methinks it's the perfect setup for my needs. Total cost under €300 for the disks plus cheapo drive bays. Welcome to the digitalization decade.
New?
2008-01-10
There's something odd when you see a new
gadget announced on a geek site ... which we've had in the
house for some two years now:
See http://www.wettershop.de/ under Designprodukte/Thermometer. It sticks on the outside of a window, BTW.
See http://www.wettershop.de/ under Designprodukte/Thermometer. It sticks on the outside of a window, BTW.
No more backups
2007-11-17
There is a feature in Mac OS X 10.5 which gets
mentioned a lot lately: Time Machine. Unfortunately,
Apple has done a good job of grabbing everyone's
attention with the user interface of this thing,
but the key advance is not the looks
but the inner workings of it all. What
TM does, is maintain hourly/daily/weekly rolling
backups. Which is great, except that until now
you always had to somehow keep system load and
disk usage in check. So normally this approach
simply doesn't scale well enough to be used on
an entire disk, no matter how clever you wrap up
rsync, etc. Which means you end up with a
diverse set of backup strategies. Messy,
tedious, and brittle, alas.
Well, times have changed. TM is revolutionary, because its overhead is proportional to the amount of change. Every hour, my (quiet) external HD spins up, rattles for a few seconds, and then spins down again after 5 minutes (not 10, see "man pmset"). All you need is a disk with say 2..3x as much free space as the backed-up area. One detail to take care of is to not include big changing files such as VMware/Parallels disk images and large active databases in the backup, because TM backs up per file. So I put all those inside /Users/Shared and exclude that entire area.
Oh, and TM does the right thing: it suspends and resumes if its external disk is off-line for a while, such as with a laptop on the road.
Note that attached disks cannot protect from malicious software and major disasters such as a fire - but this is something an occasional swap with an off-site disk can take care of.
I no longer "do" backups. I plugged external disks into the two main Macs here and TM automatically asked for permission to use them. End of story.
Well, times have changed. TM is revolutionary, because its overhead is proportional to the amount of change. Every hour, my (quiet) external HD spins up, rattles for a few seconds, and then spins down again after 5 minutes (not 10, see "man pmset"). All you need is a disk with say 2..3x as much free space as the backed-up area. One detail to take care of is to not include big changing files such as VMware/Parallels disk images and large active databases in the backup, because TM backs up per file. So I put all those inside /Users/Shared and exclude that entire area.
Oh, and TM does the right thing: it suspends and resumes if its external disk is off-line for a while, such as with a laptop on the road.
Note that attached disks cannot protect from malicious software and major disasters such as a fire - but this is something an occasional swap with an off-site disk can take care of.
I no longer "do" backups. I plugged external disks into the two main Macs here and TM automatically asked for permission to use them. End of story.
Security?
2007-07-13
Installed Vista Ultimate under VMware Fusion today
(do I need it? no, but it's good to know a bit about
it and for testing) - it nearly fills an 8 Gb disk
image, and it gobbles up 1 Gb of RAM. I won't bore
you with my opinion on those two facts...
VMware has a quick setup whereby it asks your name and a password and then lets the whole setup run with no further questions asked. After the whole install process, the system comes up logged in and ready to go - which is great.
Except that I did not enter a password, which leads to a system which does an auto-login with a password I do not know. It's definitely not the empty string. I don't even know who is to blame, VMware or Vista... After trying a few things and googling a bit, I was about to conclude that a full re-install would be my only option. No info on special boot keys to bypass/reset things, the built-in help says you need to know the admin password (or have a rescue disk, which... needs an admin password to be created). It all makes sense of course, but I wasn't getting anywhere with all this and left with a setup I couldn't administer.
But guess what: as admin, you can create a new user with admin rights, and it doesn't ask for your password! So I created a temporary user with full admin rights and with a known password, and switched to it. Then I reset the original admin's password - bingo. Did this after all the latest updates were applied, btw.
In other words: anyone can do anything on a machine running Vista if the current user is an administrator, without ever having to re-confirm the knowledge of that admin's password: simply create another admin and switch to it.
Pinch me. How many years has Vista been in development? How long has it been out as official release? On how many systems has it been installed?
VMware has a quick setup whereby it asks your name and a password and then lets the whole setup run with no further questions asked. After the whole install process, the system comes up logged in and ready to go - which is great.
Except that I did not enter a password, which leads to a system which does an auto-login with a password I do not know. It's definitely not the empty string. I don't even know who is to blame, VMware or Vista... After trying a few things and googling a bit, I was about to conclude that a full re-install would be my only option. No info on special boot keys to bypass/reset things, the built-in help says you need to know the admin password (or have a rescue disk, which... needs an admin password to be created). It all makes sense of course, but I wasn't getting anywhere with all this and left with a setup I couldn't administer.
But guess what: as admin, you can create a new user with admin rights, and it doesn't ask for your password! So I created a temporary user with full admin rights and with a known password, and switched to it. Then I reset the original admin's password - bingo. Did this after all the latest updates were applied, btw.
In other words: anyone can do anything on a machine running Vista if the current user is an administrator, without ever having to re-confirm the knowledge of that admin's password: simply create another admin and switch to it.
Pinch me. How many years has Vista been in development? How long has it been out as official release? On how many systems has it been installed?
Off-site storage
2007-06-26
Just came across an interesting utility called
JungleDisk. It creates a virtual
disk on the desktop which is automatically
encrypted and backed up to Amazon's S3 service. The disk uses local
storage as cache for fast local use, and is
accessed via WebDAV - which is how it can figure
out what to send to S3 in the background. It
only sends changes, AFAICT. This looks like an
eminently practical solution - even for larger
datasets, such as a personal photo collection.
Haven't used it - beyond a quick tryout. JD only caches files up to a (configurable) limit, the rest gets pulled from S3. Also, it looks like offline mode is not supported (yet, apparently) - though setting the cache big enough to always hold everything might do the trick.
Haven't used it - beyond a quick tryout. JD only caches files up to a (configurable) limit, the rest gets pulled from S3. Also, it looks like offline mode is not supported (yet, apparently) - though setting the cache big enough to always hold everything might do the trick.
The browser
2007-05-09
To follow up on my Flex
lock-in post, see this article: the language of the
future could indeed be JavaScript.
This does not imply what we should all jump ship and move to JS - just like Windows being dominant is no reason to adopt it.
This does not imply what we should all jump ship and move to JS - just like Windows being dominant is no reason to adopt it.
Dead pixel
2007-04-27
Not so long ago, one of my Mac's pixels went black.
Fortunately, the other 7,857,999 are ok :)
Fortunately, the other 7,857,999 are ok :)
Flex lock-in
2007-04-26
Adobe's Flex provides a way to push
XML-based "code" from a server into a Flash
player running on the browser. There's a nice
online demo. It looks like a useful
approach - the Flash player is nearly ubiquitous
by now.
I can't help but think that the Tk browser plugin could have filled this spot ages ago. And much more, by now.
The news is that Flex will be open sourced before the end of the year. Think about it: open source code, written for a presentation engine that is fully controlled by Adobe. Clever. It's not easy to come up with scenario's whereby you get to lock-in open source developments.
In other words: to conquer the world, get into the browser. JavaScript and Flash did. Is everything else becoming irrelevant?
Update: another player in this field is Microsoft with Silverlight.
I can't help but think that the Tk browser plugin could have filled this spot ages ago. And much more, by now.
The news is that Flex will be open sourced before the end of the year. Think about it: open source code, written for a presentation engine that is fully controlled by Adobe. Clever. It's not easy to come up with scenario's whereby you get to lock-in open source developments.
In other words: to conquer the world, get into the browser. JavaScript and Flash did. Is everything else becoming irrelevant?
Update: another player in this field is Microsoft with Silverlight.
New disk
2006-11-23
Disk failures
2006-11-02
Yesterday, the system disk of "teevie", my AMD64
Linux box (which also hosts a range of other OS'es as
VMware images) started running into hard disk errors.
The drive was getting terribly hot, so I took it out of the 3.5" drive slot to get some more free-air cooling. Problems went away long enough to recover all partitions with a couple of restarts (and many hours of patience). Thank you Knoppix and rsync, for being there when I needed you.
As it so happens today a big new external HD arrived (Maxtor OneTouch III), so I have been busy making full backups of all the main machines around here. No fun, but I guess I got away with the occasional when-I-think-of-it full backup style I've been doing for years now. It's not the work lost that I fear (I really do backup my active files a lot), but the amount of time it takes to restore a well-running system after serious hardware failures. Copying these big disks takes forever.
The failing IBM Deskstar 120 Gb drive worked for about 5 years without a hitch, so it really did well. Luckily, there's a spare 80 Gb around here which can take its place - but it sure takes a lot of time to shove those gigs around and get all the settings just right again!
The drive was getting terribly hot, so I took it out of the 3.5" drive slot to get some more free-air cooling. Problems went away long enough to recover all partitions with a couple of restarts (and many hours of patience). Thank you Knoppix and rsync, for being there when I needed you.
As it so happens today a big new external HD arrived (Maxtor OneTouch III), so I have been busy making full backups of all the main machines around here. No fun, but I guess I got away with the occasional when-I-think-of-it full backup style I've been doing for years now. It's not the work lost that I fear (I really do backup my active files a lot), but the amount of time it takes to restore a well-running system after serious hardware failures. Copying these big disks takes forever.
The failing IBM Deskstar 120 Gb drive worked for about 5 years without a hitch, so it really did well. Luckily, there's a spare 80 Gb around here which can take its place - but it sure takes a lot of time to shove those gigs around and get all the settings just right again!
VoodooPad
2006-09-28
Convergence
2006-02-09
The
Slimserver is an MP3 jukebox which plays nice with
iTunes music collections (and it does not require a
Squeezebox player to be useful). Works nicely (coded
in Perl, not deployed as Starkit, oh well). And now
it looks like the Nokia 770 could become a
nice
front-end for it.
When I have time for it, am having way too much fun with Vlerq right now!
When I have time for it, am having way too much fun with Vlerq right now!
Humanoid
2005-12-14
Seeing
these movies
gives me a
very odd sensation. How can this little machine look
so "real"? Amazing achievement. It all looks benign,
for now...
Safe sleep
2005-12-04
The Apple
Powerbook is now able to do a hybrid
suspend/hibernate, as described in
this article. What it
means is that nothing changes in the normal case. But
even if a battery runs flat (mine have been losing
capacity over the years), saved state is not
compromised: startup will
still bring it
back.
I love that second smile effect. This is what creates a truly loyal customer base.
I love that second smile effect. This is what creates a truly loyal customer base.
PowerGUI
2005-11-29
An example
of how Quicksilver
and a
web-based system can be used together. The video
mentioned on this
page illustrates
the workflow dynamics of it all. I'm still looking
for ways to get more of my computer work streamlined,
and mice just don't cut it: no way to automate things
into your spinal cord (read: effortlessly) when it
takes aiming and visual feedback to get anything
done.
Darwinports GUI
2005-11-09
DarwinPorts
is a
collection of nearly 3000 ports of various Unix/Linux
software packages for Mac OS X. And there's a GUI for
it, called Port
Authority. Written in
Tcl. Looking inside, PA appears to be using Tk, Tile,
Tablelist, and Critcl - cool!
No[kia] brainer
2005-11-08
CVStrac
2005-10-06
CVSTrac is a
bug tracking system tied to CVS which is delightfully
simple to set up and use: there's a self-contained
static executable for Linux which works right out of
the box. Written by Richard Hipp, author of SQLite
(which is used in CVSTrack). Absolutely top notch.
G5 vs. X86, OSX vs Linux
2005-06-04
A
comparison which
confirms what I've been seeing, but for which I never
had any real data to back it up: the Mac is not as
snappy at the core level as a Linux/x86 combo.
Wouldn't ever want to go back to anything else though
- in terms of helping me get the work done, my 1 GHz
PowerBook remains in a league of its own.
NanoBlogger
2005-04-19