You may observe files with the correct ownership (chown) and permissions (chmod u+rw) that are write-locked. In my experience, the files were editable with sudo, and modification via the Finder would prompt for an administrator password. In these situations, you may be dealing with additional file attributes.

Access Control Lists (ACLs)

Your file may have an ACL that prevents writing by the current user. To view ACLs for a file, issue ls -le. Use chmod to modify ACLs. Run chmod -N to remove the ACL.

File Flags

Running “Get Info…” on a file and checking “Locked” is functionally equivalent to running chflags uchg [file]. Show flags in directory listings using ls -lO. See also: SetFile.

Extended Attributes (metadata)

For good measure, we should also mention extended attributes. xattr modifies attributes (xattr -r -c to recursively remove), and ls -l@ will display attributes.

I don’t know of any specific metadata that would prevent file modification.

All Together, Now

adam@fsck:~:0$ mkdir /tmp/foo && cd /tmp/foo && touch bar
adam@fsck:/tmp/foo:0$ xattr -w com.sixohthree.test "o hai" bar
adam@fsck:/tmp/foo:0$ chmod +a 'everyone deny write,append,delete' bar
adam@fsck:/tmp/foo:0$ chflags uchg bar
adam@fsck:/tmp/foo:0$ ls -le@O
total 0
-rw-r--r--@ 1 adam  wheel  uchg 0 Dec  8 09:18 bar
        com.sixohthree.test     5
 0: group:everyone deny write,delete,append

The uchg flag will prevent all file modifications, including changing ACLs and extended attributes.

adam@fsck:/tmp/foo:0$ date >> bar
-bash: bar: Operation not permitted
adam@fsck:/tmp/foo:1$ chflags nouchg bar
adam@fsck:/tmp/foo:0$ date >> bar
-bash: bar: Operation not permitted
adam@fsck:/tmp/foo:1$ chmod -N bar
adam@fsck:/tmp/foo:0$ date >> bar
adam@fsck:/tmp/foo:0$

Calendar

• All-day events are always shown, rather than being part of the scrolling midnight-to-midnight view.
• Much more responsive.

Clock

• The Clock now supports alarms.

Photos & Video

• Thumbnails are now larger, showing 3 photos and growing to 4 photos when the pane is maximized, rather than always showing four thumbnails and increasing whitespace when maximizing.

Maps

• Maps is now driven by Google Maps. Haha, kidding, it’s still Bing.

Some miscellaneous things I’m hoping are now fixed:

• Messaging app would occasionally refuse to go online. Required reboot.
• Email app would sometimes show blank message bodies. Required relaunching app.
• Photos & Video was unable to play large video files. Initial test is not promising.

Only time will tell on some of those.

Oh, old and new user-agent string:

Mozilla/5.0 (hp-tablet; Linux; hpwOS/3.0.0; U; en-US) AppleWebKit/534.6 (KHTML, like Gecko) wOSBrowser/233.72 Safari/534.6 TouchPad/1.0
Mozilla/5.0 (hp-tablet; Linux; hpwOS/3.0.2; U; en-US) AppleWebKit/534.6 (KHTML, like Gecko) wOSBrowser/234.40.1 Safari/534.6 TouchPad/1.0

Anyway, three cheers to HP for pushing an update one month after the tablet’s release! Let’s keep the momentum going.

Before the Storm: Finnix

I had no experience with Finnix before this upgrade, but it was exactly what I needed throughout the process. It’s built into the Linode dashboard so you can use it as a recovery console if things go wrong. I also used it as my boot disk while cloning the Linode hard disk to my VM.

The Backup

rsync is my weapon of choice, since it allows for transport over SSH and incremental backups. I used this article to get started with some suggested rsync flags. I ran this several times throughout the process:

rsync -avzPH --numeric-ids --delete --delete-excluded --exclude-from=backup.lst root@server.example.com:/ /mnt/sda1/

Here’s my backup.lst exclusion file:

+ /dev/console
+ /dev/initctl
+ /dev/null
+ /dev/zero
- /dev/*
- /proc/*
- /sys/*
- /tmp/*
- *lost+found

After rsync, chroot to the cloned filesystem:

mount -R /proc /mnt/sda1/proc
mount -R /dev /mnt/sda1/dev
chroot /mnt/sda1 /bin/bash
vi /etc/fstab # update mountpoints. change xvda to sda1, xvdb to sda2

Install a bootloader. The Linode VPS is in Xen and doesn’t normally boot its own kernel. (I know next to nothing about Xen, but this is what I’ve gleaned.)

apt-get install grub
mkdir -p /boot/grub
cp -r /usr/lib/grub/i386-pc/{stage1,stage2,e2fs_stage1_5} /boot/grub
apt-get install linux
echo defoptions=vga=791 >>/boot/grub/menu.lst
update-grub

Reboot. Make sure you kill your cron jobs while the VM is on, as many are probably inappropriate for a backup server.

sudo service cron stop

I recommend taking a snapshot of your backup, here. It may actually be faster to just rsync again, but it’s nice to have a complete backup at the ready.

I used a couple commands to update IP addresses on my backup so I could more accurately test services post-upgrade:

sudo ~/bin/ack --follow -al '207\.192\.74\.235' /etc | sudo xargs sed -i.bak 's/207\.192\.74\.235/172.16.226.130/g'
sudo ~/bin/ack --follow -al '69\.164\.216\.5' /etc | sudo xargs sed -i.bak 's/69\.164\.216\.5/172.16.226.131/g'

The Upgrade

The following notes were taking during my test upgrade, and referenced during the real upgrade. I referenced the Ubuntu Community Documentation project’s “Upgrading to Ubuntu 10.04 LTS” page. There is also a Linode article on upgrading to Ubuntu 10.04.

On the first pass, python-setuptools killed my install. This was fixed by manually reinstalling before upgrade:

sudo dpkg -r python-setuptools && apt-get install python-setuptools

Ensure the update manager is installed, and issue the upgrade command:

sudo apt-get install update-manager-core
sudo do-release-upgrade

I encountered conflicts in the following packages, where I had modified local files. Also included is my resolution, which may or may not be helpful to anyone else.

• /etc/securetty — keep currently installed version; package adds some unneeded ones, and removes tty0 (but does include the Xen ones added by Linode)
• /etc/sysctl.conf — use new; uncomment “kernel.prink = 4 4 1 7” and add “fs.inotify.max_user_watches = 524288“
• /etc/mysql/my.cnf — kept my version
• /etc/dhcp3/dhclient.conf — kept Linode version: “request subnet-mask, broadcast-address, time-offset, routers, domain-name, domain-name-servers, host-name, ntp-servers;“
• /etc/dbconfig-common/config — nothing meaningful in the diff; use new
• /etc/dovecot/dovecot.conf — kept mine
• /etc/default/openvpn — was missing; installed package version
• /etc/snmp/snmpd.conf — kept mine
• /etc/apache2/apache2.conf — kept mine, as well as other apache2 files (but I did use the new mime.types file)
• /etc/stunnel/stunnel.conf — kept mine
• /etc/default/stunnel4 — kept mine
• /etc/udev/rules.d/75-persistent-net-generator.rules — kept installed, looked like the GOTO="persistent_net_generator_end" at the start was intentional

Got a warning about vulnerable SSL keys generated by older versions of Ubuntu; solution is to regenerate your keys. The dialog recommended using openssl-vulnkey and openvpn-vulnkey to test.

Other than having to resolve the file conflicts, the upgrade was very smooth. I found some notes about fixing the Xen serial console, and syslog breaking in 10.04. I also upgraded to a newer paravirt kernel in the Linode dashboard.

python-setuptools error

More about that error with python-setuptools: this thread was helpful. From /var/log/apt/term.log:

Log started: 2010-10-06  08:35:15
Setting up python-setuptools (0.6c9-0ubuntu1) ...
pycentral: pycentral pkginstall: not overwriting local files
pycentral pkginstall: not overwriting local files
dpkg: error processing python-setuptools (--configure):
 subprocess post-installation script returned error exit status 1
Errors were encountered while processing:
 python-setuptools
Log ended: 2010-10-06  08:35:15

Per the article, ran “apt-get remove python-setuptools” and “apt-get install python-setuptools”

Post-upgrade Recompiling

Something required apt-get install libltdl-dev. Useful info for this post, no?

I recompiled all my custom stuff, including nginx, memcached, mod_php and php-fpm, and mysqld.

The Background

For several years now, I have used a series of AppleScripts to automate enabling and disabling of the Mac OS X screen saver password. The scripts were pulled from a thread on the MacWorld forums and have worked well, save for one detail: there is quite a lot of overhead in launching a compiled AppleScript just to execute two commands in the shell:

do shell script "defaults -currentHost write com.apple.screensaver askForPassword -int 0"
do shell script "/Users/adam/bin/notif"

Under heavy load (ie. with all my normal apps, plus VMware Fusion running one or more virtual machines) I often wait 30 seconds or more to enable or disable the password.

Steam was recently ported to Mac OS X, and it has the ability to create application shortcuts on your Desktop. Digging into one of these applications, you see the following hierarchy:

adam@fsck[0]:/Applications:0$ find "Team Fortress 2.app"
Team Fortress 2.app
Team Fortress 2.app/Contents
Team Fortress 2.app/Contents/Info.plist
Team Fortress 2.app/Contents/MacOS
Team Fortress 2.app/Contents/MacOS/run.sh
Team Fortress 2.app/Contents/Resources
Team Fortress 2.app/Contents/Resources/shortcut.icns

Here’s Info.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>CFBundleExecutable</key>
    <string>run.sh</string>
    <key>CFBundleIconFile</key>
    <string>shortcut.icns</string>
    <key>CFBundleInfoDictionaryVersion</key>
    <string>1.0</string>
    <key>CFBundlePackageType</key>
    <string>APPL</string>
    <key>CFBundleSignature</key>
    <string>????</string>
    <key>CFBundleVersion</key>
    <string>1.0</string>
</dict>
</plist>

And the good stuff, run.sh:

#!/bin/bash
# autogenerated file - do not edit

open steam://run/440

This was a pretty big “wow” moment for me. No AppleScript, no .command files (which launch Terminal.app), just pure, instant command line gratification. appify was born to quickly convert shell scripts into applications.

I have one thing of note to log, so of course I’ll post it here.

Bash Escape Sequences

On opening iTerm for the first time, I was greeted with this:

Last login: Mon Apr  5 11:29:31 on ttys001
\e]1;fsckadam@fsck:~:0$

You can see a bit of gibberish after the date, but before my prompt. So, I guess my old escape sequences stopped working in Snow Leopard. I have a set of functions that set window titles using escape sequences. Changing \e and \a to their octal equivalents worked for me. So these lines in .bashrc:

function setlabel { echo -ne "\e]1;$1\a"; }
function settitle { echo -ne "\e]2;$1\a"; }

Became this:

function setlabel { echo -ne "\033]1;$1\007"; }
function settitle { echo -ne "\033]2;$1\007"; }

The device’s controversy speaks to its importance. Apple, maligned and ridiculed for so many years, is now a force. The iPod is ubiquitous, the iPhone is commonplace, and MacBooks are around every corner. Many of the content creators I follow work in an Apple world, so Apple’s implementation is noticed and does matter.

I like the iPad as a consumer device, in that sense that I would like to consume media on it. It’s a device that’s just begging to be touched. I can foresee many hours spent browsing the web, looking at photos, and watching movies with an iPad in my hands. I do all these things on my MacBook Pro, and I see the iPad as a better way to accomplish this subset of tasks.

The iPad is dangerous as a primary computing device, at least for the younger generation. Despite the announced iWork apps, it’s still a device of consumption, and consumption does not engage the brain the same way creation does. Sure you can blog, and there is potential that someone will create video, audio, and other creative apps, but those don’t exist today. It’s very important to me that my son has the opportunity to pursue whatever his interests may be, technical or not. The iPad has very little to offer in this regard.

It’s also a device that makes it easy for you to spend money, with the iBookstore, the App Store, and the iTunes Store in easy reach. Unfortunately, Apple is forced to bargain with content producers, and it’s usually the consumer that loses out. The ability to “buy” many sorts of media is disingenuous: DRM puts an expiration date on your entertainment. Authentication servers get turned off, formats become obsolete, and you have to repurchase relicense something that, at one time, you thought you owned. (Reminds me of the VHS to DVD to HD-DVD to whoops-HD-DVD-lost-to-Blu-ray transition, but that’s a whole different marketing strategy.)

For me, the iPad is still in “do want” territory. I think it will be a great toy around the house, my MacBook will still be there when I need to do some real work, and my Palm Pre will fill in the gaps.

This creates an svn less subcommand that runs svn diff with any other arguments you provided, paging the result through less(1). It also provides a custom environment variable to svn ci, instructing Vim to split screen, set some options suitable for a read-only diff, and run svn diff on all files included in the default commit message, i.e. the list of changed files. Finally, it wraps svn propdiff to highlight modifications to a property.

Any unrecognized subcommands (update, checkout, etc.) are passed through unchanged. The automatic diff is only applied to the ci subcommand, so the svn commit variant can be used when a diff is not required.

I could not find a way to remove the partition via the Mac OS X Disk Utility, so I resorted to some terminal work. By clobbering the first few bytes of the disk’s partition map, I essentially made the whole disk look like random garbage, and TrueCrypt no longer presented a warning. This will cause data loss, so be careful.

First, find the drive’s device number using Disk Utility. Right click on the drive (not one of the partitions) and select “Information.”

Look for “Disk Identifier” in the property list. In this case, my drive is called disk3, which corresponds to /dev/disk3 on the filesystem. Tomorrow it might be disk4 or disk6, so don’t make any assumptions here. Failure to find the correct drive number in this step will cause you to erase the wrong drive in a future step.

Open up your terminal and umount this disk’s volumes. Note the asterisk, which should be included in your command.

• sudo umount -f /dev/disk3*

Unmounting drives in the Finder will make the disk unavailable in /dev, and we need this block device if we’re going to clobber the filesystem.

Once the partitions are unmounted, we can write junk to the partition table and essentially unformat the drive. This is the dangerous part, so don’t screw it up or you will destroy the wrong drive, lose all your data, and seriously hate life. Replace disk3 with your drive identifier!

• sudo dd if=/dev/zero of=/dev/disk3 bs=1 count=1024

After writing zeroes to the front of the drive, TrueCrypt happily made an encrypted volume on the disk.

That process at the top of the list? SQL Developer.