Addendums

Virtualization is really useful for webservers, which are pretty low-resource but are good to have in numbers for the sake of redundancy. So you can make much better use of a resource-rich system by virtualizing it and putting several webservers on it - if that’s your particular task of the moment.

That’s one perk for the marriage of virtualization and webservers. The one I like the best, though, is easy duplication. See, setting up a webserver on FreeBSD takes a LOT of time. I know there are some great advantages to being able to directly compile all your software via ports (more sophisticated configurations, etc.), but we simply aren’t using any of those advantages with regards to our dependencies. It also strikes me as pretty crazy that basic programs like bash, vim, and screen don’t come pre-installed. You can literally take a nap while waiting for vim to install.

As a result, getting a single webserver set up can take up to a full workday - between time-consuming port installations and myriad CPAN dependencies (our web app has a Perl foundation). And it hurts the programmer’s aesthetic of “don’t repeat yourself” to do this same webserver setup across several machines - although this is exactly what I’ve had to do in the past. Luckily, we can almost entirely skip that hefty setup and install with VirtualBox.

There are two ways to go about this - by exporting and then importing a VM under a new name, or by cloning a virtual hard drive and attaching the clone to a newly created VM. I actually prefer the latter, because the configuration of the VM itself isn’t necessarily a step I want to skip - but the installation of all the software on the hard drive is. I suspect that most people would prefer the import / export option, but I just have more of a mostly-inexplicable attachment to hd cloning (much like how I have always preferred while loops over for loops despite their individual strengths / weaknesses). Anyhow, I’ll show how both are done in this entry.

To start out with: make sure the vm you are either exporting or whose hard drive you are cloning is powered off. That said, you won’t need to detach the hard drive you are cloning from the VM, if you decide to clone.

Here’s how I’d export my vm “addievm”. Note that you have to specify the name of the file you’re going to output to, and .ovf is the common extension.


VBoxManage export addievm --output addievm.ovf


Exporting isn’t instantaneous, but you’ll see a little text prompt iterating from 0% to 100% to give you an idea of your progress. If you’ve done any configuration on your machine since setting it up (like installing the operating system and some software), this can take a few minutes, and longer for systems with very large hard drives.

The export (for a single machine - you can actually export multiple machines at once, but I’m not particularly interested in it) will give you three files - in this case, they’ll be addievm.ovf, addievm.mf, and addievm.vmdk. The .ovf has the VM configuration, and the .vmdk is the virtual hard drive (a format that is compatible with virtualization platforms like VMWare, more info here). The .mf file contains the UUIDs of the virtual machine and the virtual hard drive. You’ll want to keep all three in the same place (i.e. at the same directory level) so VirtualBox can locate the additional two files (.mf and .vmdk) during an import of the .ovf. Note also that the .vmdk takes the name of the virtual hard drive itself, not the name you have given to the .ovf file - so if I had a hard drive named addiehd.vdi instead of addievm.hdi and ran the same command as above, my resulting files would be addievm.ovf, addievm.mf, and addiehd.vmdk.

To import - in this use case, to import as a copy on the same machine as the original - you can run the VBoxManage import command in the same location that you exported to. You’ll want to start with this command:


VBoxManage import -n addievm.ovf


This command doesn’t actually perform the import. Instead, because of the use of the -n option (–dry-run also works), you’ll get a description of what the import will look like with its defaults set, and the options you can use to change those defaults. Here’s what mine looks like for addievm:


Disks: vmdisk1 52428800000 -1 http://www.vmware.com/specifications/vmdk.html#sparse addievm.vmdk 1261356544 -1
Virtual system 0:
0: Suggested OS type: "FreeBSD_64"
(change with "--vsys 0 --ostype "; use "list ostypes" to list all possible values)
1: Suggested VM name "addievm_1"
(change with "--vsys 0 --vmname ")
2: Number of CPUs: 1
(change with "--vsys 0 --cpus ")
3: Guest memory: 1024 MB
(change with "--vsys 0 --memory ")
4: Network adapter: orig Bridged, config 3, extra type=Bridged
5: CD-ROM
(disable with "--vsys 0 --unit 5 --ignore")
6: IDE controller, type PIIX4
(disable with "--vsys 0 --unit 6 --ignore")
7: Hard disk image: source image=addievm.vmdk, target path=/home/addie/.VirtualBox/HardDisks/addievm.vmdk, controller=6;channel=0
(change controller with "--vsys 0 --unit 7 --controller ";
disable with "--vsys 0 --unit 7 --ignore")


The only one I really want to change in my typical use case is the name of the vm. So my command to actually import the copy will look like:


VBoxManage import addievm.ovf --vsys 0 --vmname addievm_copy


You’ll get similar output as you did with the “dummy run”, but this time it will be followed up with the text “progress bar” as with the export, incrementing from 0% to 100%. Expect a similar wait on import as you did on export.

You can also import as many times as you’d like. You’ll see from the “Hard disk image” information that the hard disk created on the import is renamed like addievm_1.vmdk, addievm_2.vmdk, etc. You aren’t able to change how this is named from the import. I really don’t like having so little control over the naming of hard disks with regards to keeping a collection of virtual machines and virtual hardware organized, but I have found a renaming workaround hack that I’ll be documenting in another post soon.

As for taking the second approach, start by creating a new VM and configuring it as you see fit. The settings don’t have to be the same for a VM with a cloned hard drive, with the exception of HD-related settings (like the operating system). Then, go ahead and clone the disk:


VBoxManage clonehd addievm.vdi addie-clone.vdi --format VDI --remember


The options for this command are very similar to those for creating a hard disk from scratch. The “–remember” option will automatically register the cloned hd with your system of virtual machines and hardware, so if you use the command


VBoxManage list hdds


the cloned hard drive will show up.

Then it’s just a matter of attaching the cloned hard drive to the new VM you’ve created. You’ll need to start up the cloned machine and change things like hostname, networking configuration, etc. so there aren’t any conflicts when both machines are running at once on the same network, but such changes are independent to the actual systems that are being cloned and are out of scope for this documentation.

As I mentioned before, one of the issues you might discover here is that, especially with import and export, you don’t have the full control over naming of your virtual hardware that you’d like. Although there is support in VBoxManage for renaming a machine, there isn’t support for renaming a piece of hardware. I’ve found ways to get around this that I’ll document in my next post; it’s a bit hacky but leaves me with a system that I feel like I actually have full control over.

The last few entries have explained, command-by-command, how to create virtual machines and prepare them for OS installation, with some details about how to unwind this work (i.e. delete everything cleanly). In the process of writing the entries, I’ve learned new details about how to attach / remove storage devices (like hard drives and DVDs) in VirtualBox 3.1.2, which requires an extra layer of abstraction.

I thought I’d stick all the commands together in sequence for easy reference. Of course, I’m doing the same thing as a shell script right now to create some real machines of my own. As a result, I’m using the “-q” option in all commands to suppress the standard headers that appear after the typical VBoxManage command is run.

Creating a VM, attaching a hard drive and OS ISO, enabling networking and VRDP
This particular VM has 50 GB hard drive, 1024 MB RAM, FreeBSD 8.0 for AMD 64 as its OS, main bridged interface named eth0, and VRDP port 3390.


VBoxManage -q createhd --filename addievm.vdi --size 50000 --format VDI --variant Standard

VBoxManage -q storageattach addievm storagectl "IDE Controller" --port 1 --device 0 --type dvddrive --medium /home/addie/8.0-RELEASE-amd64-disc1.iso


After running all of these commands, you should have a VM that is ready to be started up for OS install.

If something went horribly wrong, you may want to remove all your work and start again (I’ve had to do this more times than I’d care to admit). So here are the commands for unwinding all of the work you’ve just done without leaving extra traces lying around (the one thing left registered after this process will be the DVD used for install; I don’t remove it because I end up re-using it soon enough).

Removing the VM I just created - and its attachments


VBoxManage -q storageattach addievm storagectl "IDE Controller" --port 0 --device 0 --type hdd --medium none

VBoxManage -q unregistervm addievm --delete


That’s all for the summary - now on to new VirtualBox tasks!

This is at once one of the simpler and one of the most crucial of these posts. Your virtual machine is going to be pretty useless unless you have an operating system installed on it, and specifying the OS type when creating the machine doesn’t take care of the OS install for you.

The two things that deserve documentation in this process are the act of mounting the install ISO onto the VM (and unmounting it after install), and accessing the VM remotely to be able to view the interface for the OS install. The actual details of any specific OS install will be left out here because they’re outside of the scope of this documentation.

The caveats introduced in the Tricky Bits post apply here considerably, so if I’m seeming too brief on some details, refer to that post for clarification.

To get started with mounting an ISO, download and copy over the relevant file to your user’s home directory. In the case of my “addievm”, I’m installing a copy of FreeBSD 8.0 for a 64-bit processor, and that’s the filename I’ll be using in these examples.

In older versions of VirtualBox, here is how I would create the virtual representation of the DVD and then attach it to the machine:


VBoxManage registerimage dvd 8.0-RELEASE-amd64-bootonly.iso
VBoxManage list dvds
VBoxManage modifyvm addievm --dvd {UUID for ISO}


I found that I always had to use the UUID to get a successful result with this; there isn’t a good “name” to reference like with a VM or a harddrive. You can get the UUID really quickly with the “list” command that comes second in this series.

In newer versions of VirtualBox, where you are attaching storage media to a controller versus directly to the virtual machine, the command is slightly different. This command assumes that the controller has been created and the hard drive has been attached to port 0; essentially the commands listed in Tricky Bits.


VBoxManage storageattach addievm --storagectl "IDE Controller" --port 1 --device 0 --type dvddrive --medium /home/addie/8.0-RELEASE-amd64-bootonly.iso


Now that the DVD is mounted, we can start the machine using the commands listed in VBoxManage and VBoxHeadless. But there’s no way to access the interface using a headless server. This is where VRDP comes into play.

In Creating and deleting VMs, I enabled VRDP and assigned it a port number. This port is important for accessing the VM remotely.

I use Ubuntu Desktop for my development workstation and found that the default remote desktop application which comes with the system doesn’t work for RDP support. So I installed the program rdesktop from Synaptic, and it does the job. To launch, type into your terminal:


rdesktop {host ip}:{guest port}


Presuming my host is on a local network with static ip 192.168.0.50, and the vrdp port for my VM is 3390, I’d run:


rdesktop 192.168.0.50:3390


If all goes well, a UI for your interface will pop up and you can work through the setup accordingly.

Once installation is complete you’ll want to unmount the DVD so the system stops booting into the installation. In older versions of VirtualBox:


VBoxManage modifyvm addievm --dvd none


In newer versions:


VBoxManage storageattach addievm --storagectl "IDE Controller" --port 1 --device 0 --type dvddrive --medium emptydrive


If you’d like to suspend VRDP support at any time (maybe you want to move back to the OSE version), make sure you have networking and SSH configured on your virtual machine so you can access it without needing a graphical interface.

Next entry will jump into more sophisticated work: cloning virtual hard drives in order to save time on configuration of near-identical systems.

Since writing my last post (VirtualBox on the command line: Creating and deleting VMs), I’ve discovered some tricky bits that anybody relying on the post will run into if using the newest version of VirtualBox CSE (3.1.2 at the time of this writing). My past documentation was based on an older version of Ubuntu and an older version of VirtualBox. The work I’ve been doing the last week is on newer versions of both the operating system and the software, and there are some important differences that I’d like to account for here.

In some cases, things are a little easier, and in other cases, they seem more complicated. In the case of the more complicated steps, this is a sacrifice for greater power and control in the VM; unfortunately all my simpler setups won’t be taking advantage of these new features.

Three issues to discuss here.

One. Open Source Edition (OSE) vs Closed Source Edition (CSE).

The version of VirtualBox which comes with Synaptic on Ubuntu Karmic Koala is the Open Source Edition. Unfortunately, if you’re going to be configuring virtual machines on a headless server, you’ll need to use the Closed Source Edition (CSE). In my case, this is because I need RDP enabled. With a headless server, I don’t have a way to access the virtual machine for its operating system install without RDP. Using RDP, I can access the virtual machine from a computer that does have a graphical output, and do the operating system install / configure networking so I can ssh into the machine in the future.

If you’re really a die-hard for the OSE, there is a way to get around this, but it will require that you do all of the creation and setup of a virtual machine (i.e., what I have been documenting here) on a non-headless operating system. So you’ll create your vms with a computer that has the VirtualBox GUI, do everything that you need a graphical interface for (operating system install and initial network configuration), and then export the vm you’ve just created. Upload it to the headless server, and import it into your copy of VirtualBox on that server. According to documentation I’ve been reading while doing my research, this should work and allow you to get around installing the closed source version on your headless server.

In my case, it’s just easier to install the closed source version. RDP is occasionally useful post-OS install too - for instance, the times that the virtual machine I am ssh’ed into stop responding. Often I can load up a remote desktop of this VM and I’ll see a swap space error or some other diagnostic that allows me to figure out what exactly went wrong instead of “machine just froze”.

With this all explained, I’ll try to document where things are “OSE only” or “CSE only” and if I haven’t made note of it, it hopefully works in both editions.

One note with regards to the stuff I’ve already documented is that the CSE uses a “vboxusers” group to determine who can administrate virtualboxes and who cannot. So if you want to be able to run any of the commands I’ve documented with CSE, your user will need to be part of the “vboxusers” group first.

Two. Attaching Virtual Hard Drives, and Storage Control.

This is the one that really got me today! When I entered the command I listed in the last entry to add a virtual hard disk to a freshly-created VM:


VBoxManage modifyvm addievm --hda addievm.vdi


I got this error:


ERROR: Could not find a storage controller named 'IDE Controller'
Details: code VBOX_E_OBJECT_NOT_FOUND (0x80bb0001), component Machine, interface IMachine, callee nsISupports
Context: "AttachDevice(Bstr("IDE Controller"), 0, 0, DeviceType_HardDisk, uuid)" at line 556 of file VBoxManageModifyVM.cpp


I did some research online and discovered the solution. My understanding of what’s changed here: the newest version of VBoxManage allows for vastly improved control and flexibility with regards to controllers. Introduced to VBoxManage are a couple options related to these new powers: storagectl and storageattach. The drawback to this new amount of control is that the old “default” controller configuration no longer exists - you need to create it yourself.

The solution I found no longer uses the “modifyvm –hda” command. Instead, it looks like this:


VBoxManage storagectl addievm --name "IDE Controller" --add ide
VBoxManage storageattach addievm --storagectl "IDE Controller" --port 0 --device 0 --type hdd --medium addievm.vdi


The IDE controller, which used to be available sans configuration, is now created and attached to the VM using the first command. The second command attaches the hard drive to the controller.

I’ll be revisiting these commands in my next post, when I look more closely at the configuration required for operating system install.

Three. Networking.

Networking has never been my strong suit, but with the VirtualBox GUI, getting the guest machine to use the host machine’s internet was as simple as a matter of clicks. This was harder to do on the command line. On the old install of VirtualBox that I was working with up until this week, configuring a bridged connection between the host interface and the guest interface required a lot of additional work, which is documented here (see the heading “Host Networking in Ubuntu 8.04 and older”). The particular thing that actually spurred the upgrade to newer versions of Ubuntu and VirtualBox was networking - adding additional bridge ports to the bridge interface didn’t fix the networking problems I was having, despite following the documentation word-for-word. I read that newer versions of Ubuntu / VirtualBox don’t even require the complicated setup of a bridge interface and several additional bridge port interfaces, and I was sold on upgrading.

Ideally, setting up your network then looks something like it did in the last post (in this case, “eth0″ is the default network interface on the host machine):


VBoxManage modifyvm addievm --nic1 bridged
VBoxManage modifyvm addievm --bridgeadapter1 eth0


… but when I started up my OS install (the FreeBSD bootonly disk downloads the rest of the install and therefore requires an internet connection), my internet wasn’t working. The solution was to install some missing modules:


sudo modprobe vboxnetflt


This is documented in the same place I referenced earlier, Ubuntu documentation: VirtualBox Networking.

…

Now that these tricky bits are documented, it’s time to move on to attaching an ISO to the VM to allow for OS install, and using VRDP to facilitate that install. That will be the next entry, and hopefully with fewer corrections and caveats now that I’m up-to-date on my software.

I am creating some new VMs on a fresh install of Ubuntu 9.10 (Karmic/Kreepy Koala), so now is a great time to extract and organize each step used to create a new VM on the command line, and double-check all my documentation in real time. Here we go.

As with my last post, the machine I’m creating in these comments is called “addie_vm” but can be replaced with the machine name of your choice.

I typically start by creating the virtual hard disk for my VM. To keep things from getting confusing, I always name the hard disk with the same name as my VM. Note that renaming virtual hard disks is MUCH harder than renaming a VM, so name carefully!

VBoxManage createhd --filename addie_vm.vdi --size 100000 --format VDI --variant Standard

Some notes: VDI is a standard virtualbox format and what virtual hard disks default to in the GUI. Size is specified in MB, so what I am creating here is a 100 GB hard disk. The “Standard” variant means that the actual space used on your physical hard drive will grow to a cap of 100 GB versus taking up those 100 GB from the get-go. If you’d like your virtual hard drive to take up its capacity on the physical hard drive, use the “Fixed” variant instead (This is a useful option if you’re planning very carefully about how the entire host system’s resources are being used).

Next, create the VM with some basic features configured. The VMs I am creating will have FreeBSD installed on them, so I run:

VBoxManage createvm --name addie_vm --ostype FreeBSD_64 --register

The –register arg allows you to skip a step; you won’t have to run VBoxManage registervm to get the system to recognize your new machine. If you’re using another OS, you’ll need to include a specific string specifying your OS type after the –ostype arg. You can see your options by running:

VBoxManage list ostypes

Now, glue two pieces together, as it were - attach your hard drive to the VM you just created.*

VBoxManage modifyvm addie_vm --hda addie_vm.vdi

At this point the “basics” are in place, but here are a couple of options I set before I go about installing the OS on the new VM (and I will explore some of these in greater detail later on):


VBoxManage modifyvm addie_vm --memory 1024


This command sets the RAM for your machine - the number following –memory should be in megabytes. The VM will be created with a default of 128 MB RAM, which is a bit on the small side, so adjusting it before you get to tinkering around with your machine is recommended. As with hard disk space, you’ll want to know the total RAM of your host system, and the total RAM of your other VMs, and plan accordingly. If too much host RAM is allocated, and the host system is being used for anything other than running VMs, you’ll run into some serious performance problems on your host machine.

Enabling networking is also important. Networking is one of my weakest areas knowledge-wise, and past versions of VirtualBox have made getting internet working on VMs pretty complicated, especially if you are using only the command line. If you’re using an older copy of VirtualBox (2.0), you’ll probably want to look elsewhere for documentation, starting with help.ubuntu.com’s documentation.

Luckily, things are easier in newer versions of VirtualBox. I use a bridged network connection, so the VM uses the same hardware as my host machine. To see the available bridged interfaces, run:

VBoxManage list bridgedifs

If you’re going with system defaults with regards to networking, you’ll probably be using “eth0″. Interface name in hand, you’ll then run:


VBoxManage modifyvm addie_vm --nic1 bridged
VBoxManage modifyvm addie_vm --bridgeadapater1 eth0


The first command configures your machine to use a bridged adapter. The second command points the adapter to the appropriate interface. The “1″ in –nic1 and –bridgeadapter1 specifies that this is the first adapter; you can configure multiple adapters if needed.

Finally, and I will revisit this later, I like enabling vrdp - the VirtualBox Remote Desktop Protocol - and assigning a port for the machine in question (the port is optional - VBoxHeadless will assign one for you if you don’t have one already).  Note that these options are only available in the closed source version of VirtualBox (unfortunately).


VBoxManage modifyvm addie_vm --vrdp on
VBoxManage modifyvm addie_vm --vrdpport 3390


To close, let’s trash my creation. Getting rid of a VM is actually somewhat tricky from the command line, because you must first detach the VM from its hard drive, and to make sure you aren’t cluttering up the VirtualBox configuration and leaving around garbage, you’ll want to unregister both VM and hard disk. Here’s the series of steps. Start with your VM powered off:

VBoxManage modifyvm addie_vm --hda1 none

This command disconnects the virtual hard drive from your virtual machine.

You can go ahead and unregister the hard drive and VM now:


VBoxManage unregistervm addie_vm --delete
VBoxManage closemedium disk addie_vm.vdi


You’re done with the VM now - thanks to the –delete flag, your VM is gone. The hard drive is still lingering though, and if you want to save disk space, you’re now in the position to safely delete it. Assuming your VirtualBox configuration lives in .VirtualBox under your home directory, you can remove it this way:


rm ~/.VirtualBox/HardDisks/addie_vm.vdi


… and that’s all for this entry. Next up: cloning hard drives so you don’t have to go through system setup on several machines in a row.

* If you’re using VirtualBox CSE 3.1.x, you may run into issues here - I have.   I’ll try to write about resolving them in a future post; if I do I’ll link to it in this aside.

This is the first of many entries I’m hoping to write on using VirtualBox on the command line (i.e. with the server edition of your favorite unix-based OS; in my case Ubuntu) after having about a year’s worth of experience using the GUI version of the software. I’m no expert with VirtualBox, but I’ve found that it’s been a tremendous aid with my work, second only to git’s branching in allowing me to switch rapidly between several projects at once with minimal overhead.

Documenting usage of the software on the command line is great territory for some blog posts too. I keep pulling out the same note-covered, four-page printout of the VBoxManage usage string, because extenuating circumstances keep requiring machine administration. Instead of adding to the mess note-wise, I figured I’d do a little cleanup of my documentation and summarize the commands I’m using the most often.

The two command-line programs I’m using to administer VirtualBox are VBoxManage and VBoxHeadless. VBoxManage is the be-all, end-all program for creating, deleting, and configuring VMs and all their components. VBoxHeadless is the program used to run VMs on a headless server (such as Ubuntu Server Edition, like I’m using). It’s the server-edition equivalent of using the program VirtualBox on the command line, which is the “standard”, GUI version (and what you’ll see in your process list if you’re running a vm on the GUI version). (If you try running VirtualBox on the command line with a headless machine, like I sometimes do when I’m feeling especially tired and not thinking straight, you’ll get X11-related complaints.)

I inherited some pre-configured vms with this server, so the first thing I needed to do was learn how to start and stop my machine. Starting a virtual machine is possible with VBoxHeadless or with VBoxManage; most of the documentation I read recommended VBoxHeadless for my use case.

Here’s how I start a virtual machine called “addie_vm” on the command line:

VBoxHeadless -s addie_vm &

-s is a shortened version of the option –startvm, which is hopefully self-explanatory. You follow it up with either the vm name or its UUID (a unique identifier for the VM that I use in other circumstances but is much less human-readable).

You will want to run the process in the background using the ampersand, or you’ll lose access to your terminal.

Here’s the rough equivalent with VBoxManage (it appears to start up a few additional processes along with VBoxHeadless):

VBoxManage startvm addie_vm --type headless

Running in the background isn’t needed in this case.

I typically power off my virtual machine using the command

VBoxManage controlvm addies_vm poweroff

This is obviously preferrable to killing the VM process, which is the equivalent of manually powering off or unplugging your physical machine. Which is not to say I haven’t had to do either on occasion.

Next entry I will document how I’ve been creating and removing virtual machines and their hard disks, something I’ve had to do a lot of in recent days.

Share in the comments if there’s anything about this documentation that is missing to the detriment of my system; there are so many options within a program like VBoxManage that it’s easy to miss details (or glaze over them when reading documentation).

Thanks to Code N Splode, I’ve known about the Geek Feminism wiki for awhile. It’s a really amazing resource - it concisely addresses the issues facing women in tech (specifically Free and Open Source Software, or FOSS) in a way that is knowledgeable and matter-of-fact. Nothing puts the air back into my lungs quite like seeing excuses for sexist behavior de-legitimized through a good bingo card. As a person who wants to give thought to every voice in discussion (counter-productive in the case of a lot of comment threads on gender in FOSS), it’s nice to have some shortcuts via the wiki: “This is an excuse, and this is why it doesn’t work”.

I was really delighted to see the wiki’s inevitable growth into a blog after this year’s OSCON (where Kirrily Robert, nee Skud, imho a woman in FOSS we are incredibly lucky to have for how gloriously well she handles the topic). I don’t read it regularly yet, but everytime I refer to the site I see intelligent, confident women addressing the issues of being a woman in tech or a woman in FOSS. Several times, women from my own community (Portland, OR) have appeared. It’s encouraging to see this community emerge online, to both establish presence and provide support to geek feminists on the margins (i.e. less involved, like myself).

In this year’s political atmosphere, we’ve seen our country’s latent unresolved issues with race and gender come out a bit more publicly, oftentimes in uglier ways than we still thought possible (I think specifically of Obama’s presidency and the Senate confirmation of Sonia Sotomayor to the Supreme Court). This racial / gender hiccup on society’s part has been simultaneously revolting and encouraging - encouraging because I’d like to think that those who thought there wasn’t a problem might be woken up and pushed to activism. I’d like to think that this year’s ugliness will have repercussions in coming years, to bring our discourse, beliefs, and behavior to a higher plane.

The Open Source community has been following a similar plotline in regards to re-opening one of its ugliest wounds - the gender inequity issue. I really thank the Geek Feminism community for documenting the List of Incidents from this year and years past. Taken together, these incidents reveal that tech and especially FOSS have a much longer way to go than we could have anticipated even a year ago.

The incidents themselves are only one part of the problem - the bigger issue is typically the fallout, where the backlash against those who call out sexist behavior is astounding. Anytime I read up on these incidents, I’m typically derailed for a good day or so by fist-clenching anger and disbelief. I’m incredibly discouraged to find out that, when a subset of men within the community are called upon to explore their feelings on these issues, they come up on the wrong side of the fence. I’m disheartened that much of the software I use on a daily basis - be it in the workplace or at home - is made by some people I probably wouldn’t want to associate with in “real life”.

This isn’t just a gender issue, either. My parents raised both of their children to be adults and professionals, which meant a certain level of conduct not just in the workplace, but in adult society. Most of adult society has safeguards in place to punish or discipline those who aren’t able to follow that level of conduct - but the FOSS community (and tech in general, as I’ve found through my three years in industry) has revealed itself to be shockingly free of a lot of those safeguards. People who have never learned to behave like adults seem to run wilder in the land of tech, and even those who’ve been chastised for it seem to be slower to learn their lessons, if ever. It makes for a gut-churning contrast to the more civilized parts of our society.

These incidents - and the fallout afterwards - bring forth a LOT of complicated feelings for me, things I will be unlikely to blog about coherently much of the time. And this is one of the biggest reasons I am grateful for Geek Feminism, because I finally have a pool of references to people who are saying what I think and feel, and creating a growing community of likeminded individuals.

I expect that I will be sharing many a blog post from Geek Feminism and entries from other sites that it links to. Not just to voice my support, but to continue to add to the dissemination of this discussion to the community at large. The people who voiced their opposition to sexist comments and behavior in the FOSS community this year did the right thing, despite the backlash. Consider me another person saying “hear, hear!” and compelling my peers to do likewise.

My ability to get a technical blog going has been hampered by my inability to stick to any habit lately; it’s something that I’m working on, but I think the trickiest thing is identifying the habit I want to train myself into that’s the biggest priority (the queue is about 20+ “new habits to adopt” long). Try to do too many at once and you’re doomed to fail.

Tumult in my work and personal life has of course added to my inability to focus technically. In particular, I haven’t been doing a lot of coding since I last posted (in May). Instead, I’ve been doing a lot of internationalization work on a very nuts-and-bolts level; climbing full-bore into the creepy abyss that is the overlap between sys work and developer work; taking on the interpersonal drama that is characteristic to any developing job but rarely makes its way into the job description.

I’d like to say I have some interesting insights on internationalization work or on adapting a git-based deploy infrastructure to a multi-system scale, but as I’m still mucking about I don’t think my thoughts are all there.

I’ve hardly written code in the last few months, let alone this year at all, which probably has a lot to do with my enthusiasm for tech being so unfocused. I finally had the opportunity to crack into some Perl code this weekend, which - it being Perl - only reminded me of how quickly my mood can plummet from 100 to zero looking at the stuff. I’d like to overcome the gag reflex, though, so I’ve tried to persist - currently reading the “Symbol Tables and Typeglobs” entry in Mastering Perl, and digging around a bit in Perl Design Patterns.

Right now, my biggest issue with using object-oriented design patterns in Perl is how ugly the implementation is in comparison to other OO languages (it doesn’t help that Perl is an OO language as an add-on). I’m finding that grasping these concepts requires digging into additional advanced concepts that can be really daunting. I understand that the point of design patterns is for code readability / maintainability, but if advanced Perl mastery is required to make sense of code that uses these patterns, then only those initiated to such practices will actually find the code readable. At work, only a small portion of our code base uses some of these concepts, so my hope is to augment this code with comments directing our developers to the documentation where these concepts are covered. In a world where we all have pre-existing advanced knowledge, leaving notes like “the concept being used here is x” is probably redundant, but that seems to be unrealistic. Finding a happy medium between elegant code and clues for those who don’t yet grasp the voodoo seems to be an important, albeit tricky, step.

Socially, recently, the big tech community events have been:

• Open Source Bridge in June - met and interacted with tech friends old and new; delighted myself by discovering that I actually found a lot of the tech talks super interesting, not just the social ones (until now, I don’t think I had enough of a technical foundation in industry to take on a new topic out of the blue). I also happened to meet our newest dev hire at work for OS Bridge, and so it’s been beneficial on a workplace level too.
• I presented at Code N Splode for the first time in August - on puzzles, coding competitions, and puzzle hunting - the edges of the geek world that have drawn me more than free-time coding projects (although I’d like to do more of those).
• I competed with three other local programmers (Maria, Reid, and Igal) in the Playdash event, a puzzle hunt put on by now-local puzzlehunting aficionados Team Snout. Looking forward to seeing Portland become a puzzlehunting city.

Now that I feel like I’ve caught up a bit - hopefully I’ll try to get back into the habit by throwing out some small bits and pieces - in particular, “RTing” things that I’ve found interesting on other tech blogs, on the days I happen to stumble upon them.

In the meantime, I have unpublished my long rambling post from post-barcamp, mostly because I was embarassed at how long and rambling it was.  I am aiming to be more like the fantastic female tech bloggers I get to see online every day, and they’re pretty good with saying it right concisely.

Thanks to Curtis and Erik for pointing out that the nonsense text I was just happily overlooking at the bottom of my blog posts was actually a php error signaling that my wordpress configuration was broken. BarCamp apparently really wiped me out, because I slept 12+ hours, and debugging in that post-sleep haze was not pleasant, but I think I have a fix in place (one that requires a change every time I change my theme - as a programmer this makes me cranky!).

So comment away, but please be kind, because I think I was sleep-addled enough last night where every point I made inspired 3 more points, and it was pretty hard to keep it under control. Not to say I have trouble enough with that when I’m well-rested. But I’m building confidence with this whole tech blog thing right now and would like my blog posts to be slim and sexy - last night’s was not, but I hope it is a gem nonetheless, one I can keep there without feeling too self-conscious.

So, there’s a great initiative afoot this year to make a bit of a public splash in the interwebs via the Ada Lovelace Day Pledge, which is:

I will publish a blog post on Tuesday 24th March about a woman in technology whom I admire but only if 1,000 other people will do the same.

Cool thing: the pledge is 600+ people over the limit as of the time of this post.  I haven’t even formally made the pledge yet (I suppose I’ll do it simultaneously while writing this, since I’m going ahead with the deal) because I didn’t know how much of my tech blog would be up and running by the day - and although this blog is barely crawling, this is good fodder for getting started.

I really can’t speak personally about one woman in technology specifically.  Much of my experience as a female programmer is probably symptomatic of the experience at large, which is to say that all of my best programmer pals are men and although I have deeply craved a mentor / friend who is female to help me through the constant struggles, the pickings are slim.  Just because you are the only women in X group doesn’t mean that you are instantly best friends for being women.

Although I do want to acknowledge that impulse on my part.  Suw’s blog post about this initiative mentions:

But what to do? Well, let’s pull back from the anger a little, and start to look instead at why it might be that women feel less secure in their abilities than most men, and what might help change that. Undoubtedly it’s a complex issue, but recent research may shed some light: Psychologist Penelope Lockwood discovered that women need to see female role models more than men need to see male ones.

Throughout my history as a female programmer I have ached for female role models, and it’s required quite a bit of restraint to mask my desparation for such things.  I have the feeling many of those I have wanted to serve as my role models had no idea that I looked up to them.  Of course, that’s the whole point of this movement.  Acknowledging the importance of role models and our own potential to be those role models, even if we don’t see ourselves as “good enough” or in the proper position to fulfill that role.

Identity has been a major struggle for me as a programmer.  It strikes me that the bulk of most varied, interesting, fascinating people I have met are programmers, and yet in the same stroke I feel like the space for my own person within the technology personality-space is extremely restricted.  How does this culture draw in such a varied array of absorbing, brilliant people, but still as a whole firmly reinforce its most negative stereotypes as a socially-bumbling, misogynist, difficult population?  This is a discussion for another time, but I think it ties in importantly: my greatest grief and greatest joy as a programmer have related to group identification.

The positive in group identification: In late college, when I met some of my best friends in the world (yes, programmers) through interning at the Southern California Earthquake Center, I delighted in the ways my geekitude matched up with that of my friends.  I wore my identification as a programmer like a badge of honor, and delighted in talking shop with tech friends and seeing the look of pure bafflement on the faces of non-tech friends in the room.  There is an immense amount of my personality that is satisfied by being a programmer.

Unfortunately that group identification has existed during the minority of my time as a programmer.  Sometimes it was a stubborn, nearly prideful “I’m not like these people,” like when I took programming classes in high school (and inevitably I had to eat my words a bit when some initially offputting programmer turned out to be a delight).  Mostly it was mournful, unnerving “I don’t belong here,” due to not seeing any aspect of myself in classmates, co-workers, and peers.  This feeling of being a square peg in a round hole lasted for the majority of my college career and only intensified when I entered the industry, completely overwhelming me several times in the last two years.  It’s only recently that I’ve begun to start feeling my way around and acknowledged my legitimacy as a member of the tech community, whether or not I can identify with those I see around me at all times.  As I mentioned in my introductory posts, I’ve finally come to terms with my legitimacy even though I don’t have the raw numbers of “people exactly like me” to help me be more comfortable with it.  “It all starts with you”, and other such pithy sayings.

To wrap this up, I do want to acknowledge the women who have had a positive role in my time as a programmer.  Women are so rare in tech that nearly every one I’ve encountered has served as a role model.  I think a lot of my lack of confidence in embracing them in that role - or asking them to actively serve in that role - is because I’ve seen a tremendous number of women who are hesitant to emphasize their gender in a professional context.  I’ve seen enough of these successful women expressing their hesitance to reach out in a gender-specific way that it’s made me scared to ask them for help.  But even when I have not been able to ask people for help they often still have influence on me as an observer.

@ School:

The Computer & Information Science program at the University of Oregon had four female faculty when I attended, which is an incredibly high number (especially since the gender ratio among undergrads was something like one female student to every twenty males.  Yes, I was often the sole woman in my classes).  This meant that I took higher-level courses from three different women, which was an incredible blessing.  My computer architecture professor, Dr. Jan Cuny, left for a position at the NSF, helping with diversity efforts, not long after I took her class.  I’ve seen her give a few rockin’ speeches at conferences like Tapia and Grace Hopper.  Meanwhile, my thesis advisor, Dr. Sarah Douglas led her classes and my thesis process with a disciplined, incredibly knowledgeable hand.  I received some of the greatest bang for the buck from the three classes I took from her,  and she helped me immensely with my senior thesis, instructing me in both the areas where I didn’t need to stress out so much, and the areas where I needed to shape up and do better-quality work.

@ Work:

Female co-workers doing the same job as me can be counted on two hands for all three companies I’ve worked for in the two and a half years I’ve been in industry, but they’ve all had a positive impact (and I’m especially happy that I’ve stayed in touch with those I worked with in my first Portland job). Nori, from my time at the big G, was the only woman on my 30-person team when I joined, so just by circumstance I looked up to her.  Beyond just that mere circumstance, though, she established herself as and remains one of my biggest girl geek crushes, and I continue to admire the incredible presence she creates, even if from afar.  Since returning to Portland, I met the best mentor-coworker I’ve had to date, Amy, who continues to impress pretty much everyone with her ability to go above and beyond in the work she does, and always took the time to give a thorough and considerate response to every question I had.

In the Community:

And finally, the group that I found out about Ada Lovelace Day through, Code N Splode. They conveniently have a meeting tonight!  Not only have Code N Splode meetings been a great way to keep in touch with former coworkers (aforementioned), but they’ve exposed me to some of the local developer community’s strongest voices, including some incredibly active and passionate women.  It has been so encouraging to see local female tech types who are passionate about technology and helping women’s representation in the community / increasing the confidence of the women who are already there.  And I can say that just watching them in action giving news updates at meetings, presenting a technical speech, or just chewing the fat at the “cd /usr/local/gin” which follows has had a tremendous amount to do with my own sense of daring-do increasing in recent months.  Most blatantly this blog.  I’m looking forward to continuing to cultivate my new girl geek crushes with this group - they’re a living example of the positive things you can do as a female programmer when you care about the problem and then make a point of doing something about it - even if it’s as simple as acknowledging that you may be a positive influence or unintentional role model for another female programmer.

I could go on and on (orgs like Anita Borg, DevChix, Systers, etc. - my SCEC internship - diversity-themed conferences) - but it’s late.  The only reason I’m up at 3 am on a Monday night is because I took an extended nap with my freshly-returned-from-the-hospital cat earlier (she got into some lily pollen, which can cause renal failure).  And, time aside, I find myself feeling increasingly nervous as my blog posts start to (inevitably) grow.  Because, although I have a lot to say on these topics, I’m not yet fully confident expressing it to an undefined crowd.  But in the spirit of this day and the other people following suit in fulfilling their pledges, I’m hoping that this entry is of some value to another female tech type at some other point in her journey.

Inevitably, lots more on women in technology later.  When you’re the only female in the crowd (or one of just a few) for as long as I have been, and your brain goes off all sorts of analytical directions while observing such trends, you end up developing a bit of an obsession.  Some women want to de-emphasize their gender in professional terms, but I can’t help but focus on it.  It is not the only symbol of my feeling different from the rest of the tech community, but it is the most blatant symbol - the others are obviously a lot more subjective and fuzzy.  And the time has come to work towards celebrating being different in this community - my unique voice - because isn’t that the point of all these diversity efforts in the first place?