Subversion hosting, CVS hosting, Trac hosting, Bugzilla hosting and software collaboration Providing hosted Subversion, CVS, Trac and Bugzilla repositories

June 10, 2008

New FreeBSD “Work In Progress” Mailing List

Filed under: Misc — Tags: — Greg Larkin @ 1:51 pm

Hi all,

I just got a message on the FreeBSD-hackers mailing list about a new mailing list where “works in progress” will be detailed: FreeBSD-wip-status.  By subscribing to this mailing list, you will keep up with the cutting edge of FreeBSD development and learn about the many ways that FreeBSD is used out in the world.

Here’s an excerpt from the announcement:

Hello everybody,

Subscribe today!!  Tell all your friends, co-workers and everybody you
know who’s interested in FreeBSD.  Post this on your blog! ;)


if you’ve been around lately you might already know about our effort to
provide periodical status reports about ongoing FreeBSD related WIPs.
The past reports can be found at:

This system has some shortcomings however.  The biggest one is that due to
the collection overhead and the high rate of exciting development under
the FreeBSD umbrella most of the reports are somewhat outdated by the
time they get to “print”.  Other projects never are announced in the
status reports because they happen in-between and fall through the

In order to improve this situation we have created a new mailing list that
should receive status reports from now on.

The full announcement can be found here:

Bookmark and Share

Keep in touch,
Greg, LLC

Call me - Greg Larkin: Offline

May 8, 2008

Getting ElasticDrive to Work Under FreeBSD

Filed under: Operating Systems — Tags: , , — Greg Larkin @ 2:38 pm

Hi everyone,

I recently learned of the ElasticDrive application that implements a disk device backed by Amazon S3 and other kinds of storage services. To determine whether I could easily deploy an infinitely-scalable disk device on my FreeBSD servers, I downloaded ElasticDrive and proceeded to install it on a FreeBSD 7.0 virtual machine.

There are a few prerequisites, such as FUSE and Python setuptools, but those are easily installed with the FreeBSD ports system. If I can get ElasticDrive to work, it will be handy to package it as a port for one-command installation.
In the mean time, I installed it manually and edited the /etc/elasticdrive.cfg to point to my Amazon S3 account. I reduced the size of the S3 backing store to a 100Mb disk:

#Path to fuse is the path to the device mount folder. NOT your fuse libraries

After I fired up the tool, I saw my ElasticDrive file:
fbsd70# ls -al /mnt/elasticdrive
total 102403
drwxr-xr-x  2 root  wheel          0 Dec 31  1969 .
drwxr-xr-x  4 root  wheel        512 May  7 13:36 ..
-rw-r--r--  1 root  wheel  104857600 Dec 31  1969 ed0
-r--r--r--  1 root  wheel        391 Dec 31  1969 stats
fbsd70# du -sh /mnt/elasticdrive
100M    /mnt/elasticdrive

Cool – this looks promising!

FreeBSD uses the mdconfig command to create a file-backed loop device, similar to the Linux losetup command. This is too easy:

fbsd70# mdconfig -a -t vnode -f /mnt/elasticdrive/ed0 -u 0
fbsd70# ls -al /dev/md0*
crw-r-----  1 root  operator    0, 101 May  8 12:32 /dev/md0
crw-r-----  1 root  operator    0, 106 May  8 12:32 /dev/md0a
crw-r-----  1 root  operator    0, 107 May  8 12:32 /dev/md0c
fbsd70# fdisk /dev/md0
******* Working on device /dev/md0 *******
parameters extracted from in-core disklabel are:
cylinders=12 heads=255 sectors/track=63 (16065 blks/cyl)

parameters to be used for BIOS calculations are:
cylinders=12 heads=255 sectors/track=63 (16065 blks/cyl)

fdisk: invalid fdisk partition table found
Media sector size is 512
Warning: BIOS sector numbering starts with sector 1
Information from DOS bootblock is:
The data for partition 1 is:
sysid 165 (0xa5),(FreeBSD/NetBSD/386BSD)
start 63, size 192717 (94 Meg), flag 80 (active)
beg: cyl 0/ head 1/ sector 1;
end: cyl 11/ head 254/ sector 63
The data for partition 2 is:
The data for partition 3 is:
The data for partition 4 is:

Ok, let’s create a filesystem and test it out:
fbsd70# newfs /dev/md0a
/dev/md0a: 100.0MB (204784 sectors) block size 16384, fragment size 2048
using 4 cylinder groups of 25.00MB, 1600 blks, 3200 inodes.
super-block backups (for fsck -b #) at:
160, 51360, 102560, 153760
cg 0: bad magic number

Woops – what’s this now?! Looking at the newfs source code, this error is generated when data read back from the disk doesn’t match data written to the disk.

I’ve tried to figure out what the problem is using truss and other techniques, and I’ve contacted ElasticDrive to see if they have any ideas. So far, nothing has worked.

If anyone else has an idea how to troubleshoot this, let me know.

Bookmark and Share

Keep in touch,
Greg, LLC

Call me - Greg Larkin: Offline

April 26, 2008

Who’s Going To BSDCan 2008?

Filed under: Operating Systems — Tags: , , — Greg Larkin @ 4:15 pm

Hi everyone,

I just registered for the BSDCan 2008 conference in Ottawa, ON, Canada on 5/16 and 5/17. I’m really looking forward to meeting some of the folks that I’ve exchanged emails with over the past couple of years while learning and using FreeBSD and maintaining some ports.

Is anyone reading this post going to be at the conference, too? If you want to meet up, drop me a line and let me know. I hope to leave the conference with a lot of new FreeBSD knowledge and hear about how others use it and what to expect in future versions.

I’m also just looking forward to getting back to Ottawa. We’ve mostly visited there in the winter for the Winterlude and cross-country skiing at Parc Gatineau. If you enjoy freezing your tail off in mid-February (-30F at night not unheard of!) and skating on the Rideau Canal with 50,000 other fun-loving souls, you’ve got to visit!

I imagine that the warmer months will be very nice as well, and I do hope to see some of the Tulip Festival while I’m there.

Bookmark and Share

Keep in touch,
Greg, LLC

Call me - Greg Larkin: Offline

April 20, 2008

Updated FreeBSD 7.0 VMware Image Now Available

Filed under: Operating Systems — Tags: , , — Greg Larkin @ 4:22 pm

Hi everyone,

I updated the ports tree and repackaged the FreeBSD 7.0 VMware virtual machine that turned out to be corrupted in my earlier release. I apologize for anyone who was inconvenienced, and I still don’t know how part of one of the disks was corrupted, since the source image was fine.

Anyway, you can find the new download here: VMware 7.0 virtual machine with ZFS enabled

If you run into any problems, please contact me.

Bookmark and Share

Keep in touch,
Greg, LLC

Call me - Greg Larkin: Offline

April 18, 2008

Update: Need Help with VMware Workstation Running on Microsoft Vista

Filed under: Operating Systems — Tags: , — Greg Larkin @ 5:36 pm

Hi everyone,

In a previous post, I asked for assistance about getting a FreeBSD 7.0 virtual machine running on VMware Workstation on MS Windows Vista. Well, it turns out I didn’t need any help after all – the torrent file was corrupted!

Somehow, one of the 2Gb slices of the VM’s boot disk was truncated to ~1.4Gb. I don’t know how it happened – perhaps during the ZIP compression. It’s very odd, because the rest of the slices were fine.

Anyway, I’m preparing a new torrent, and I’ll post the link when it’s ready.

Bookmark and Share

Keep in touch,
Greg, LLC

Call me - Greg Larkin: Offline

Routing Between Virtual Machines on Separate Physical Servers

Filed under: Operating Systems — Tags: , — Greg Larkin @ 5:30 pm

Hi everyone,

A while back, I was setting up some virtual machines, each with a public and host-only network interface. I like using the host-only interface for high-performance VM-to-VM communications and then to expose the WAN IP to the Internet for public access.

I soon ran into a problem, though. What happens if a VM on one physical server needs to make an internal connection to one on another physical server? The first VM has to use the public IP address of the second VM, but I still want the connection to be essentially private and not route the packets outside of the firewall-protected LAN.

Here’s a diagram showing the VMware Server configuration with virtual machines and host-only networks (click to enlarge):

Network Diagram in VMware Server Environment

For the following examples, let’s use these IP addresses for the physical servers and virtual machines:

Hostname WAN IP LAN IP Gateway IP

Notice that the LAN IPs are repeated for VMs on different physical hosts. That’s because the VMs are connecting to a host-only network on each physical machine and are independent of each other. Therefore, they can use the same IP numbers without conflicting.

Ok, so assume VM1 is a public-facing web server, and VM4 is a database server with no external services exposed except for ssh. Since there’s no way for VM1 to use VM4′s LAN IP address to connect to the database – the IPs are the same – VM1 has to use VM4′s WAN IP.

Let’s now trace where a packet as it originates from VM1 bound for VM4. This table shows the path that the packet takes, along with the source address as it passes through each hop.

Hop # Hostname Source Addr Dest Addr
1 VM1
2 PH1
3 GW1

Woops – the packet has made it out to the gateway router (GW1)! That’s not what we want, because the packets exchanged between any VM should stay on the inside of the firewall. The same problem occurs if any VM on one physical machine connects to a VM on the other physical machine.

Luckily, this easily fixed by adding some static routes to each physical machine to help guide the packets where they need to go. Instead of allowing packets bound for a VM to use the default route and get sent to the gateway router, a static route redirects them to the next hop that we specify.

On RHEL4, the /etc/sysconfig/static-routes file controls any required extra routes. On PH1, the file will look like:

any host gw
any host gw
any host gw

The file on PH2 looks like:
any host gw
any host gw
any host gw

This simply means “when sending a packet to host X (e.g., first send it to host Y (e.g., because it has the information about how to get there”.

Once the static routes are installed, the packet trace now looks like:

Hop # Hostname Source Addr Dest Addr
1 VM1
2 PH1
3 PH2
4 VM4

Of course, this can be extended across many virtual machines and physical machines, and it always helps to trace the packet and the network routes at each hop to determine where it’s going next. The traceroute command is your friend!

Bookmark and Share

Keep in touch,
Greg, LLC

Call me - Greg Larkin: Offline

April 15, 2008

Keeping VMware Management Log Files Under Control – Take 2

Filed under: Operating Systems — Tags: , , — Greg Larkin @ 7:12 am

Hi everyone,

In a previous post, I postulated a solution for rotating the Apache log files produced by the VMware MUI application. Unfortunately, the MUI version of Apache doesn’t work well with logrotate, as I discovered over the weekend.

After receiving an alert on the cell phone in the middle of the night that the MUI was down, I discovered this in the /var/log/vmware-mui/error_log file:

[Sun Apr 13 04:02:03 2008] [error] Could not destroy the stale named pipe directory: /var/run/vmware//httpd/21157\n
[Sun Apr 13 04:02:03 2008] [error] VMWARE PANIC: \nNOT_IMPLEMENTED F(4023):707\n
[Sun Apr 13 04:02:03 2008] [error] Panic: Could not allocate temporary context.\n

Ok, perhaps there’s a better solution than logrotate and one that I’m already using with Apache in all of the virtual machines. A while back, I discovered the httplog tool, which builds on the premise of the standard rotatelogs tool that’s part of the Apache distribution. Since it doesn’t restart Apache when it rotates its log files, I suspected it might work well with the MUI.

Since the MUI is running on a RHEL4 system, I created an RPM of httplog instead of installing directly from the source distro. After many years working with Linux and FreeBSD, I much prefer everything to be installed from an RPM or a port, just for sanity’s sake and managing future upgrades.

The next step is to integrate the httplog command into the MUI’s httpd.conf file, found in /usr/lib/vmware-mui/apache/conf. If you’d like to update your file, download this patch file, and run the following commands (after installing httplog):

rm -f /etc/logrotate.d/httpd.vmware
/etc/rc.d/init.d/httpd.vmware stop
patch -b --verbose  /usr/lib/vmware-mui/apache/conf/httpd.conf < httpd.conf.httplog-patch
/etc/rc.d/init.d/httpd.vmware start

After starting up httpd.vmware, your /var/log/vmware-mui directory should look something like this:
total 16
lrwxrwxrwx  1 root root   36 Apr 14 17:12 error_log -> /var/log/vmware-mui/200815-error_log
-rw-r--r--  1 root root  186 Apr 14 17:12 200815-error_log
lrwxrwxrwx  1 root root   41 Apr 14 17:12 ssl_engine_log -> /var/log/vmware-mui/200815-ssl_engine_log
lrwxrwxrwx  1 root root   42 Apr 14 17:16 ssl_request_log -> /var/log/vmware-mui/200815-ssl_request_log
lrwxrwxrwx  1 root root   37 Apr 14 17:16 access_log -> /var/log/vmware-mui/200815-access_log
-rw-r--r--  1 root root  396 Apr 14 17:31 200815-ssl_request_log
-rw-r--r--  1 root root 3820 Apr 14 17:31 200815-ssl_engine_log
-rw-r--r--  1 root root  328 Apr 14 17:31 200815-access_log

I’ve got my httplog process configured to encode the week number into the Apache log filenames, rotate them each week and compress the rotated logs. If you want to automatically remove compressed logs after a certain number of weeks, a simple cron job accomplishes that.

httplog accepts many different filename configurations, so man httplog for more information or post questions here.

Bookmark and Share

Keep in touch,
Greg, LLC

Call me - Greg Larkin: Offline

April 14, 2008

Small Tweak To Get Httplog To Build On RHEL4

Filed under: Operating Systems — Tags: , — Greg Larkin @ 5:32 pm

Hi everyone,

In a future post to follow very shortly, I’ll discuss a change to the VMware MUI application that relies on the httplog tool to perform Apache log rotation.

The first thing I had to do was get httplog installed on the RHEL4 system here. I quickly discovered that the source distribution wasn’t too good at detecting the proper version of zlib and configuring the Makefile accordingly. If you don’t have exactly zlib v1.1.3, the configure script assumes that you don’t have it at all. If you have a later version of zlib (like RHEL4 does), you have to do some hand-editing of the source code to build the tool.

I found an httplog SRPM online, but the .spec file wasn’t quite up to snuff. I added some dependencies and tweaked the configure script a bit. If you just want to install the tool, download the httplog RPM, or you can make additional tweaks with the httplog SRPM.

Bookmark and Share

Keep in touch,
Greg, LLC

Call me - Greg Larkin: Offline

April 10, 2008

Need Help with VMware Workstation Running on Microsoft Vista

Filed under: Operating Systems — Tags: , — Greg Larkin @ 11:02 am

Hi everyone,

A couple of weeks ago, I made a FreeBSD 7.0 + ZFS VMware image torrent available on the BitTorrent Tracker. A reader of this blog contacted me a couple of days ago about a boot problem he’s having with the image, and I was wondering if anyone out there can reproduce it and/or provide a solution? Here are the particulars:

  • Host OS: Microsoft Windows Vista
  • VMware Workstation version 6.0.3 build – 80004
  • FreeBSD 7.0 + ZFS VMware image (torrent)

Initially, it appears that the VMware image ZIP did not unpack correctly, because this error is displayed:

VMware Workstation Error Message

However, checking the directory where the ZIP file was unpacked shows that the file does exist:

Virtual Machine Directory Listing

I tried to debug the problem from here since I don’t have Vista installed anywhere, but nothing has worked yet. He sees the same problem with VMware Player, too, so I wonder if it’s something to do with Vista permissions or some other security setting.

If someone has an idea what the problem could be, let me know.

Bookmark and Share

Keep in touch,
Greg, LLC

Call me - Greg Larkin: Offline

April 7, 2008

Keeping VMware Management Log Files Under Control

Filed under: Operating Systems — Tags: , — Greg Larkin @ 2:01 pm

Hi everyone,

I recently upgraded the production servers to VMware Server 1.0.5 and also upgraded the VMware MUI package. The MUI (Web-based Management Interface) is useful when you need to restart a VM, reallocate VM memory and perform other maintenance tasks, but you don’t have access to the VMware Server Console or VirtualCenter.

The MUI is driven by Apache 1.3.31, and as such, it generates the familiar log files:


However, after a while, the log directory tends to fill up:
# ls -larS
total 78988
drwxr-xr-x  14 root root     4096 Apr  7 11:58 ..
drwxr-xr-x   2 root root     4096 May  9  2007 .
-rw-r--r--   1 root root    53985 Mar 24 09:20 error_log
-rw-r--r--   1 root root  8280230 Apr  7 12:59 access_log
-rw-r--r--   1 root root  9955524 Apr  7 12:59 ssl_request_log
-rw-r--r--   1 root root 62473978 Apr  7 12:59 ssl_engine_log

Ok, it’s only 78Mb so far, but why wait until the logs fill up the disk? Since the VMware Server host is running RHEL4, it came pre-installed with logrotate, and an existing configuration for the standard Apache log rotation can be easily adapted for the VMware Server MUI. Just place the following in /etc/logrotate.d/httpd.vmware:
/var/log/vmware-mui/*log {
        /bin/kill -HUP `cat /var/run/ 2>/dev/null` 2> /dev/null || true

The default settings in /etc/logrotate.conf also take effect during rotation and you can enable log file compression and length of retention in there.

Bookmark and Share

Keep in touch,
Greg, LLC

Call me - Greg Larkin: Offline
Pages: Prev 1 2 3 4 5 6 Next

Powered by WordPress