VirtualBox serial console on Mac OS X

I normally access a VirtualBox VM serial console through minicom in GNU/Linux, but for some reason it didn't worked on Mac OS X, so I researched about how can I access the serial console.

VirtualBox (and VMware for what I saw on the pages [1]) map the serial device of the virtual machine to a "Named Pipe" (actually a UNIX Domain Socket) which can be accessed using netcat or minicom. I tried the nc -U variant as stated in the page but I had no luck making it work because the mac ports version of netcat does *not* support attaching to UNIX domain sockets [2].

The screen man page [3] describe how to attach to a existing tty device but the file is a named pipe so the program cannot do its magic with it. There were some pages describing how to use socat [4] to map a UNIX domain socket to a tty device (actually a PTY) [5] [6] [7]. I also found a couple VMware forum posts with useful links [8] [9].

Thanks to this I can happily run the following two commands to attach to the serial console

% socat -d -d ./Thesis.ttyS0 PTY
2012/05/29 01:08:00 socat[29713] N opening connection to LEN=16 AF=1 "./Thesis.ttyS0"
2012/05/29 01:08:00 socat[29713] N successfully connected from local address LEN=16 AF=1 ""
2012/05/29 01:08:00 socat[29713] N successfully connected via
2012/05/29 01:08:00 socat[29713] N PTY is /dev/ttys007
2012/05/29 01:08:00 socat[29713] N starting data transfer loop with FDs [3,3] and [4,4]
In this case the desired PTY is /dev/tty007 and then in another terminal window

% screen /dev/ttys007

I wanted to do this as painful (automated) as possible, but there were a few problems:

1. The PTY device allocated on Mac OS X is subject to the number of terminals currently being used, so it is a variable device.
2. I have to run two commands in order to get the PTY and attach to it (socat, then screen)
3. Simple shell magic can't work because socat outputs the messages to stderr.

I decided to do some shell magic to automate the task in several steps

1. Map the UNIX domain socket to a PTY
2. Somehow, get the PTY name (the tricky part)
3. Once the name is known, attach to the PTY

Map the socket to a PTY

Same as above, still the messages are output to stderr

socat -d -d ./Thesis.ttyS0 PTY

Get the PTY name

The device name is printed to stderr and simply by doing a 2>&1 and pipe it through a sed or awk instance will do the trick. NO!!!, for some reason (still unknown to me) sed and grep got stuck and even if they got the apropriate input they didn't send anything to the screen.

I managed to solve the problem by redirecting the socat output to a file (which might also be a FIFO if you are interested), and then pointing grep to get the desired line and piping that output to sed to clean the text and get the desired PTY name.

Making it work altogether

Since the device is dynamically allocated and the socat output may vary, this kind of magic can be done, yes it might also be done with an sed [10] or awk script [11], but there was 2 or 3 AM and I just made it work.

screen `socat -d -d ./Thesis.ttyS0 PTY 2>&1 | tee /tmp/x &>/dev/null & grep '.*N\ PTY\ is\ ' /tmp/x | sed -e 's/.*N\ PTY\ is\ //g'`

So the above line does the following:

1. Map the socket
2. Two choices here, I chose the first one because was faster but YMMV.
1.  Use 2>&1 | tee /tmp/x &>/dev/null and redirect the output to a file and optionally to the terminal (I did it for debugging but wasn't interested in keeping it).

socat -d -d ./Thesis.ttyS0 PTY 2>&1 | tee /tmp/x &>/dev/null & grep '.*N\ PTY\ is\ ' /tmp/x | sed -e 's/.*N\ PTY\ is\ //g'


1. Use &> /tmp/x & sleep 1 to give time to socat to write in the file and then read it to get the device name.

socat -d -d ./Thesis.ttyS0 PTY &> /tmp/x & sleep 1 ; grep '.*N\ PTY\ is\ ' /tmp/x | sed -e 's/.*N\ PTY\ is\ //g' 


3. Attach to the screen
   screen `all the above thing in backquotes to execute it before`

Again the final one-liner

screen `socat -d -d ./Thesis.ttyS0 PTY 2>&1 | tee /tmp/x &>/dev/null & grep '.*N\ PTY\ is\ ' /tmp/x | sed -e 's/.*N\ PTY\ is\ //g'`

--

  = ^ . ^ =

[1] http://wiki.networksecuritytoolkit.org/nstwiki/index.php/Console_Output_and_Serial_Terminals

[2] http://fixunix.com/slackware/537945-nc-does-not-support-unix-domain-socket.html

[3] http://linux.die.net/man/1/screen

[4] http://www.dest-unreach.org/socat/

[5] http://www.linuxsmiths.com/blog/?p=312

[6] http://blackmagic02881.wordpress.com/2007/02/05/linux-serial-console-how-to-with-vmware-server/

[7] http://thewayeye.net/2009/december/4/connecting-virtual-machines-serial-console-os-x-and-vmware-fusion[8] http://communities.vmware.com/thread/33528

[9] http://communities.vmware.com/thread/28508

[11] http://linux.die.net/man/1/sed 

[10] http://linux.die.net/man/1/awk

Update: I ported the script to make it work with Linux, check out the new post for details and also the @Github gist.
--
= ^ . ^ =