Removing a disk from a RAID 1 array

I’m in the process of building a new NAS server to replace our existing one which is getting a bit long in the tooth. As usual I’ve opted to install Ubuntu Server LTS as the OS. In my current NAS I’ve got the system partition RAID 1’d across all the drives, this means that should one fail I’ve still got 4 copies.

Setting up the new NAS I’ve put the system partition on a separate SSD from the main data drives. I’ve currently only got the one SSD but I wanted the option of RAID 1’ing the system drive for redundancy should I decide to get another SSD in future.

The Ubuntu installer will quite happily let you create a RAID 1 MD device with 2 disks but only select a single drive to belong to it. This would be the equivalent of the following mdadm command:

mdadm --create /dev/md2 --level=1 --raid-devices=2 /dev/loop1 missing

However, when you boot the OS will consider that array degraded:

mdadm -D /dev/md2
/dev/md2:
Version : 1.2
Creation Time : Fri Jan 16 21:24:44 2015
Raid Level : raid1
Array Size : 32704 (31.94 MiB 33.49 MB)
Used Dev Size : 32704 (31.94 MiB 33.49 MB)
Raid Devices : 2
Total Devices : 1
Persistence : Superblock is persistent

Update Time : Fri Jan 16 21:26:58 2015
State : clean, degraded
Active Devices : 1
Working Devices : 1
Failed Devices : 0
Spare Devices : 0

Name : lupin:2 (local to host lupin)
UUID : eccabf13:879f4196:1c47de11:3059e0b8
Events : 6

Number Major Minor RaidDevice State
0 7 1 0 active sync /dev/loop1
1 0 0 1 removed

It seems that you can’t remove the “missing” disk from the array even with

mdadm /dev/md2 -r detached

I eventually found the solution to this problem. It seems (counter-intuitively) that to remove a device from the array you have to “grow” that array as follows:

mdadm /dev/md2 --grow --force --raid-devices=1

This resulted in the missing / removed / non-existent device being removed from the array and it being considered clean:

mdadm -D /dev/md2
/dev/md2:
Version : 1.2
Creation Time : Fri Jan 16 21:24:44 2015
Raid Level : raid1
Array Size : 32704 (31.94 MiB 33.49 MB)
Used Dev Size : 32704 (31.94 MiB 33.49 MB)
Raid Devices : 1
Total Devices : 1
Persistence : Superblock is persistent

Update Time : Fri Jan 16 21:32:17 2015
State : clean
Active Devices : 1
Working Devices : 1
Failed Devices : 0
Spare Devices : 0

Name : lupin:2 (local to host lupin)
UUID : eccabf13:879f4196:1c47de11:3059e0b8
Events : 11

Number Major Minor RaidDevice State
0 7 1 0 active sync /dev/loop1

This took me a while to figure out so I hope this might help someone else out there who’s trying to achieve the same thing.

Dropbox on Ubuntu Server

I’ve been using the Dropbox Linux client on my server for a while now, it’s very handy for sync’ing files to & from my various systems.

I recently installed a new server and hit the problem that following the usual instructions:


cd ~ && wget -O - "https://www.dropbox.com/download?plat=lnx.x86_64" | tar xzf -
~/.dropbox-dist/dropboxd

resulted in Dropbox not starting.

The problem (according to its crash log) was that it was trying to link against libgtk which I didn’t have installed on my (headless) Ubuntu Server 14.04 installation.

After a bit of messing around I realized that the problem was that I had enabled ForwardX11 on my PuTTY sessions and so dropboxd thought that I had an X11 session available. When I disabled X11 forwarding the problem went away & dropboxd ran as normal.

Unrar – Delete archive(s) after extraction

Due to lack of disk space on a server I needed a mechanism for automatically deleting RAR archives after their contents had been extracted. I couldn’t find any particularly neat way to do this with a shell script so I knocked up the following quick & dirty patch for unrar.

The patch applies cleanly to version 5.1.6 of the unrar source as follows:

wget 'http://www.rarlab.com/rar/unrarsrc-5.1.6.tar.gz'
tar -xvzf unrarsrc-5.1.6.tar.gz
cd unrar
wget 'http://simon.aldrich.eu/download/unrar/unrar-deleteOnExtract.patch'
patch -p1 < unrar-deleteOnExtract.patch
make

The compiled unrar binary will have the following extra option:

de  Delete archive(s) after extracting

Which you can use with the -e or -x commands e.g.:

unrar x -de test.part01.rar

Hopefully this might be useful to someone other than just me. Although, obviously, if extraction fails for any reason you will have deleted all the earlier volumes in the archive set. My suggestion would be to use the -t command to test your archive(s) first. I take no responsibility for any screw-ups you may cause yourself if you use this patch – caveat emptor!

Hacking the Motorola Blink 1 Baby Monitor (Part 2)

Ok, so it’s been quite some time since I posted my first efforts at “hacking” the Motorola Blink 1 Baby Monitor. Suffice to say we’ve been quite busy for a while & I’ve only just gotten around to actually plugging it in again now that our son is with us & at an age where we’re starting to think about being able to put him down in his crib & go into another room.

Anyway, I powered it on for the first time since August today and it asked to perform a firmware upgrade. I though ‘Aha! I’ll capture what it’s up to and see if I can work out where it downloads new firmware from’ but I inadvertently messed-up my tcpdump session & didn’t actually capture anything while it was upgrading. Furthermore, as I should have known from reading the comments here it seems that Motorola have disabled the landing page for the onboard web-server in the new firmware version (08-050) and it now just gives you a 404.

Well obviously this couldn’t be allowed to stand. Suffice to say if you capture all network traffic from the Blink when it powers on you’ll see it makes some web requests to a Monitor Everywhere ‘OTA’ server. It seems this is how it determines if there’s a firmware upgrade to be downloaded & with a bit of jiggery-pokery you too can download bmfwromfs_08_050.tar.gz which contains the latest firmware.

Unpacking the gzip’d tarball you’ll see there is a binary file ‘conprog.bin’ which I’m pretty sure is the kernel image  (2.6.17.14 since you ask) and a file ‘rootfs.bin’ which is a romfs image file containing the root file system for the camera.

You can mount this under Linux using the command:

mount -t romfs -o loop <path to rootfs.bin> <mount-point>

I’ve only just got this going tonight so I’ve yet to have a real poke around in there but for everyone who’s looking for the web interface point your brower at:

http://<camera-ip>/blinkhome.html

– or –

http://<camera-ip>/index2.html

to get it back. Incidentally the pages aren’t quite the same, so worth looking at both!

 

Hacking the Motorola Blink 1 Baby Monitor (Part 1)

As I mentioned before I’ve recently acquired a Motorola Blink 1 Wi-Fi baby monitor. My plan was to reverse engineer some of its functions in order to integrate it with our existing home automation. My findings to date indicate that it should be fairly easy to interface with the monitor without using the Monitor Everywhere App.

The first step was just to point a web browser at the camera’s IP address which paid off by giving me the following (with no authentication incidentally – be careful if you might be putting one of these on a publicly accessible IP):

Motorola Blink 1 Web InterfaceYou get a web interface with Javascript based video streaming & control Pan & Tilt as well as a reading of the current temperature & controls for contrast, brightness & image resolution. Two resolutions are available: QVGA (320×240) & VGA (640×480). On this page the video stream is provided by using some Javascript to repeatedly load a JPEG image from the URL:

http://<camera-ip>/?action=snapshot

and replacing the existing page image with it. Control of pan & tilt is also done with AJAX HTTP requests, I’ve not done any experimenting with those yet so I’ll leave them for another time. It is also possible to read the temperature sensor value in °C by requesting the following URL:

http://<camera-ip>/?action=command&command=value_temperature

 

The link for “Video + Audio” takes you to an identical looking page but instead of using Javascript to provide the video it using a Java applet to playback an MJPEG stream with audio. The Java applet appears to be based on the Cambozola streaming image viewer by Andy Wilcock. The URL for the MJPEG stream is:

http://<camera-ip>/?action=appletvastream

It seems possible to playback the video stream perfectly happily using VLC (just point it at the URL) but so far I can’t get it to play any audio with VLC. Unjar’ing the cambozola.jar from the Blink makes me suspect there could be an ADPCM audio stream (there is an ADPCMDecoder.class in the JAR) but it’s not obvious from capturing the stream’s URL with wget – if anyone knows anything about audio encoding in MJPEG streams I’d be most interested to hear about it.

One final point of interest from the web interface is that the page title identifies the Blink 1 as the MBP2000W which I’m guessing is the Motorola internal model number for the device.

Having explored the web interface a bit I thought I’d look into what sort of network traffic the Blink was generating. Having whipped out arpspoof & tcpdump I can confirm that the Blink is chatty little thing. On start-up it registers itself with the Monitor Everywhere website (it seems to use its own MAC address as the key), it seems to generate lots of ICMP traffic (mostly echo requests) & there’s definitely some STUN stuff going on (presumably to allow access to the camera from outside of your local network). I’ve got some network captures that I’ll be spending some time analysing in the near future.

One thing I did notice is that it’s fairly easy to receive alerts from the camera when it detects a noise. It just sends out a UDP subnet broadcast on port 51110. The contents of the UDP packet are:

VOX:<Camera MAC Address>

You can test this for yourself just by using netcat to listen on UDP port 51110 like so:

netcat -l -u 51110

That’s all I’ve got so far but I’ll post some more info when I’ve got some. In the meantime, happy hacking!

Motorola Blink1 Wi-Fi Baby Monitor

Discovering that we were expecting our first child at the end of the year naturally meant it was time to shop for exciting new gadgets that will make parenting a walk in the park (ha!). Top of that list came the acquisition of a baby monitor.

We’d seen a wireless video baby-monitor (with pan & tilt) that some friends of ours had bought. It looked ideal as it came with a neat little portable display unit & had all the required features (night vision, sound alert, talkback, temperature monitor etc). Unfortunately on further investigation I discovered that it transmitted video over the unregulated 2.4GHz band which meant that when the baby monitor was on it stomped all over their Wi-Fi signal. This seemed like a bit of a show-stopper to me.

Fortunately there are a few video baby monitors on the market which transmit their video over Wi-Fi meaning they should play nicely with the Wi-Fi. After a bit of research I decided to go with the Motorola Blink 1, it has all the features we wanted (temperature & sound alert, talkback) and also has an Apple & Android app (called Monitor Everywhere) which allows you to view the camera on your phone or tablet at home and also when away from home (should you so wish). You can also log into the Monitor Everywhere website to view your camera from any web browser. We bought ours from Amazon for £75 and got a £25 discount for joining Amazon Family which meant we got the Blink for £50 – bargain!

The slight downside to the Blink is that it doesn’t come with a dedicated video display. Obviously the intention is that you’ll use your phone or tablet to monitor it so you don’t need one. We decided to set up Katy’s Nexus 7 (which rarely gets used at home) as a display using the official charging dock which allows it to charge while sitting in the correct orientation for viewing video.

So far the experience is pretty positive. The set-up was straightforward & all done via the app (in this case the Android one) including adding the monitor to our Wi-Fi network. Once on the Wi-Fi it automatically downloaded & installed a new firmware update. After that it just appeared as a camera available to be viewed in the app. My two complaints about the current version of the app are:

  1. It doesn’t keep the screen of the device on when you’re viewing the camera
  2. If there’s no background noise the audio stream from the camera makes a periodic “purring” noise

Because it’s a Wi-Fi camera, it should be possible to reverse-engineer some (or all) of its functions and maybe make it do some cool stuff. Things I’m thinking of so far are:

  • Pause playback on the media centre / squeezebox when the monitor detects a noise
  • Provide some visual cue (LED strip or something) when noise is detected
  • Log & graph the temperature in the nursery
  • Automatically launch the app & turn on the screen of the Nexus when noise is detected

I’m sure there’s a bunch of other stuff it could also do. Anyone got any suggestions?Anyway, I’ll be documenting my discoveries here and hopefully they’ll be useful to other Geek Dads / Dads-to-be.

Network remote for Onkyo A/V Receivers

I recently purchased an Onkyo TX-NR515 A/V receiver & amplifier. One of my main reasons for choosing this amp was that it supports network remote control via its ethernet port. This enables me to shut the amp away out of sight in an A/V cabinet and control it using the Onkyo Remote Android app.

Being a geek though it wasn’t sufficient for me to be able to control it via app. I wanted to make the amp “magically” turn itself on & select the correct input source when I turned on my HTPC or Squeezebox Touch (more on this soon).

Thanks to Tom Gutwin‘s excellent work of finding Onkyo’s protocol specification & documenting his efforts to produce a Java eISCP client it was pretty easy to produce a little Linux command-line utility which sends remote commands to an Onkyo amp.

The utility is written in C and should compile cleanly with GCC on Linux (it may work on other platforms, I haven’t tried it). Usage is as follows:

./onkyo-iscp <amp hostname or ip> <ISCP command> <command parameter>

For example:

./onkyo-iscp onkyo.home.lan PWR 01

will send a “power on” message to the amp. There’s currently no error checking on the command or parameters, it assumes you know what you’re sending to the amp (read the protocol spec for a list of commands & parameters). It also doesn’t read any data from the amp, I might get around to implementing this eventually (if it turns out that I need it).

Source tarball is here: onkyo-iscp.tar.gz

Changing Pageant’s default behaviour

As a Unix developer, I spend most of my working day with lots of terminals open to various Linux & Solaris development or test servers

For many years I’ve used the excellent PuTTY by Simon Tatham for all my SSH’ing needs. In addition to PuTTY, I also use Pageant to make launching new terminal sessions as easy as right clicking an icon in the system-tray and choosing the machine.

The one complaint that I have with Pageant is its default double-click behaviour. My typical use case for Pageant is:

  • I start Pageant when logging into Windows and automatically load my SSH key (by supplying the filename of the private-key file as a command-line argument to Pageant).
  • I mostly launch previously saved PuTTY sessions from Pageant’s “Saved Sessions” menu.
  • Occasionally (once or twice a day) I’ll need to launch a PuTTY session to a machine which isn’t in my “Saved Sessions” (usually because saving them all would make the “Saved Sessions” menu overly long).

Pageant’s default behaviour when double-clicking the tray icon is to launch the “View Keys” dialog. What I’d really like it to do is to launch PuTTY (i.e.: what the “New Session” menu item does).

Thankfully Pageant (indeed all of the PuTTY suite) is open-source so it was relatively straightforward to download the source, make my simple change and recompile Pageant.

In case anyone else shares my use case I’m making this modified Pageant available:

Pageant

You can also download a patch file containing these changes which can be applied to Pageant v0.62 here.

App Idea : SMS Spam Reporter

I’m probably not alone in getting quite a few “spam” SMS messages on my phone. Lately they’ve all been about PPI mis-selling but in the past they’ve been about dodgy loans, accident claims etc etc

Unfortunately all these messages come from mobiles that have been bought to bulk-send SMS spam and aren’t registered or legislated for in any way. The only recourse that you might possibly have against these are to report the text to your mobile phone operator in the hope that they will block that number so that others aren’t inconvenienced.

You obviously need to include the sender’s number when forwarding the spam text to 7226 or (87226 if you’re on Vodafone). On my phone, the ‘forward’ function doesn’t do this automatically. So here’s my great app idea:

  • You set up a “blacklist” of words that usually appear in spam texts (e.g.: “PPI”, “claim” etc)
  • When your phone receives a text with one of the blacklisted words it automatically pops up a dialog which shows you the text and asks you if you’d like to report it as spam.
  • The app edits the text to include the original sender’s number and automatically forwards it to your operator’s SMS spam reporting number.
  • The app then deletes the text from your text inbox.

It doesn’t sound too hard does it? It might even result in these spam merchants getting their comeuppance. So why can’t I find an app that will do this? All the anti-spam apps I can find want to filter messages out of my inbox and become my messaging app. I don’t want that, I just want to report these sons of guns.

If I get a bit of free time this will be my first Android app (unless someone else makes it first).