inside Habbie's mind

I have roughly documented my procedure below; the two scripts you will need are in a git repository. When you follow these steps, you will end up with a fully functional Debian chroot in which you can use apt-get to install additional software. We will also setup telnet running outside of the chroot, accessible from localhost, for easy root access to the NSA itself without going through the backdoor sequence every time.

Of course, this is just a guide, written according to my taste and preferences. Feel free to do things your way; I’d love a comment to hear what you did differently.

One final note: this setup only arranges for ssh and localhost telnet to be started on boot; other daemons you install inside Debian will not get automatic startups. I believe you can replace the ssh line in init.sh with something else to get a full Debian startup, but for now I have no need for it.

1. grab a working Debian box (I used Debian 6/squeeze amd64), and run ‘sudo /usr/sbin/debootstrap --foreign --arch=armel --variant=minbase --include=openssh-server squeeze ./nas-debootstrap/ ftp://ftp.nl.debian.org/debian’. This creates a Debian environment in ./nas-debootstrap/. Tar it up and copy it to the NSA (in any share will do).
2. open telnet on the NSA310 - log in as administrator on the webinterface and browse to ‘/zyxel/cgi-bin/remote_help-cgi?type=backdoor’. This opens telnet (port 23). The URL never finishes loading for me, but you can just cancel it. Username is root; password is your admin password.
3. on my box, the main storage dir is /i-data/370f61a5 which seems a bad thing to standardise on; /etc/zyxel/storage/sysvol is a symlink to it, and we’ll use that from here on.
4. in /etc/zyxel/storage/sysvol I made a dir .debian; I will refer to this dir as .debian from now on.
5. extract the tarball and rename it to .debian/root.
6. type ‘chroot .debian/root /debootstrap/debootstrap --second-stage’ to finish the debootstrap. Then, set a root password: ‘chroot .debian/root passwd root’. Note: after this step you can enter your Debian setup by saying ‘chroot .debian/root’ to look around a bit
7. ‘mkdir .debian/root/data’ for mounting our actual storage.
8. outside of the chroot, grab init.sh and telnet.sh from the git repo and put them in .debian and chmod +x them. Put chroot-initin .debian/root/etc and chmod +x it too.
9. ‘cd /usr/local/zy-pkgs/etc/init.d; ln -s /etc/zyxel/storage/sysvol/.debian/init.sh DEBIAN; ln -s /etc/zyxel/storage/sysvol/.debian/telnet.sh LOTELNET’

Now, you should have a Debian setup that starts when you boot your NAS. If everything looks okay, type ‘reboot’ to give it a shot! A reboot on mine (with or without Debian) takes about two minutes, so don’t panic!

When it is up again, try ‘ssh root@nsa310’ replacing nsa310 with the IP or perhaps internal hostname you have. This should drop you into a bash shell inside your Debian setup!

One last thing: to use apt-get, you’ll need a sources.list configuration for it. I recommend using Debian Sources List Generator for this.

As it turns out, many people (Viagénie, Bill Manning, Ryan Rawdon) knew about this trick already. Somebody (I forget who/where) pointed out that this trick worked well for e164.arpa mapping too. Indeed, I have used this trick to do statistics on other sparse but deep zones years ago.

But, to many other people, this idea was totally novel. Marc ‘van Hauser’ Heuse added a tool based on this idea to his thc-ipv6 IPv6 hacker toolkit, and Patrik ‘nevdull77’ Karlsson committed an nmap script employing the same technique.

Simon Arlott (pointed out to me by Ryan Rawdon) took it one step further – he also wrote a tool to mitigate this trick in a very clever way. Check out the README and ip6dnshide.py in his ip6walk github repo. The trick works with ldns (1.6.12)+NSD (3.2.10) but BIND9 (9.9.0b1) rejects the resulting zone file due to non-terminal wildcards.

All in all, the idea was not new but it deserved some exposure. I am glad I was able to provide that :)

I have added some progress reporting to my implementation and at least one issue has been filed against it. I do not intend to develop this further (although I may at some point write a more parallel version), as at least three other implementations exist, and at least two of those are likely to see more usage than my script anyway.

The method is best explained using an example. This example uses the IPv6 Address Prefix Reserved for Documentation.

Let’s assume that within 2001:DB8::/32 there is 2001:DB8:80::/48 which has reverse DNS hosted in a zone called 0.8.0.0.8.b.d.0.1.0.0.2.ip6.arpa.

Now, given the zone name (let’s call it X to be slightly easier on the eyes), we can query 0.X, 1.X, 2.X etc. up to and including f.X. Most of these queries will return an NXDOMAIN rcode; this means the name does not exist, but very importantly, this can usually be construed to mean that no longer name exists either. Suppose that in this case, two of the names (0.X and f.X) do not return NXDOMAIN – instead they return NOERROR. This means the nameserver has a reason to not deny existence, and in this case, that reason is that a longer name exists.

We tested 16 prefixes of X and have rejected 14 due to NXDOMAIN; by that, we have reduced our search space from a full /48 to 1/8th of a /52 – a 128-fold reduction with just 16 queries. This trend continues at every step.

Note that whether this works depends on how a nameserver handles requests for these shortened names (the technical term is ‘empty non-terminals’). BIND and NSD return NOERROR and NXDOMAIN as described above; tinydns and PowerDNS in most setups will not.

As said above, I could not find any previous work applying this feature of DNS to the mapping of reverse zones. If you do know of anything, let me know and I will update this post.

For reasons yet unknown, my Mac system disk lost its partition table (GPT) yesterday. This is how I recovered:

Finding

I booted an old copy of my OS X Install from USB and downloaded testdisk. I then used it to scan for any partitions it would recognise. Results (warning: copied by hand from photo):

Disk /dev/disk0 - 512 GB / 476 GiB - CHS 1000215216 1 1
     Partition        Start        End    Size in sectors
 P DOS_FAT_32            40     409639     409600 [EFI]
 P HFS            998945640 1000215175    1269536

The DOS_FAT_32 partition contains EFI data; the HFS partition contains OS X Lion Recovery data. Obviously, testdisk did not recognise my FileVault partition. However, it is quite likely our data partition is right in between these two!

Fixing

As testdisk does not support writing Mac/GPT partition tables, I took a phone camera photo of the results above and booted into the gparted liveCD (easier than building or finding parted/gdisk binaries for OS X, I think). With both my troubled drive and my old install connected on USB, it should be easy to compare details and get everything right.

As it turns out, gparted (the GNOME GUI for parted) is extremely limited. I could not even find how to define sector boundaries for partitions, and the partition type list is quite short. The CLI version is slightly better – it allows sector specifications but is unaware of most Apple-related GPT GUIDs.

This is how I, inefficiently, set my partition table straight:

1. Use gparted to make a new GPT table.
2. Use parted to make 3 partitions, using the numbers from testdisk and squeezing partition 2 right in between (starting one sector after end of partition 1, ending one sector before start of partition 3). I verified the snug fit with my old install.
3. Try to set types and names right using parted, and failing.
4. Trying the same thing with gdisk (which is on the gparted live CD) and succeeding. See below for correct settings.

If you’re doing this, I recommend going with gdisk straight away.

Partition map

1. sector 40-409639 (409600 sectors, 200.0 MiB): partition type EFI, label ‘EFI System Partition’
2. sector 409640-? (? sectors, ? GiB): partition type Apple Core Storage, label whatever
3. sector ?-? (1269536 sectors, 619.9 MiB): partition type Apple boot, label ‘Recovery HD’

For non-FileVault users, I expect the type of partition 2 is HFS+ and the label does matter (cosmetically).

Wikipedia has a very useful list of partition type GUIDs. gdisk knows about all these numbers, though.

Update, Jan 12th 2012: it happened again. This time, I went straight for gdisk instead of bothering with (g)parted. gdisk immediately offered to restore my ‘backup MBR’ but did mention something about a broken checksum. I said ‘yes’, pushed ‘w’ for write. All done.

It turns out this is a firmware issue with my Samsung 830 SSD. Of course, the firmware updater does not work on a Mac.

This script fixes the CSV up in a few ways:

• it puts a single quote character in front of all timestamp fields - without it, Numbers will interpret the dates and, for me, it is confused about day field vs. month field
• it sorts the CSV by start time, ascending instead of descending

Code: (download here)

#!/usr/bin/env python
import csv
import sys
import operator

r = csv.reader(sys.stdin)
rows=[]
for row in r:
        row[5] = "'%s" % row[5]
        row[6] = "'%s" % row[6]
        row[7] = "'%s" % row[7]
        rows.append(row)

rows = [rows[0]] + sorted(
       rows[1:],
       key=operator.itemgetter(5)
)

w = csv.writer(sys.stdout)
for row in rows:
        w.writerow(row)

<?php
        // create empty array $a
	$a = array();
        // append two items to it
	$a[] = "a";
	$a[] = "b";
        // remove last item using array_pop
	array_pop($a);
        // append another item
	$a[] = "c";

	$b = array();
	$b[] = "a";
	$b[] = "b";
        // remove last item using unset
	unset($b[1]);
	$b[] = "c";
        // print key for value "c"
	print array_search("c", $a). "\n";
	print array_search("c", $b). "\n";
?>

Exhibit B:

<?php
        // create pre-filled key-less array $a
	$a=array('a','b');
        // dump it
	print_r($a);
        // json-dump it
	print json_encode($a)."\n";
        // add key/value item to array
	$a['x']='c';
	print_r($a);
	print json_encode($a)."\n";
?>

(I’m on PHP 5.3.4).

• Nambu: lost DM support last week. No streaming. Often errors during refresh. Otherwise, best client ever.
• Twitter.app: looks like shit. Does not work with HyperDock. Can’t find the keyboard shortcut to switch accounts.
• Echofon: looks like utter shit. Multiple account support seems hackish.
• Kiwi: looks good but is extremely buggy. Frequently fails to load (or display) timelines or DMs at all.
• YoruFukurou: terrible name. Reasonable looks. Acceptable keyboard shortcuts. Streaming support. Multi-account support at least has working notifications. I’m sticking with YoruFukurou for now.

In my previous blog post I wondered

I don’t know what the mathematical implications of having the last few bits of a private key are, but it can’t be good.

As it turns out, for DSA, quite bad.

In short, this pam_env symlink issue, in some cases, allows an attacker to lift enough private key data from a DSA key to make brute-forcing the rest feasible.

For all details, see my article. Comments welcome on this post, or via e-mail as noted in the article.

70 packages can be updated.
35 updates are security updates.

However, in the course of this investigation I noticed that /etc/update-motd.d is basically a bunch of shell scripts that get run as root on every ssh login by any user. Scary, no?

Remembering that being able to inject some environment variables was always a great way to break out of ‘restricted’ shells that were written in (ba)sh, I set out to look at methods of influencing the environment these scripts would be run in. .ssh/environment, .pam_environment and OpenSSH’s SendEnv all turned out to be smart enough to only do their work after update-motd was done, sadly. I was out of ideas.

The code of pam_env.so did tell me another interesting thing – ~/.pam_environment is opened and read as root, without any dropping of privileges. This suggested that symlinking it to some unreadable file (of the right format, i.e. consisting of VAR=value lines) would compromise the data in that file.

A proof of concept was simple enough (newline inserted because my blog layout is too narrow):

root@vps6001:~# cat /etc/env2
ENV2=bla
...
peter@vps6001:~$ ls -al .pam_environment 
lrwxrwxrwx 1 peter peter 9 May 12 22:13
                   .pam_environment -> /etc/env2
peter@vps6001:~$ ls -al /etc/env2
---------- 1 root root 9 May 12 22:13 /etc/env2
peter@vps6001:~$ set | grep -i env
ENV2=bla

Effective targets for this trick do exist. DirectAdmin (a commercial control panel) stores MySQL login information in a suitable file, and many daemons that support LDAP expect a password stored in a similar way. Also, base64-encoded files (like SSL and ssh keys) that happen to need padding such that they end in == expose their last line this way. I don’t know what the mathematical implications of having the last few bits of a private key are, but it can’t be good.

Some internet searching turned out that this issue was previously discovered by other people.

Ubuntu 10.04 and Debian 6 are still vulnerable to this issue. Debian has a report on file but has not acted on it yet. Ubuntu doesn’t seem to know at all.

I emailed security@ for both distributions, Debian responded within minutes pointing me to the report they already had. I’m waiting for a response from Ubuntu.

Workaround: add user_readenv=0 to every pam_env.so listing in /etc/pam.d.

Load average

---

Originally defined as the difference between available system memory and free memory.

…

Load average can also be calculated for car engines; a common value is 2.2.