Migration from Drupal to Blogger

(update: Migrated the code to GitHub and implemented minor improvements.)

It has finally happened: this blog is migrated away from Drupal to Blogger. My reason to move towards Blogger (and thus not away from Drupal) is very simple: No need to patch/update the application.
An important thing for me is that I wanted to keep all my blogposts, timestamps and comments. Unfortunately it looks like most people move away from Blogger towards Drupal and the web is full of code and information to export your data from Blogger in XML and then import it into Drupal.
But information how to upload everything into Blogger was nihil.
So I wrote a php script to do the export while keeping:

• posts
• comments
• tags / categories 
• publishing date

However there are a few quircks.

• Comments are (partially) anonymized because of a security feature of Blogger
• URLs are not customizable, so you will create dead links
• Images are not changed or imported. So manual work is still necessary

To use this script first create your blog into Blogger, create a test posts and export it to XML. Then run my php script and copy paste the output towards the bottom of the XML, where your test post is located.
Save the file and import it again in Blogger. It usually takes some time, but in the end you get the message that everything is imported correctly.

The code to do this is located here: https://github.com/cvandeplas/inet_scripts/blob/master/drupal_to_blogger.php .

Book review: BackTrack 5 Wireless Penetration Testing

Just before my holiday I got a new mail from Packt publishing to read a new book of theirs about Wireless Penetration Testing. Perfect to read on a sunny beach.

As this book is directed towards beginners I tried to read and review it with beginners eyes. Like their other book I was positively surprised to see a name I knew. The author Vivek Ramachandran not only gave a Wireless Pentesting training at BruCON, but is also known for his work on wireless security.

Content
The book has nine chapters starting with info how to build your lab, and what kind of hardware is required to more advanced attacks like Mis-Association, Caffe Latte, and breaking WPA-Enterprise.

I wouldn't compare this book to a standard book you read, because this book would be more a training manual teaching you some (basic) theory and then giving you lab exercises (or vice-versa). This is a great thing for geeks like me that remember by doing, and not by reading.

The disappointing bit was the lack of cryptographic theory. I think it is rather important to not only learn to use a tool with its command line options, but it's also important to know what the differences are between PTW and FMS attacks, and why it's possible to do ARP replays while the packets are encrypted. (Answer: because an ARP packet has a fixed length it can be recognized even being encrypted.)

As I am more experienced half of the book was a quick read, however the second half was a lot more pleasing as it taught me things I didn't know. (or forgot because of a lack of practice)

Conclusion
If you don't have experience with Wireless Cracking/Penetration Testing this book is definitely a must-read. I do advice however that you open Wikipedia and the site of Aircrack when reading trough WLAN Encryption Flaws (Chapter 4) to better understand the cryptographics.
Don't forget to buy a wireless card supporting monitor mode and packet injection while ordering this (e)book.

If you want to read a bit have a look at the free sample chapter.

RTBF TV Series downloader

Some time ago I wrote a simple script to automagically download TV episodes from the "revoir" functionality from the website of the RTBF.
That first script was rather unstable, so I analyzed the HTTP flow occurring while playing a video manually and wrote a lot more stable script that seems to work for some time.
The rtbf_tv_series_downloader.py script is available on a github repository.

How is it working?

1. The XML feed with the latest episodes is fetched.
2. From that file the unique id is extracted.
3. That unique id is used to download the JSON file for that episode.
4. In that JSON file a full download url is available.
5. That file is downloaded and saved to the disk. Only if it was not yet on the disk.

Python global variables

Some things in python are weird, especially when considering global variables. Let's take the following code where we define two global variables (string and dict) and change their value inside the function.

dictionaryVar = {'A':"original"}
stringVar = "original"
globalStringVar = "original"

def aFunction():
    global globalStringVar
    dictionaryVar['A']="changed"
    stringVar = "changed"
    globalStringVar="changed"
    return dictionaryVar, stringVar, globalStringVar

print "Output of the function is:"
a = aFunction()
print "Dictionary   : ",
print a[0]
print "String       : "+a[1]
print "Global String: "+a[2]
print "\nGlobal variables are now: "
print "Dictionary   : ",
print dictionaryVar
print "String       : "+stringVar
print "Global String: "+globalStringVar

And now when running the code we see the following output (Python 2.6.6) we see the following:

$ python tmp/foo.py 
Output of the function is:
Dictionary   :  {'A': 'changed'}
String       : changed
Global String: changed

Global variables are now: 
Dictionary   :  {'A': 'changed'}
String       : original
Global String: changed

So the conclusion is:

• Global strings changed in a function are returned correctly, and not changed outside the scope of the function. (expected)
• Global dictionaries changed in a function are returned correctly, but they are also changed outside the scope of the function. (not expected)
• Global strings, declared as global (in the function), changed in a function are returned correctly, and are also changed outside the scope of the function. (expected)

Book review: BackTrack 4: Assuring Security by Penetration Testing

Recently Packt publishing contacted me to ask me if I would like to review their BackTrack 4 book. Being an avid user of this distribution, and wondering what a book about BackTrack would look like, I accepted the offer.

A few days before BackTrack 5 came out the book finally arrived in my mailbox. As I also had the opportunity to play with BackTrack 5 during the time I read the book, I should be able to see how useful it is now BT5 is out.

A suprise
A first surprise was when I read the first pages about the authors and reviewers. Peter Van Eeckhoutte, also known as corelanc0d3r (from Corelan Team), is one of the three reviewers of this book. Seeing his name in this book gave me a good feeling about what I was going to encounter. (no no, it's not because he's Belgian)

Content
The book is divided into twelve chapters, with the first chapter an introduction to the BackTrack distribution, the various forms, how to configure the basics, update the system and make your own version of the live CD. The second chapter (free sample) gives an overview of various penetration testing methodologies, including the OSSTMM, ISSAF, OWASP, ... but also a BackTrack pentesting process in ten consecutive steps: Target Scoping, Information Gathering, Target Discovery, Enumerating Target, Vulnerability Mapping, Social Engineering, Target Exploitation, Privilege Escalation, Maintaining Access, and last but not least Documentation and Reporting. 

If you already used BackTrack before you will certainly recognize some of these names in the menu's of the BT4 menu ... and even more from the BT5 menu ... 

The next ten chapters first elaborates each step in some detail, to then dive into the real usage of each of the tools delivered with BT. So what options and arguments you need to do your job. This review won't go into detail into each chapter as it can be considered as an "enumeration of many tools". Many tools I already knew, but also many I discovered while reading.

At the end there's the very-much needed chapter about Documentation and Reporting ... a step often hated by techies. The book tries to convince you of the utility of your report and helps you by giving some tips and tricks with a sample table of contents to start with.

Downsides
Unfortunately no book is perfect and the thing that I really missed was a discussion of IPv6 tools, and examples with IPv6 IP addresses. Fortunately there's still that rather old Uninformed article from H D Moore to fill the gap.

Also be careful not to read the whole book at once, as your brain risks a buffer overflow if you do.

Conclusion
As this book is really focused on the BackTrack distribution the authors knew they wouldn't need to fill pages on how to install these hundreds of tools, but instead they could concentrate on explaining what every tool does and how to use them.
Of course you can't expect to have an extremely deep dive into each one of the tools, knowing that the book discusses around 100 of them. But they found a good equilibrium by going deeper with the more important tools available, with for example the five practical examples of exploitation with metasploit. (db_nmap, snmp scanner, vnc scanner, iis6 webdav attack, bind/reverse shell and meterpreter and msfpayload)

I already know what I'll do with this book: First put my name in it, then lend it to some friends who will certainly learn a lot from it and finally make sure I get it back (that's why I put my name in it) to use it as a later quick reference. An eBook version is available with a discount if you have the paper-version, and I'm hesitating to buy that one for the sake of mobility.

So if you're interested to buy the book, you can do that here.

BruCON Call For Papers

2009 was the first edition of BruCON, a non-profit conference meant to unite all the people in and around Belgium interested in discussing computer security, privacy and computer technology related topics. It was a great first edition thanks to the help of the sponsors and many volunteers.

I'm happy that I'll be able to play a (more significant) role in the organization of the second edition.

Do you have an interesting topic to present or a cool workshop? Have a look at the Call of Papers here.

I'm going to FOSDEM

Change files on the read-only filesystem of your Android phone

I am currently working on an small application that needs to load kernel modules at the startup of the Android phone. I could eventually start up an Activity or Service using a trigger on the BOOT_COMPLETED_ACTION, (howto), but this creates some complexity as I need to load compcache kernel modules requiring lots of free memory.
Using a boot script is much better.
(Un)fortunately an application cannot change things in the /system partition as it is mounted in read only.

# mount
rootfs on / type rootfs (ro)
tmpfs on /dev type tmpfs (rw,mode=755)
devpts on /dev/pts type devpts (rw,mode=600)
proc on /proc type proc (rw)
sysfs on /sys type sysfs (rw)
tmpfs on /sqlite_stmt_journals type tmpfs (rw,size=4096k)
/dev/block/mtdblock3 on /system type yaffs2 (ro)
/dev/block/mtdblock5 on /data type yaffs2 (rw,nosuid,nodev)
/dev/block/mtdblock4 on /cache type yaffs2 (rw,nosuid,nodev)
/dev/block/mmcblk0p2 on /system/sd type ext2 (rw,noatime,nodiratime,errors=continue)
/dev/block//vold/179:1 on /sdcard type vfat (rw,dirsync,nosuid,nodev,noexec,uid=1000,...)

Fortunately, as I have root support on my phone, I can simply remount the /system partition as rw, do my change and then remount it back to ro.
Here is how you do this in java code:

public static void saveCommandsToBootFile(String script, String filename) {
 // first remount filesystem in rw
 // save the file
 // remount the filesystem back to ro
 String command = 
  "mount -o remount,rw /system \n" +
  "echo '" + script.replace("'", "\\'") + "' > " + filename + " \n" +
  "mount -o remount,ro /system \n";
 executeCommand(command);
}

public static void executeCommand(String command) {
 Log.d(MainActivity.LOG_TAG, "Executing the following commands: \n" + command);
 Process process;
 try {
  process = Runtime.getRuntime().exec("su -c sh");
  DataOutputStream os = new DataOutputStream(process.getOutputStream());
  //DataInputStream osRes = new DataInputStream(process.getInputStream());
  os.writeBytes(command); os.flush();
  // and finally close the shell
  os.writeBytes("exit\n"); os.flush();
  process.waitFor();
 } catch (IOException e) {
  e.printStackTrace();
 } catch (InterruptedException e) {
  e.printStackTrace();
 } 
}

Some remarks you could have:

• I didn't use java to write the file: Indeed, my java application runs in a limited environment and has no rights to write to /system/, even mounted rw. I would need to write the file temporary somewhere else, to then move it back to the final location. This looks a little to complex.
• I escape the ' quote in the script to prevent my echo foo > bar failing.
• An uncontrolled filename could result in command injection as root !  (Thanks to Steve Nugen from UNO to report that!)

Traffic Stats for TunnelDroid

I just released a new version of TunnelDroid adding support for traffic statistics. Go to the Market to update...

DNS speed analysis - the true story

Today I was curious about the results of Namebench, or in other words the speed of my dns resolving.
It is interesting to know that I run my own recursive DNS server in a DMZ of my home network. This DNS server will never forward queries to my provider or other public recursive DNS, but will do the necessary to resolve the addresses completely by itself.

Table with the result:

(underlined my own DNS)

IP	Descr.	Hostname	Avg (ms)	Min	Max	Err	NoAns	Notes
62.6.40.162	BT-70 GB	indnsc70.ukcore.bt.net.	55.16	20.98	635.58	1	1
156.154.70.1	UltraDNS	rdns1.ultradns.net.	62.61	18.01	711.56	1	0	NXDOMAIN Hijacking
195.27.150.42	Cable & Wireless DE-2		89.10	21.24	899.27	1	1
8.8.8.8	Google Public DNS	google-public-dns-a.google.com.	89.15	18.47	1843.15	1	1	Replica of Google Public DNS-2 [8.8.4.4]
208.67.220.220	OpenDNS	resolver2.opendns.com.	89.81	17.75	1720.89	2	0	NXDOMAIN Hijacking
212.159.13.150	Plus-2 GB	ns2.plus.net.	100.23	23.36	927.12	9	1
157.193.71.1	Ugent-3 BE	ugdns3.ugent.be.	105.39	12.17	1077.03	1	1
78.47.115.194	Cesidio-A DE	a-root.cesidio.net.	141.89	25.35	1480.88	0	1
62.153.158.62	T-Online SUL DE	dns00.sul.t-online.de.	147.66	25.93	905.30	3	1
10.107.4.1	10.107.4.1		170.16	2.22	1246.37	0	1	Your current primary DNS server
8.8.4.4	Google Public DNS-2	google-public-dns-b.google.com.	0.00	0.00	0.00	0		Slower replica of Google Public DNS [8.8.8.8]

A first conclusion you can make is that my DNS server is extremely sloow. The tool even says that the fastest DNS is 209% Faster than mine! The numbers don't lie: average : 170.16 ms, fastest: 2.22 ms , worst: 1246.37 ms.

So my DNS can be extremely fast, only 2.2 ms,
but also extremely slow.

It's very simple to explain if you know the address resolution mechanism represented in this wikipedia image:
So a new query, that my DNS server didn't see before, will take a long time because he will need to perform many queries over my 'slow' internet line. But a cached query will have no delay because I stay on my LAN.

Response Distribution Graph

Let's take a look at the graph plotting the Response Distribution (in the first 200ms)

In this graph we see that in 37% of the times my DNS server is the fastest. But where does Namebench takes his DNS data? From my Firefox history: it has 59312 entries.