Kreuzworträtsel (German)

Hier ein Kreuzworträtsel, dass zum Beispiel verwendet werden kann, um die Bewerberanzahl bei einer WG-Besichtigung zu reduzieren 😉. Das Kreuzworträtsel wurde mit dem Python-Kreuzworträtselgenerator genxword erzeugt. Viel Spaß beim Rätseln! Download als .pdf hier.

[Download als Pdf]

Simple Experiment on Git Scalability

Have you ever experienced big git repositories growing slow? I was curious to learn more about the scalability of git and ran the following simple experiment:

In a script I made 300 commits and in every commit I added one file with random data to the repository. Moreover, I randomly changed one of the existing files. Then I checked out the old commits in a non-consecutive order. The time for every git operation (git status, git add, git checkout) was measured and is visualized in the following plots. The time required for git commit is negligible in comparison to the other operations and it is therefore omitted in the plots. The size of the file that was added to the repository in every commit was varied in different runs of the experiment (80 chars x 100 lines, 80 chars x 1000 lines, 80 chars x 10,000 lines). The times presented for git checkout are averaged over the times required for the checkouts of all the individual commits.

1. File with 80 chars x 100 lines added to the repository in every commit.

2. File with 80 chars x 1000 lines added to the repository in every commit.

3. File with 80 chars x 10,000 lines added to the repository in every commit.

The experiment was run in a ramdisk in order to decrease the time required for the overall experiment. I assume that the measured times presented here are higher by a constant factor (~10) for git use with a hard disk.

In the first run of the experiment with 'small' files (80 chars x 100 lines) the operations git status, git add and git checkout seem to scale very linear with the number of commits.

In the second run of the experiment with 'medium' file sizes (80 chars x 1000 lines) more noise on the times required for git add and git status is observed, which make it a bit hard to identify an overall trend. Interestingly, the time required for git add and git status seem to  scale constantly for 'large' file sizes (80 chars x 10,000 lines). The average time required for git checkout seems to scale very linearly in all three runs of the experiment with the number of commits in the repository.

In the second (third) run of the experiment the file size is increased by a factor of 10 (100). As can be observed from the plots, this increases the general times measured for git checkout, but only by a factor of  ~3 (~6).

The Python scripts that were used for the experiment can be downloaded here. Git version 2.7.4 was used for the experiment.

Installing Maps on a Garmin Etrex 30

I use a Garmin Etrex 30 for outdoor navigation and Geocaching. Amazingly detailed maps can be downloaded as compatible Garmin images on this Openstreetmap website.

I have found the USB connection of the Etrex 30 to be a bit shaky. The Garmin image files can be quite large (~Gigabytes). Copying the files can take some time and it also gets stuck occasionally. If a corrupt Garmin image is written to the device, it will not be recognized on it.

A simple way to alleviate this problem is by using rsync for the synchronization. It will still take quite some time, but rsync makes sure that the images are copied correctly. Here is an example how to use rsync:

rsync --progress gmapsupp_germany.img  /media/path_to_my/GARMIN_micro_sd/Garmin/

Simple Backup Script

Here is a simple backup script using Duplicity:

#!/bin/bash

#install duplicity:
#sudo apt-get install duplicity
#sudo apt-get install python-pip
#sudo pip install paramiko

export PASSPHRASE=MYSECRETPASSPHRASE
duplicity --num-retries 20 /home/myHomeDir sftp://USERNAME@MYSERVER//home/myHomeDir/duplicity
unset PASSPHRASE

#amend the following to your crontab for regular backups at 3.am (issuing 'crontab -e')
#0 3 * * *  /path/to/script/duplicity_backup.sh

QR codes for wireless lan access

Do you share your wireless lan with friends and family and still want to maintain a policy of frequently changing your wireless lan password? They might find it tedious to type a new password into their mobile devices on every visit.

A way to make this more convenient is to use a QR Code, which can be scanned to make a device connect to your wireless lan automatically. This actually works for many mobile devices, especially Android smartphones.

A simple Python script that can be used to create such a QR code can be downloaded here. It requires qrencode and Imagemagick. An Android QR code reader, which I have found useful is Barcode Scanner.

The script generates a printable pdf that looks like the following:

  

Find the shortest path to lift­-off!

Can you solve the riddle (pdf version here). The sha256sum of the solution is 38700dfad5711976e2f7aeab31013f04aed8c83118a1ef892f6d23bdfe94460. Good Luck!

Legible Plots with Matplotlib and LibreOffice

Have you ever struggled presenting scientific plots in LibreOffice? A lot of the time the fonts come out too small or it can happen that thin lines disappear in scaled figures. Often plots are not originally created to be shown on slides. A pragmatic solution to create more legible plots with Matplotlib for LibreOffice slides is to take the dimensions that a plot should have on the final slide into account when creating the plot.

LibreOffice offers 12 standard layouts, as can be seen below:

Not all of these layouts are suitable to present plots, but for those who are, the dimensions of the layout boxes should be taken into account to create legible plots. E.g. in Matplotlib this can be accomplished with the following command:

matplotlib.rc('figure', figsize= [width,height]) 

Moreover, it is advantageous to modify some other style options to make a plot more legible. Some style options which I found helpful in the past are the following:

matplotlib.rc('lines', linewidth=4.0) #thick lines
matplotlib.rc('font',  size=18) #big font
matplotlib.rcParams.update({'font.sans-serif': 'Liberation Sans'}) #adjust font w/ slide font

In my opinion .png is one of the few trustworthy output formats for plots. Generally producing plots in vector formats (e.g. .svg) would be nicer, but at least I have experienced these being rendered incorrectly (LibreOffice 4.3.6.2 was used at the time of this writing).

An example slide with these style options applied is shown below:

The full script that was used to create the sample plots can be downloaded here. It can also be used to create sample plots for the other LibreOffice layouts.

Medisana ViFit connect Review

In the post I want to share some experiences with the Medisana ViFit connect, which is an activity tracker. It counts steps and bins them in intervals of 15min. Memory of the device is sufficient for 15 days of recording and battery lifetime is around 6 days. It has an OLED display that saves you from having to wear an additional wrist watch.

Medisana outlines that for optimal usage the activity tracker should be wrist worn. I disagree with this as my intuition is that the activity tracker overestimates the number of steps when wrist worn, especially when you spend a big portion of the day sitting.

Bluetooth 4.0 is used to upload recorded data to a smartphone, tablet, etc.. The synchronization is slow and not always successful. The app does not build an offline database, so internet connection is required while synchronizing. Moreover, synchronization always has to be explicitly initiated and is not executed as a background service.
The activity tracker has a Micro USB port for recharging.  Interestingly, hooking it up to a Linux PC loaded a driver for this USB to UART bridge. Maybe this leaves some potential for hacking? It would be great to be able to download the activity data without having to upload them to the cloud first. Another idea would be to reduce the binning interval of the activity tracker to eg. 1min to be able to draw conclusions about the types of activity (eg. running, biking, hiking, sitting, etc.) from the recorded data.

In my opinion the accompanying website cloud.vitadock.com fails to provide much insightful information (Maybe there is a legal requirement that prevents them from interpreting the activity data for the user?). It is possible to export the activity data as .csv files and I find it helpful to create two additional plots as follows:
Firstly, a histogram showing the averaged activity for all days that usually falls within the individual times of the day. An exemplary plot is shown in Fig. 1 based on mock data. It is also interesting to compare working days and weekend days in this plot.

Fig. 1.: Relative activity over time of the day

Secondly, a scatter plot showing the correlation between sleep time and the active time during the next day as shown in Fig. 2. Again this plot is based on mock data.

Fig. 2.: Active time versus sleep time

Graphviz Diagrams

The following images show regular 5x5 grids with a nearest neighbor, ring or complete topology:

I recently found out that drawing these kind of images is easily possible using Graphviz and Python. A small script that automates creation of these images with arbitrary number of rows and columns can be downloaded here. The script creates a .svg file, which can be conveniently postprocessed with Inkscape. Usage of the script is as follows:

python ./graphviz_grid.py <num_rows> <num_cols> <ring|nn|compl> <outfilename.svg>

Measure your fame!

It recently occurred to me that Google Plus shows the amount of accesses on every G+ profile. Greatly amazed that Google Plus facilitates so much narcissism, I wanted to track the amount of accesses to my profile over time.

Here are two small and very simple scripts that can be used to record and visualize the amount of accesses for a G+ profile.

g+.py downloads the current amount of accesses from a G+ profile page. I recommend making a cron job that executes it daily. It is also possible to track other people's 'fame', since the amount of accesses of a G+ profile is public.

g+viz.py simply visualizes the recorded amount of accesses over time. The practical value of these plots is that they can be used to estimate how much resonance is created by the activity on a given profile.

Spiral Matrix

Have you ever tried to assign integer values to a square matrix in spiral order? Eg. for a 5x5 matrix this would look as follows:

1	2	3	4	5
16	17	18	19	6
15	24	25	20	7
14	23	22	21	8
13	12	11	10	9

If this is what you have been lying wake at night for, here comes an iterative solution using Python/Numpy:

#!/usr/bin/env python

""" create a spiral array """

import numpy

def spiral(n):

     mat = numpy.zeros([n,n])

    dirs = [

        numpy.array([0,1]),  #right

        numpy.array([1,0]),  #down

        numpy.array([0,-1]), #left

        numpy.array([-1,0]), #up

    ]

    ptr = numpy.array([0,-1])

    ptr_prime = ptr

    cur_dir_ind = 0

    for i in range(1,n*n+1):

        #adjust direction if necessary

        while(True):

                ptr_prime = ptr + dirs[cur_dir_ind]

                if ptr_prime[0]<0 or ptr_prime[0]>=n or ptr_prime[1]<0 \\ 

                   or ptr_prime[1] >= n:

                        cur_dir_ind = (cur_dir_ind + 1) % len(dirs)

                elif mat[ptr_prime[0],ptr_prime[1]] > 0.0:

                        cur_dir_ind = (cur_dir_ind + 1) % len(dirs)

                else:

                    break

        #update ptr;

        ptr = ptr_prime

        mat[ptr[0],ptr[1]] = i

    

    return mat

7 Wonders

This post is about the board game 7 Wonders. Sometimes there is a bit of confusion about calculating the victory points from the green cards (scientific structures). Each of the green cards has a symbol (tablet, compass or gear). The player receives 7 victory points for a set of all three symbols and additionally the number of the symbols squared for every symbol. Moreover there is a purple 'joker' card (guild) that can count as any type of green card.

The following few lines of Python can be used to compute the maximum amount of victory points that can be obtained for a given amount of tablet, compass, gear and joker cards. It does this by recursively finding the best allocation for the purple joker cards.

 

#!/usr/bin/env python

""" compute max nr of points for science cards """

#type1 : gear

#type2 : tablet

#type3 : compass

def get_score(type1, type2, type3):

    a = type1**2+type2**2+type3**2

    b = min([type1, type2, type3]) * 7 #sets

   return a+b

def max_points(type1=0, type2=0, type3=0, joker=0):

    if joker==0:

        return get_score(type1, type2, type3)

    else:

        return max([

            max_points(type1+1,type2 , type3 , joker-1),

            max_points(type1 ,type2+1, type3 , joker-1),

            max_points(type1 ,type2 , type3+1, joker-1)

        ])

Exercise Sounds with SuperCollider

Imagine you are instructing a group of athletes doing circuit training. The circuit training consists of stress and relaxation phases, which will be repeated in multiple series. At the end of a series there is usually a longer relaxation phase ('Serienpause').

To determine beginning and end of the individual phases, it is straight forward to use a stopwatch. However, the stopwatch will draw away most of your attention and you will not be able to correct the athletes any longer. Therefore, an acoustic signal is desirable that indicates the individual phases. The waveform of the signal might look as shown below:

The pulses indicate the seconds during the stress phase and the noisy parts mark the relaxation phase.

Such an audio signal can be synthesized using a software synthesizer, eg. SuperCollider. An exemplary audio file that was created with SuperCollider can be listened to here. It consists of 3 series with 2 stress phases (30s), a relaxation phase (20s) and a longer relaxation phase at the end of the series (30s).

The SuperCollider code, that was used to generate the audio file, can be downloaded here. It can be easily adapted to account for other durations of the individual phases.

Presentations and Age-related Vision Changes

It is well known that people's vision degrades as they age. Imagine you are making slides for a presentation and want to optimize them for an audience with potential vision difficulties.

This script can be used to estimate how an elderly person  might perceive your slides (needs ImageMagick, GMIC). An example slide is shown below.

The script applies a general blur to the slides, a yellow tint to account for yellowing of the lens and reduced color vision. At last, it applies a vignette to account for the loss of peripheral vision.

Presenting a Bibtex Bibliography on a Website

Imagine you are given a long Bibtex file, that needs to be presented on a website. In order to achieve some flexibility in presenting the data, a MySQL database is to be used. This rules out alternatives such as bibtex2html, which creates static html pages.

A Python script that can be used to parse the Bibtex file and write it into the database can be downloaded here. Pybtex is used to conveniently parse the Bibtex file. The database layout for the different publication types (article, inproceedings, incollection, etc.) is hardcoded into the script, but should be easily adaptable to your needs.

Ordering Pictures by Dissimilarity

Imagine you want to post some pictures on the web, eg. for a social media post or a picture gallery on the web. Sometimes the context defines how the pictures have to be ordered, eg. in chronological order, but sometimes you also have the freedom to order the pictures, such that the presentation becomes more interesting.

The question is whether it is possible to automatically find an ordering for the pictures that makes the presentation more interesting to the user. The idea of this post is that the pictures should be ordered to maximize dissimilarity between consecutive pictures. It should catch the the viewers attention if the visual impression of two consecutive pictures is as different as possible.

A metric is required to formalize 'dissimilarity'. Different options exist here, but one approach is to look at the histograms of every picture and quantify the similarity between two pictures as the statistical distance between their histograms.

A script that implements this idea can be downloaded here (needs OpenCV). The Bhattacharyya distance is used as one example of a statistical distance. An example of a picture set in 'boring order' and a picture set that has been sorted as outlined above is shown below:

Lenna in 'boring order'

Lenna in 'interesting order'

DDNS and IPv6?

Ever tried setting up DDNS with IPv6? I recently had to learn that there is a remarkable range of tools and services that do not work with IPv6.

ddclient does not support IPv6, nor does inadyn.
inadyn-mt claims to support IPv6, but if that is true, it is at least hard to configure.

Also not all DDNS-Services offer IPv6 support with freedns.afraid.org being one notable exception. This service also allows setting the IPv6 via a URL.

A pragmatic solution to get DDNS working with IPv6 could be running the following Python script periodically using a cronjob:

#!/usr/bin/env python

''' update ipv6 record on freedns.afraid.org '''

import netifaces

import subprocess

import sys

iface_name = "eth0"

pwd_hash = "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqr"

try:

addrs = netifaces.ifaddresses(iface_name)

ipv6_str = addrs[netifaces.AF_INET6][0]['addr']

#check if link local adress

if ipv6_str[0:5] == "fe80:":

raise Exception()

except:

sys.exit("could not determine ipv6 address");

subprocess.call(

[

"wget",

"-q",

"--read-timeout=0.0",

"--waitretry=5",

"--tries=400",

"https://freedns.afraid.org/dynamic/update.php?"+pwd_hash+"&address="+ipv6_str

]

)

The script can also be downloaded here. It requires the package netifaces, which can be installed using pip. Moreover wget needs to be installed. You need to adjust the interface name and the password hash manually.
$> sudo apt-get install python-pip
$> sudo pip install netifaces

I suggest simply copying the script to /opt, making it executable and adding a cronjob for it.
$> sudo crontab -e

add the following line to the crontab
 */30 * * * *    /opt/ipv6_update.py

and restart the cron daemon
$> sudo /etc/init.d/cron restart

If you are using a network manager, eg. wicd or network-manager, the Python script above can be hooked in there. For wicd the script would have to be copied to
/etc/wicd/scripts/postconnect

Experiences from running a Tor intermediate relay

In this post I want to share some basic experiences from running a Tor intermediate relay for 5 months.

Resource Usage

Fortunately Tor's hardware requirements are modest. An old  ASUS EeePC 1101HA netbook with an Intel Atom CPU Z520 @ 1.33GHz and 1GB of RAM was used for the Tor node and found to be sufficient. The node was used in conjunction with a simple cable internet connection (10Mbit/s downstream; 1Mbit/s upstream). Amazingly and despite the cheap hardware, the system worked stable and no system crashes occurred.

Tor's  memory usage is about 210MB and the memory usage of the entire system is only about 250MB.

The  bandwidth usage was around 100kBytes/s on average and a little more than 1GByte/day was up- and downloaded. The bandwidth is mostly consumed in bursts since Tor is designed to reduce latency and then the system goes idle to not exceed the bandwidth limit. The relay was also used to mirror directory information.

The temperature of the CPU was around 27°C, which is not so much above ambient temperature. This conveniently reduces aging and noise from the fan.

Power Consumption

An attempt was made to reduce power consumption by

• uninstalling unnecessary services (eg. CUPS)
• configuring DPMS to turn off the screen quickly eg. turning off the screen after 10s:  $> xset dpms 10 0 0

• alternatively it is possible to turn off the screen using vbetool dpms off , if no X server is present. Strangely the screen will automatically turn itself back on after some time and a cron job is required to keep it turned off permanently.
• shutting down Wifi and Bluetooth (Fn+... on the keyboard) 
• using powertop for further optimizations

Assuming a (probably too high) power consumption of 15W and a price for electricity of 0.2€/kWh, the energy bill is increased by around 26€ per year through this act of altruism. For a future relay, I intend to use an old smartphone or tablet computer, which will reduce power consumption significantly.

Configuration

The configuration was pretty usual. Some minor things to be mentioned here:

To be able to use the tor relay from the local network, eg.
SocksPort 192.168.178.42:910
SocksPolicy accept 192.168.178.0/16
It seems Tor will not start automatically after reboot, if these lines are in the config file.

To view connection information (eg. open circuits) from arm:
DisableDebuggerAttachment 0

To fix the exit node to a specific country:
ExitNodes de 

Software 

Not much software was required:

• Debian 7 (Wheezy) was used as OS.
• wicd-curses was used as a network manager for convenient configuration of the network interface
• Tor was installed from the Debian repositories.
• X was necessary for DPMS (energy management); alternatively vbetool is sufficient to turn the screen off.
• OpenBox was used as a resource friendly window manager and for convenience
• The hardware clock of the EeePC seems to be drifting considerably. Therefore it was necessary to install ntpd to keep system clock in sync.
• some other optional tools (arm, htop, powertop, sensors, unattended-upgrades)

Statistics about the Tor node can be obtained through Atlas and arm. arm crashes after some time, if left running.

arm needs to be started as the user running tor eg.
$> sudo -u yourusername-tor arm

Wget, Cookies and Firefox

Did you ever want to automatically (mass)-download data from a website, where a login is required, eg. a wiki or a social network? If the website stores a session cookie on your computer, it might be possible to download content automatedly using Wget.

It is possible to pass Wget a cookie file as a parameter. This might look like the following:
wget --keep-session-cookies --load-cookies=cookies.txt -p -k https://someurl.org/protected/site_01.htm

An example of a cookie file might look as follows (use tabs instead of spaces!):
# HTTP cookie file.
someurl.org  TRUE  /  FALSE  1391671828  someurlUserID  42
someurl.org  TRUE  /  FALSE  1391671828  someurlUserName Peter
someurl.org  TRUE  /  FALSE  1391671828  someurlToken  d3d3fdsere
someurl.org  TRUE  /  FALSE  -1  someurl_session  g8furfv99dmp1

After logging in on the respective website, you can conveniently view the necessary cookies in Firefox.

date can be used to convert the expiration time of the cookies in Firefox to the format used in the wget cookie files, eg. by issuing:
date -d "Wed 12 Mar 2014 01:31:42 PM CET" +%s

Installing Tizen SDK on Ubuntu with OpenJDK

Today I tried installing the Tizen SDK (tizen-sdk-ubuntu64-v2.2.71.bin) on Ubuntu 12.04. The installation script exited complaining that it requires Oracle JDK instead of OpenJDK ("OpenJDK is not supported. Try again with Oracle JDK.").

This is a bit annoying, because OpenJDK comes with Ubuntu per default and Oracle JDK is not in the repositories any more. It seems, the installer's requirement is merely a policy and not an actual technical requirement. The installation does succeed even with the OpenJDK, if the following lines are commented out in the installation script:

# check the default java as OpenJDK ##
if [ "ubuntu" = "${OS_NAME}" ] ; then
    CHECK_OPENJDK=`java -version 2>&1 | egrep -e OpenJDK`
    if [ -n "${CHECK_OPENJDK}" ] ; then
        echo "${CE} OpenJDK is not supported. Try again with Oracle JDK. ${CN}"
        exit 1
    fi
fi

The installation and basic usage of the IDE seem to work without problems after this. The OpenJDK version used was 1.6.0_27.