Downgrading Subversion from 1.8 to 1.7 in MacPorts

bash-3.2# cd /tmp
bash-3.2# svn co http://svn.macports.org/repository/macports/trunk/dports/devel/subversion --revision 108493
A    subversion/files
A    subversion/files/patch-Makefile.in.diff
A    subversion/files/patch-osx_unicode_precomp.diff
A    subversion/files/config_impl.h.patch
A    subversion/files/servers.default
A    subversion/Portfile
Ausgecheckt, Revision 108493.
bash-3.2# cd subversion/
bash-3.2# port install
--->  Computing dependencies for subversion
--->  Fetching archive for subversion
--->  Attempting to fetch subversion-1.7.10_1.darwin_12.x86_64.tbz2 from http://mse.uk.packages.macports.org/sites/packages.macports.org/subversion
--->  Attempting to fetch subversion-1.7.10_1.darwin_12.x86_64.tbz2.rmd160 from http://mse.uk.packages.macports.org/sites/packages.macports.org/subversion
...
--->  Scanning binaries for linking errors: 100.0%
--->  No broken files found.
bash-3.2# port installed subversion
The following ports are currently installed:
  subversion @1.7.10_1
  subversion @1.8.1_1 (active)
bash-3.2# port activate subversion @1.7.10_1
--->  Computing dependencies for subversion
--->  Deactivating subversion @1.8.1_1
--->  Cleaning subversion
--->  Activating subversion @1.7.10_1
--->  Cleaning subversion

Nice Vagrant Script for setting up a LDAP VM for Testing

I found a nice Setup Script at https://github.com/gschueler/vagrant-rundeck-ldap for creating a working LDAP VM with a basic

vagrant up

You just need to do some more steps before:

  • make sure you have installed Version 4.2.12 of Virtualbox! (the latest version – 4.2.14 – breaks vagrant)
  • get your vagrant setup from here
  • download the necessary ubuntu vagrant box:
    vagrant box add precise32 http://files.vagrantup.com/precise32.box

The LDAP port will be mapped to port 3890 in your local machine (the host where virtualbox is installed)
You can login (e.g. with Apache Directory Studio) with the following settings:

Encryption: none
Host: localhost
Port: 3890
User: dc=Manager,dc=example,dc=com
Pass: password

Hacking just for Fun: using Bookmarklets

So there are a handful of webtools using Bookmarklets for their services. The first i know was del.icio.us for saving a Webpage to your del.icio.us bookmarks. Another famous service is Instapaper (it uses internally read it later pocket, but that is another Story). I have a special service in mind, i want to create using a Bookmarklets, before i start, i played around with the Bookmarklet from Instapaper.

The Magic is just a normale HTML A Tag, in whith some javascript is embedded:

javascript:function%20iprl5(){var%20d=document,z=d.createElement('scr'+'ipt'),b=d.body,l=d.location;try{if(!b)throw(0);d.title='(Saving...)%20'+d.title;z.setAttribute('src',l.protocol+'//www.instapaper.com/j/foobar?u='+encodeURIComponent(l.href)+'&t='+(new%20Date().getTime()));b.appendChild(z);}catch(e){alert('Please%20wait%20until%20the%20page%20has%20loaded.');}}iprl5();void(0)

If we unwrap that script we got

function iprl5(){
    var d=document,
    z=d.createElement('scr'+'ipt'),
    b=d.body,
    l=d.location;
    try{
        if(!b)
            throw(0);
        d.title='(Saving...) '+d.title;
        z.setAttribute('src',l.protocol+'//www.instapaper.com/j/foobar?u='+encodeURIComponent(l.href)+'&t='+(new Date().getTime()));
        b.appendChild(z);
    }catch(e){
        alert('Please wait until the page has loaded.');
    }
}
iprl5();
void(0)

The command basically just creates a script tag in the active DOM-Tree and preloads some Javascript-File. The File is then executed by the Browsers JS Runtime.
I started a litte Demo Project. You find it here on GitHub. It is based on Play! 1.2.4 (Installation Guide here).

ATM there are just two JS Templates. The first one (app/views/Application/bookmarklet.js) just contains the source for the Script-Tag itself. The second (app/views/Application/input.js) will be loaded then after the Javascript behinde that link is called.

Hacking just for Fun: Raid5 in Java

I was just curious how easy it might be to write a RAID5 compatible Outputstream in Java? Just a few Lines. For sure it is not the most elegante solution. Especially if you see the nice possibility to integrate one Outputstream within another… so maybe two Raid5s into one Raid0 Stream? (would be RAID50) then. Reading is missing ;-).

/**
 * Raid5Stream
 * 31.03.2012
 * @author Philipp Haussleiter
 *
 */
package de.javastream.jraid;

import java.io.File;
import java.io.FileOutputStream;
import java.io.IOException;
import java.io.OutputStream;

public class Raid5Stream extends OutputStream {

    public final static int PARITY_MARKER = 1;
    public final static int DATA_MARKER = 0;
    private File disks[];
    private FileOutputStream streams[];
    private int mode = 0;
    private final static int MAX_MODE = 2;
    private int[] buffer = new int[2];

    public Raid5Stream(File disks[]) throws IOException {
        super();
        if (disks.length != 3) {
            throw new RuntimeException("we need a disk count of x times 3");
        }
        int i = 0;
        this.disks = new File[disks.length];
        this.streams = new FileOutputStream[disks.length];
        for (File f : disks) {
            if (!f.exists() && !f.createNewFile()) {
                throw new RuntimeException(f.getAbsolutePath() + " does not exists and cannot be created!");
            } else {
                System.out.println("using " + f.getAbsolutePath() + "\n");
                this.disks[i] = f;
                this.streams[i] = new FileOutputStream(f);
            }
            i++;
        }
        writerMarker();
    }

    private void writerMarker() throws IOException {
        for (int i = 0; i < this.streams.length; i++) {
            if (i % 3 == 0) {
                this.streams[i].write(PARITY_MARKER);
            } else {
                this.streams[i].write(DATA_MARKER);
            }
        }
    }

    @Override
    public void write(int i) throws IOException {
        switch (mode) {
            case MAX_MODE:
                this.streams[mode].write(buffer[0] ^ buffer[1]);
                mode = 0;
            default:
                this.buffer[mode] = i;
                this.streams[mode].write(i);
                mode++;
        }
    }
}

Using it is easy as:

...
public class App {

    public static void main(String[] args) throws IOException {
        File raid5_disk1 = new File("raid5.disk1");
        File raid5_disk2 = new File("raid5.disk2");
        File raid5_disk3 = new File("raid5.disk3");
        long count = 0;
        Raid5Stream raid5Stream = new Raid5Stream(new File[]{raid5_disk1, raid5_disk2, raid5_disk3});
        FileInputStream in = new FileInputStream("/dev/random");
        BufferedInputStream bis = new BufferedInputStream(in);
        BufferedOutputStream raid5bos = new BufferedOutputStream(raid5Stream);
        while(count < 1024*1024*2){
            raid5bos.write(bis.read());
            count++;
        }
        raid5bos.close();
        bis.close();
    }
}

Good sources for more reading are the Wikipedia Articles about RAID and XOR.

Run local/remote terminal commands with java using ssh

Sometimes you need to use some CLI-Tools before you want to create or search for a native JNI Binding.
So there is a common way, using the Java Process-Class. But then you might meet two problems i had to face in the past during several problems:

  1. There are (a really small) number of CLI-Tools, that giving no constant output over the STD-OUT (the standard output the Process-Class uses for output)
  2. There is no “elegant” way to implement a process call into your project.

To solve this Problem I created a basic HelperClass, that calls the System over SSH (with the Convenience to work remote and the side-effect to always get STD-compatible output).
I am primarely using it for a fun project SAM i started some months ago to try to create a Management-Tool for Unices and Windows with a very low client-side footprint.

The first Class is used to capsulate the basic SSH Calls:

 
// some imports... 
 public class SystemHelper {
    private Runtime r;
    private String sshPrefix = "";

    // call with $user and 127.0.0.1 to run local command.     
    public SystemHelper(String user, String ip) {
        r = Runtime.getRuntime();
        sshPrefix = " ssh -o UserKnownHostsFile=/dev/null -o StrictHostKeyChecking=no " + user + "@" + ip;
    }

    public void runCommand(String command, ProcessParser pp) {
        try {
            Logger.info("running: " + this.sshPrefix + " " + command);
            Process p = r.exec(this.sshPrefix + " " + command);
            InputStream in = p.getInputStream();
            BufferedInputStream buf = new BufferedInputStream(in);
            InputStreamReader inread = new InputStreamReader(buf);
            BufferedReader bufferedreader = new BufferedReader(inread);
            pp.parse(bufferedreader);
            try {
                if (p.waitFor() != 0) {
                    Logger.info("exit value = " + p.exitValue());
                }
            } catch (InterruptedException e) {
                System.err.println(e);
            } finally {
                // Close the InputStream                 
                bufferedreader.close();
                inread.close();
                buf.close();
                in.close();
            }
        } catch (IOException ex) {
            Logger.error(ex.getLocalizedMessage());
        }
    }
}

ProcessParser is an interface that defines the methode parse, accepting a BufferedReader for parsing the output of the Process. Unfortunately there is no timeout ATM to kill a hanging SSH-Call.

 public interface ProcessParser { 
     public void parse(BufferedReader bufferedreader); 
 } 

The most basic (Output-)Parser looks like this:

 
    public String getPublicSSHKey() {
        SimpeOutputPP so = new SimpeOutputPP();
        String command = "cat ~/.ssh/id_rsa.pub";
        runCommand(command, so);
        if (!so.getOutput().isEmpty()) {
            return so.getOutput().get(0);
        }
        return "";
    }

This returns just the public SSH-Key of the current user. I implemented some more parsers for the output of apt (dpkg), rpm and pacman. You can find them in the github project here.

Mass Conversion of different video files with handbreak-CLI

I am currently converting all my videos to a fixed format using handbrake. If you have a lot of videos to convert, the UI version of handbrake is not always the best solution. Handbrake offers a CLI, sometimes you it needs to be installed separately.

It has a lot of options ( https://trac.handbrake.fr/wiki/CLIGuide ). So the best is to go with your preferred preset.
I want to encode all video files in a given folder to the preset “universal”.
So i wrote a short bash script, to to the work:


#!/bin/bash
# Folder Setup
OUT_FOLDER=/home/media/out
IN_FOLDER=/home/media/in
DONE_FOLDER=$OUT_FOLDER/videos_done

if [ ! -f $DONE_FOLDER  ] ; then mkdir -p $DONE_FOLDER; fi
if [ ! -f $OUT_FOLDER  ] ; then mkdir -p $OUT_FOLDER; fi

#All Extensions of the input files
EXTS=( mp4 flv )
#Find Handbrake CLI
HB=`which HandBrakeCLI`

for ext in ${EXTS[@]}; do
	echo "for $ext"
	for FILENAME in `ls $IN_FOLDER/*.$ext`; do
		#echo $FILENAME
		$HB -i $FILENAME  -o $OUT_FOLDER/`basename "$FILENAME" .$ext`.mp4  --preset="Universal"
		mv $FILENAME $DONE_FOLDER/`basename "$FILENAME"`
	done
done

Another use-case might the conversion of some videos to flv for a web playback (or the other way round flv->mp4 for playback on iOS Devices)

setup your public SSH key to another UNIX Host

Normally you would prefer to use your public ssh key for login into a remote linux machine.
I created a script to perform the basic steps for inserting your public key into the hosts authorized_keys files.

The script looks like this:

#!/bin/bash

HOST=$1;
echo ">> setup your ssh keys for $HOST"
echo ""
echo ">> creating ssh keys on $HOST if necessary"
echo "(you need to enter your password)"
echo ""
ssh $HOST 'if [ ! -d ~/.ssh  ] ; then ssh-keygen -t rsa; fi'
echo ""
PUBKEY=`cat ~/.ssh/id_dsa.pub`
echo "=========================================================="
echo "your id_dsa.pub:"
echo "$PUBKEY"
echo "=========================================================="
echo ""
echo ">> transfering your public ssh key"
scp ~/.ssh/authorized_keys $HOST:~/.ssh/authorized_keys
ssh $HOST 'chmod 600 ~/.ssh/authorized_keys'
echo ""
echo ">> login with your public key"
echo "(should work without a password)"
ssh $HOST

A typical run might look like this:

imotep:~ philipp$ setupssh philipp@192.168.178.55
>> setup your ssh keys for philipp@192.168.178.55

>> creating ssh keys on philipp@192.168.178.55 if necessary
(you need to enter your password)

The authenticity of host '192.168.178.55 (192.168.178.55)' can't be established.
RSA key fingerprint is ...
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '192.168.178.55' (RSA) to the list of known hosts.
philipp@192.168.178.55's password:
Enter file in which to save the key (/home/philipp/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Generating public/private rsa key pair.
Created directory '/home/philipp/.ssh'.
Your identification has been saved in /home/philipp/.ssh/id_rsa.
Your public key has been saved in /home/philipp/.ssh/id_rsa.pub.
The key fingerprint is:
... philipp@debian
The key's randomart image is:
+--[ RSA 2048]----+
|...              |
+-----------------+

==========================================================
your id_dsa.pub:
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx......xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
==========================================================

>> transfering your public ssh key
philipp@192.168.178.55's password:
authorized_keys                                                                         100%  610     0.6KB/s   00:00    

>> login with your public key
(should work without a password)
Linux debian 2.6.26-2-amd64 #1 SMP Thu Nov 25 04:30:55 UTC 2010 x86_64

The programs included with the Debian GNU/Linux system are free software;
the exact distribution terms for each program are described in the
individual files in /usr/share/doc/*/copyright.

Debian GNU/Linux comes with ABSOLUTELY NO WARRANTY, to the extent
permitted by applicable law.
Last login: Sun Jan 23 17:31:16 2011 from imotep.fritz.box
philipp@debian:~$

Preparation for counter update

So after a long time i plan to give my counter-engine (^^) an update.
The main Problem is, that the counter itself was perfect for ~2000 counts. But now with 5000 count, the presentation of the results is just not readable anymore.
So i plan the following things to do during the next days (i hope i will finish this during tomorrow):

  • using a new drawing method: instead of drawing the whole graph, i will draw just time-slices of a particular periode. So e.g. if you want to plot the current graph with this method, it would be necessary to draw 24 slices with a month periode.
  • to get read of lots of code within the counter/presentation script, i want to use the build-in date-calculation functions of mysql.
    So e.g.
    SELECT something FROM `counts` WHERE DATE_SUB(CURDATE(),INTERVAL 30 DAY) <= date_col; Should be perfect :-).
  • To use this functions, i have to use the build-in datetime Datatype. At the moment i am storring all counts with a unix timestamp. To make the move, i have to do three steps:
    1. creating a new field: ALTER TABLE `counts` ADD `created` DATETIME NOT NULL;
    2. migrate the old dates: UPDATE `counts` SET created = from_unixtime(unixtime);
    3. finally i had to change to insert statement within the counter script:
      “INSERT INTO `counts` ( … user_string, created) VALUES … ‘”.$user_string.”‘, NOW()) “;

More next time…

Nexenta CP 2 RC1 & Java 6

root@sunny:/tank/home# apt-get install sun-java6-jre

root@sunny:/tank/home# java -version
dl failure on line 685Error: failed /usr/lib/jvm/java-6-sun-1.6.0.10/jre/lib/i386/client/libjvm.so, because ld.so.1: java: fatal: libCrun.so.1: open failed: No such file or directory

root@sunny:/tank/home# apt-get install sunwlibc

root@sunny:/tank/home# java -version
java version “1.6.0_10”
Java(TM) SE Runtime Environment (build 1.6.0_10-b33)
Java HotSpot(TM) Client VM (build 11.0-b15, mixed mode)