Tuesday, January 7, 2020

Checking on Rule Utility


Currently churning through a bunch of rules to see if they're as good as I hope they are. (If you aren't measuring it, it might just be wishful thinking.)

So, run hashcat with --debug-mode=4 --debug-file=foo and make sure you have an empty pot file if you want representantive data.

Post process with this:

#!/bin/perl

while ($line=<STDIN>)
{
    chomp($line);

    if ($line=~m/^([^:]+):(.+):([^:]+)$/) {
        $line=$2;
    }

    if (!defined($f{$line})) {
        $f{$line}=1;
    } else {
        $f{$line}=$f{$line}+1;
    }
}

foreach my $rule (sort { $f{$b} <=> $f{$a} } keys %f) {
    if ($f{$rule}>1) {
        printf "%s,%s\n", $rule, $f{$rule};
    }
}

Which gives you rule,# occurrences if you run the debug file through it. Then you can plot log_10 of the frequencies and see which ones are actually any use.

This is provisional data, looks like a decent run will take a week or two...
(HIBP data, Top258Million-probable.txt dict)