I wrote something like this some time ago, but this version is much better, if only because it's in python. Basically, it's a script that highlights standard input based on arguments passed to it.

But how is that useful? Well imagine that you've dumped the contents of a file to standard output, maybe even piped it through grep, and/or sed etc. Oftentimes you're still left with a lot of text and it's hard to find what you're looking for. If only there was a way to highlight arbitrary portions of the text with some colour...

Here's what you do:

$ cat somefile | highlight.py some strings

You'll be presented with the same body of text, but with the word "some" highlighted everywhere in light blue and "strings" highlighted in light green. The script can support up to nine arguments which will show up in different colours. I hope someone finds it useful.

#!/usr/bin/env python3
# -*- coding: utf-8 -*-

import sys,re

colours = [
    "\033[1;34m", # light blue
    "\033[1;32m", # light green
    "\033[1;36m", # light cyan
    "\033[1;31m", # light red
    "\033[1;33m", # yellow
    "\033[0;32m", # green
    "\033[0;36m", # cyan
    "\033[0;33m", # brown
    "\033[1;35m", # pink
    "\033[0m"     # none
]

args = sys.argv[1:]

# Strip out arguments exceeding the maximum
if len(args) > 9:
    print("\n%sWARNING: This script only allows for a maximum of 9 arguments.%s\n\n" % (colours[4], colours[9]), file=sys.stderr)
    args = args[0:8]

while True:
    line = sys.stdin.readline()
    colour = 0
    for arg in args:
        line = re.sub(
            r"(%s)" % (arg),
            "%s%s%s" % (colours[colour], "\g<1>", colours[9]),
            line
        )
        colour = colour + 1
    if line == '':
        break
    try:
        print(line.rstrip("\n"))
    except:
        pass

Wil Wheton posted to Twitter today a request for an easy way to fetch all of one's tweets and store them locally. Someone might want to do that if they want a personal archive, or if they're interested in porting their data over to a Free implimentation like Laconica. Whatever your reasoning, here's a quick and dirty way to do it:

for i in {1..999}; do
  curl -s "http://twitter.com/statuses/user_timeline.xml?screen_name=your_screen_name&count=200&page=$i" | grep '<text>' | sed -e 's/^ *<text>\(.*\)<\/text>/\1/'
  sleep 2
done

Just hit "ctrl-c" when you hit your first post ever.

I wrote something rather fun today and I thought that I'd share it here. It's a Python module that you can use to interact with PHP products. Specifically, it's a reproduction of PHP's http_build_query() and parse_ini_file() functions that act as PHP does according to PHP's own way of doing things.

This means that if you've written an API server (as we have) in PHP that makes use of things like the above, you can interact with it using Python as your scripting language with little effort.

Examples:

from php import parse_ini_file

config = parse_ini_file("/path/to/config.ini")
print config["sectionName"]["keyName"]

This would give you the value for keyName in the section called sectionName in your config.ini file.

from php import http_build_query

somedata = {
  "keyname": "valuename",
  "otherkey": 123,
  "anotherkey": [1,2,3,{"seven": "eight"}]
}
print http_build_query(somedata)

This would give you:

otherkey=123&keyname=valuename&anotherkey[1]=2&anotherkey[0]=1&anotherkey[3][seven]=eight&anotherkey[2]=3&

The code was fun to write, and I'm guessing that it'll be useful to others so I'm posting it here. If you do end up using it, lemme know by posting a comment here eh?

You can download it here: php.py.

When I mentioned this to some other coworkers, they pointed out that I'm not the only one trying to get some of PHP's odd functionality into Python. Another developer has mimicked PHP's serialize() functions in the form of a Python module. I wonder if there are any other cases where this kind of stuff might be useful.

I stumbled upon an ugly PHP bug today and thought that I would share. While PHP is supposed to be a untyped language, this isn't always the case. The following code snippet for example does not do what you might expect:

switch ($output->status)
{
  case 0: $output->status = 'fail'; break;
  case 1: $output->status = 'ok';   break;
  case 2: $output->status = 'stub'; break;
}

With this code, passing in a string such as "ok", $output->status is set to "fail". This is due to what I assume to be a bug in PHP's lack of keeping everything untyped. For some reason, it would seem that PHP parses $output->status as an integer (therefore all strings return as 0) and then compares them to the list. If however you change the cases to strings:

switch ($output->status)
{
  case '0': $output->status = 'fail'; break;
  case '1': $output->status = 'ok';   break;
  case '2': $output->status = 'stub'; break;
}

Everything works as expected. Pretty lame if you ask me, but there it is.

Big thanks to Corey for brightening my day with this one:

1. Disbelief
   • "Who wrote this!?
2. Anger
   • "I'm not cleaning this up!"
3. Bargaining
   • "Okay, we'll fix up this module if you promise we'll just rewrite everything else."
4. Depression
   • "This is never going to get any better."
5. Acceptance
   • "I'll just create a wrapper..."

I just read a nifty post on monkeycycle about how to geocode an spreadsheet with free tools from Google and Yahoo and it occurred to me that this is probably the kind of thing people go looking for so I thought that I'd post my latest shiny new bit of code here.

I call it a cascading geocoder. The idea being that most of the time, a single geocoding service is pretty good, but sometimes it goes down, and other times it can't understand the address. For the purposes of the project I'm working on, this wasn't permissible, so I wrote some code that attempts to code an address first with Google, then if that fails, it uses geocoder.ca's engine.

It's fully object oriented and very clean. It's also GPL. Download it here if you're interested ^_^

I've been assigned to a junior-level programmer here at the office to teach her how to write code for the server I've been labouring on over the past six months. The system is my brainchild, my baby and it's with a mix of relief and aprehension that I'm taking on this new apprentice for this project.

And then I saw this at the top of her first class file:

<?php



	/**
	*
	*   Author: Coworker's Name (coworker@donatgroup.com)
	*  Licence: GPL-3 "Information wants to be free"

Granted, she forgot to capitalise "Free" but it's a pretty good start ;-)

Yeah, I know it's late, but I got obsessed and couldn't turn away. Now I have a really slick python script that pulls down the latest Questionable Content strip and adds it to my collection to read later:

#!/usr/bin/env python

import os, sys, re
import pycurl, urllib2

destination = "/path/to/qc/repository"

class QC:

  def __init__(self):

    self.contents = ''
    self.baseurl  = 'http://questionablecontent.net/'


  def callback(self,buf):

    self.contents = self.contents + buf


  def end(self):

    c = pycurl.Curl()
    c.setopt(c.URL, self.baseurl)
    c.setopt(c.WRITEFUNCTION, self.callback)
    c.perform()
    c.close()

    try:
      return int(re.search('.*\/comics\/(\d+)\.png', self.contents).group(1))
    except:
      print ""
      print "  The regular expression no longer works."
      print "  You might want to check into that."
      print ""
      exit()


  def fetch(self,n):

    strip = urllib2.build_opener().open(self.baseurl + 'comics/' + str(n) + ".png").read()

    f = "%s%s%04d%s" % (destination, "/", n, ".png")

    fout = open(f, "wb")
    fout.write(strip)
    fout.close()


# Main -----------------------------------------------------------------------

qc = QC()

for i in range(1, qc.end() + 1):
  f = "%s%s%04d%s" % (destination, "/", i, ".png")
  if not os.path.exists(f):
    print "Fetching strip #" + str(i)
    qc.fetch(i)

I just wrote this nifty wrapper for Simpletest, a unit testing suite for PHP and thought that I'd share it here.

Basically, the text output for Simpletest is ugly. And it's kinda lacking in output for those of us who are visually gratified. I like to see my tests pass and I like to see something green when everything is ok. Mock me if you like, I dig the pretty :-)

To use this, just put the following block into a file and call it something like TextReporterWithPasses.php and then, where your test suite script looks something like this:

  $test = new GroupTest('MTV API Layer');
  // Some $test->addTestFile() stuff
  $test->run();

Do this instead:

  require 'TextReporterWithPasses.php';
  $test = new GroupTest('MTV API Layer');
  // Some $test->addTestFile() stuff
  $test->run(new TextReporterWithPasses());

And behold the pretty ;-) Here's the code:

<?



  /**
  *
  *   Author: Daniel Quinn (daniel.quinn@donatgroup.com)
  *  License: GPL-3 "Information wants to be Free"
  * Function: Prettifies Simpletest's text output
  *
  */


  class TextReporterWithPasses extends TextReporter
  {

    private $_colours;
    private $_testCount;
    private $_errors;

    public function __construct()
    {
      parent::__construct();

      $this->_testCount = 0;
      $this->_errors = array();
      
      $this->_setColours();
    }


    public function paintPass($message)
    {
      $this->_passes++;
      print ($this->_testCount == 0) ? '  .' : '.';
      $this->_manageWrapping();
    }


    public function paintFail($message)
    {
      $this->_fails++;
      $error = new stdClass();
      $error->message = $message;
      $error->breadcrumb = "\tin ". implode("\n\tin ", array_reverse($this->getTestList()));

      $this->_errors[] = $error;

      print $this->_colours['red-light'];
      print ($this->_testCount == 0) ? '  x' : 'x';
      print $this->_colours['grey'];

      $this->_manageWrapping();

    }


    public function paintHeader($name)
    {
      if (!SimpleReporter::inCli())
      {
        header('Content-type: text/plain');
      }

      print "\n  ". $this->_colours['white'] . $name . $this->_colours['grey'] . "\n  ----------------------------------------------------------------------------\n";

      flush();

    }


    public function paintFooter()
    {

      if ($this->getFailCount() + $this->getExceptionCount() == 0)
      {
        print "\n\n  ". $this->_colours['white'] .'[ '. $this->_colours['green-light'] .'w00t!'. $this->_colours['white'] ." ]\n";
      }
      else
      {
        print "\n\n". $this->_colours['red-light'] ."  The result was less awesome than you might have hoped :-(\n\n". $this->_colours['red'];
        foreach ($this->_errors as $e)
        {
          print "  $e->message";
        }
        print "\n";
      }

      $status = array(
        'run' => $this->getTestCaseProgress() ."/". $this->getTestCaseCount(),
        'passes' => $this->getPassCount(),
        'failures' => $this->getFailCount(),
        'exceptions' => $this->getExceptionCount()
      );

      print $this->_colours['white'];

      print "\n  Test cases run: $status[run], Passes: $status[passes], Failures: $status[failures], Exceptions: $status[exceptions]\n\n";

      print $this->_colours['none'];

    }


    // Private methods -------------------------------------------

    private function _setColours()
    {
      $this->_colours = array(
        'black'       => "\033[0;30m",
        'blue'        => "\033[0;34m",
        'blue-light'  => "\033[1;34m",
        'green'       => "\033[0;32m",
        'green-light' => "\033[1;32m",
        'cyan'        => "\033[0;36m",
        'cyan-light'  => "\033[1;36m",
        'red'         => "\033[0;31m",
        'red-light'   => "\033[1;31m",
        'purple'      => "\033[0;35m",
        'brown'       => "\033[0;33m",
        'grey'        => "\033[1;30m",
        'grey-light'  => "\033[0;37m",
        'pink'        => "\033[1;35m",
        'yellow'      => "\033[1;33m",
        'white'       => "\033[1;37m",
        'none'        => "\033[0m"
      );
    }


    private function _manageWrapping()
    {
      // Poor man's wrapping
      if (++$this->_testCount == 76)
      {
        print "\n";
        $this->_testCount = 0;
      }
    }


  }


?>

Isn't it cool that I work for a company that supports the Free distribution of code?

I'm finally understanding what it's like to write code for Drupal. Essentially, you have to imagine the least efficient way of doing something, then look for how to do that... 'cause that's exactly what Drupal does.

I mean, why would anyone want to create additional columns to a user table when they could just put a bunch of serialised data into a string in the user table? It's just so easy! Why didn't I think of that?

Oh, right, 'cause I'm not insane.