Programming - Silas Sewell

Push: Func Module to Run Arbitrary Python Code

Func is a nifty, although not yet polished, Python-based service for running tasks on many hosts at once. It lets you do things like restart Apache instances, run yum updates or provision virtual machines.

You can accomplish theses tasks via the command line interface

func web-*.example.net call service restart httpd

or through the Python API

import func.overlord.client as fc
 
client = fc.Client('web-*.example.net')
 
print client.service.restart('httpd')

One of the annoying issues I’m dealing with right now is the distribution of Func modules, which for various reasons isn’t as simple as packaging the modules and pushing them to all the hosts (it is, just not in my network).

My solution is Push, a Func module which lets you instantly run Python code on any or all hosts in your network.

Example usage:

import func.overlord.client as fc
 
client = fc.Client('web-*.example.net')
 
source = """
def main():
  return 'Hello World'
"""
 
print client.push.code(source)

I realize this isn’t the most elegant solution, but its very useful and extremely simple.

I also realize that I could use the copyfile module to push modules via Func, but I haven’t found a way to restart funcd via Func (required to initialize modules), so until that happens I need a solution which doesn’t require restarting Func.

Get Push Module

Posted November 18th, 2008 in Programming, Systems Administration | No Comments

C GMP Hello World on Leopard (OS X 10.5)

A simple GMP Hello World example in C.

Makefile

all:
        gcc -o gmp_hello_world gmp_hello_world.c -lgmp -m64
 
clean:
        rm gmp_hello_world

gmp_hello_world.c

/*
 * GMP Hello World on OS X 10.5
*/
#include <gmp.h>
#include <stdio.h>
 
int main() {
  mpz_t add_total1, add_total2, sub_total1, sub_total2, mul_total1, mul_total2;
  mpz_t num1, num2;
 
  // Initialize variables
  mpz_init_set_str(num1, "345192567923875922375736284875732", 10);
  mpz_init_set_str(num2, "937929298382994742939293857584837", 10);
  mpz_init(add_total1);
  mpz_init(add_total2);
  mpz_init(sub_total1);
  mpz_init(sub_total2);
  mpz_init(mul_total1);
  mpz_init(mul_total2);
 
  // Do arithmetic
  mpz_add(add_total1, num1, num2);
  mpz_add_ui(add_total2, num1, 10);
  mpz_sub(sub_total1, num1, num2);
  mpz_sub_ui(sub_total2, num1, 10);
  mpz_mul(mul_total1, num1, num2);
  mpz_mul_ui(mul_total2, num1, 10);
 
  // Display results
  gmp_printf("Add Total 1: %Zd\n", add_total1);
  gmp_printf("Add Total 2: %Zd\n", add_total2);
  gmp_printf("Subtract Total 1: %Zd\n", sub_total1);
  gmp_printf("Subtract Total 2: %Zd\n", sub_total2);
  gmp_printf("Multiply Total 1: %Zd\n", mul_total1);
  gmp_printf("Multiply Total 2: %Zd\n", mul_total2);
 
  // Free space
  mpz_clear(num1);
  mpz_clear(num2);
  mpz_clear(add_total1);
  mpz_clear(add_total2);
  mpz_clear(sub_total1);
  mpz_clear(sub_total2);
  mpz_clear(mul_total1);
  mpz_clear(mul_total2);
 
  return 0;
}

Link: gmp_hello_world

Posted October 19th, 2008 in Mathematics, Programming | No Comments

Persistent Queuing in Python

Peafowl is a Python port of Ruby’s Starling.

Posted September 26th, 2008 in Programming | No Comments

Universal Feed Parser to JSON

My attempt at wrangling a Universal Feed Parser object into JSON.

import feedparser
 
def json_make_normal(obj):
    if type(obj) in [str, unicode, int, float, bool, dict, set, list, tuple]:
        return obj
    try: return dict(obj)
    except: pass
    try: return list(obj)
    except: pass
    return None
 
def json_handle(obj):
    obj = json_make_normal(obj)
    if type(obj) in [str, unicode]:
        obj = obj.replace('\\', '\\\\')
        obj = obj.replace('"', '\\"')
        obj = obj.replace('\b', '\\\b')
        obj = obj.replace('\f', '\\\f')
        obj = obj.replace('\n', '\\\n')
        obj = obj.replace('\r', '\\\r')
        obj = obj.replace('\t', '\\\t')
        return '"%s"' % obj
    elif type(obj) in [int, float]:
        return obj
    elif type(obj) is bool:
        if obj: return 'true'
        else: return 'false'
    elif type(obj) is type(None):
        return 'null'
    elif type(obj) is dict:
        temp = ''
        for key in obj.keys():
            temp += '%s:%s, ' % (json_handle(key), json_handle(obj[key]))
        return '{%s}' % temp[:-2]
    elif type(obj) in [set, list, tuple]:
        temp = ''
        for value in obj:
            temp += '%s, ' % json_handle(value)
        return '[%s]' % temp[:-2]
    return 'null'
 
data = feedparser.parse('http://digg.com/rss/index.xml')
 
print json_handle(data)

Link: feed_parser_to_json.py

Update: According to John Paulett jsonpickle now supports Universal Feed Parser objects. Thanks for letting me know!

Posted May 5th, 2008 in Programming | No Comments

Post a Get

I’ve created a PHP script called Post a Get which can be copied to any PHP capable server and used to created a POST request using GET syntax.

Let’s say you use GoDaddy as your domain registrar and you would like to see if a domain is available using Firefox keywords, but they don’t offer a GET search.

For a traditional keyword, you could construct a GET request like:

https://www.godaddy.com/gdshop/registrar/search.asp&amp;checkAvail=1&amp;fblur=1&amp;tld=.com&amp;domainToCheck=%s

But as of right now Firefox doesn’t offer a way to create POST keywords. What Post a Fix does is allow you to construct a POST request using GET syntax and pass it to the Post a Get code.

So if you wanted to submit the above GET as a POST, you would construct the URL and append the special __action parameter with target URL. So the final URL would look something like:

http://www.example.org/postaget/index.php?__action=https://www.godaddy.com/gdshop/registrar/search.asp&amp;checkAvail=1&amp;tld=.com&amp;domainToCheck=%s

Posted March 30th, 2008 in Programming, Projects | No Comments