Object factory using the Python import mechanism

Today something related to Python programing … These snippets were rescued from a forgotten repository into my old Inspiron 6000.

Speaking clearly, this post is about one app (purpose of it is not relevant) which internal handlers class can be setting in configuration time. This idea is cool when you want minimize hookups in your app. For example, this solution is a easy way when, in a hypothetical future, a third developer want to extend your app with a new handler. In this scenario, this developer only should build the handler as a external module and load it in the python classpath.

First step, I build my Factory class. These objects class create DataSources objects:

class DataSourceFactory (object):

def create (self,handlerClassname, object_uid=None, key_value=None, \
handler_options={}):

...
modulename, classname = handlerClassname.rsplit('.',1)
module = __import__(modulename, {}, {}, classname)
handler_class = getattr (module,classname)
ds_handler = handler_class()
for k,v in handler_options.items():
setattr(ds_handler, k, eval(v))

ds = DataSource(ds_handler)
ds.key_value = key_value
ds.object_uid = object_uid

return ds

You note, two things in the previous code:

  • The create function received a string with the classname of the handler
  • I use getattr and __import__ ( object reflexion ) for instantiate the hadler objects received as parameter

The classname, in my app, is setting in the app configuration file. This file is a standard Python config file:

[origin]
datasource_handler='syncldap.datasources.LdapDataSourceHandler'
key_value='sid'
object_uid='sid'
...

This confs are loaded into de app using the RawConfigParser:

def create_Synchronizer(self,config_filename):
  # RawConfigParser not interpolate attribute values
  cfg = self.ConfigParser.RawConfigParser()
  cfg.readfp(file(config_filename))

# DataSourceFactory
data_source_factory = self.datasources.DataSourceFactory()
# Load class name of origin handler
origin_data_source_handler_classname = \
eval (cfg.get('origin','datasource_handler'))
# For example: 'syncldap.datasources.LdapDataSourceHandler'

# Load origin options
origin_handler_options = dict (cfg.items('opt:origin_handler'))
origin_key_value = eval \
  (cfg.get('origin','key_value'))

origin_object_uid = eval \
  (cfg.get('origin','object_uid'))

# Creating origin source
origin_source = \
  data_source_factory.create(origin_data_source_handler_classname, \
  origin_object_uid, origin_key_value, origin_handler_options)

ClusterSSH: A GTK parallel SSH tool

From some time ago, I’m using a amazing admin tool named clusterSSH (aka cssh).

With this tool (packages available for GNU/Debian like distributions, at least), we
can interact simultaneously against a servers cluster
. This is very useful,
when your are making eventual tasks in similar servers
(for example, Tomcat Cluster nodes, … ) and you want execute the same intructions
in all of them.

cssh

My config (~/.csshrc) file for cssh is look like to the default settings:

auto_quit=yes
command=
comms=ssh
console_position=
extra_cluster_file=~/.clusters <<<<<<<<<
history_height=10
history_width=40
key_addhost=Control-Shift-plus
key_clientname=Alt-n
key_history=Alt-h
key_paste=Control-v
key_quit=Control-q
key_retilehosts=Alt-r
max_host_menu_items=30
method=ssh
mouse_paste=Button-2
rsh_args=
screen_reserve_bottom=60
screen_reserve_left=0
screen_reserve_right=0
screen_reserve_top=0
show_history=0
ssh=/usr/bin/ssh
ssh_args= -x -o ConnectTimeout=10
telnet_args=
terminal=/usr/bin/xterm
terminal_allow_send_events=-xrm ‘*.VT100.allowSendEvents:true’
terminal_args=
# terminal_bg_style=dark
terminal_colorize=1
terminal_decoration_height=10
terminal_decoration_width=8
terminal_font=6×13
terminal_reserve_bottom=0
terminal_reserve_left=5
terminal_reserve_right=0
terminal_reserve_top=5
terminal_size=80×24
terminal_title_opt=-T
title=CSSH
unmap_on_redraw=no
use_hotkeys=yes
window_tiling=yes
window_tiling_direction=right

The  ~/.clusters file is the file which defined the concrete clusters (see man ):

# home cluster
c-home tor@192.168.1.10 pablo@192.168.1.11

# promox-10.40.140
promox-10.40.140 10.40.140.17 10.40.140.18 10.40.140.19 10.40.140.33 10.40.140.17 10.40.140.18 10.40.140.33

# kvm-10.41.120
kvm-10.41.120 10.41.120.17 10.41.120.18

When I want work with c-home cluster, we execute de cssh as following:

# cssh c-home

In addition, I have written a tiny python script that automatized the cluster lines generation. This script is based in pararell executed ICMP queries. Thats is cool when your servers are deploying in a big VLAN or the number of them is big. In this cases, we can execute my script to found the servers.

# ./cssh-clusterline-generator.py -L 200 -H 250 -d mot -n 10.40.140 >> ~/.clusters

# mot-10.40.140-range-10-150
mot-10.40.140-range-10-150 10.40.140.17 10.40.140.19 10.40.140.32 10.40.140.37

Finally, … the script:

import os
from threading import Thread
from optparse import OptionParser

class Thread_(Thread):
def __init__ (self,ip):
Thread.__init__(self)
self.ip = ip
self.status = -1
def run(self):

res = os.system(“ping -c 1 %s > /dev/null” % self.ip)
res_str = “Not founded”

self.status = res

threads_=[]
ips = “”

parser = OptionParser()
parser.add_option(“-n”, “–net”, dest=”network”, default=”10.121.55″,
help=”Class C Network”, metavar=”NETWORK”)
parser.add_option(“-L”, “–lowrange”, dest=”lowrange”, default=”1″,
help=”Low range”, metavar=”LOW”)
parser.add_option(“-H”, “–highrange”, dest=”highrange”, default=”254″,
help=”High range”, metavar=”HIGH”)
parser.add_option(“-d”, “–deploy”, dest=”deploy”, default=”Net”,
help=”Deploy name”, metavar=”DEPLOY”)
parser.add_option(“-v”, “–verbose”, dest=”verbose”,
default=False, action=”store_true”,
help=”Verboise mode”)

(options, args) = parser.parse_args()

low_range = int(options.lowrange)
high_range = int(options.highrange)
net=options.network
deploy_id = options.deploy
verbose=options.verbose

for i in range (low_range, high_range+1):
ip = net + “.” + str(i)
h = Thread_(ip)
threads_.append(h)
h.start()

for h in threads_:
res_str = “Not founded”
h.join()
count=0

if h.status == 0:
count = count + 1
res_str = “FOUNDED”
if verbose:
print “Looking host in %s … %s” % (h.ip, res_str)
ips += h.ip + ” ”

if verbose:
print “Finished word. %s host founded” % count

print “”
print “# ” + deploy_id + “-” + net + “-range-” + str(low_range) + “-” + str(high_range)
line = deploy_id + “-” + net + “-range-” + str(low_range) + “-” + str(high_range) + ” ” + ips
print line

Working with github.com

One of my pending aims is try to work with git (SCM). Normally, when I am developing , I work with SVN or BZR. From some time ago, I use git only as a way to get source tarballs and nothing else, but now, I think that can try to use git in some minor project to obtain a real experience opinion.
In next lines, I am going to write some little notes about use of git into github.com.
I think that this notes are useful for me, primarly, but can be useful for other git-beginners, too.

Global setup:

Download and install Git
git config –global user.name “Pablo Saavedra”
git config –global user.email saavedra.pablo@gmail.com
Add your public key at github.com

Next steps, creating a new repository:

mkdir awstats-update
cd awstats-update
git init
touch README
git add README
git commit -m ‘first commit’
git remote add origin git@github.com:psaavedra/awstats-update.git
git push origin master

Other way, existing Git Repo:

cd existing_git_repo
git remote add origin git@github.com:psaavedra/awstats-update.git
git push origin master

The last way, importing from Subversion:

git-svn can be used to import as well. Note that there may be issues if you have branches or tags (they won’t be imported over). If you only have a trunk, like many svn repositories, this method should work for you without issue.
First, be sure to create your repository on GitHub
$ git svn clone -s SVN_REPO_URL LOCAL_DIR
$ cd LOCAL_DIR
$ git remote add origin git@github.com:GITHUB_USERNAME/REPO_NAME.git
$ git push origin master
Note that the -s switch implies that your svn repo is set up with the standard branches/tags/trunk structure.
git-svn adds a note to every commit message it copies over from svn. To get rid of that note, add –no-metadata to the command.
You can pull in later commits by running git-svn rebase from within your repo. If you ever lose your local copy, just run the import again with the same settings and you’ll get another working directory with all the necessary git-svn settings saved.

Refers

awstats-update

Finally, I have achieved to push my first git repository at github.com. This repo is a set  formed by silly (but usefull) tools: awstats-update.

Basically, this tools are a couple de scripts which can be executed as cronjobs. These, periodically, are checking  if exist new sites enabled in the apache/nginx configuration and building the corresponding awstats configuration file for them.

The repository can be accessed from:

git://github.com/psaavedra/awstats-update.git

Red5: The OpenSource RTMP server

Recently, I started to work with Red5 RTMP server. Red5 is a OpenSourceRTMP server based on Spring/Tomcat/Java technologies and this is the alternative to the privative Adobe FMS (Flash Media Server) .

Currently, Red5 project has reachend the 0.9 milestone of the Red5 roadmap. You can the latest feautures added to Red5 in: http://red5.org/wiki/Changelog.  Here’s a brief description of the features in Red5:

  • Streaming Audio/Video (FLV and MP3)
  • Recording Client Streams (FLV only)
  • Shared Objects
  • Live Stream Publishing (live h264 supported now)
  • Remoting ( AMF0/ AMF3)

You also need to know 2 concepts:

  • Red5 is a Java aplication server framework that provide all the logic for FLV reflection.
  • You will deploy aplications over Tomcat or stanalone Red5 launcher. This aplication will use the red5 jar library for implement the application logic.

Red5 community work primarly with Java progaming language. In anyway, you can work with another supported language to deploy the application and hadlers. For example: jython, jruby or action script. You can obtain more info in http://osflash.org/red5 and http://red5.org/