Django List Installed Tables and Models

Get Database connection
# Django 1.2
from django.db import connections

connection = connections['default']

# Django 1.1.1<=

from django.db import connection

# Get Database tables

tables = connection.introspection.table_names()

# Get Database models

modelclasses = connection.introspection.installed_models(tables)

# Get Django model name and app name
for model in list(modelclasses):
    print model._meta.app_label, model._meta.module_name

# A better way to get Django models
from django.db.models.loading import get_models, get_app
get_app('app_label')  # returns an app class given a label
get_models()  # all models 
get_models(application)  # models belonging to app class

# Get a model from a string
from django.db import models
models.get_model(*"myapp.mymodel".split('.')) # returns model class

# Get all Python modules

import sys
sys.modules

http://www.slideshare.net/ubernostrum/django-in-depth?src=related_normal&rel=3017822

Use alternate Mirror for Python pypi installs (when pypi goes down)

Specify alternate mirror from the command line:
pip install -i http://d.pypi.python.org/simple $PACKAGE

Specify alternate mirror in ~/.pip/pip.conf:
[global]
index-url = http://d.pypi.python.org/simple

More mirrors:
http://b.pypi.python.org/
http://c.pypi.python.org/
http://d.pypi.python.org/
http://e.pypi.python.org/
http://f.pypi.python.org/

Lists of mirrors:
http://pypi.python.org/mirrors/
http://www.pypi-mirrors.org/


Source:
http://jacobian.org/writing/when-pypi-goes-down/

Query Django Admin Log

Show Django Admin Log entries


select u.username, u.first_name, u.last_name, l.action_time, c.name, c.app_label,
       l.object_id as "Object Modified", l.object_repr, l.action_flag, l.change_message
from (auth_user u inner join django_admin_log l on u.id = l.user_id )
      inner join django_content_type c on c.id = l.content_type_id
where l.object_id = '9'  /* filter by object id if desired */
       and l.action_time > to_date('2010-10-04 01:00','YYYY-MM-DD HH:MI')  /* filter by date range */
order by l.action_time;

Change Default Linux Shell

Needs sudo privileges: to change the default shell:

chsh

http://forums.devshed.com/unix-help-35/how-to-change-default-shell-52749.html

Host a Test email server for Django header debugging

Set the following in settings.py
EMAIL_HOST="localhost"
EMAIL_PORT="1025"

Run the following to host the test webserver

python -m smtpd -n -c DebuggingServer localhost:1025

http://docs.djangoproject.com/en/dev/topics/email/ 

Django Management Commands

Django syncdb without prompts
python manage.py syncdb --noinput

http://docs.djangoproject.com/en/dev/ref/django-admin/

Bash customize autocomplete

http://www.linuxjournal.com/content/more-using-bash-complete-command

Dynamic Fieldsets in Django Admin

Problem: the fieldsets option for the admin interface (classes defined in admin.py) requires that you explicitly list every single field in the model in the fieldset list that you pass to it.  This gets tedious if you add a field to a model, as you also have to add it to this admin.py.  Not very DRY.

Solution:
In models.py
from django.db import models
class My(models.Model):
    a = models.CharField(max_length=10)
    b = models.CharField(max_length=10)
    c = models.CharField(max_length=10)
    d = models.CharField(max_length=10)
    e = models.CharField(max_length=10)
    f = models.CharField(max_length=10)
    g = models.CharField(max_length=10)
    h = models.CharField(max_length=10)

In admin.py
from django.contrib import admin
from django.forms.models import fields_for_model
from f import models
class MyAdmin(admin.ModelAdmin):
    def __init__(self, *args, **kwargs):
        super(MyAdmin, self).__init__(*args, **kwargs)
        all_fields = set(fields_for_model(models.My))
        fieldset1_fields = ('e', 'f',)
        fieldset2_fields = ('g', 'h',)
        fieldset_fields = set(fieldset1_fields) | set(fieldset2_fields)
        rest_fields = list(all_fields - fieldset_fields)
        self.fieldsets = (
            (None, {
                'fields': rest_fields
            }),
            ('Fieldset 1', {
                'classes': ('collapse',),
                'fields': fieldset1_fields
            }),
            ('Fieldset 2', {
                'classes': ('collapse',),
                'fields': fieldset2_fields
            }),
        )
    models = models.My
admin.site.register(models.My, MyAdmin)


Thanks for the help lorochka85

Drill through a Login with Python

import urllib2
import urllib
import BeautifulSoup

# build opener with HTTPCookieProcessor
o = urllib2.build_opener( urllib2.HTTPCookieProcessor() )
urllib2.install_opener( o )

# assuming the site expects 'user' and 'pass' as query params
p = urllib.urlencode( { 'username': 'myusername', 'password': 'mypassword' } )

# perform login with params
f = o.open( 'https://www.mysite.com/login', p )
data = f.read()
f.close()

# second request should automatically pass back any
# cookies received during login... thanks to the HTTPCookieProcessor
f = o.open( 'http://www.mysite.com/home/' )
data = f.read()
f.close()

soup = BeautifulSoup.BeautifulSoup(data)

http://www.nomadjourney.com/2009/03/automatic-site-login-using-python-urllib2/

Python String format

Python Format dates into strings
from datetime import date
d1 = date(2010,01,23)
d2 = date(2010,01,23)
"%s - %s" % (d1.strftime("%m / %d / %Y"), d1.strftime("%m / %d / %Y"))

Python Unicode Convert

Convert weird unicode string characters to string:
import unicodedata
unicodedata.normalize('NFKD',unicode_string).encode('ascii','ignore')

Django module for helping convert things to ascii strings:
from django.utils.encoding import smart_str
smart_str('weird string goes here')

Another hack for removing unicode errors:
unicode('my weird string', errors='ignore')


http://docs.python.org/howto/unicode.html#the-unicode-type

matplotlib basics

Simple Plot
import matplotlib.pyplot as plt
plt.plot([1,2,3,4])
# plot a set of points (1,1) and (3,2)
plt.plot([1,3],[1,2],'ro')
# Or plot as a bunch of connect line segments
plt.plot([2,4,6],[2,4,6])
# set a label for the x and y axis
plt.ylabel('some numbers')
plt.xlabel('joe rocks')
# set the axis [xmin, xmax, ymin, ymax ]
plt.axis([0,6,0,5])
# 'show' the graph
plt.show()


Plot a function
As far as I can tell matplotlib cannot plot a continuous function.  
Instead, create a range of distinct numbers:

import numpy as np
import matplotlib.pyplot as plt
# create a range of numbers from 0 to 5. increment by .1
x = np.arange(0.,5.,.1)
# apply a function to that 
y = np.sin(x)
# plot with extra display params
plt.plot(x,y,linewidth=2.0, label='joeplot', color='blue')
plt.title(r'$\alpha_i > \beta_i$', fontsize=20)
plt.axis([0,5,-1,1])
plt.grid(True)
plt.show()

Make a cool heatmap
import numpy as np
import matplotlib.pyplot as plt
#x = y = np.linspace(-5, 5, 12)
# create a grid of 12 X coordinates and 12 Y coordinates 
# these coordinates will be used to represent specific locations on 
# the grid.  the meshgrid() basically creates a nice uniform coordinate grid
X,Y = np.meshgrid([1,2,3,4],[1,2,3])
# X,Y = np.meshgrid(x, y)
# these ravel functions basically make the 2d arrays created above into lists
x = X.ravel()
y = Y.ravel()
plt.subplot(111)
plt.hexbin(x,y,C=[1,2,3,3,2,2,3,3,2,3,3,4], gridsize=30)
cb = plt.colorbar()
cb.set_label('Heat Value')
plt.axis([x.min(), x.max(), y.min(), y.max()])
plt.grid(True)


http://stackoverflow.com/questions/2369492/generate-a-heatmap-in-matplotlib-using-a-scatter-data-set

Case Study Notes: install pyqrcode in a virtualenv on OSX

Kind of a buggy install.

mkdir qrcode; cd qrcode
virtualenv --distribute --no-site-packages ve
source ./ve/bin/activate
echo "pil" > requirements.pip
echo "http://svn.apache.org/repos/asf/lucene/pylucene/trunk/jcc" >> requirements.pip

wget http://downloads.sourceforge.net/pyqrcode/pyqrcode-0.2.1.tar.gz?use_mirror=

cd pyqrcode-0.2.1

# EDIT pyqrcode-0.2.1/Makefile
# change the call to jcc from
GENERATE=python -m jcc --jar $(LIBFILE) \
# TO
GENERATE=python -m jcc.__main__ --jar $(LIBFILE) \

make
make egg

# it will create qrcode-0.2.1-py2.6-macosx-10.6-universal.egg
# Edit qrcode-0.2.1-py2.6-macosx-10.6-universal.egg/qrcode/__init__.py
# change the line
_qrcode._setExceptionTypes(JavaError, InvalidArgsError)
# TO
_qrcode._set_exception_types(JavaError, InvalidArgsError)

pip -E ./ve/ install qrcode-0.2.1-py2.6-macosx-10.6-universal.egg

# http://pyqrcode.sourceforge.net/
# http://www.mail-archive.com/pythonmac-sig@python.org/msg09864.html
# http://mail-archives.apache.org/mod_mbox/lucene-pylucene-dev/200904.mbox/%3C49EEECAC.7070606@cheimes.de%3E

Installing MySQL-python (MySQLdb) in a virtualenv on OSX 10.6

There were two problems with the MySQL-Python package that I had when installing on OS X 10.6
1) First, an error saying "EnvironmentError: mysql_config not found" when setup.py is run (through pip or easy_install)
2) Second, an error saying "ImportError: dynamic module does not define init function(init_mysql) when importing MySQLdb" when 'import MySQLdb" is issued

Causes:
1) This is caused by the build script not being able to find a MySQL program called mysql_config. This program is used to determine metadata about the mysql install. For example, the command "mysql_config --cflags" reports the flags used to build mysql.
2) This is caused by a mismatch between the architectures that MySQL was built with and the architecture that MySQL-python installer is trying to build and install MySQL-python as. To find what arch MySQL was built as, run "mysql_config --cflags" (this was i386 in my case). OSX tries to build MySQL-python as x86_64. Therefore, there is a arch mismatch.

Solution:
1) Locate the mysql_config binary location and add the path to the MySQL-python/site.cfg file as a config directive: "mysql_config = /opt/local/bin/mysql_config5"
2) Use the arch x86_64 (64 bit) version of MySQL compiled through MacPorts, not a i386 (32 bit) version downloaded off the internet and installed as a dmg.

Summary:
Assuming the MacPorts version of MySQL is installed, both issues can be solved by adding the mysql_config = /opt/local/bin/mysql_config5 config directive to the MySQL-python/site.cfg file.

# SCRIPTED SOLUTION: Create a script called setup.sh and add the following lines
virtualenv --distribute --no-site-packages ve
source ./ve/bin/activate
pip install -E ./ve -r requirements.1.pip # other deps
export ARCHFLAGS="-arch x86_64"
pip install -E ./ve MySQL-python
echo "mysql_config = /opt/local/bin/mysql_config5" >> ./ve/build/MySQL-python/site.cfg
pip install -E ./ve MySQL-python


Resources:
http://groups.google.com/group/python-virtualenv/msg/cf4f3117faea476b?pli=1
http://birdhouse.org/blog/2009/02/21/python-mysql-connections-on-mac-os/
http://stackoverflow.com/questions/2111283/how-to-build-64-bit-python-on-os-x-10-6-only-64-bit-no-universal-nonsense

Python Pip Usage

Basic PIP install
pip install somepackage

Basic PIP uninstall
pip uninstall somepackage

Requirements file
# cat requirements.txt
MyApp
Framework==0.0.1
Library>=0.2

Installing using a requirements file
pip install -r requirements.pip

PIP Freezing requirements
pip freeze # lists all packages and the specific version installed. Useful for migrating

PIP install into virtualenv env
pip install -E ./env
pip install -E ./env -r requirements.pip # using a requirements file

Basic script for creating a virtualenv and installing requirements
virtualenv --distribute --no-site-packages ve
source ./ve/bin/activate
pip install -E ./ve -r requirements.pip
pip install -E ./ve -r requirements-test.pip

http://heisel.org/blog/2009/11/21/django-hudson/
http://pip.openplans.org/#freezing-requirements

Case Study Notes: install djangobb in virtualenv

Ubuntu 10.04 python 2.6 django 1.1

sudo apt-get build-dep python-psycopg2
sudo aptitude install python-dev

cd Sites
virtualenv --no-site-packages env
. ./env/bin/activate
pip install pil
pip install markdown2 Markdown
pip install django-registration
pip install djapian
pil install xapian
pip install psycopg2
easy_install -i http://downloads.egenix.com/python/index/ucs4/ egenix-mx-base
pip install http://code.djangoproject.com/svn/django/tags/releases/1.1.2/
hg clone http://hg.djangobb.org/djangobb/ djangobb
cd env/lib/python2.6/site-packages/
cp /usr/lib/python2.6/dist-packages/_xapian.so .
cp /usr/lib/python2.6/dist-packages/xapian.py .

# configure settings.py to use postgresql_psycopg2
# comment out the openid stuff

## http://www.saltycrane.com/blog/2009/07/using-psycopg2-virtualenv-ubuntu-jaunty/

Postgresql Basic Commands

Login to postgresql:
psql -d mydb -U myuser -W
psql -h myhost -d mydb -U myuser -W
psql -U myuser -h myhost "dbname=mydb sslmode=require" # ssl connection

Default Admin Login:
sudo -u postgres psql -U postgres
sudo -u postgres psql

List databases on postgresql server:
psql -l [-U myuser] [-W]

Turn off line pager pagination in psql:
\pset pager

Determine system tables:
select * from pg_tables where tableowner = 'postgres';

List databases from within a pg shell:
\l

List databases from UNIX command prompt:
psql -U postgres -l

Describe a table:
\d tablename

Quit psql:
\q

Switch postgres database within admin login shell:
\connect databasename

Reset a user password as admin:
alter user usertochange with password 'new_passwd';

Show all tables:
\dt

List all Schemas:
\dn

List all users:
\du

Load data into posgresql:
psql -W -U username -H hostname < file.sql

Dump (Backup) Data into file:
pg_dump -W -U username -h hostname database_name > file.sql

Increment a sequence:
SELECT nextval('my_id_seq');

Create new user:
CREATE USER jjasinski WITH PASSWORD 'myPassword';
# or
sudo -u postgres createuser jjasinski -W

Change user password:
ALTER USER Postgres WITH PASSWORD 'mypass';

Grant user createdb privilege:
ALTER USER myuser WITH createdb;

Create a superuser user:
create user mysuper with password '1234' SUPERUSER
# or even better
create user mysuper with password '1234' SUPERUSER CREATEDB CREATEROLE INHERIT LOGIN REPLICATION;
# or
sudo -u postgres createuser jjasinski -W -s

Upgrade an existing user to superuser: 
alter user mysuper with superuser;
# or even better
alter user mysuper with SUPERUSER CREATEDB CREATEROLE INHERIT LOGIN REPLICATION

Show Database Version:
SELECT version();

Change Database Owner:
alter database database_name owner to new_owner;

Copy a database:
CREATE DATABASE newdb WITH TEMPLATE originaldb;
http://www.commandprompt.com/ppbook/x14316

View Database Connections:
SELECT * FROM pg_stat_activity;

View show data directory (works on 9.1+; not on 7.x):
show data_directory;

Show run-time parameters:

show all;
select * from pg_settings;

Show the block size setting:
# show block_size;
 block_size
------------
 8192
(1 row)

Show stored procedure source:
SELECT prosrc FROM pg_proc WHERE proname = 'procname'

Grant examples:
# readonly to all tables for myuser
grant select on all tables in schema public to myuser;
# all privileges on table1 and table2 to myuser
grant all privileges on table1, table2, table3 to myuser;

Restore Postgres .dump file:
pg_restore --verbose --clean --no-acl --no-owner -h localhost -U myuser -d mydb latest.dump
source

Find all active sessions and kill them (i.e. for when needing to drop or rename db)
Source: http://stackoverflow.com/questions/5408156/how-to-drop-a-postgresql-database-if-there-are-active-connections-to-it

# Postgres 9.2 and above
SELECT pg_terminate_backend(pg_stat_activity.pid) 

FROM pg_stat_activity 

WHERE pg_stat_activity.datname = 'TARGET_DB' 

 AND pid <> pg_backend_pid();

# Postgres 9.1 and below
SELECT pg_terminate_backend(pg_stat_activity.procpid) 

FROM pg_stat_activity 

WHERE pg_stat_activity.datname = 'TARGET_DB' 

AND procpid <> pg_backend_pid();


Re-read postgres config without dropping connections (i.e. if postgres.conf or pg_hba.conf changes)
Source: http://www.heatware.net/databases/postgresql-reload-config-without-restarting/

/usr/bin/pg_ctl reload
or
SELECT pg_reload_conf();

Per user query logging (logs to to postgres logs):
alter role myuser set log_statement = 'all';


Sources:
http://www.devdaily.com/blog/post/postgresql/log-in-postgresql-database
http://forums.devshed.com/postgresql-help-21/how-do-you-turn-off-more-scroll-lock-at-psql-174831.htm
http://www.cyberciti.biz/faq/howto-add-postgresql-user-account/
http://archives.postgresql.org/pgsql-general/1998-08/msg00050.php
http://stackoverflow.com/questions/876522/creating-a-copy-of-a-database-in-postgres
http://stackoverflow.com/questions/1137060/where-does-postgresql-store-the-database

Recycle bin script

crm()
{
# pass this function the name of the directory that
# you want to backup and remove 
if [ -d "$1" ]
then
  cd "${1}/../"
else
  echo "Directory does not exist"
  return 1
fi

PWD=`pwd`
BKPFILES=`find $PWD -type -f`
BKPDIR="$HOME/.Trash"
daten=`date +%Y.%m.%d-%H.%M.%S`

for i in $BKPFILES
do
  echo "$i"
  DIRNAME="$BKPDIR"`echo ${i%/*}`
  FILENAME=`echo ${i##*/}`
  mkdir -p "$DIRNAME"
  cp -pf "$i" "${DIRNAME}/${FILENAME}.${daten}"
  #NEWFILE=`echo "$i" | sed 's|[ \/]|_|g'`
  print "$i"  "$NEWFILE"   " $DIRNAME"
done
rm -rf "$1"
cd - 
}

OSX Color LS

Color LS for Mac OX
alias ls='ls -G'

sftp notes

sftp using alternate port:
sftp -oPort=2222 username@host.com

sftp using alternate private key file:

sftp -o IdentityFile=/custom/file/location.pem username@host.com

http://www.unix.com/shell-programming-scripting/43334-sftp-scripting-help-required.html

Basic Python Development Server setup with Ubuntu 10.04

Quick setup guide

# config vi command line
echo "set -o vi" >> ~/.bashrc
. ~/.bashrc

# install packages
aptitude install vim openssh-server apache2 python-virtualenv python-mysqldb mysql-server libapache2-mod-php5 libapache2-mod-wsgi eclipse subversion nmap ubuntu-restricted-extras g++ git-gui virtualbox-ose php5-cli mdadm fabric python-dev mercurial

# eclipse development plugins
install pydev for eclipse
install subclipse for eclipse

# firefox development plugins
install firebug for firefox @ http://getfirebug.com/
install web developer for firefox @ https://addons.mozilla.org/en-US/firefox/addon/60/

# add google repo
deb http://dl.google.com/linux/deb/ stable non-free
aptitude update

cd /opt/
wget http://www.djangoproject.com/download/1.1.2/tarball/
wget http://www.djangoproject.com/download/1.2.1/tarball/
tar -xvf Django-1.1.2.tar.gz
tar -xvf Django-1.2.1.tar.gz
cd /usr/lib/python2.6/dist-packages
ln -s /opt/Django-1.2.1/django/ django

# mysql workbench download (64 bit version)
aptitude install libzip1 python-pysqlite2 # deps needed
http://dev.mysql.com/get/Downloads/MySQLGUITools/mysql-workbench-oss-5.2.22-1ubu1004-amd64.deb/from/http://mirror.services.wisc.edu/mysql/
dpkg -i mysql-workbench-oss-5.2.22-1ubu1004-amd64.deb

# setup postgresql
aptitude install postgresql pgadmin3 python-psycopg2

sudo su -
passwd postgres
su postgres
psql template1

The last instruction should open the psql shell, where you can run the following:

ALTER USER postgres WITH ENCRYPTED PASSWORD 'mypassword';

# Setup NX server (Google NeatX server)
sudo apt-get install python-software-properties && sudo add-apt-repository ppa:freenx-team
sudo apt-get update
sudo apt-get install neatx-server

## NOTE: if neatx gives a weird error, delete the session dirs on the server:
## /var/lib/neatx/sessions/some_dir/

https://help.ubuntu.com/community/FreeNX
http://programmingzen.com/2007/12/26/installing-django-with-postgresql-on-ubuntu/

Simple Network Test Script

This is a simple script to log network outages on a local network

root@ubuntu:/mnt/root/usr/local/bin# more lifeline
#!/bin/bash

PING='/bin/ping'
EXTHOST1='www.google.com'
EXTHOST2='sun.iwu.edu'
INTHOST='192.168.1.1'
LOG='/var/log/network_outages'
WAITTIME=120

echo "NETWORK STATUS SCRIPT: Started " `date` >> $LOG
while [ 1=1 ]
do
   dater=`date +%Y.%m.%d-%H.%M.%S`
   $PING -q -c1 $EXTHOST1
   ret=$?
   echo "RET: $ret"
   if [ $ret -ne 0 ]
   then
     echo "EXTERNAL_PING(1): outage detected $dater" >> $LOG
   fi

   dater=`date +%Y.%m.%d-%H.%M.%S`
   $PING -q -c1 $EXTHOST2
   ret=$?
   echo "RET: $ret"
   if [ $ret -ne 0 ]
   then
     echo "EXTERNAL_PING(2): outage detected $dater" >> $LOG
   fi

   $PING -q -c1 $INTHOST
   ret=$?
   echo "RET: $ret"
   if [ $ret -ne 0 ]
   then
     echo "INTERNAL_PING: outage detected $dater" >> $LOG
   fi

   sleep $WAITTIME
done

Simple HEX Editor

View file in Hex Editor
od -cx filename

Bash String Operatinos

Remove the last four characters in a string
echo "somefile.txt" | awk 'sub("....$","")

Remove file extension from a string
ls -1 | sed 's/\(.*\)\..*/\1/'

Remove PREFIX from SOMEPATH
SOMEPATH="/home/myuser/usr/bin/"
PREFIX="/home/myuser/"
echo ${SOMEPATH#$PREFIX}
Returns: usr/bin/

Remove shortest match of PREFIX from SOMEPATH
SOMEPATH="/home/sub/subhome/myuser/usr/bin/"
PREFIX="/*/myuser/"
echo ${SOMEPATH#$PREFIX}
Returns: usr/bin/

Parsing parts of a file:
foo=/tmp/my.dir/filename.tar.gz
To get: /tmp/my.dir (like dirname)
path = ${foo%/*}
To get: filename.tar.gz (like basename)
file = ${foo##*/}
To get: filename
base = ${file%%.*}
To get: tar.gz
ext = ${file#*.}

Removing first 8 characters of a string:
echo $var | cut -c9-

http://unstableme.blogspot.com/2008/03/printremove-first-some-characters-of.html
http://tldp.org/LDP/LGNET/18/bash.html
http://docstore.mik.ua/orelly/unix/upt/ch09_07.htm
http://unstableme.blogspot.com/2007/12/removing-last-two-characters-bash.html
http://www.unix.com/shell-programming-scripting/40360-remove-file-extension.html

Oracle Login Context Information

Find the Database instance you are logged into
SELECT sys_context('USERENV', 'DB_NAME') FROM dual;
SELECT sys_context('USERENV', 'INSTANCE_NAME') FROM dual;

Find the operating system user hosting the Oracle session
SELECT sys_context('USERENV', 'OS_USER') FROM dual;

Find the Oracle user currently logged into
SELECT sys_context('USERENV', 'SESSION_USER') FROM dual;

http://www.psoug.org/reference/sys_context.html

Python Virtual Env

Download and install via apt-get, ports, easy_install, etc.

Locate the install directory (usually site-packages) and find the virtualenv.py script.
Alternatively, the package manager may have installed a virtualenv script into your PATH.

I will use ENV to refer to /path/to/python/virtual/env
I will use virtualenv.py to refer to /path/to/script/virtualenv.py

Create a new Virtual Environment
python virtualenv.py ENV

(optional) Activate (set PATH and PYTHONPATH) for new Virtual ENV
source ENV/bin/activate

Location of Virtual Env python binary
ENV/bin/python

Location of Virtual ENV python site-packages dir
ENV/lib/python2.x/site-packages/

Location of Virtual ENV python easy_install script
ENV/bin/easy_install


http://pypi.python.org/pypi/virtualenv

curl Usage

Send a POST

curl -u joe:1234 -X POST http://localhost:8000/api/1.0/NU1yWdRM5JBjnZpZX/message/ -d "message=test"

-u = username and password

-d = post parameters

-X = request method

Send a POST using data from standard in:
echo "mykey=myvalue" | curl -X POST -d @-  https://mysite.com/

Show verbose output: 
curl -v http://mysite.com/

Ignore SSL key errors:
curl -k http://mysite.com/

Combining them: 
echo "TEST=1
TEST2=2
" curl -X POST -vk -d @- http://mysite.com/

netbios name lookups

Find NetBIOS name from IP
Windows:
netstat -a 192.168.1.22

Linux:
nbtstat 192.168.1.22 # ntbstat separate package

Find IP from NetBIOS name
Windows:
nbtstat -a host.example.com

Linux:
nmblookup host.example.com

http://www.irongeek.com/i.php?page=security/ipinfo

MacPorts Usage

Update MacPorts itself
sudo port selfupdate
sudo port -d selfupdate # debug

Updates the port tree with new versions definitions
sudo port sync

List available ports
sudo port list

Search ports
port search [keyword]

Lookup package info (desc, maintainer, etc)
port info [package]

Find package dependencies
port deps [package]

Install package
sudo port install [package]
sudo port -v install [package] #verbose

Clean out build files and tarballs
port clean --all [pakcage]

Uninstall a package
sudo port uninstall [package]

Show port contents
port contents [package]

List installed packages
port installed

List outdated ports
port outdated

Upgrade specific packages
port upgrade [package]
port upgrade outdated #updates all outdated packages


http://guide.macports.org/chunked/using.html

DJango Model Inheritance with subclasses

http://www.djangosnippets.org/snippets/1034/
http://www.djangosnippets.org/snippets/1031/
http://adam.gomaa.us/blog/2009/feb/16/subclassing-django-querysets/
http://docs.djangoproject.com/en/dev/ref/contrib/contenttypes/

Python List Operations (map, for comprehensions)

EXAMPLE 1
# Create a list
>>> l = [1,2,3,4,5,6]

# iterate through the list using a for comprehension
>>> [i for i in l]
[1, 2, 3, 4, 5, 6]

# square elements in the list using a for comprehension
>>> [i**2 for i in l]
[1, 4, 9, 16, 25, 36]

# square only even elements in the list using a for comprehension
>>> [i**2 for i in l if i % 2 == 0]
[4, 16, 36]

# iterate through the list using map
>>> map(lambda w: w, l)
[1, 2, 3, 4, 5, 6]

# square each element of the list using map
>>> map(lambda w: w**2, l)
[1, 4, 9, 16, 25, 36]

# or use map to call a separately defined function to# iterate through the list
>>> def squa(x):
...... return x**2

>>> map(squa, l)
[1, 4, 9, 16, 25, 36]

# use map function to call a separately defined function that takes 2 args
>>> l2 = map(lambda w: (w,2),l)
>>> l2
[(1, 2), (2, 2), (3, 2), (4, 2), (5, 2), (6, 2)]

>>> def pow(base, expo):
...... return base**expo

>>> map(lambda (x,y): pow(x,y), l2)
[1, 4, 9, 16, 25, 36]

# filter a list
>>> filter(lambda w: w > 2, l)
[3, 4, 5, 6]

EXAMPLE 2
# define a simple class to play with
>>> class A(object):
...... def __init__(self, x):
........ self.x = x

# create simple list of object instances of the class
>>> l = [A(1), A(2), A(3)]

# use a for (list) comprehension to iterate through the list
>>> [i.x for i in l]
[1, 2, 3]

# use a for comprehension to iterate with a condition
>>> [i.x for i in l if i.x > 1]
[2, 3]

# use map to apply a function to every element in the list
>>> map(lambda w: w.x, l)
[1, 2, 3]

>>> map(lambda w: w.x * w.x, l)
[1, 4, 9]

EXAMPLE 1

# Create a list

>>> l = [1,2,3,4,5,6]

# iterate through the list using a for comprehension

>>> [i for i in l]

[1, 2, 3, 4, 5, 6]

# square elements in the list using a for comprehension

>>> [i**2 for i in l]

[1, 4, 9, 16, 25, 36]

# square only even elements in the list using a for comprehension

>>> [i**2 for i in l if i % 2 == 0]

[4, 16, 36]

# iterate through the list using map

>>> map(lambda w: w, l)

[1, 2, 3, 4, 5, 6]

# square each element of the list using map

>>> map(lambda w: w**2, l)

[1, 4, 9, 16, 25, 36]

# or use map to call a separately defined function to# iterate through the list

>>> def squa(x):

...... return x**2

>>> map(squa, l)

[1, 4, 9, 16, 25, 36]

# use map function to call a separately defined function that takes 2 args

>>> l2 = map(lambda w: (w,2),l)

>>> l2

[(1, 2), (2, 2), (3, 2), (4, 2), (5, 2), (6, 2)]

>>> def pow(base, expo):

...... return base**expo

>>> map(lambda (x,y): pow(x,y), l2)

[1, 4, 9, 16, 25, 36]

# filter a list

>>> filter(lambda w: w > 2, l)

[3, 4, 5, 6]

EXAMPLE 3

# split a list into chunks

>>> map(None, *(iter(range(10)),) * 3)

[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, None, None)]

http://stackoverflow.com/questions/1335392/iteration-over-list-slices

Django South Migrations

Install Django South

- Place the untared 'south' folder into your Django applicaion

- Modify the project's settings.py and add 'south' to the INSTALLED_APPS list

- For new Django projects:

---1) initialize any Django apps or models to monitor with South.

----a) See Initializing Applications below

---2) run ./manage.py syncdb

---3) run ./manage.py migrate

To view help on startmigration:  # South 7.x+ uses the schemamigration command instead

./manage.py startmigration

Usage: ./manage.py startmigration appname migrationname [--initial] [--auto]

[--model ModelName] [--add-field ModelName.field_name] [--freeze] [--stdout]

Initializing Applications - needed whenever you want to track more Django apps

# this creates a 'migrations' folder in your Django app directory

./manage startmigration app_name comment --initial  # updates for South 7.x+
./manage schemamigration app_name comment --initial # Init's all models in application
./manage startmigration app_name --initial # updates for South 7.x+

./manage schemamigration app_name --initial # comment is optional here

Adding South with no data and no existing models.py objects:
# 1) syncdb as normal
python manage.py syncdb
# 2) Start an application if needed
python manage.py startapp appname
# 3) create a migration file with a blueprint to create tables
python manage.py startmigration appname --initial  # updates for South 7.x+
python manage.py schemamigration appname --initial
# 4) Apply the blueprint and create the tables
python manage.py migrate appname

# 5) Repeat steps 3 and 4 for each additional app if needed

Adding south with no data and existing models.py objects:
# 1) syncdb as normal if needed
python manage.py syncdb
# 2) create a migration file with a blueprint to create tables
python manage.py startmigration appname --initial # updates for South 7.x+
python manage.py schemamigration appname --initial
# 3) Apply the blueprint but it doesn't actually create the tables
python manage.py migrate appname --fake
# 4) Repeat steps 2 and 3 for each additional app if needed

Applying changes after making changes to a model (repeat for each app):
# Creates a migration 'blueprint' file and guesses what should take place
python manage.py startmigration appname migrationname --auto # updates for South 7.x+
python manage.py schemamigration appname migrationname --auto
# OR create the blueprint file but it will be empty
python manage.py startmigration appname migrationname  # updates for South 7.x+
python manage.py schemamigration appname migrationname
# make any modifications needed
# migrate the tables in the app
python manage.py migrate appname
# OR migrate all tables in all apps
python manage.py migrate

Adding a new model to an app already managed with south:
./manage startmigration app_name comment --model m1 --model m2 # updates for South 7.x+
./manage schemamigration app_name comment --model m1 --model m2 # inits specific models

List available migrations
python manage.py migrate --list

http://south.aeracode.org/wiki/ConvertingAnApp

Oracle Space queries

Get Used space for current user
SELECT sum(bytes)/(1024 * 1024) as "Used (MB)"
FROM user_SEGMENTS
ORDER BY 1 desc

Get tablespace usage for all users using that tablespace
select owner, sum(bytes)/power(2,20)mb
from dba_extents
where tablespace_name = 'TABLESPACE'
group by owner
order by 2 desc;

Get Total tablespace available for current user
select tablespace_name,round(sum(bytes) / (1024 * 1024),2) "Tablespace SIZE (MB)"
from user_free_space
group by tablespace_name

Get space of individual tables/indexes (objects) in a schema
select sum(bytes) / (1024 * 1024) as MB,owner,segment_type, segment_name
from dba_segments s
where owner = 'SCHEMA'
group by owner,segment_type, segment_name
order by MB desc

Get USER_ dictornary Tables
SELECT table_name, comments
FROM dictionary
WHERE table_name LIKE 'USER_%'
ORDER BY table_name;

http://snipplr.com/view/4748/get-a-list-of-all-the-user-tables-oracle/
http://www.freelists.org/post/oracle-l/dba-extents-vs-dba-segments,8

CSS Pre Tag formatting

Make pre tags wrap as expected

pre {
white-space: pre-wrap;       /* css-3 should we be so lucky... */
white-space: -moz-pre-wrap;  /* Mozilla, since 1999 */
white-space: -pre-wrap;      /* Opera 4-6  */
white-space: -o-pre-wrap;    /* Opera 7  */
word-wrap: break-word;       /* Internet Explorer 5.5+ */
_white-space: pre;   /* IE only hack to re-specify in addition to
word-wrap  */
}

http://archivist.incutio.com/viewlist/css-discuss/55677
http://bavotasan.com/tutorials/how-to-wrap-text-within-the-pre-tag-using-css/

Relative to Absolute Path in Shell Script

Simple bash function to convert relative paths to absolute pats
fun_abs
{ echo “`cd \`dirname $1\`; pwd`/`basename $1`” }

http://www.robertpeaslee.com/index.php/converting-a-relative-path-to-an-absolute-path-in-bash/

UPDATE: unfortuately the above only works for directories that exist - the perl hack will get around this

fun_abs
{FILE=`$PERL -e "use File::Spec::Functions qw[rel2abs];print rel2abs('$1');"`; echo "$FILE"; }

Get File Extension in Shell Script

Returns the last file extension in the file name
echo "thisfile.txt.log"awk -F . '{print $NF}' # returns "log"

http://liquidat.wordpress.com/2007/09/29/short-tip-get-file-extension-in-shell-script/

Correct CSS PNG Color Mismatch

Copied from link below
Mac OS X will render the images using the color profile they got stored, actually png gamma correction would be more exact. This is not immediately apparent because at a glance the files may look identical but using the pngs over a CSS background color often reveals an unpleasant sight on the mac, especially on Safari.
To prevent such a color mismatch either use a different image format or strip the png gamma correction. In the case of the latter, pngcrush does a pretty good job.
I was pleasantly surprised that I could install it with ease on Fedora ("yum install -y pngcrush"). To use it run:

pngcrush -rem gAMA -rem cHRM -rem iCCP -rem sRGB png-file-name optimized-png-file-name


http://www.viseztrance.com/2009/02/fixing-the-png-color-mismatch-on-mac-os-x.html

vim settings

# start vim with an alternate vimrc location
vim -u filename

# Turn compatibility mode off via set command
:set nocompatible

# view special characters in a vim session
:set list
:set nolist # disable

# set search result hilighting
:set hlsearch

# use different visible characters to see special characters in vim
:set listchars=eol:$,tab:>-,trail:~,extends:>,precedes:< :set list # view line numbers
:set nu
:set nonu # disable

# enable linewrap (on by default)
:set wrap
:set nowrap #disable

# indenting source code (setup)
:set et # expand tabs to spaces
:set sw=4
:set smarttab

# Manually choose the language for syntax highlighting

:setf language
:setf html  # example

# The available languages for syntax highlighting 

# can be found in the vim install directory.  

# In my system, it was /usr/share/vim/vim72/syntax


# indenting source code (usage)
1) in command mode press v to and arrows to select a range of lines
2) after selecting, press ==

http://vim.wikia.com/wiki/Indenting_source_code

# indent multiple lines (not necessarily source code)
1) in command mode press v and arrows to select a range of lines
2) type shift >
shift < will shift a block back

# compatibility mode
# by default, vim will enter into compatibility mode when loading.
# Disable this behavior one of two ways
# 1) Create a .vimrc in your home directory
# 2) Within the vim command line, type :set nocompatible

# trim whitespace at end of line
:1,$s/[ <tab>]*$//

# get vim to return the previous position when editing files

" Tell vim to remember certain things when we exit
"  '10 : marks will be remembered for up to 10 previously edited files
"  "100 : will save up to 100 lines for each register
"  :20 : up to 20 lines of command-line history will be remembered
"  % : saves and restores the buffer list
"  n... : where to save the viminfo files
set viminfo='10,\"100,:20,%,n~/.viminfo

" when we reload, tell vim to restore the cursor to the saved position
augroup JumpCursorOnEdit
au!
autocmd BufReadPost *
\ if expand(":p:h") !=? $TEMP |
\ if line("'\"") > 1 && line("'\"") <= line("$") |
\ let JumpCursorOnEdit_foo = line("'\"") |
\ let b:doopenfold = 1 |
\ if (foldlevel(JumpCursorOnEdit_foo) > foldlevel(JumpCursorOnEdit_foo - 1)) |
\ let JumpCursorOnEdit_foo = JumpCursorOnEdit_foo - 1 |
\ let b:doopenfold = 2 |
\ endif |
\ exe JumpCursorOnEdit_foo |
\ endif |
\ endif
" Need to postpone using "zv" until after reading the modelines.
autocmd BufWinEnter *
\ if exists("b:doopenfold") |
\ exe "normal zv" |
\ if(b:doopenfold > 1) |
\ exe "+".1 |
\ endif |
\ unlet b:doopenfold |
\ endif
augroup END

http://vim.wikia.com/wiki/Restore_cursor_to_file_position_in_previous_editing_session

http://ubuntuforums.org/showthread.php?t=789327
http://www.oualline.com/vim-cook.html
http://stackoverflow.com/questions/1675688/make-vim-show-all-white-spaces-as-a-character
http://jamesreubenknowles.com/set-vim-syntax-language-270

svn+ssh on alternate port

Setup the config
# Add the following lines to ~/.subversion/config
# must be put in the [tunnels] section of the config file
sshnew=ssh -l user -p 2222

Execute the svn:
svn co svn+sshnew://servername.com/path/to/repository/

Djanog show model query

Show the raw SQL generated by a model access: 

# DEBUG needs to be set True 

>>> from django.db import connection 

>>> connection.queries [{'sql': 'SELECT polls_polls.id,polls_polls.question,polls_polls.pub_date FROM polls_polls', 'time': '0.002'}]

sql # The raw SQL statement 
time # How long the statement took to execute, in seconds.

Reset the queries list returned above:
>>> from django import db
>>> db.reset_queries()

Show SQL for a given query (newer django version; 1.5+?)
>>> Product.objects.all().query.sql_with_params()

Show SQL for a given query (older django version; 1.2-1.5+?)
>>> Product.objects.all().query.as_sql()

http://docs.djangoproject.com/en/dev/faq/models/
http://docs.djangoproject.com/en/dev/faq/models/#how-can-i-see-the-raw-sql-queries-django-is-running

http://stackoverflow.com/questions/2314920/django-show-log-orm-sql-calls-from-python-shell

Secure Django Location with htaccess file (webfaction)

Basic steps to secure a Django account using htaccess file
note, this examples assumes apache 2.2, in which the basic auth apache module changed.
note, this example uses a WebFaction account but can be applied to any Django Apache install.

1) Execute this command to create an htpassword file
htpasswd -c /home/my_account/webapps/evesch/apache2/conf/.mypasswds my_user

2) Execute this command to create a htgroups file
echo "managers: my_user" > /home/my_account/webapps/evesch/apache2/conf/.mygroups

3) Modify your httpd.conf file to look like this.
############# contents of /home/my_account/webapps/evesch/apache2 #####
ServerRoot "/home/my_account/webapps/evesch/apache2"

LoadModule dir_module modules/mod_dir.so
LoadModule env_module modules/mod_env.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule mime_module modules/mod_mime.so
LoadModule python_module modules/mod_python.so
LoadModule rewrite_module modules/mod_rewrite.so

# added by me (joe)
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so


KeepAlive Off
Listen 7637
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
CustomLog logs/access_log combined
ServerLimit 2

<Location "/">
PythonHandler django.core.handlers.modpython
PythonPath "['/home/my_account/webapps/evesch', '/home/my_account/webapps/evesch/lib/python2.5'] + sys.path"
SetEnv DJANGO_SETTINGS_MODULE myproject.settings
SetHandler python-program

# added by me (joe)
AuthType Basic
AuthName "Under Construction"
AuthUserFile /home/my_account/webapps/evesch/apache2/conf/.mypasswds
AuthGroupFile /home/my_account/webapps/evesch/apache2/conf/.mygroups
Require group managers
</Location>

##############################################

4) restart your apache server and web browser
/home/my_account/webapps/evesch/apache2/bin/stop
/home/my_account/webapps/evesch/apache2/bin/start


http://forum.webfaction.com/viewtopic.php?id=2363

Show RAM Memory Usage (Linux)

Show RAM used by given user (in MB)
ps -u your_username -o rss,command | grep -v peruser | awk '{sum+=$1} END {print sum/1024}'

Show RAM used by individual process (in KB)
ps -u your_username -o rss,pid,command


http://forum.webfaction.com/viewtopic.php?id=2356

Compile httpd (Apache)

Http Configure options

./configure --enable-so --with-mpm=worker --prefix=/opt/httpd/http2.2/ --enable-rewrite --enable-alias --with-port=8080 --enable-cgi --enable-spelling | tee jjj.configure | tee jjj.configure
make | tee jjj.make
make install | tee jjj.make

mod_python Configure options

./configure --with-apxs=/opt/httpd/http2.2/bin/apxs --with-python=/opt/python2.6/bin/python
make
make install


http://cavedoni.com/2005/django-osx

Change keyboard in X

http://ubuntu-tutorials.com/2008/01/31/changing-the-system-keyboard-mapping-on-ubuntu-dvorak-vs-qwerty/
http://xorg.freedesktop.org/archive/X11R7.0/doc/html/setxkbmap.1.html

Find Usage

"Grep" through each file in a directory:
ls xargs grep -i 'STRING'

List all directories and subdirectories in a directory
find . -type d

List all files in a directory no recursion
# workaround for "parameter list is too long".
find . \( ! -name . -prune \) -name 'STRING*' -print
find . -maxdepth 1 -name 'STRING*' -print # GNU find only

Remove all files in a directory no recursion
# workaround for "parameter list is too long".
find . \( ! -name . -prune \) -name 'STRING*' -print -exec rm {} \;

Find a file with a given inode and delete it
ls -lai # lists the inodes next to the files
find . -inum 12345 -exec rm {} \; # finds and removes by inode

http://www.faqs.org/qa/qa-1381.html
http://sial.org/howto/shell/

Django Model Many-to-One

Define the models

from django.db import models

class Reporter(models.Model):
.. first_name = models.CharField(max_length=30)
.. last_name = models.CharField(max_length=30)
.. email = models.EmailField(blank=True, null=True)

def __unicode__(self):
.. return u"%s %s" % (self.first_name, self.last_name)

class Article(models.Model):
.. headline = models.CharField(max_length=100)
.. pub_date = models.DateField()
.. reporter = models.ForeignKey(Reporter, blank=True, null=True)

def __unicode__(self):
.. return self.headline

.. class Meta:
.... ordering = ('headline',)

------------------------------------------------
Add some objects
# import models from core.models
>>> from core.models import Reporter, Article
>>> from datetime import datetime # needed for inserting current time
# add some reporters
>>> r1 = Reporter(first_name="Joe", last_name="Jaz")
>>> r1.save()
>>> r2 = Reporter(first_name="Jane", last_name="Smith")
>>> r2.save()
# add some articles
>>> a1 = Article(headline="Article1", pub_date=datetime(2008,10,14))
>>> a1.reporter = r1 # can add separately
>>> a1.save()
>>> a2 = Article(headline="Article2", pub_date=datetime(2008,10,15), reporter=r2)
>>> a2.save()
# show an article's reporter
>>> a1.reporter
<Reporter: Joe Jaz>
>>> a2.reporter.first_name
'Jane'
# create an article "under" a Reporter via the Reporter object
# the 'article_set' method is generated on model creation using the lower-case model name
# the create method automatically does a save()
>>> a3 = r1.article_set.create(headline="Article3", pub_date=datetime.now())
>>> a3.headline
'Article3'
>>> a3
<Article: Article1> # returns object with Article type
>>> a3.reporter
<Reporter: Joe Jaz> # returns object with Reporter type - the article reporter
# create an article with no reporter and then add it to a particular reporters set of articles
# the add method automatically does a save()
>>> a4 = Article(headline="Article4", pub_date=datetime.now())
>>> r2.article_set.add(a4)
# show all of the given reporter's articles
>>> r2.article_set.all()
[<Article: Article2>, <Article: Article4>]
# count the number of articles that belong to a reporter
>>> r2.record_set.count()
2
>>> r1.record_set.count()
2
# associate article 4 with reporter 1 (instead of reporter 2)
>>> r1.article_set.add(a4)
# we can assign it back and use different syntax this time
>>> a4.reporter = r2
>>> a4.save()
# we can assign multiple articles to a reporter at a time
# this syntax calls save automatically
>>> r1.article_set = [a2, a4]
# we can create an article and assign it with a reporter_id instead of the reporter object
# the reporter_id field is generated with the model
>>> a5 = Article(headline="Article5",pub_date=datetime.now(),reporter_id=r1.id)
>>> a5.save()
# or reporter_id can be a string
>>> a6 = Article(headline="Article6",pub_date=datetime.now(), reporter_id="1")
>>> a6.save()
# Do a query to find reporters of article 1 and article 3
>>> Reporter.objects.filter(article__in=[a1,a3])

http://www.djangoproject.com/documentation/models/many_to_one/

Django OneToMany & ManyToMany Recursive Models

Define the models in core.models.

from django.db import models

class Place(models.Model):
.. name = models.CharField(max_length=50);
.. parent = models.ForeignKey('self', null=True);

.. def __unicode__(self):
.... return "%s > %s" % (self.name,self.parent )

class Fan(models.Model):
.. name = models.CharField(max_length=8)
.. idol = models.ManyToManyField('self', null=True)

.. def __unicode__(self):
.... return self.name

----------------------------------------------------------
Interact with a Foreign Key (One to Many)

# import Place model
from core.models import Place
# add a Place and save
world = Place(name="Earth", parent=None)
world.save()
# add two Places with the world as a parent
country1 = Place(name="US", parent=world)
country1.save()
country2 = Place(name="UK", parent=world)
country2.save()
# add two Places with the US as a parent
city1 = Place(name="Chicago")
city1.parent = country1 # make the association through an update
city1.save()
city2 = Place(name="New York", parent=country1) # associate through insert
city2.save()
# select all the places
Place.objects.all()

------------------------------------------------------
Interact with a Many to Many model

from core.models import Fan

# Define 2 people
person1 = Fan(name="Joe")
person1.save()
person2 = Fan(name="Jane")
person2.save()
# associate the two people
person1.idol.add(person2)

------------------------------------------------------

Java Jar Commands

Creating the File structure
# simple suggested structure
project_directory/
\
_ bin/ # binary files
\_ byte_file.class # compiled java bytecode
_src/ # source files (optional)
_META-INF/ # standard location for manifest file
\_MANIFEST.MF # manifest file containing jar variables

Editing the MANIFEST.MF
The manifest can contain variables such as the following:
Main-Class: class_name # name of a class with a "main" method that will run when jar is run.
Class-path: path/to/classes/in/package/ # path start at root of package

Create the Jar
jar -cvfm jar_name.jar path/to/manifest.mf dir/to/class/files/
jar -cvfm project.jar META-INF/MANIFEST.MF bin/
-c # compress into a jar
-v # verbose output
-f # compress to a file specified on the command line (versus to stdout)
-m # specify a path to a manifest file.

Execute the Jar
java -jar jar_name.jar

Extract a Jar file
jar -xvf jar_name.jar