Python Pip Usage

Basic PIP install
pip install somepackage

Basic PIP uninstall
pip uninstall somepackage

Requirements file
# cat requirements.txt
MyApp
Framework==0.0.1
Library>=0.2

Installing using a requirements file
pip install -r requirements.pip

PIP Freezing requirements
pip freeze # lists all packages and the specific version installed. Useful for migrating

PIP install into virtualenv env
pip install -E ./env
pip install -E ./env -r requirements.pip # using a requirements file

Basic script for creating a virtualenv and installing requirements
virtualenv --distribute --no-site-packages ve
source ./ve/bin/activate
pip install -E ./ve -r requirements.pip
pip install -E ./ve -r requirements-test.pip

http://heisel.org/blog/2009/11/21/django-hudson/
http://pip.openplans.org/#freezing-requirements

Case Study Notes: install djangobb in virtualenv

Ubuntu 10.04 python 2.6 django 1.1

sudo apt-get build-dep python-psycopg2
sudo aptitude install python-dev

cd Sites
virtualenv --no-site-packages env
. ./env/bin/activate
pip install pil
pip install markdown2 Markdown
pip install django-registration
pip install djapian
pil install xapian
pip install psycopg2
easy_install -i http://downloads.egenix.com/python/index/ucs4/ egenix-mx-base
pip install http://code.djangoproject.com/svn/django/tags/releases/1.1.2/
hg clone http://hg.djangobb.org/djangobb/ djangobb
cd env/lib/python2.6/site-packages/
cp /usr/lib/python2.6/dist-packages/_xapian.so .
cp /usr/lib/python2.6/dist-packages/xapian.py .

# configure settings.py to use postgresql_psycopg2
# comment out the openid stuff

## http://www.saltycrane.com/blog/2009/07/using-psycopg2-virtualenv-ubuntu-jaunty/

Postgresql Basic Commands

Login to postgresql:
psql -d mydb -U myuser -W
psql -h myhost -d mydb -U myuser -W
psql -U myuser -h myhost "dbname=mydb sslmode=require" # ssl connection

Default Admin Login:
sudo -u postgres psql -U postgres
sudo -u postgres psql

List databases on postgresql server:
psql -l [-U myuser] [-W]

Turn off line pager pagination in psql:
\pset pager

Determine system tables:
select * from pg_tables where tableowner = 'postgres';

List databases from within a pg shell:
\l

List databases from UNIX command prompt:
psql -U postgres -l

Describe a table:
\d tablename

Quit psql:
\q

Switch postgres database within admin login shell:
\connect databasename

Reset a user password as admin:
alter user usertochange with password 'new_passwd';

Show all tables:
\dt

List all Schemas:
\dn

List all users:
\du

Load data into posgresql:
psql -W -U username -H hostname < file.sql

Dump (Backup) Data into file:
pg_dump -W -U username -h hostname database_name > file.sql

Increment a sequence:
SELECT nextval('my_id_seq');

Create new user:
CREATE USER jjasinski WITH PASSWORD 'myPassword';
# or
sudo -u postgres createuser jjasinski -W

Change user password:
ALTER USER Postgres WITH PASSWORD 'mypass';

Grant user createdb privilege:
ALTER USER myuser WITH createdb;

Create a superuser user:
create user mysuper with password '1234' SUPERUSER
# or even better
create user mysuper with password '1234' SUPERUSER CREATEDB CREATEROLE INHERIT LOGIN REPLICATION;
# or
sudo -u postgres createuser jjasinski -W -s

Upgrade an existing user to superuser: 
alter user mysuper with superuser;
# or even better
alter user mysuper with SUPERUSER CREATEDB CREATEROLE INHERIT LOGIN REPLICATION

Show Database Version:
SELECT version();

Change Database Owner:
alter database database_name owner to new_owner;

Copy a database:
CREATE DATABASE newdb WITH TEMPLATE originaldb;
http://www.commandprompt.com/ppbook/x14316

View Database Connections:
SELECT * FROM pg_stat_activity;

View show data directory (works on 9.1+; not on 7.x):
show data_directory;

Show run-time parameters:

show all;
select * from pg_settings;

Show the block size setting:
# show block_size;
 block_size
------------
 8192
(1 row)

Show stored procedure source:
SELECT prosrc FROM pg_proc WHERE proname = 'procname'

Grant examples:
# readonly to all tables for myuser
grant select on all tables in schema public to myuser;
# all privileges on table1 and table2 to myuser
grant all privileges on table1, table2, table3 to myuser;

Restore Postgres .dump file:
pg_restore --verbose --clean --no-acl --no-owner -h localhost -U myuser -d mydb latest.dump
source

Find all active sessions and kill them (i.e. for when needing to drop or rename db)
Source: http://stackoverflow.com/questions/5408156/how-to-drop-a-postgresql-database-if-there-are-active-connections-to-it

# Postgres 9.2 and above
SELECT pg_terminate_backend(pg_stat_activity.pid) 

FROM pg_stat_activity 

WHERE pg_stat_activity.datname = 'TARGET_DB' 

 AND pid <> pg_backend_pid();

# Postgres 9.1 and below
SELECT pg_terminate_backend(pg_stat_activity.procpid) 

FROM pg_stat_activity 

WHERE pg_stat_activity.datname = 'TARGET_DB' 

AND procpid <> pg_backend_pid();


Re-read postgres config without dropping connections (i.e. if postgres.conf or pg_hba.conf changes)
Source: http://www.heatware.net/databases/postgresql-reload-config-without-restarting/

/usr/bin/pg_ctl reload
or
SELECT pg_reload_conf();

Per user query logging (logs to to postgres logs):
alter role myuser set log_statement = 'all';


Sources:
http://www.devdaily.com/blog/post/postgresql/log-in-postgresql-database
http://forums.devshed.com/postgresql-help-21/how-do-you-turn-off-more-scroll-lock-at-psql-174831.htm
http://www.cyberciti.biz/faq/howto-add-postgresql-user-account/
http://archives.postgresql.org/pgsql-general/1998-08/msg00050.php
http://stackoverflow.com/questions/876522/creating-a-copy-of-a-database-in-postgres
http://stackoverflow.com/questions/1137060/where-does-postgresql-store-the-database

Recycle bin script

crm()
{
# pass this function the name of the directory that
# you want to backup and remove 
if [ -d "$1" ]
then
  cd "${1}/../"
else
  echo "Directory does not exist"
  return 1
fi

PWD=`pwd`
BKPFILES=`find $PWD -type -f`
BKPDIR="$HOME/.Trash"
daten=`date +%Y.%m.%d-%H.%M.%S`

for i in $BKPFILES
do
  echo "$i"
  DIRNAME="$BKPDIR"`echo ${i%/*}`
  FILENAME=`echo ${i##*/}`
  mkdir -p "$DIRNAME"
  cp -pf "$i" "${DIRNAME}/${FILENAME}.${daten}"
  #NEWFILE=`echo "$i" | sed 's|[ \/]|_|g'`
  print "$i"  "$NEWFILE"   " $DIRNAME"
done
rm -rf "$1"
cd - 
}

OSX Color LS

Color LS for Mac OX
alias ls='ls -G'

sftp notes

sftp using alternate port:
sftp -oPort=2222 username@host.com

sftp using alternate private key file:

sftp -o IdentityFile=/custom/file/location.pem username@host.com

http://www.unix.com/shell-programming-scripting/43334-sftp-scripting-help-required.html

Basic Python Development Server setup with Ubuntu 10.04

Quick setup guide

# config vi command line
echo "set -o vi" >> ~/.bashrc
. ~/.bashrc

# install packages
aptitude install vim openssh-server apache2 python-virtualenv python-mysqldb mysql-server libapache2-mod-php5 libapache2-mod-wsgi eclipse subversion nmap ubuntu-restricted-extras g++ git-gui virtualbox-ose php5-cli mdadm fabric python-dev mercurial

# eclipse development plugins
install pydev for eclipse
install subclipse for eclipse

# firefox development plugins
install firebug for firefox @ http://getfirebug.com/
install web developer for firefox @ https://addons.mozilla.org/en-US/firefox/addon/60/

# add google repo
deb http://dl.google.com/linux/deb/ stable non-free
aptitude update

cd /opt/
wget http://www.djangoproject.com/download/1.1.2/tarball/
wget http://www.djangoproject.com/download/1.2.1/tarball/
tar -xvf Django-1.1.2.tar.gz
tar -xvf Django-1.2.1.tar.gz
cd /usr/lib/python2.6/dist-packages
ln -s /opt/Django-1.2.1/django/ django

# mysql workbench download (64 bit version)
aptitude install libzip1 python-pysqlite2 # deps needed
http://dev.mysql.com/get/Downloads/MySQLGUITools/mysql-workbench-oss-5.2.22-1ubu1004-amd64.deb/from/http://mirror.services.wisc.edu/mysql/
dpkg -i mysql-workbench-oss-5.2.22-1ubu1004-amd64.deb

# setup postgresql
aptitude install postgresql pgadmin3 python-psycopg2

sudo su -
passwd postgres
su postgres
psql template1

The last instruction should open the psql shell, where you can run the following:

ALTER USER postgres WITH ENCRYPTED PASSWORD 'mypassword';

# Setup NX server (Google NeatX server)
sudo apt-get install python-software-properties && sudo add-apt-repository ppa:freenx-team
sudo apt-get update
sudo apt-get install neatx-server

## NOTE: if neatx gives a weird error, delete the session dirs on the server:
## /var/lib/neatx/sessions/some_dir/

https://help.ubuntu.com/community/FreeNX
http://programmingzen.com/2007/12/26/installing-django-with-postgresql-on-ubuntu/

Simple Network Test Script

This is a simple script to log network outages on a local network

root@ubuntu:/mnt/root/usr/local/bin# more lifeline
#!/bin/bash

PING='/bin/ping'
EXTHOST1='www.google.com'
EXTHOST2='sun.iwu.edu'
INTHOST='192.168.1.1'
LOG='/var/log/network_outages'
WAITTIME=120

echo "NETWORK STATUS SCRIPT: Started " `date` >> $LOG
while [ 1=1 ]
do
   dater=`date +%Y.%m.%d-%H.%M.%S`
   $PING -q -c1 $EXTHOST1
   ret=$?
   echo "RET: $ret"
   if [ $ret -ne 0 ]
   then
     echo "EXTERNAL_PING(1): outage detected $dater" >> $LOG
   fi

   dater=`date +%Y.%m.%d-%H.%M.%S`
   $PING -q -c1 $EXTHOST2
   ret=$?
   echo "RET: $ret"
   if [ $ret -ne 0 ]
   then
     echo "EXTERNAL_PING(2): outage detected $dater" >> $LOG
   fi

   $PING -q -c1 $INTHOST
   ret=$?
   echo "RET: $ret"
   if [ $ret -ne 0 ]
   then
     echo "INTERNAL_PING: outage detected $dater" >> $LOG
   fi

   sleep $WAITTIME
done