Python Virtual Env

Download and install via apt-get, ports, easy_install, etc.

Locate the install directory (usually site-packages) and find the virtualenv.py script.
Alternatively, the package manager may have installed a virtualenv script into your PATH.

I will use ENV to refer to /path/to/python/virtual/env
I will use virtualenv.py to refer to /path/to/script/virtualenv.py

Create a new Virtual Environment
python virtualenv.py ENV

(optional) Activate (set PATH and PYTHONPATH) for new Virtual ENV
source ENV/bin/activate

Location of Virtual Env python binary
ENV/bin/python

Location of Virtual ENV python site-packages dir
ENV/lib/python2.x/site-packages/

Location of Virtual ENV python easy_install script
ENV/bin/easy_install


http://pypi.python.org/pypi/virtualenv

curl Usage

Send a POST

curl -u joe:1234 -X POST http://localhost:8000/api/1.0/NU1yWdRM5JBjnZpZX/message/ -d "message=test"

-u = username and password

-d = post parameters

-X = request method

Send a POST using data from standard in:
echo "mykey=myvalue" | curl -X POST -d @-  https://mysite.com/

Show verbose output: 
curl -v http://mysite.com/

Ignore SSL key errors:
curl -k http://mysite.com/

Combining them: 
echo "TEST=1
TEST2=2
" curl -X POST -vk -d @- http://mysite.com/

netbios name lookups

Find NetBIOS name from IP
Windows:
netstat -a 192.168.1.22

Linux:
nbtstat 192.168.1.22 # ntbstat separate package

Find IP from NetBIOS name
Windows:
nbtstat -a host.example.com

Linux:
nmblookup host.example.com

http://www.irongeek.com/i.php?page=security/ipinfo

MacPorts Usage

Update MacPorts itself
sudo port selfupdate
sudo port -d selfupdate # debug

Updates the port tree with new versions definitions
sudo port sync

List available ports
sudo port list

Search ports
port search [keyword]

Lookup package info (desc, maintainer, etc)
port info [package]

Find package dependencies
port deps [package]

Install package
sudo port install [package]
sudo port -v install [package] #verbose

Clean out build files and tarballs
port clean --all [pakcage]

Uninstall a package
sudo port uninstall [package]

Show port contents
port contents [package]

List installed packages
port installed

List outdated ports
port outdated

Upgrade specific packages
port upgrade [package]
port upgrade outdated #updates all outdated packages


http://guide.macports.org/chunked/using.html

DJango Model Inheritance with subclasses

http://www.djangosnippets.org/snippets/1034/
http://www.djangosnippets.org/snippets/1031/
http://adam.gomaa.us/blog/2009/feb/16/subclassing-django-querysets/
http://docs.djangoproject.com/en/dev/ref/contrib/contenttypes/

Python List Operations (map, for comprehensions)

EXAMPLE 1
# Create a list
>>> l = [1,2,3,4,5,6]

# iterate through the list using a for comprehension
>>> [i for i in l]
[1, 2, 3, 4, 5, 6]

# square elements in the list using a for comprehension
>>> [i**2 for i in l]
[1, 4, 9, 16, 25, 36]

# square only even elements in the list using a for comprehension
>>> [i**2 for i in l if i % 2 == 0]
[4, 16, 36]

# iterate through the list using map
>>> map(lambda w: w, l)
[1, 2, 3, 4, 5, 6]

# square each element of the list using map
>>> map(lambda w: w**2, l)
[1, 4, 9, 16, 25, 36]

# or use map to call a separately defined function to# iterate through the list
>>> def squa(x):
...... return x**2

>>> map(squa, l)
[1, 4, 9, 16, 25, 36]

# use map function to call a separately defined function that takes 2 args
>>> l2 = map(lambda w: (w,2),l)
>>> l2
[(1, 2), (2, 2), (3, 2), (4, 2), (5, 2), (6, 2)]

>>> def pow(base, expo):
...... return base**expo

>>> map(lambda (x,y): pow(x,y), l2)
[1, 4, 9, 16, 25, 36]

# filter a list
>>> filter(lambda w: w > 2, l)
[3, 4, 5, 6]

EXAMPLE 2
# define a simple class to play with
>>> class A(object):
...... def __init__(self, x):
........ self.x = x

# create simple list of object instances of the class
>>> l = [A(1), A(2), A(3)]

# use a for (list) comprehension to iterate through the list
>>> [i.x for i in l]
[1, 2, 3]

# use a for comprehension to iterate with a condition
>>> [i.x for i in l if i.x > 1]
[2, 3]

# use map to apply a function to every element in the list
>>> map(lambda w: w.x, l)
[1, 2, 3]

>>> map(lambda w: w.x * w.x, l)
[1, 4, 9]

EXAMPLE 1

# Create a list

>>> l = [1,2,3,4,5,6]

# iterate through the list using a for comprehension

>>> [i for i in l]

[1, 2, 3, 4, 5, 6]

# square elements in the list using a for comprehension

>>> [i**2 for i in l]

[1, 4, 9, 16, 25, 36]

# square only even elements in the list using a for comprehension

>>> [i**2 for i in l if i % 2 == 0]

[4, 16, 36]

# iterate through the list using map

>>> map(lambda w: w, l)

[1, 2, 3, 4, 5, 6]

# square each element of the list using map

>>> map(lambda w: w**2, l)

[1, 4, 9, 16, 25, 36]

# or use map to call a separately defined function to# iterate through the list

>>> def squa(x):

...... return x**2

>>> map(squa, l)

[1, 4, 9, 16, 25, 36]

# use map function to call a separately defined function that takes 2 args

>>> l2 = map(lambda w: (w,2),l)

>>> l2

[(1, 2), (2, 2), (3, 2), (4, 2), (5, 2), (6, 2)]

>>> def pow(base, expo):

...... return base**expo

>>> map(lambda (x,y): pow(x,y), l2)

[1, 4, 9, 16, 25, 36]

# filter a list

>>> filter(lambda w: w > 2, l)

[3, 4, 5, 6]

EXAMPLE 3

# split a list into chunks

>>> map(None, *(iter(range(10)),) * 3)

[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, None, None)]

http://stackoverflow.com/questions/1335392/iteration-over-list-slices

Django South Migrations

Install Django South

- Place the untared 'south' folder into your Django applicaion

- Modify the project's settings.py and add 'south' to the INSTALLED_APPS list

- For new Django projects:

---1) initialize any Django apps or models to monitor with South.

----a) See Initializing Applications below

---2) run ./manage.py syncdb

---3) run ./manage.py migrate

To view help on startmigration:  # South 7.x+ uses the schemamigration command instead

./manage.py startmigration

Usage: ./manage.py startmigration appname migrationname [--initial] [--auto]

[--model ModelName] [--add-field ModelName.field_name] [--freeze] [--stdout]

Initializing Applications - needed whenever you want to track more Django apps

# this creates a 'migrations' folder in your Django app directory

./manage startmigration app_name comment --initial  # updates for South 7.x+
./manage schemamigration app_name comment --initial # Init's all models in application
./manage startmigration app_name --initial # updates for South 7.x+

./manage schemamigration app_name --initial # comment is optional here

Adding South with no data and no existing models.py objects:
# 1) syncdb as normal
python manage.py syncdb
# 2) Start an application if needed
python manage.py startapp appname
# 3) create a migration file with a blueprint to create tables
python manage.py startmigration appname --initial  # updates for South 7.x+
python manage.py schemamigration appname --initial
# 4) Apply the blueprint and create the tables
python manage.py migrate appname

# 5) Repeat steps 3 and 4 for each additional app if needed

Adding south with no data and existing models.py objects:
# 1) syncdb as normal if needed
python manage.py syncdb
# 2) create a migration file with a blueprint to create tables
python manage.py startmigration appname --initial # updates for South 7.x+
python manage.py schemamigration appname --initial
# 3) Apply the blueprint but it doesn't actually create the tables
python manage.py migrate appname --fake
# 4) Repeat steps 2 and 3 for each additional app if needed

Applying changes after making changes to a model (repeat for each app):
# Creates a migration 'blueprint' file and guesses what should take place
python manage.py startmigration appname migrationname --auto # updates for South 7.x+
python manage.py schemamigration appname migrationname --auto
# OR create the blueprint file but it will be empty
python manage.py startmigration appname migrationname  # updates for South 7.x+
python manage.py schemamigration appname migrationname
# make any modifications needed
# migrate the tables in the app
python manage.py migrate appname
# OR migrate all tables in all apps
python manage.py migrate

Adding a new model to an app already managed with south:
./manage startmigration app_name comment --model m1 --model m2 # updates for South 7.x+
./manage schemamigration app_name comment --model m1 --model m2 # inits specific models

List available migrations
python manage.py migrate --list

http://south.aeracode.org/wiki/ConvertingAnApp

Oracle Space queries

Get Used space for current user
SELECT sum(bytes)/(1024 * 1024) as "Used (MB)"
FROM user_SEGMENTS
ORDER BY 1 desc

Get tablespace usage for all users using that tablespace
select owner, sum(bytes)/power(2,20)mb
from dba_extents
where tablespace_name = 'TABLESPACE'
group by owner
order by 2 desc;

Get Total tablespace available for current user
select tablespace_name,round(sum(bytes) / (1024 * 1024),2) "Tablespace SIZE (MB)"
from user_free_space
group by tablespace_name

Get space of individual tables/indexes (objects) in a schema
select sum(bytes) / (1024 * 1024) as MB,owner,segment_type, segment_name
from dba_segments s
where owner = 'SCHEMA'
group by owner,segment_type, segment_name
order by MB desc

Get USER_ dictornary Tables
SELECT table_name, comments
FROM dictionary
WHERE table_name LIKE 'USER_%'
ORDER BY table_name;

http://snipplr.com/view/4748/get-a-list-of-all-the-user-tables-oracle/
http://www.freelists.org/post/oracle-l/dba-extents-vs-dba-segments,8

CSS Pre Tag formatting

Make pre tags wrap as expected

pre {
white-space: pre-wrap;       /* css-3 should we be so lucky... */
white-space: -moz-pre-wrap;  /* Mozilla, since 1999 */
white-space: -pre-wrap;      /* Opera 4-6  */
white-space: -o-pre-wrap;    /* Opera 7  */
word-wrap: break-word;       /* Internet Explorer 5.5+ */
_white-space: pre;   /* IE only hack to re-specify in addition to
word-wrap  */
}

http://archivist.incutio.com/viewlist/css-discuss/55677
http://bavotasan.com/tutorials/how-to-wrap-text-within-the-pre-tag-using-css/

Relative to Absolute Path in Shell Script

Simple bash function to convert relative paths to absolute pats
fun_abs
{ echo “`cd \`dirname $1\`; pwd`/`basename $1`” }

http://www.robertpeaslee.com/index.php/converting-a-relative-path-to-an-absolute-path-in-bash/

UPDATE: unfortuately the above only works for directories that exist - the perl hack will get around this

fun_abs
{FILE=`$PERL -e "use File::Spec::Functions qw[rel2abs];print rel2abs('$1');"`; echo "$FILE"; }

Get File Extension in Shell Script

Returns the last file extension in the file name
echo "thisfile.txt.log"awk -F . '{print $NF}' # returns "log"

http://liquidat.wordpress.com/2007/09/29/short-tip-get-file-extension-in-shell-script/

Correct CSS PNG Color Mismatch

Copied from link below
Mac OS X will render the images using the color profile they got stored, actually png gamma correction would be more exact. This is not immediately apparent because at a glance the files may look identical but using the pngs over a CSS background color often reveals an unpleasant sight on the mac, especially on Safari.
To prevent such a color mismatch either use a different image format or strip the png gamma correction. In the case of the latter, pngcrush does a pretty good job.
I was pleasantly surprised that I could install it with ease on Fedora ("yum install -y pngcrush"). To use it run:

pngcrush -rem gAMA -rem cHRM -rem iCCP -rem sRGB png-file-name optimized-png-file-name


http://www.viseztrance.com/2009/02/fixing-the-png-color-mismatch-on-mac-os-x.html

vim settings

# start vim with an alternate vimrc location
vim -u filename

# Turn compatibility mode off via set command
:set nocompatible

# view special characters in a vim session
:set list
:set nolist # disable

# set search result hilighting
:set hlsearch

# use different visible characters to see special characters in vim
:set listchars=eol:$,tab:>-,trail:~,extends:>,precedes:< :set list # view line numbers
:set nu
:set nonu # disable

# enable linewrap (on by default)
:set wrap
:set nowrap #disable

# indenting source code (setup)
:set et # expand tabs to spaces
:set sw=4
:set smarttab

# Manually choose the language for syntax highlighting

:setf language
:setf html  # example

# The available languages for syntax highlighting 

# can be found in the vim install directory.  

# In my system, it was /usr/share/vim/vim72/syntax


# indenting source code (usage)
1) in command mode press v to and arrows to select a range of lines
2) after selecting, press ==

http://vim.wikia.com/wiki/Indenting_source_code

# indent multiple lines (not necessarily source code)
1) in command mode press v and arrows to select a range of lines
2) type shift >
shift < will shift a block back

# compatibility mode
# by default, vim will enter into compatibility mode when loading.
# Disable this behavior one of two ways
# 1) Create a .vimrc in your home directory
# 2) Within the vim command line, type :set nocompatible

# trim whitespace at end of line
:1,$s/[ <tab>]*$//

# get vim to return the previous position when editing files

" Tell vim to remember certain things when we exit
"  '10 : marks will be remembered for up to 10 previously edited files
"  "100 : will save up to 100 lines for each register
"  :20 : up to 20 lines of command-line history will be remembered
"  % : saves and restores the buffer list
"  n... : where to save the viminfo files
set viminfo='10,\"100,:20,%,n~/.viminfo

" when we reload, tell vim to restore the cursor to the saved position
augroup JumpCursorOnEdit
au!
autocmd BufReadPost *
\ if expand(":p:h") !=? $TEMP |
\ if line("'\"") > 1 && line("'\"") <= line("$") |
\ let JumpCursorOnEdit_foo = line("'\"") |
\ let b:doopenfold = 1 |
\ if (foldlevel(JumpCursorOnEdit_foo) > foldlevel(JumpCursorOnEdit_foo - 1)) |
\ let JumpCursorOnEdit_foo = JumpCursorOnEdit_foo - 1 |
\ let b:doopenfold = 2 |
\ endif |
\ exe JumpCursorOnEdit_foo |
\ endif |
\ endif
" Need to postpone using "zv" until after reading the modelines.
autocmd BufWinEnter *
\ if exists("b:doopenfold") |
\ exe "normal zv" |
\ if(b:doopenfold > 1) |
\ exe "+".1 |
\ endif |
\ unlet b:doopenfold |
\ endif
augroup END

http://vim.wikia.com/wiki/Restore_cursor_to_file_position_in_previous_editing_session

http://ubuntuforums.org/showthread.php?t=789327
http://www.oualline.com/vim-cook.html
http://stackoverflow.com/questions/1675688/make-vim-show-all-white-spaces-as-a-character
http://jamesreubenknowles.com/set-vim-syntax-language-270

svn+ssh on alternate port

Setup the config
# Add the following lines to ~/.subversion/config
# must be put in the [tunnels] section of the config file
sshnew=ssh -l user -p 2222

Execute the svn:
svn co svn+sshnew://servername.com/path/to/repository/

Djanog show model query

Show the raw SQL generated by a model access: 

# DEBUG needs to be set True 

>>> from django.db import connection 

>>> connection.queries [{'sql': 'SELECT polls_polls.id,polls_polls.question,polls_polls.pub_date FROM polls_polls', 'time': '0.002'}]

sql # The raw SQL statement 
time # How long the statement took to execute, in seconds.

Reset the queries list returned above:
>>> from django import db
>>> db.reset_queries()

Show SQL for a given query (newer django version; 1.5+?)
>>> Product.objects.all().query.sql_with_params()

Show SQL for a given query (older django version; 1.2-1.5+?)
>>> Product.objects.all().query.as_sql()

http://docs.djangoproject.com/en/dev/faq/models/
http://docs.djangoproject.com/en/dev/faq/models/#how-can-i-see-the-raw-sql-queries-django-is-running

http://stackoverflow.com/questions/2314920/django-show-log-orm-sql-calls-from-python-shell

Secure Django Location with htaccess file (webfaction)

Basic steps to secure a Django account using htaccess file
note, this examples assumes apache 2.2, in which the basic auth apache module changed.
note, this example uses a WebFaction account but can be applied to any Django Apache install.

1) Execute this command to create an htpassword file
htpasswd -c /home/my_account/webapps/evesch/apache2/conf/.mypasswds my_user

2) Execute this command to create a htgroups file
echo "managers: my_user" > /home/my_account/webapps/evesch/apache2/conf/.mygroups

3) Modify your httpd.conf file to look like this.
############# contents of /home/my_account/webapps/evesch/apache2 #####
ServerRoot "/home/my_account/webapps/evesch/apache2"

LoadModule dir_module modules/mod_dir.so
LoadModule env_module modules/mod_env.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule mime_module modules/mod_mime.so
LoadModule python_module modules/mod_python.so
LoadModule rewrite_module modules/mod_rewrite.so

# added by me (joe)
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so


KeepAlive Off
Listen 7637
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
CustomLog logs/access_log combined
ServerLimit 2

<Location "/">
PythonHandler django.core.handlers.modpython
PythonPath "['/home/my_account/webapps/evesch', '/home/my_account/webapps/evesch/lib/python2.5'] + sys.path"
SetEnv DJANGO_SETTINGS_MODULE myproject.settings
SetHandler python-program

# added by me (joe)
AuthType Basic
AuthName "Under Construction"
AuthUserFile /home/my_account/webapps/evesch/apache2/conf/.mypasswds
AuthGroupFile /home/my_account/webapps/evesch/apache2/conf/.mygroups
Require group managers
</Location>

##############################################

4) restart your apache server and web browser
/home/my_account/webapps/evesch/apache2/bin/stop
/home/my_account/webapps/evesch/apache2/bin/start


http://forum.webfaction.com/viewtopic.php?id=2363

Show RAM Memory Usage (Linux)

Show RAM used by given user (in MB)
ps -u your_username -o rss,command | grep -v peruser | awk '{sum+=$1} END {print sum/1024}'

Show RAM used by individual process (in KB)
ps -u your_username -o rss,pid,command


http://forum.webfaction.com/viewtopic.php?id=2356

Compile httpd (Apache)

Http Configure options

./configure --enable-so --with-mpm=worker --prefix=/opt/httpd/http2.2/ --enable-rewrite --enable-alias --with-port=8080 --enable-cgi --enable-spelling | tee jjj.configure | tee jjj.configure
make | tee jjj.make
make install | tee jjj.make

mod_python Configure options

./configure --with-apxs=/opt/httpd/http2.2/bin/apxs --with-python=/opt/python2.6/bin/python
make
make install


http://cavedoni.com/2005/django-osx

Change keyboard in X

http://ubuntu-tutorials.com/2008/01/31/changing-the-system-keyboard-mapping-on-ubuntu-dvorak-vs-qwerty/
http://xorg.freedesktop.org/archive/X11R7.0/doc/html/setxkbmap.1.html

Find Usage

"Grep" through each file in a directory:
ls xargs grep -i 'STRING'

List all directories and subdirectories in a directory
find . -type d

List all files in a directory no recursion
# workaround for "parameter list is too long".
find . \( ! -name . -prune \) -name 'STRING*' -print
find . -maxdepth 1 -name 'STRING*' -print # GNU find only

Remove all files in a directory no recursion
# workaround for "parameter list is too long".
find . \( ! -name . -prune \) -name 'STRING*' -print -exec rm {} \;

Find a file with a given inode and delete it
ls -lai # lists the inodes next to the files
find . -inum 12345 -exec rm {} \; # finds and removes by inode

http://www.faqs.org/qa/qa-1381.html
http://sial.org/howto/shell/