Relative to Absolute Path in Shell Script

Simple bash function to convert relative paths to absolute pats
fun_abs
{ echo “`cd \`dirname $1\`; pwd`/`basename $1`” }

http://www.robertpeaslee.com/index.php/converting-a-relative-path-to-an-absolute-path-in-bash/

UPDATE: unfortuately the above only works for directories that exist - the perl hack will get around this

fun_abs
{FILE=`$PERL -e "use File::Spec::Functions qw[rel2abs];print rel2abs('$1');"`; echo "$FILE"; }

Get File Extension in Shell Script

Returns the last file extension in the file name
echo "thisfile.txt.log"awk -F . '{print $NF}' # returns "log"

http://liquidat.wordpress.com/2007/09/29/short-tip-get-file-extension-in-shell-script/

Correct CSS PNG Color Mismatch

Copied from link below
Mac OS X will render the images using the color profile they got stored, actually png gamma correction would be more exact. This is not immediately apparent because at a glance the files may look identical but using the pngs over a CSS background color often reveals an unpleasant sight on the mac, especially on Safari.
To prevent such a color mismatch either use a different image format or strip the png gamma correction. In the case of the latter, pngcrush does a pretty good job.
I was pleasantly surprised that I could install it with ease on Fedora ("yum install -y pngcrush"). To use it run:

pngcrush -rem gAMA -rem cHRM -rem iCCP -rem sRGB png-file-name optimized-png-file-name


http://www.viseztrance.com/2009/02/fixing-the-png-color-mismatch-on-mac-os-x.html

vim settings

# start vim with an alternate vimrc location
vim -u filename

# Turn compatibility mode off via set command
:set nocompatible

# view special characters in a vim session
:set list
:set nolist # disable

# set search result hilighting
:set hlsearch

# use different visible characters to see special characters in vim
:set listchars=eol:$,tab:>-,trail:~,extends:>,precedes:< :set list # view line numbers
:set nu
:set nonu # disable

# enable linewrap (on by default)
:set wrap
:set nowrap #disable

# indenting source code (setup)
:set et # expand tabs to spaces
:set sw=4
:set smarttab

# Manually choose the language for syntax highlighting

:setf language
:setf html  # example

# The available languages for syntax highlighting 

# can be found in the vim install directory.  

# In my system, it was /usr/share/vim/vim72/syntax


# indenting source code (usage)
1) in command mode press v to and arrows to select a range of lines
2) after selecting, press ==

http://vim.wikia.com/wiki/Indenting_source_code

# indent multiple lines (not necessarily source code)
1) in command mode press v and arrows to select a range of lines
2) type shift >
shift < will shift a block back

# compatibility mode
# by default, vim will enter into compatibility mode when loading.
# Disable this behavior one of two ways
# 1) Create a .vimrc in your home directory
# 2) Within the vim command line, type :set nocompatible

# trim whitespace at end of line
:1,$s/[ <tab>]*$//

# get vim to return the previous position when editing files

" Tell vim to remember certain things when we exit
"  '10 : marks will be remembered for up to 10 previously edited files
"  "100 : will save up to 100 lines for each register
"  :20 : up to 20 lines of command-line history will be remembered
"  % : saves and restores the buffer list
"  n... : where to save the viminfo files
set viminfo='10,\"100,:20,%,n~/.viminfo

" when we reload, tell vim to restore the cursor to the saved position
augroup JumpCursorOnEdit
au!
autocmd BufReadPost *
\ if expand(":p:h") !=? $TEMP |
\ if line("'\"") > 1 && line("'\"") <= line("$") |
\ let JumpCursorOnEdit_foo = line("'\"") |
\ let b:doopenfold = 1 |
\ if (foldlevel(JumpCursorOnEdit_foo) > foldlevel(JumpCursorOnEdit_foo - 1)) |
\ let JumpCursorOnEdit_foo = JumpCursorOnEdit_foo - 1 |
\ let b:doopenfold = 2 |
\ endif |
\ exe JumpCursorOnEdit_foo |
\ endif |
\ endif
" Need to postpone using "zv" until after reading the modelines.
autocmd BufWinEnter *
\ if exists("b:doopenfold") |
\ exe "normal zv" |
\ if(b:doopenfold > 1) |
\ exe "+".1 |
\ endif |
\ unlet b:doopenfold |
\ endif
augroup END

http://vim.wikia.com/wiki/Restore_cursor_to_file_position_in_previous_editing_session

http://ubuntuforums.org/showthread.php?t=789327
http://www.oualline.com/vim-cook.html
http://stackoverflow.com/questions/1675688/make-vim-show-all-white-spaces-as-a-character
http://jamesreubenknowles.com/set-vim-syntax-language-270

svn+ssh on alternate port

Setup the config
# Add the following lines to ~/.subversion/config
# must be put in the [tunnels] section of the config file
sshnew=ssh -l user -p 2222

Execute the svn:
svn co svn+sshnew://servername.com/path/to/repository/

Djanog show model query

Show the raw SQL generated by a model access: 

# DEBUG needs to be set True 

>>> from django.db import connection 

>>> connection.queries [{'sql': 'SELECT polls_polls.id,polls_polls.question,polls_polls.pub_date FROM polls_polls', 'time': '0.002'}]

sql # The raw SQL statement 
time # How long the statement took to execute, in seconds.

Reset the queries list returned above:
>>> from django import db
>>> db.reset_queries()

Show SQL for a given query (newer django version; 1.5+?)
>>> Product.objects.all().query.sql_with_params()

Show SQL for a given query (older django version; 1.2-1.5+?)
>>> Product.objects.all().query.as_sql()

http://docs.djangoproject.com/en/dev/faq/models/
http://docs.djangoproject.com/en/dev/faq/models/#how-can-i-see-the-raw-sql-queries-django-is-running

http://stackoverflow.com/questions/2314920/django-show-log-orm-sql-calls-from-python-shell

Secure Django Location with htaccess file (webfaction)

Basic steps to secure a Django account using htaccess file
note, this examples assumes apache 2.2, in which the basic auth apache module changed.
note, this example uses a WebFaction account but can be applied to any Django Apache install.

1) Execute this command to create an htpassword file
htpasswd -c /home/my_account/webapps/evesch/apache2/conf/.mypasswds my_user

2) Execute this command to create a htgroups file
echo "managers: my_user" > /home/my_account/webapps/evesch/apache2/conf/.mygroups

3) Modify your httpd.conf file to look like this.
############# contents of /home/my_account/webapps/evesch/apache2 #####
ServerRoot "/home/my_account/webapps/evesch/apache2"

LoadModule dir_module modules/mod_dir.so
LoadModule env_module modules/mod_env.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule mime_module modules/mod_mime.so
LoadModule python_module modules/mod_python.so
LoadModule rewrite_module modules/mod_rewrite.so

# added by me (joe)
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so


KeepAlive Off
Listen 7637
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
CustomLog logs/access_log combined
ServerLimit 2

<Location "/">
PythonHandler django.core.handlers.modpython
PythonPath "['/home/my_account/webapps/evesch', '/home/my_account/webapps/evesch/lib/python2.5'] + sys.path"
SetEnv DJANGO_SETTINGS_MODULE myproject.settings
SetHandler python-program

# added by me (joe)
AuthType Basic
AuthName "Under Construction"
AuthUserFile /home/my_account/webapps/evesch/apache2/conf/.mypasswds
AuthGroupFile /home/my_account/webapps/evesch/apache2/conf/.mygroups
Require group managers
</Location>

##############################################

4) restart your apache server and web browser
/home/my_account/webapps/evesch/apache2/bin/stop
/home/my_account/webapps/evesch/apache2/bin/start


http://forum.webfaction.com/viewtopic.php?id=2363

Show RAM Memory Usage (Linux)

Show RAM used by given user (in MB)
ps -u your_username -o rss,command | grep -v peruser | awk '{sum+=$1} END {print sum/1024}'

Show RAM used by individual process (in KB)
ps -u your_username -o rss,pid,command


http://forum.webfaction.com/viewtopic.php?id=2356

Compile httpd (Apache)

Http Configure options

./configure --enable-so --with-mpm=worker --prefix=/opt/httpd/http2.2/ --enable-rewrite --enable-alias --with-port=8080 --enable-cgi --enable-spelling | tee jjj.configure | tee jjj.configure
make | tee jjj.make
make install | tee jjj.make

mod_python Configure options

./configure --with-apxs=/opt/httpd/http2.2/bin/apxs --with-python=/opt/python2.6/bin/python
make
make install


http://cavedoni.com/2005/django-osx

Change keyboard in X

http://ubuntu-tutorials.com/2008/01/31/changing-the-system-keyboard-mapping-on-ubuntu-dvorak-vs-qwerty/
http://xorg.freedesktop.org/archive/X11R7.0/doc/html/setxkbmap.1.html

Find Usage

"Grep" through each file in a directory:
ls xargs grep -i 'STRING'

List all directories and subdirectories in a directory
find . -type d

List all files in a directory no recursion
# workaround for "parameter list is too long".
find . \( ! -name . -prune \) -name 'STRING*' -print
find . -maxdepth 1 -name 'STRING*' -print # GNU find only

Remove all files in a directory no recursion
# workaround for "parameter list is too long".
find . \( ! -name . -prune \) -name 'STRING*' -print -exec rm {} \;

Find a file with a given inode and delete it
ls -lai # lists the inodes next to the files
find . -inum 12345 -exec rm {} \; # finds and removes by inode

http://www.faqs.org/qa/qa-1381.html
http://sial.org/howto/shell/

Django Model Many-to-One

Define the models

from django.db import models

class Reporter(models.Model):
.. first_name = models.CharField(max_length=30)
.. last_name = models.CharField(max_length=30)
.. email = models.EmailField(blank=True, null=True)

def __unicode__(self):
.. return u"%s %s" % (self.first_name, self.last_name)

class Article(models.Model):
.. headline = models.CharField(max_length=100)
.. pub_date = models.DateField()
.. reporter = models.ForeignKey(Reporter, blank=True, null=True)

def __unicode__(self):
.. return self.headline

.. class Meta:
.... ordering = ('headline',)

------------------------------------------------
Add some objects
# import models from core.models
>>> from core.models import Reporter, Article
>>> from datetime import datetime # needed for inserting current time
# add some reporters
>>> r1 = Reporter(first_name="Joe", last_name="Jaz")
>>> r1.save()
>>> r2 = Reporter(first_name="Jane", last_name="Smith")
>>> r2.save()
# add some articles
>>> a1 = Article(headline="Article1", pub_date=datetime(2008,10,14))
>>> a1.reporter = r1 # can add separately
>>> a1.save()
>>> a2 = Article(headline="Article2", pub_date=datetime(2008,10,15), reporter=r2)
>>> a2.save()
# show an article's reporter
>>> a1.reporter
<Reporter: Joe Jaz>
>>> a2.reporter.first_name
'Jane'
# create an article "under" a Reporter via the Reporter object
# the 'article_set' method is generated on model creation using the lower-case model name
# the create method automatically does a save()
>>> a3 = r1.article_set.create(headline="Article3", pub_date=datetime.now())
>>> a3.headline
'Article3'
>>> a3
<Article: Article1> # returns object with Article type
>>> a3.reporter
<Reporter: Joe Jaz> # returns object with Reporter type - the article reporter
# create an article with no reporter and then add it to a particular reporters set of articles
# the add method automatically does a save()
>>> a4 = Article(headline="Article4", pub_date=datetime.now())
>>> r2.article_set.add(a4)
# show all of the given reporter's articles
>>> r2.article_set.all()
[<Article: Article2>, <Article: Article4>]
# count the number of articles that belong to a reporter
>>> r2.record_set.count()
2
>>> r1.record_set.count()
2
# associate article 4 with reporter 1 (instead of reporter 2)
>>> r1.article_set.add(a4)
# we can assign it back and use different syntax this time
>>> a4.reporter = r2
>>> a4.save()
# we can assign multiple articles to a reporter at a time
# this syntax calls save automatically
>>> r1.article_set = [a2, a4]
# we can create an article and assign it with a reporter_id instead of the reporter object
# the reporter_id field is generated with the model
>>> a5 = Article(headline="Article5",pub_date=datetime.now(),reporter_id=r1.id)
>>> a5.save()
# or reporter_id can be a string
>>> a6 = Article(headline="Article6",pub_date=datetime.now(), reporter_id="1")
>>> a6.save()
# Do a query to find reporters of article 1 and article 3
>>> Reporter.objects.filter(article__in=[a1,a3])

http://www.djangoproject.com/documentation/models/many_to_one/

Django OneToMany & ManyToMany Recursive Models

Define the models in core.models.

from django.db import models

class Place(models.Model):
.. name = models.CharField(max_length=50);
.. parent = models.ForeignKey('self', null=True);

.. def __unicode__(self):
.... return "%s > %s" % (self.name,self.parent )

class Fan(models.Model):
.. name = models.CharField(max_length=8)
.. idol = models.ManyToManyField('self', null=True)

.. def __unicode__(self):
.... return self.name

----------------------------------------------------------
Interact with a Foreign Key (One to Many)

# import Place model
from core.models import Place
# add a Place and save
world = Place(name="Earth", parent=None)
world.save()
# add two Places with the world as a parent
country1 = Place(name="US", parent=world)
country1.save()
country2 = Place(name="UK", parent=world)
country2.save()
# add two Places with the US as a parent
city1 = Place(name="Chicago")
city1.parent = country1 # make the association through an update
city1.save()
city2 = Place(name="New York", parent=country1) # associate through insert
city2.save()
# select all the places
Place.objects.all()

------------------------------------------------------
Interact with a Many to Many model

from core.models import Fan

# Define 2 people
person1 = Fan(name="Joe")
person1.save()
person2 = Fan(name="Jane")
person2.save()
# associate the two people
person1.idol.add(person2)

------------------------------------------------------

Java Jar Commands

Creating the File structure
# simple suggested structure
project_directory/
\
_ bin/ # binary files
\_ byte_file.class # compiled java bytecode
_src/ # source files (optional)
_META-INF/ # standard location for manifest file
\_MANIFEST.MF # manifest file containing jar variables

Editing the MANIFEST.MF
The manifest can contain variables such as the following:
Main-Class: class_name # name of a class with a "main" method that will run when jar is run.
Class-path: path/to/classes/in/package/ # path start at root of package

Create the Jar
jar -cvfm jar_name.jar path/to/manifest.mf dir/to/class/files/
jar -cvfm project.jar META-INF/MANIFEST.MF bin/
-c # compress into a jar
-v # verbose output
-f # compress to a file specified on the command line (versus to stdout)
-m # specify a path to a manifest file.

Execute the Jar
java -jar jar_name.jar

Extract a Jar file
jar -xvf jar_name.jar

SSH Client Persistance

Keep SSH connections from timing out
On your client, add this line to your /etc/ssh/ssh_config:
ServerAliveInterval 30

Sound Card Info

Check what ALSA soundcards are available on a Linux system:
cat /proc/asound/cards

Check what features each ALSA card has:
cat /proc/asound/devices
# note that the left number column corresponds to a soundcard in the above list

List sound cards detected
aplay -l

Set the default sound card:
# Enter this into /etc/asound.conf or .asoundrc

pcm.!default {
type hw
card 1
}
ctl.!default {
type hw
card 1
}

Using alsamixer to select which sound card to affect: 
alsamixer -c <sound_card_number_from_proc> 

Using mplayer with a non-default sound device
mplayer -ao oss:/dev/dsp *.mp3 # device 0
mplayer -ao alsa:device=hw=0.0 *.mp3 # device 0
mplayer -ao oss:/dev/dsp1 *.mp3 # device 1
mplayer -ao alsa:device=how=1.0 *.mp3 # device 1


http://seehuhn.de/pages/alsa
http://ubuntuforums.org/showthread.php?t=747054

Sendmail/Postfix Info

Sendmail view mailq:
mailq
sendmail -bp

Sendmail view verbose info:
sendmail -v

Postfix flush mail queue:
sudo postfix flush
sudo postfix -f

Remove all mail in queue: 
sudo postsuper -d ALL

Remove all mail in deferred queue: 
sudo postsuper -d ALL deferred

Postfix Info
http://www.akadia.com/services/postfix_mta.html

http://www.cyberciti.biz/tips/howto-postfix-flush-mail-queue.html
http://www.unix.com.ua/orelly/networking/tcpip/ch10_08.htm
http://shinphp.blogspot.com/2011/09/ubuntu-clean-sendmail-queue.html

Join UNIX Command

Join takes two sorted text files and joins them together on common keys, similar to the Join SQL statements.

Basic join syntax:
join file1 file2

Specify a delimiter other than the default of whitespace (for input and output):
join -t: file1 file2 # sets the delimiter to be a colon


Join on specific columns in each file:
join -1 1 -2 1 file1 file2 # this will make file1's first column join with the file2's first column.

Join on a specific column and output specific columns:
join -11 -21 -o 1.1 1.2 2.1 2.2 file1 file2 # the -o options takes space-seperated arguments that take the form of file.column to specify only particular columns.

Specify filler text in a result column when two keys do not match between files:
join -e "none" file1 file2 -o 1.1 1.2 2.1 2.2 # the -e option needs to be used with -o

http://www.softpanorama.org/Tools/join.shtml
http://www.computerhope.com/unix/ujoin.htm

Compile Python 2.5 From Source

Download from Python web page
http://www.python.org/

UnTAR and enter the directory
tar -jxvf Python-2.5.2.tar.bz2
cd Python-2.5.2

Configure using an alternate install dir
(use ./configure -h for config options)
./configure --prefix=/opt/python/

Build and install the software
make | tee makelog
make test | tee maketestlog
make install | tee makeinstalllog
This will create a python binary. If you specified the prefix, it will put it in the bin/ directory under the prefix directory. However, this compiles the command line python version. To get it working with Apache, preform the following.

Download mod_python
http://www.modpython.org/

Untar and enter into the directory
tar -zxvf mod_python-3.3.1.tgz
cd mod_python-3.3.1

Configure mod_python.
It needs to know where python binary to use, and the apache build tool apxs is located during the configure.

./configure \
--with-apxs=/usr/sbin/apxs \ # the apache build tool
--with-python=/opt/python25/bin/python2.5 \ # path to python binary
| tee output.config
make | tee output.make
make install | tee output.makeinstall

make install should place a binary module in /etc/httpd/modules
(or /usr/lib/httpd/modules)
The module will be called mod_python.so

Configure Apache
You will need to tell Apache to load the module by adding the following line in the Apache configuration file (httpd.conf)

LoadModule python_module modules/mod_python.so
<Directory /path/to/somewhere >
  AddHandler mod_python .py
  PythonHandler mptest
  PythonDebug On
</Directory>

Imaging Support
To be able to manipulate images with python, you need the PIL library. Download it from http://www.pythonware.com/products/pil/
After untaring and entering the directory, install by: python setup.py install
It will copy files into your default (`which python`) python install.

http://www.modpython.org/live/current/doc-html/inst-apacheconfig.html

Basic apache .htaccess authentication

To restrict access to a specific directory
Create a .htaccess file in the directory that you want to restrict
AuthUserFile /path/to/.mypasswds
AuthGroupFile /dev/null
AuthName "Title for password box"
AuthType Basic
require user username
require user user2 # you can add more users by setting multiple require statements
# Require valid-user  # you can allow any user listed in password files.

To generate an encrypted password
Execute this shell command which will create the file .mypasswds with a hashed password entry
htpasswd -c /path/to/.mypasswds username
New password: password
Re-type new password: password

Enable the configuration
The module mod_auth must be loaded in Apache2 config
LoadModule auth_module modules/mod_auth.so
Edit the Apache2 config files to set the AllowOverride for the directory you want to protect
<Directory "/var/www/html">
Options Indexes FollowSymLinks
AllowOverride AuthConfig
</Directory>

http://www.yolinux.com/TUTORIALS/LinuxTutorialApacheAddingLoginSiteProtection.html
http://engr.oregonstate.edu/computing/web/43
http://httpd.apache.org/docs/2.0/howto/auth.html