Linux Info

Copy File through Compressed Tar

2012-02-09T12:06:00.000-06:00

Copy and Compress File through an ssh connection
tar jcvf - myfile.txt | ssh myhost.com "cat > myfile.tar.bz2"

Django Session Introspection

2012-02-09T12:04:00.000-06:00

Get Data Stored in a Session
from django.contrib.sessions import models
s = models.Session.objects.get(session_key='05adfa345a54525cad912')
s.get_decoded()

2011-11-19T13:11:00.001-06:00

ImageMatik

Convert all TIFs in a directory to JPEGs
mogrify -format jpg -quality 50 *.tiff

http://linuxuser32.wordpress.com/2007/06/16/batch-image-convert-scale-thumbnail-jpegs-pdf/

2011-11-10T17:31:00.001-06:00

Python: lxml insert element before and after given element

Based on: http://stackoverflow.com/questions/7474972/python-lxml-append-element-after-another-element

from lxml import etree

xml = etree.parse('old_xml.xml')

count = 1

for pp in xml.xpath('/response/items/item'):

prod_id = etree.Element('new_element') # this element will be inserted

prod_id.text = "%s" % count

nav = pp.find('name') # we will insert before/after this element

div = nav.getparent()

pp.insert(pp.index(nav), prod_id) # insert before 'name'

pp.insert(pp.index(nav) + 1, prod_id) # insert after 'name'

count += 1

out = open('new_xml.xml','w')

out.write(etree.tostring(xml, pretty_print=True))

out.close()

2011-11-10T17:19:00.001-06:00

Python: Detect Charset and convert to Unicode

Converting data to Unicode via chardet
http://stackoverflow.com/questions/2686709/encoding-in-python-with-lxml-complex-solution

import chardet
from lxml import html

f = open('my.xml')

content = f.read()

f.close()

encoding = chardet.detect(content)['encoding']
if encoding != 'utf-8':
content = content.decode(encoding, 'replace').encode('utf-8')

Converting Data to Unicode via UnicodeDammit

http://lxml.de/elementsoup.html

from BeautifulSoup import UnicodeDammit

converted = UnicodeDammit(content)

if not converted.unicode:

raise UnicodeDecodeError(

"Failed to detect encoding, tried [%s]",

', '.join(converted.triedEncodings))

# print converted.originalEncoding

return converted.unicode

Credit goes to Ian Bicking and others on the lxml team

2011-11-08T12:37:00.001-06:00

Using lxml xpath to get elements with a default namespace

xml_string="""

<b>
<r>
<Value>string</Value>

</r>

<r>

<Value>string</Value>

</r>

</b>

<e>

<string>string1</string>

<string>string2</string>

</e>

</a>

"""

from lxml import etree

xml = etree.fromstring(xml_string)

xml.xpath('/w3:a/w3:b/w3:r/w3:Value/text()', namespaces={'w3':'http://www.w3.org/1999/xhtml'})

SQLAlchemy Notes

2011-09-22T22:00:00.001-05:00

Summarized From
http://www.sqlalchemy.org/docs/core/tutorial.html

This covers the basics of the Non-orm usage of SQLAlchemy

# Define connection engine
from sqlalchemy import create_engine
engine = create_engine('sqlite:///:memory:', echo=True)

# Create some database tables
from sqlalchemy import Table, Column, Integer, String, MetaData, ForeignKey
metadata = MetaData()
users = Table('users', metadata,
Column('id', Integer, primary_key=True),
Column('name', String),
Column('fullname', String),
)

addresses = Table('addresses', metadata,
Column('id', Integer, primary_key=True),
Column('user_id', None, ForeignKey('users.id')),
Column('email_address', String, nullable=False)
)

metadata.create_all(engine)

# Insert into the tables
ins = users.insert()
ins.values(name='jack', fullname='Jack the Pumpkin king')

conn = engine.connect()
result = conn.execute(ins)
result.inserted_primary_key

# Multiple insert into the tables
conn.execute(addresses.insert(), [
{'user_id': 1, 'email_address' : 'jack@yahoo.com'},
{'user_id': 1, 'email_address' : 'jack@msn.com'},
{'user_id': 2, 'email_address' : 'www@www.org'},
{'user_id': 2, 'email_address' : 'wendy@aol.com'},
])

result = engine.execute(users.insert(), name='fred', fullname="Fred Flintstone")

# Select from the tables
from sqlalchemy.sql import select

# Select all fields in user table
s = select([users])
result = conn.execute(s)
list(result)

# Select name and fullname columns from user table
s = select([users.c.name, users.c.fullname])

# Join two user and address tables
# Cartesian product
list(conn.execute(select([users, addresses])))
# Join on id columns
list(conn.execute(select([users, addresses], users.c.id==addresses.c.user_id)))

# create conditions for the where clause
from sqlalchemy.sql import and_, or_, not_
print and_(users.c.name.like('j%'), users.c.id==addresses.c.user_id, )

s = select([(users.c.fullname + "--" + addresses.c.email_address).label('title')])
print conn.execute(s).fetchall()

print users.join(addresses)
print users.join(addresses, addresses.c.email_address.like(users.c.name + '%'))

s = select([users.c.fullname], from_obj=[
users.join(addresses, addresses.c.email_address.like(users.c.name + '%'))
])
print conn.execute(s).fetchall()

# Delete records based on a condition
conn.execute(users.delete().where(users.c.name > 'm'))

Django: Quickly testing an authenticated ajax

2011-09-01T21:53:00.000-05:00

Make a authenticated post with the django test client and mark simulate an ajax post:
from django.test import Client

c = Client()

c.login(username='jjasinski', password='xxxx')

print c.post('/user/posts/1/',{},HTTP_X_REQUESTED_WITH='XMLHttpRequest')

Self-signed SSL Certificate

2011-08-10T17:59:00.001-05:00

http://articles.slicehost.com/2007/12/19/ubuntu-gutsy-self-signed-ssl-certificates-and-nginx

Using Rsync to copy file Structure

2011-07-01T00:29:00.000-05:00

Simple rsync copy on local filesystem:
rsync -r -t -v source/ dest

rsync over ssh:
rsync -r -t -v username@source.com:/path/to/src/ /path/to/dest

rsync copy shortcut to archive:
rsync -av username@source.com:/path/to/src/ /path/to/dest

Common Flags:
-a = archive copy. Equivalent to flags -rlptgoD
-r = recursive copy
-t = save timestamp
-v = verbose copy
-vv = more verbose
-u = don't copy over existing files newer than source
-l = preserve symlinks
-p = preserve premissions
-g = preserve group
-o = preserve owner
-D = preserve devices (superuser only)

http://www.cyberciti.biz/tips/linux-use-rsync-transfer-mirror-files-directories.html
http://en.wikipedia.org/wiki/Rsync

Git INFO Common Commands

2011-06-02T23:52:00.012-05:00

Configure Git client options:
# show config options
git config --list # show all
git config --global --list # show only global
git config --local --list # show only local

# --global option impacts options at a global level
# --local option impacts options at a local level (.git/config)
# set username/email associated with client
git config --global user.email "joe@example.com"
git config --global user.name "Joe Jaz"

# add color syntax highlighting
git config --global color.status auto
git config --global color.branch auto

# set default pager and editor
git config --global core.editor vim # or set the GIT_EDITOR environment variable
git config --global core.pager less # or set the GIT_PAGER environment variable

# display an option value
git config group.value
git config user.name

Start a new repository and import:
# In the root folder to add to version control:
git init
git init --bare # creates a new repository (with no working copy)
git add . # add everything in directory to repo
git commit -m 'initial commit'

Making commits:
# commit specific files
git commit file1 file2 -m 'comment'

# commit all 'added' files
git commit -a -m 'comment'

# change a commit message
git commit --amend -m 'New Message'

Create new repository by cloning existing one:
git clone --bare . path_or_url_to_git_dir.git

Setting the remote repository (to push and pull from):
git remote add short_name path_or_url_to_git_dir.git
git remote add origin git://github.com/imagescape/iscape-authlog.git

Show remote repositories available:
git remote
git remote -v # show the url
git remote show origin # show detail information about origin

Remove a remote repository:
git remove repo_name

Rename a remote repository:
git rename old_repo_name new_repo_name

Push and Pull from remote repository:
git push origin # push recent commits to remote repository called "origin"
git pull origin # pull updates from origin

Revert changes in working copy:
git clean -n
git clean --dry-run

Show file differences:
git diff # show differences between working copy and any files staged for commit
git diff --staged # compares staged changes with last commit
git diff --cached # same as above for git older than 1.6.1

Show File/s History:
git log filename # Show single file versions

# Show number of lines changed in file/s
git log -stat filename # single file
git log -stat # multiple files

# show log summaries on one line
git log --pretty=oneline # Pre-defined format
git log --pretty=format:"%h - %cD (%an) %s" # custom format (see below)
git log --pretty=format:"%h %s" --graph

Pre-Defined Pretty formats:
oneline
short
medium
full
fuller
email
raw
format:string # where string is

Format Strings
%H: commit hash
%h: abbreviated commit hash
%T: tree hash
%t: abbreviated tree hash
%P: parent hashes
%p: abbreviated parent hashes
%an: author name
%aN: author name (respecting .mailmap, see git-shortlog(1) or git-blame(1))
%ae: author email
%aE: author email (respecting .mailmap, see git-shortlog(1) or git-blame(1))
%ad: author date (format respects --date= option)
%aD: author date, RFC2822 style
%ar: author date, relative
%at: author date, UNIX timestamp
%ai: author date, ISO 8601 format
%cn: committer name
%cN: committer name (respecting .mailmap, see git-shortlog(1) or git-blame(1))
%ce: committer email
%cE: committer email (respecting .mailmap, see git-shortlog(1) or git-blame(1))
%cd: committer date
%cD: committer date, RFC2822 style
%cr: committer date, relative
%ct: committer date, UNIX timestamp
%ci: committer date, ISO 8601 format
%d: ref names, like the --decorate option of git-log(1)
%e: encoding
%s: subject
%f: sanitized subject line, suitable for a filename
%b: body
%B: raw body (unwrapped subject and body)
%N: commit notes
%gD: reflog selector, e.g., refs/stash@{1}
%gd: shortened reflog selector, e.g., stash@{1}
%gs: reflog subject
%Cred: switch color to red
%Cgreen: switch color to green
%Cblue: switch color to blue
%Creset: reset color
%C(…): color specification, as described in color.branch.* config option
%m: left, right or boundary mark
%n: newline
%%: a raw %
%x00: print a byte from a hex code
%w([[,[,]]]): switch line wrapping, like the -w option of git-shortlog(1).

Show git branches available:
git branch

Show all branches including remote branches:
git branch -a

Create new branch:
git branch new_branch

Change branches:
git checkout new_branch

Delete branch:
git branch -D new_branch

Merge branch into current copy:
git merge branch_to_merge_in

Resolve a Merge:
1) Edit file to resolve
2) git add conflict_file
3) git commit

Undo a Merge: # restores state to pre-merge
git reset --heard HEAD

References:
http://jonas.nitro.dk/git/quick-reference.html
http://www.vogella.de/articles/Git/article.html#git_stagingindex
http://book.git-scm.com/3_basic_branching_and_merging.html

Python Process Info

2011-05-15T16:05:00.001-05:00

Pipe the output of one process into the input of another via python:
import subprocess
p1 = subprocess.Popen(['echo','joe'], shell=False,
stdout=subprocess.PIPE)
p2 = subprocess.Popen(['cat','-n'], stdin=p1.stdout,
stdout=subprocess.PIPE)
p2.stdout.read()

Common Open ID Urls

2011-05-15T16:01:00.001-05:00

Taken From http://stackoverflow.com/questions/1116743/where-can-i-find-a-list-of-openid-provider-urls

Google https://www.google.com/accounts/o8/id
Yahoo https://me.yahoo.com
Flickr http://www.flickr.com/username
AOL http://openid.aol.com/username
Blogspot https://www.blogspot.com/
LiveJournal http://username.livejournal.com/
Wordpress https://username.wordpress.com/
VerisignLabs https://pip.verisignlabs.com/
MyOpenID https://www.myopenid.com/
MyVidoop https://myvidoop.com/
ClaimID https://claimid.com/username
Technorati https://technorati.com/people/technorati/username/

SVN List Codes

2011-05-03T00:17:00.000-05:00

Codes Displayed when doing an "svn stat"
U = Working file was updated
G = Changes on the repository were automatically merged into the working copy
M = Working Copy is modified
C = This file conflicts with the version in the repo
? = This file is not under version control
! = This file is under version control but is missing or incomplete
A = This file will be added to version control after commit
A+ = This file will be moved after commit
D = This file will be deleted after commit
I = This item is being ignored (due to the svn:ignore porperty)
L = This file is locked
X = Item is unversioned, but used by an externals definition
~ = Messed up item (file when it should be dir). resolve by moving and doing "svn up"

http://knaddison.com/technology/svn-status-code-cheat-sheet

Whoosh Python Search Notes

2011-04-26T18:59:00.004-05:00

Baisc Whoosh Search Walkthrough

# Import required
import os
from whoosh.index import create_in
from whoosh.fields import *

# Create a "Schema" object to store text to search
# You can create custom search fields such as "title", "domain", "path"
# and "content".
schema = Schema(title=TEXT(stored=True), domain=ID(stored=True),
                              path=ID(stored=True), content=TEXT)

# Check to see if a search index directory exists; if not, create one
if not os.path.exists("search_test"):
   os.mkdir("search_test")

# Create a search index in the search index directory
ix = create_in("search_test", schema, indexname="testindex")
writer = ix.writer()

# Index some documents
writer.add_document(title=u"Document 1", domain=u"www.mysearch.com",
   path=u"/mywebsite", content=u'Article')
writer.add_document(title=u"Document 2", domain=u"www.mysearch.com",
   path=u"/mywebsite/2/article/1", content=u'Document a')
writer.commit()

# Running search queries
from whoosh.qparser import QueryParser
import whoosh.index as index
# Check to see if the given search index exists
if index.exists_in("search_test", indexname="testindex"):
   # open the search index
   ix = index.open_dir("search_test", indexname="testindex")
   # generate a search query based on some string.
   # Here is creates a query looking for "Document" in the "content"
   # search index field.
   query = QueryParser("content", ix.schema).parse(u'*Document*')
   with ix.searcher() as searcher:
   # do the actual search
   results = searcher.search(query)
   results

# Running search queries
from whoosh.qparser import QueryParser
import whoosh.index as index
if index.exists_in("search_test", indexname="testindex"):
   ix = index.open_dir("search_test", indexname="testindex"))
   query = QueryParser("domain", ix.schema).parse(u'www.mydomain.com')
   with ix.searcher() as searcher:
   results = searcher.search(query)
   results

# Running search queries
from whoosh.qparser import QueryParser, MultifieldParser
import whoosh.index as index
from whoosh.query import *
if index.exists_in("search_test", indexname="testindex"):
   ix = index.open_dir("search_test")
   query = MultifieldParser(["domain","title","content"], schema=ix.schema).parse(u"Document OR Article")
   with ix.searcher() as searcher:
   results = searcher.search(query)
   results

Different way to alter a Postgres table

2011-04-26T18:52:00.000-05:00

Alter Postgres Tables via metadata
(Thanks Louis R.)

select atttypmod from pg_attribute where attrelid = 'TABLE1'::regclass and attname = 'title';
# then
update pg_attribute set atttypmod = 200+4 where attrelid = 'TABLE1'::regclass and attname = 'title';

http://sniptools.com/databases/resize-a-column-in-a-postgresql-table-without-changing-data

Quick Setup Instructions for SVN Compile and Local Install

2011-04-21T14:58:00.000-05:00

Steps to make a custom build of SVN and place it in non-system directory.

Make a place to store the new copy:
mkdir -p /home/myuser/customsvn/src/
cd /home/myuser/customsvn/src/

Download from Apache Subversion; place in local install src directory:
ls -la /home/myuser/customsvn/src/
subversion-1.6.xx.tar.bz2
subversion-deps-1.6.xx.tar.bz2

Untar subversion then subversion-deps:
tar -jxvf subversion-1.6.xx.tar.bz2
tar -jxvf subversion-deps-1.6.xx.tar.bz2

Configure Command (Configure without Apache stuff):
./configure --prefix=/home/myuser/customsvn/bin/svn/ --with-apxs=no --disable-mod-activation --with-serf=no --with-berkeley-db=no | tee configure.jjj.log

SVN Merging For Version 1.4-

2011-04-19T12:09:00.006-05:00

Merging for svn 1.4- is a bit different than for newer versions.
This is what I have found that works for merging a trunk to a branch:
~~svn merge \~~
  ~~svn+ssh://svn.example.com/repo/myproject/branches/newbranch@HEAD \~~
  ~~svn+ssh://svn.example.com/repo/myproject/trunk@HEAD \~~
  ~~/sites/myproject/src/myproject-workingcopy~~

*NOTE: seems weird that the trunk is the argument after the branch, but my trunk changes weren't being merged in when I did it the other way.

It seems that with svn version pre 1.4, you must specify the version where your branch was last copied or merged from. If your branch was created at version 234, then the following command would merge it.
svn merge --dry-run \
  svn+ssh://svn.example.com/export/myproject/branches/newbranchh@234 \
  svn+ssh://svn.example.com/export/myproject/trunk/@HEAD .

http://www.svnforum.org/threads/38394-1.4-merge

Django Content Type Quick Reference

2011-04-15T12:03:00.002-05:00

from django.contrib.contenttypes.models import ContentType

# Get the content type for a given model
ct = ContentType.objects.get(app_label="myapp", model="mymodel")
# or
ct = ContentType.objects.get_for_model(MyModel)

# Get the class associated with the content type
model_cls = ct.model_class()

# Filter the queryset as normal
queryset = model_cls.objects.filter()

# Get a specific object instance given a content type
ct.get_object_for_this_type(username='Guido')

http://docs.djangoproject.com/en/dev/ref/contrib/contenttypes/

Cool Python Modules

2011-04-13T16:33:00.007-05:00

String Similarity

google-diff-match-patch - The Diff Match and Patch libraries offer robust algorithms to perform the operations required for synchronizing plain text.

http://code.google.com/p/google-diff-match-patch/wiki/API

pylevenshtein - Levenshtein (edit) distance, and edit operations, string similarity...

http://code.google.com/p/pylevenshtein/

difflib - classes and functions for comparing sequences

http://docs.python.org/library/difflib.html

Database Tools

uuid - Python built-in module for creating unique ids
http://docs.python.org/library/uuid.html

django-command-extensions: UUIDField - uuid field distributed with django-comamnd-extensions.
http://code.google.com/p/django-command-extensions/

django-uuidfield - stand-alone Django UUIDField implementation.
https://github.com/dcramer/django-uuidfield

System Tools

Imp - get python internals
http://docs.python.org/library/imp.html

Django Tools

AutoSlugify - automate slug field
http://packages.python.org/django-autoslug/

Postgresql Sequences

2011-04-13T14:53:00.001-05:00

Create a new sequence:
CREATE SEQUENCE mytable_myid_seq;

Add the sequence as the incrementor for a table column:
ALTER TABLE mytable
ALTER COLUMN myid
SET DEFAULT NEXTVAL('mytable_myid_seq');

Update existing rows in table with the sequence:
UPDATE mytable
SET myid = NEXTVAL('mytable_myid_seq');

Display the sequence information:
Select * from mytable_myid_seq;

Update a sequence to be one greater than a table's id column:
select setval('mytable_myid_seq', (select max(id) + 1 from mytable));
http://railspikes.com/2009/3/6/duplicate-key-violates-unique-constraint (thanks Louis)
http://pointbeing.net/weblog/2008/03/mysql-versus-postgresql-adding-an-auto-increment-column-to-a-table.html

Django News/Code At www.djangocurrent.com

2011-04-01T17:07:00.000-05:00

I've created a more formal blog called www.djangocurrent.com, focused around Django. I'll continue to post code snippets and information on this site, but will additionally post more formal write-ups at djangocurrent.com. Check it out!

Pretty Print XML with Python

2011-03-16T10:46:00.000-05:00

Pretty Print XML

import xml.dom.minidom

# Read XML from File

xml = xml.dom.minidom.parse(xml_filename)

# Read XML from String

xml = xml.dom.minidom.parseString(xml_string)

xml.toprettyxml()

http://stackoverflow.com/questions/749796/pretty-printing-xml-in-python

Django Chain QuerySet

2011-02-25T14:32:00.000-06:00

Based off of:
http://zmsmith.com/2010/04/using-custom-django-querysets/

Find Open Ports

2011-02-25T01:23:00.002-06:00

Find open ports on local Linux machine:
sudo netstat -lnptu

http://www.techrepublic.com/blog/security/list-open-ports-and-listening-services/443

Django Testing Views

2011-01-30T14:48:00.005-06:00

Setting up the example:

--- views.py -------------------------------------
# define a view for testing
def index(request, template_name=''):
    context = {'things':MyThing.objects.all(),}
    return render_to_response(
           template_name,
           context, context_instance=RequestContext(request))

--- urls.py ---------------------------------------
# define a url for testing
urlpatterns = patterns('',
...
    url('^$', 'core.views.index',
        {'template_name': 'someindex.html'},
         name="index"),
)

Testing from a Dango shell:
import django
from django.test.client import Client
django.test.utils.setup_test_environment() # needed to see response context
client = Client()
response = client.get('/')
response.context[-1]['things']

# we now have access to the response context and can use this

# test code in a unittest tests.py file. (not yet shown)

Testing in a test.py file:
class SimpleTest(TestCase):

   def test_view_list(self):
   resp = self.client.get(reverse('some_named_url'))
   self.assertEqual(resp.status_code, 200)
   self.assertTrue('somevar' in resp.context[-1])

http://ericholscher.com/blog/2008/nov/11/practical-examples-test-django/

Favorite Plugins

2011-01-06T23:02:00.006-06:00

Firefox Plugins for Web Development
- ColorZilla # color picker
- Firebug # CSS / HTML / JS debugger
- Firecookie # cookie info for firebug
- JSONView # Pretty JSON viewer
- RESTClient # GET and POST to restful interfaces
- Selenium IDE # Create tests you can replay in the browser
- Tamper Data # Allow one to tamper with headers and POST vars the client sends
- View Source Chart # Pretty HTML viewer
- Web Developer # Web developer tools
- GMarks # google bookmars sync
- LIVE HTTP headers # view http headers
- SQLite Manager # manage SQLite db files
- YSlow # Yahoo performance tool
- SearchStatus # find website rankings

Thunderbird Plugins for Google Sync
- Lightening # Calendar plugin
- google_contacts # sync google contacts
- CalDev sync google calendar # http://www.google.com/support/calendar/bin/answer.py?hl=en&answer=99358

Eclipse Plugins
- Pydev # python web development
- Subclipse # subversion for eclipse
- Egit # git plugin for eclipse
- m2eclipse # maven plugin for Java package management

Python Date Operations

2011-01-05T10:45:00.012-06:00

Get current time:
>>> import datetime
>>> datetime.date.today()
datetime.date(2011, 1, 12)
>>> datetime.datetime.now()
datetime.datetime(2011, 1, 12, 20, 36, 21, 430985)

Adding time and timedelta:
>>> datetime.timedelta(hours=7) + datetime.datetime(2011,1,1)
datetime.datetime(2011, 1, 1, 7, 0)
>>> datetime.timedelta(days=50) + datetime.date(2011,1,1)
datetime.date(2011, 2, 20)

Subtracting dates:
>>> datetime.date(2011,1,2) - datetime.date(2011,1,1)
datetime.timedelta(1)

Convert date and time into datetime:
>>> import datetime
>>> d = datetime.date(2010,01,05)
>>> t = datetime.time(10,25)

>>>datetime.datetime.combine(d,t)

datetime.datetime(2010, 1, 5, 10, 25)

Convert String to datetime:

>>> datetime.datetime.strptime("2011-01-05","%Y-%m-%d")

datetime.datetime(2011, 1, 5, 0, 0)

Convert datetime to individual components:

>>> dt = datetime.datetime(2011,01,05,12,00)

>>> (dt.year, dt.month, dt.day, dt.hour, dt.minute)

(2011, 1, 5, 12, 0)

Convert datetime to date:

>>> dt = datetime.datetime(2011,01,05,12,00)

>>> dt.date()

datetime.date(2011, 1, 5)

Add one day to a date:
>>> dt = datetime.datetime(2011,01,05,12,00)
>>> delta = datetime.timedelta(days=1)

>>> dt + delta

datetime.datetime(2011, 1, 6, 12, 0)

Number of days in a month:
>>> import calendar
>>> calendar.monthrange(2011,1)
(5, 31)

more to come...

http://stackoverflow.com/questions/1937622/convert-date-to-datetime-in-python

http://www.linuxtopia.org/online_books/programming_books/python_programming/python_ch32s03.html

Pretty Print JSON with python

2010-12-06T18:33:00.003-06:00

Pretty Print JSON
echo '{"foo": "lorem", "bar": "ipsum"}' | python -mjson.tool

Dump Django data as pretty JSON
./manage.py dumpdata mymodule | python -mjson.tool

Prety Print JSON from Python Shell
import json
json_data = [1,2,3,{'a','b',c'}]
json.dumps(json_data, sort_keys=True, indent=4)

http://stackoverflow.com/questions/352098/how-to-pretty-print-json-script
http://docs.python.org/library/json.html

Django: Random Image/File Field name

2010-11-27T02:04:00.006-06:00

By default, Django image/file fields place uploaded files in the MEDIA_ROOT directory, under a subdirectory given by the "upload_to" model field parameter. However, the name of uploaded file is the same name as the original filename uploaded by the client. The filename can be changed by creating a custom storage manager.

---- myapp/storage.py
import os, itertools, random, string
from django.core.files.storage import FileSystemStorage

def rand_key(size):
   return "".join([random.choice(string.letters + string.digits)
   for i in range(size)])

class RandomFileSystemStorage(FileSystemStorage):

   def get_valid_name(self, name):
   file_root, file_ext = os.path.splitext(name)
   return "%s%s" % (rand_key(32), file_ext)

   def get_available_name(self, name):
   dir_name, file_name = os.path.split(name)
   file_root, file_ext = os.path.splitext(file_name)
   count = itertools.count(1)
   while self.exists(name):
   name = os.path.join(dir_name, "%s_%s%s" % (
   rand_key(32),  count.next(), file_ext))
   return name

---- myapp/models.py

from django.db import models

from myapp.storage import RandomFileSystemStorage

class Photo(models.Model):

image = models.ImageField(upload_to='media',
storage=RandomFileSystemStorage())

Setup Django RabbitMQ and Celery

2010-11-16T22:43:00.013-06:00

Rabbit Message Queue is a separate server that remotely executes tasks given to it.
Celery is a Python client program that sends tasks to the RabbitMQ.
Django-celery is a Django wrapper for Celery that makes it
work with Django more nicely. Tasks are executed in Django
view code or wherever. The tasks can be defined in the Django app,
are sent to a Celery client daemon executed by ./manage.py and
Celery serializes the task and sends it to RabbitMQ for processing.
RabbitMQ notifies Celery when it is done with each task.

1) Install deps
UBUNTU 10.04 NOTE
sudo aptitude install python-virtualenv #(recommended but not required)
sudo aptitude install rabbitmq-server

2) Create rabbitmq user and vhost
sudo rabbitmqctl add_user my_rabbit_user mypasswd
sudo rabbitmqctl add_vhost /my_vhost
sudo rabbitmqctl set_permissions -p /my_vhost my_rabbit_user "" ".*" ".*"

3) Setup python environment
virtualenv --no-site-packages testrabbitmq #(create a virtualenv, not required)
cd testrabbitmq
. ./bin/activate # (activate virtualenv, not required)
pip install django
pip install celery
pip install django-celery

4) Setup Django project
django-admin.py startproject testc # create a test project
cd testc
python ./manage.py startapp cel # create a test app

5) Create a test model
Edit cel/models.py:
from django.db import models
class MyModel(models.Model):
   field1 = models.CharField(max_length=12)
   field2 = models.CharField(max_length=12)

6) Create some test tasks:
Edit cel/tasks.py
from celery.decorators import task
@task
def add(x, y)
   return x + y

from cel import models
@task
def addmodel(x, y):
   record = models.MyModel.objects.create(field1=x, field2=y)
   record.save()
   return record

from cel import models

@task(ignore_result=True) # Celery will ignore results sent back to it

def addmodel2(x, y):

record = models.MyModel.objects.create(field1=x, field2=y)

record.save()

return record

7) configure django settings
Edit settings.py

# set django-celery autoloader

import djcelery
djcelery.setup_loader()

...

# Set database settings

...

# set information to connect to rabbitmq (broker)
BROKER_HOST = "127.0.0.1"
BROKER_PORT = 5672
BROKER_VHOST = "/my_vhost"
BROKER_USER = "my_rabbit_user"
BROKER_PASSWORD = "1234"

# add to installed apps
INSTALLED_APPS = (
   ...
   'djcelery', # django-celery
   'cel',
)

8) Syncdb
python ./manage syncdb

9) Restart the rabbitmq server (optional)
UBUNTU 10.04 NOTE - Ubuntu starts the rabbitmq
server by default and installs an init script
(/etc/init.d/rabbitmq-server start|stop). For testing,
let's stop the server and restart manually run it in the
foreground to see more output.
sudo /etc/init.d/rabbitmq-server stop
sudo rabbitmq-server start
# will display lots of output and say broker is running.
# The terminal will wait.

10) Start the celery client process
In another terminal (if using virtualenv, be sure to activate it),
in the testrabbitmq Django project, execute the following:

python ./manage.py celeryd -l info

# this will hang the terminal and set the Celery (the message client) waiting.

11) Send a message
In another terminal (if using virtualenv, be sure to activate it),
in the testrabbitmq Django project, execute the following:
python ./manage.py shell
>>> from cel import tasks
>>> result = tasks.add.delay(1, 2)
>>> result.ready() # waits until task is done
True
>>> result.state
u'SUCCESS'
>>> result.successful()
True
>>> result = tasks.add.delay(1 + 2) # will cause an error
>>> result.successful()
False
>>> result = tasks.addmodel.delay('a','b')
>>> result.successful()
True
>>> for i in range(0,1000): # stresstest
.... result = tasks.addmodel.delay('a','b')

# alternate syntax (more flexible - can pass args)
>>> result = tasks.addmodel.apply_asyc(args=['c','d'])
>>> result.successful()

# execute at a given time

>>> from datetime import datetime, timedelta

>>> result = tasks.addmodel.apply_asyc(args=['c','d'],
eta=datetime.now() + timedelta(minutes=1))

>>> result.successful()

# execute after a given number of seconds

>>> result = tasks.addmodel.apply_asyc(args=['c','d'],

countdown=3)

>>> result.successful()

# alternate syntax (can queue functions you don't own)
>>> from celery.execute import send_task
>>> tasks.addmodel.name
'cel.tasks.addmodel'
>>> result = send_task("cel.tasks.addmodel", args=['e', 'f'])
>>> result.get()

>>> from djcelery.models import TaskMeta
>>> TaskMeta.objects.count() # this table contains all results meta info
   # for tasks not defined with ignore_result=True

APPENDIX - rabbitmq commands

sudo rabbitmqctl stop # stop the rabbitmq
sudo rabbitmqctl list_users # list the available users
sudo rabbitmqctl list_vhosts # list the available vhosts
sudo rabbitmqctl list_queues
Listing queues ...
celery 544 # by default, lists the queues in the server
   # and how many messages in them
sudo rabbitmqctl list_queues [options]
   common list_queues options
   name = name of queue
   durable = queue survives server restarts
   pid = erlang process id
   messages_ready = ready to be delivered to clients
   messages = queue depth
   memory = bytes of mem consumed by erlang process

# NOTES, for any changes to the code, be sure to restart the celery client (./manage.py celeryd)

http://www.turnkeylinux.org/blog/django-celery-rabbitmq
http://ask.github.com/celery/userguide/executing.html

Tell Browser not to cache forms on back

2010-11-05T11:39:00.005-05:00

This works in IE and Firefox but not Chrome or Opera

<meta http-equiv="cache-control" content="no-cache"> 
<meta http-equiv="cache-control" content="no-store">
<meta http-equiv="expires" content="0"> 
<meta http-equiv="pragma" content="no-cache">

Django List Installed Tables and Models

2010-10-19T20:21:00.012-05:00

Get Database connection
# Django 1.2
from django.db import connections

connection = connections['default']

# Django 1.1.1<=

from django.db import connection

# Get Database tables

tables = connection.introspection.table_names()

# Get Database models

modelclasses = connection.introspection.installed_models(tables)

# Get Django model name and app name
for model in list(modelclasses):
print model._meta.app_label, model._meta.module_name

# A better way to get Django models
from django.db.models.loading import get_models, get_app
get_app('app_label') # returns an app class given a label
get_models() # all models
get_models(application) # models belonging to app class

# Get a model from a string
from django.db import models
models.get_model(*"myapp.mymodel".split('.')) # returns model class

# Get all Python modules

import sys
sys.modules

http://www.slideshare.net/ubernostrum/django-in-depth?src=related_normal&rel=3017822

Use alternate Mirror for Python pypi installs (when pypi goes down)

2010-10-06T20:46:00.000-05:00

Specify alternate mirror from the command line:
pip install -i http://d.pypi.python.org/simple $PACKAGE

Specify alternate mirror in ~/.pip/pip.conf:
[global]
index-url = http://d.pypi.python.org/simple

http://jacobian.org/writing/when-pypi-goes-down/

Query Django Admin Log

2010-10-04T14:15:00.000-05:00

Show Django Admin Log entries

select u.username, u.first_name, u.last_name, l.action_time, c.name, c.app_label,
   l.object_id as "Object Modified", l.object_repr, l.action_flag, l.change_message
from (auth_user u inner join django_admin_log l on u.id = l.user_id )
   inner join django_content_type c on c.id = l.content_type_id
where l.object_id = '9' /* filter by object id if desired */
   and l.action_time > to_date('2010-10-04 01:00','YYYY-MM-DD HH:MI') /* filter by date range */
order by l.action_time;

Change Default Linux Shell

2010-08-30T09:21:00.000-05:00

Needs sudo privileges: to change the default shell:

chsh

http://forums.devshed.com/unix-help-35/how-to-change-default-shell-52749.html

Host a Test email server for Django header debugging

2010-08-27T11:33:00.000-05:00

Set the following in settings.py
EMAIL_HOST
EMAIL_PORT

Run the following to host the test webserver

python -m smtpd -n -c DebuggingServer localhost:1025

http://docs.djangoproject.com/en/dev/topics/email/

Django Management Commands

2010-08-26T21:17:00.000-05:00

Django syncdb without prompts
python manage.py syncdb --noinput

http://docs.djangoproject.com/en/dev/ref/django-admin/

Bash customize autocomplete

2010-08-21T11:33:00.001-05:00

http://www.linuxjournal.com/content/more-using-bash-complete-command

Dynamic Fieldsets in Django Admin

2010-08-21T11:08:00.002-05:00

Problem: the fieldsets option for the admin interface (classes defined in admin.py) requires that you explicitly list every single field in the model in the fieldset list that you pass to it. This gets tedious if you add a field to a model, as you also have to add it to this admin.py. Not very DRY.

Solution:
In models.py
from django.db import models
class My(models.Model):
    a = models.CharField(max_length=10)
    b = models.CharField(max_length=10)
    c = models.CharField(max_length=10)
    d = models.CharField(max_length=10)
    e = models.CharField(max_length=10)
    f = models.CharField(max_length=10)
    g = models.CharField(max_length=10)
    h = models.CharField(max_length=10)

In admin.py
from django.contrib import admin
from django.forms.models import fields_for_model
from f import models
class MyAdmin(admin.ModelAdmin):
    def __init__(self, *args, **kwargs):
        super(MyAdmin, self).__init__(*args, **kwargs)
        all_fields = set(fields_for_model(models.My))
        fieldset1_fields = ('e', 'f',)
        fieldset2_fields = ('g', 'h',)
        fieldset_fields = set(fieldset1_fields) | set(fieldset2_fields)
        rest_fields = list(all_fields - fieldset_fields)
        self.fieldsets = (
            (None, {
                'fields': rest_fields
            }),
            ('Fieldset 1', {
                'classes': ('collapse',),
                'fields': fieldset1_fields
            }),
            ('Fieldset 2', {
                'classes': ('collapse',),
                'fields': fieldset2_fields
            }),
        )
    models = models.My
admin.site.register(models.My, MyAdmin)

Thanks for the help lorochka85

Drill through a Login with Python

2010-08-19T21:01:00.005-05:00

import urllib2
import urllib
import BeautifulSoup

# build opener with HTTPCookieProcessor
o = urllib2.build_opener( urllib2.HTTPCookieProcessor() )
urllib2.install_opener( o )

# assuming the site expects 'user' and 'pass' as query params
p = urllib.urlencode( { 'username': 'myusername', 'password': 'mypassword' } )

# perform login with params
f = o.open( 'https://www.mysite.com/login', p )
data = f.read()
f.close()

# second request should automatically pass back any
# cookies received during login... thanks to the HTTPCookieProcessor
f = o.open( 'http://www.mysite.com/home/' )
data = f.read()
f.close()

soup = BeautifulSoup.BeautifulSoup(data)

http://www.nomadjourney.com/2009/03/automatic-site-login-using-python-urllib2/

Python String format

2010-08-16T14:59:00.000-05:00

Python Format dates into strings
from datetime import date
d1 = date(2010,01,23)
d2 = date(2010,01,23)
"%s - %s" % (d1.strftime("%m / %d / %Y"), d1.strftime("%m / %d / %Y"))

Python Unicode Convert

2010-08-12T17:35:00.005-05:00

Convert weird unicode string characters to string:
import unicodedata
unicodedata.normalize('NFKD',unicode_string).encode('ascii','ignore')

Django module for helping convert things to ascii strings:
from django.utils.encoding import smart_str
smart_str('weird string goes here')

Another hack for removing unicode errors:
unicode('my weird string', errors='ignore')

http://docs.python.org/howto/unicode.html#the-unicode-type

matplotlib basics

2010-07-15T18:55:00.007-05:00


Simple Plot
import matplotlib.pyplot as plt
plt.plot([1,2,3,4])
# plot a set of points (1,1) and (3,2)
plt.plot([1,3],[1,2],'ro')
# Or plot as a bunch of connect line segments
plt.plot([2,4,6],[2,4,6])
# set a label for the x and y axis
plt.ylabel('some numbers')
plt.xlabel('joe rocks')
# set the axis [xmin, xmax, ymin, ymax ]
plt.axis([0,6,0,5])
# 'show' the graph
plt.show()


Plot a function
As far as I can tell matplotlib cannot plot a continuous function.  
Instead, create a range of distinct numbers:

import numpy as np
import matplotlib.pyplot as plt
# create a range of numbers from 0 to 5. increment by .1
x = np.arange(0.,5.,.1)
# apply a function to that 
y = np.sin(x)
# plot with extra display params
plt.plot(x,y,linewidth=2.0, label='joeplot', color='blue')
plt.title(r'$\alpha_i > \beta_i$', fontsize=20)
plt.axis([0,5,-1,1])
plt.grid(True)
plt.show()

Make a cool heatmap
import numpy as np
import matplotlib.pyplot as plt
#x = y = np.linspace(-5, 5, 12)
# create a grid of 12 X coordinates and 12 Y coordinates 
# these coordinates will be used to represent specific locations on 
# the grid.  the meshgrid() basically creates a nice uniform coordinate grid
X,Y = np.meshgrid([1,2,3,4],[1,2,3])
# X,Y = np.meshgrid(x, y)
# these ravel functions basically make the 2d arrays created above into lists
x = X.ravel()
y = Y.ravel()
plt.subplot(111)
plt.hexbin(x,y,C=[1,2,3,3,2,2,3,3,2,3,3,4], gridsize=30)
cb = plt.colorbar()
cb.set_label('Heat Value')
plt.axis([x.min(), x.max(), y.min(), y.max()])
plt.grid(True)


http://stackoverflow.com/questions/2369492/generate-a-heatmap-in-matplotlib-using-a-scatter-data-set

Case Study Notes: install pyqrcode in a virtualenv on OSX

2010-07-09T18:13:00.001-05:00

Kind of a buggy install.

mkdir qrcode; cd qrcode
virtualenv --distribute --no-site-packages ve
source ./ve/bin/activate
echo "pil" > requirements.pip
echo "http://svn.apache.org/repos/asf/lucene/pylucene/trunk/jcc" >> requirements.pip

wget http://downloads.sourceforge.net/pyqrcode/pyqrcode-0.2.1.tar.gz?use_mirror=

cd pyqrcode-0.2.1

# EDIT pyqrcode-0.2.1/Makefile
# change the call to jcc from
GENERATE=python -m jcc --jar $(LIBFILE) \
# TO
GENERATE=python -m jcc.__main__ --jar $(LIBFILE) \

make
make egg

# it will create qrcode-0.2.1-py2.6-macosx-10.6-universal.egg
# Edit qrcode-0.2.1-py2.6-macosx-10.6-universal.egg/qrcode/__init__.py
# change the line
_qrcode._setExceptionTypes(JavaError, InvalidArgsError)
# TO
_qrcode._set_exception_types(JavaError, InvalidArgsError)

pip -E ./ve/ install qrcode-0.2.1-py2.6-macosx-10.6-universal.egg

# http://pyqrcode.sourceforge.net/
# http://www.mail-archive.com/pythonmac-sig@python.org/msg09864.html
# http://mail-archives.apache.org/mod_mbox/lucene-pylucene-dev/200904.mbox/%3C49EEECAC.7070606@cheimes.de%3E

Installing MySQL-python (MySQLdb) in a virtualenv on OSX 10.6

2010-07-09T17:42:00.006-05:00

There were two problems with the MySQL-Python package that I had when installing on OS X 10.6
1) First, an error saying "EnvironmentError: mysql_config not found" when setup.py is run (through pip or easy_install)
2) Second, an error saying "ImportError: dynamic module does not define init function(init_mysql) when importing MySQLdb" when 'import MySQLdb" is issued

Causes:
1) This is caused by the build script not being able to find a MySQL program called mysql_config. This program is used to determine metadata about the mysql install. For example, the command "mysql_config --cflags" reports the flags used to build mysql.
2) This is caused by a mismatch between the architectures that MySQL was built with and the architecture that MySQL-python installer is trying to build and install MySQL-python as. To find what arch MySQL was built as, run "mysql_config --cflags" (this was i386 in my case). OSX tries to build MySQL-python as x86_64. Therefore, there is a arch mismatch.

Solution:
1) Locate the mysql_config binary location and add the path to the MySQL-python/site.cfg file as a config directive: "mysql_config = /opt/local/bin/mysql_config5"
2) Use the arch x86_64 (64 bit) version of MySQL compiled through MacPorts, not a i386 (32 bit) version downloaded off the internet and installed as a dmg.

Summary:
Assuming the MacPorts version of MySQL is installed, both issues can be solved by adding the mysql_config = /opt/local/bin/mysql_config5 config directive to the MySQL-python/site.cfg file.

# SCRIPTED SOLUTION: Create a script called setup.sh and add the following lines
virtualenv --distribute --no-site-packages ve
source ./ve/bin/activate
pip install -E ./ve -r requirements.1.pip # other deps
export ARCHFLAGS="-arch x86_64"
pip install -E ./ve MySQL-python
echo "mysql_config = /opt/local/bin/mysql_config5" >> ./ve/build/MySQL-python/site.cfg
pip install -E ./ve MySQL-python

Resources:
http://groups.google.com/group/python-virtualenv/msg/cf4f3117faea476b?pli=1
http://birdhouse.org/blog/2009/02/21/python-mysql-connections-on-mac-os/
http://stackoverflow.com/questions/2111283/how-to-build-64-bit-python-on-os-x-10-6-only-64-bit-no-universal-nonsense

Python Pip Usage

2010-06-26T15:07:00.005-05:00

Basic PIP install
pip install somepackage

Basic PIP uninstall
pip uninstall somepackage

Requirements file
# cat requirements.pip
MyApp
Framework==0.0.1
Library>=0.2

Installing using a requirements file
pip install -r requirements.pip

PIP Freezing requirements
pip freeze # lists all packages and the specific version installed. Useful for migrating

PIP install into virtualenv env
pip install -E ./env
pip install -E ./env -r requirements.pip # using a requirements file

Basic script for creating a virtualenv and installing requirements
virtualenv --distribute --no-site-packages ve
source ./ve/bin/activate
pip install -E ./ve -r requirements.pip
pip install -E ./ve -r requirements-test.pip

http://heisel.org/blog/2009/11/21/django-hudson/
http://pip.openplans.org/#freezing-requirements

Case Study Notes: install djangobb in virtualenv

2010-06-21T00:48:00.003-05:00

Ubuntu 10.04 python 2.6 django 1.1

sudo apt-get build-dep python-psycopg2
sudo aptitude install python-dev

cd Sites
virtualenv --no-site-packages env
. ./env/bin/activate
pip install pil
pip install markdown2 Markdown
pip install django-registration
pip install djapian
pil install xapian
pip install psycopg2
easy_install -i http://downloads.egenix.com/python/index/ucs4/ egenix-mx-base
pip install http://code.djangoproject.com/svn/django/tags/releases/1.1.2/
hg clone http://hg.djangobb.org/djangobb/ djangobb
cd env/lib/python2.6/site-packages/
cp /usr/lib/python2.6/dist-packages/_xapian.so .
cp /usr/lib/python2.6/dist-packages/xapian.py .

# configure settings.py to use postgresql_psycopg2
# comment out the openid stuff

## http://www.saltycrane.com/blog/2009/07/using-psycopg2-virtualenv-ubuntu-jaunty/

Postgresql Basic Commands

2010-06-20T09:10:00.039-05:00

Login to postgresql:
psql -d mydb -U myuser -W
psql -h myhost -d mydb -U myuser -W

Default Admin Login:
sudo -u postgres psql -U postgres

List databases on postgresql server:
psql -l [-U myuser] [-W]

Turn off line pager pagination in psql:
\pset pager

Determine system tables:
select * from pg_tables where tableowner = 'postgres';

List databases from within a pg shell:
\l

List databases from UNIX command prompt:
psql -U postgres -l

Describe a table:
\d tablename

Quit psql:
\q

Switch postgres database within admin login shell:
\connect databasename

Reset a user password as admin:
alter user usertochange with password 'new_passwd';

Show all tables:
\dt

List all Schemas:
\dn

List all users:
\du

Load data into posgresql:
psql -W -U username -H hostname < file.sql

Dump (Backup) Data into file:
pg_dump -W -U username -h hostname database_name > file.sql

Increment a sequence:
SELECT nextval('my_id_seq');

Create new user:
CREATE USER tom WITH PASSWORD 'myPassword';

Change user password:
ALTER USER Postgres WITH PASSWORD 'mypass';

Grant user createdb privilege:
ALTER USER myuser WITH createdb;

Show Database Version:
SELECT version();

Change Database Owner:
alter database database_name owner to new_owner;

Copy a database:
CREATE DATABASE newdb WITH TEMPLATE originaldb;
http://www.commandprompt.com/ppbook/x14316

View Database Connections:
SELECT * FROM pg_stat_activity;

http://www.devdaily.com/blog/post/postgresql/log-in-postgresql-database
http://forums.devshed.com/postgresql-help-21/how-do-you-turn-off-more-scroll-lock-at-psql-174831.htm
http://www.cyberciti.biz/faq/howto-add-postgresql-user-account/
http://archives.postgresql.org/pgsql-general/1998-08/msg00050.php
http://stackoverflow.com/questions/876522/creating-a-copy-of-a-database-in-postgres

Recycle bin script

2010-06-18T16:44:00.006-05:00


crm()
{
# pass this function the name of the directory that
# you want to backup and remove 
if [ -d "$1" ]
then
  cd "${1}/../"
else
  echo "Directory does not exist"
  return 1
fi

PWD=`pwd`
BKPFILES=`find $PWD -type -f`
BKPDIR="$HOME/.Trash"
daten=`date +%Y.%m.%d-%H.%M.%S`

for i in $BKPFILES
do
  echo "$i"
  DIRNAME="$BKPDIR"`echo ${i%/*}`
  FILENAME=`echo ${i##*/}`
  mkdir -p "$DIRNAME"
  cp -pf "$i" "${DIRNAME}/${FILENAME}.${daten}"
  #NEWFILE=`echo "$i" | sed 's|[ \/]|_|g'`
  print "$i"  "$NEWFILE"   " $DIRNAME"
done
rm -rf "$1"
cd - 
}

OSX Color LS

2010-06-17T16:10:00.002-05:00

Color LS for Mac OX
alias ls='ls -G'

sftp notes

2010-06-15T11:18:00.004-05:00

sftp using alternate port:
sftp -oPort=2222 username@host.com

sftp using alternate private key file:

sftp -o IdentityFile=/custom/file/location.pem username@host.com

http://www.unix.com/shell-programming-scripting/43334-sftp-scripting-help-required.html

Basic Python Development Server setup with Ubuntu 10.04

2010-06-15T03:08:00.014-05:00

Quick setup guide

# config vi command line
echo "set -o vi" >> ~/.bashrc
. ~/.bashrc

# install packages
aptitude install vim openssh-server apache2 python-virtualenv python-mysqldb mysql-server libapache2-mod-php5 libapache2-mod-wsgi eclipse subversion nmap ubuntu-restricted-extras g++ git-gui virtualbox-ose php5-cli mdadm fabric python-dev mercurial

# eclipse development plugins
install pydev for eclipse
install subclipse for eclipse

# firefox development plugins
install firebug for firefox @ http://getfirebug.com/
install web developer for firefox @ https://addons.mozilla.org/en-US/firefox/addon/60/

# add google repo
deb http://dl.google.com/linux/deb/ stable non-free
aptitude update

cd /opt/
wget http://www.djangoproject.com/download/1.1.2/tarball/
wget http://www.djangoproject.com/download/1.2.1/tarball/
tar -xvf Django-1.1.2.tar.gz
tar -xvf Django-1.2.1.tar.gz
cd /usr/lib/python2.6/dist-packages
ln -s /opt/Django-1.2.1/django/ django

# mysql workbench download (64 bit version)
aptitude install libzip1 python-pysqlite2 # deps needed
http://dev.mysql.com/get/Downloads/MySQLGUITools/mysql-workbench-oss-5.2.22-1ubu1004-amd64.deb/from/http://mirror.services.wisc.edu/mysql/
dpkg -i mysql-workbench-oss-5.2.22-1ubu1004-amd64.deb

# setup postgresql
aptitude install postgresql pgadmin3 python-psycopg2

sudo su -
passwd postgres
su postgres
psql template1

The last instruction should open the psql shell, where you can run the following:

ALTER USER postgres WITH ENCRYPTED PASSWORD 'mypassword';

# Setup NX server (Google NeatX server)
sudo apt-get install python-software-properties && sudo add-apt-repository ppa:freenx-team
sudo apt-get update
sudo apt-get install neatx-server

## NOTE: if neatx gives a weird error, delete the session dirs on the server:
## /var/lib/neatx/sessions/some_dir/

https://help.ubuntu.com/community/FreeNX
http://programmingzen.com/2007/12/26/installing-django-with-postgresql-on-ubuntu/

Simple Network Test Script

2010-06-14T00:57:00.002-05:00

This is a simple script to log network outages on a local network

root@ubuntu:/mnt/root/usr/local/bin# more lifeline
#!/bin/bash

PING='/bin/ping'
EXTHOST1='www.google.com'
EXTHOST2='sun.iwu.edu'
INTHOST='192.168.1.1'
LOG='/var/log/network_outages'
WAITTIME=120

echo "NETWORK STATUS SCRIPT: Started " `date` >> $LOG
while [ 1=1 ]
do
   dater=`date +%Y.%m.%d-%H.%M.%S`
   $PING -q -c1 $EXTHOST1
   ret=$?
   echo "RET: $ret"
   if [ $ret -ne 0 ]
   then
     echo "EXTERNAL_PING(1): outage detected $dater" >> $LOG
   fi

   dater=`date +%Y.%m.%d-%H.%M.%S`
   $PING -q -c1 $EXTHOST2
   ret=$?
   echo "RET: $ret"
   if [ $ret -ne 0 ]
   then
     echo "EXTERNAL_PING(2): outage detected $dater" >> $LOG
   fi

   $PING -q -c1 $INTHOST
   ret=$?
   echo "RET: $ret"
   if [ $ret -ne 0 ]
   then
     echo "INTERNAL_PING: outage detected $dater" >> $LOG
   fi

   sleep $WAITTIME
done

Simple HEX Editor

2010-05-05T17:03:00.000-05:00

View file in Hex Editor
od -cx filename

Bash String Operatinos

2010-04-20T11:21:00.016-05:00

Remove the last four characters in a string
echo "somefile.txt" | awk 'sub("....$","")

Remove file extension from a string
ls -1 | sed 's/$.*$\..*/\1/'

Remove PREFIX from SOMEPATH
SOMEPATH="/home/myuser/usr/bin/"
PREFIX="/home/myuser/"
echo ${SOMEPATH#$PREFIX}
Returns: usr/bin/

Remove shortest match of PREFIX from SOMEPATH
SOMEPATH="/home/sub/subhome/myuser/usr/bin/"
PREFIX="/*/myuser/"
echo ${SOMEPATH#$PREFIX}
Returns: usr/bin/

Parsing parts of a file:
foo=/tmp/my.dir/filename.tar.gz
To get: /tmp/my.dir (like dirname)
path = ${foo%/*}
To get: filename.tar.gz (like basename)
file = ${foo##*/}
To get: filename
base = ${file%%.*}
To get: tar.gz
ext = ${file#*.}

Removing first 8 characters of a string:
echo $var | cut -c9-

http://unstableme.blogspot.com/2008/03/printremove-first-some-characters-of.html
http://tldp.org/LDP/LGNET/18/bash.html
http://docstore.mik.ua/orelly/unix/upt/ch09_07.htm
http://unstableme.blogspot.com/2007/12/removing-last-two-characters-bash.html
http://www.unix.com/shell-programming-scripting/40360-remove-file-extension.html

Oracle Login Context Information

2010-02-03T10:22:00.003-06:00

Find the Database instance you are logged into
SELECT sys_context('USERENV', 'DB_NAME') FROM dual;
SELECT sys_context('USERENV', 'INSTANCE_NAME') FROM dual;

Find the operating system user hosting the Oracle session
SELECT sys_context('USERENV', 'OS_USER') FROM dual;

Find the Oracle user currently logged into
SELECT sys_context('USERENV', 'SESSION_USER') FROM dual;

http://www.psoug.org/reference/sys_context.html

Python Virtual Env

2010-01-02T15:01:00.003-06:00

Download and install via apt-get, ports, easy_install, etc.

Locate the install directory (usually site-packages) and find the virtualenv.py script.
Alternatively, the package manager may have installed a virtualenv script into your PATH.

I will use ENV to refer to /path/to/python/virtual/env
I will use virtualenv.py to refer to /path/to/script/virtualenv.py

Create a new Virtual Environment
python virtualenv.py ENV

(optional) Activate (set PATH and PYTHONPATH) for new Virtual ENV
source ENV/bin/activate

Location of Virtual Env python binary
ENV/bin/python

Location of Virtual ENV python site-packages dir
ENV/lib/python2.x/site-packages/

Location of Virtual ENV python easy_install script
ENV/bin/easy_install

http://pypi.python.org/pypi/virtualenv

curl Usage

2009-12-11T02:22:00.002-06:00

Send a POST

curl -u joe:1234 -X POST http://localhost:8000/api/1.0/NU1yWdRM5JBjnZpZX/message/ -d "message=test"

-u = username and password

-d = post parameters

-X = request method

netbios name lookups

2009-12-08T12:35:00.002-06:00

Find NetBIOS name from IP
Windows:
netstat -a 192.168.1.22

Linux:
nbtstat 192.168.1.22 # ntbstat separate package

Find IP from NetBIOS name
Windows:
nbtstat -a host.example.com

Linux:
nmblookup host.example.com

http://www.irongeek.com/i.php?page=security/ipinfo

MacPorts Usage

2009-10-28T01:38:00.003-05:00

Update MacPorts itself
sudo port selfupdate
sudo port -d selfupdate # debug

Updates the port tree with new versions definitions
sudo port sync

List available ports
sudo port list

Search ports
port search [keyword]

Lookup package info (desc, maintainer, etc)
port info [package]

Find package dependencies
port deps [package]

Install package
sudo port install [package]
sudo port -v install [package] #verbose

Clean out build files and tarballs
port clean --all [pakcage]

Uninstall a package
sudo port uninstall [package]

Show port contents
port contents [package]

List installed packages
port installed

List outdated ports
port outdated

Upgrade specific packages
port upgrade [package]
port upgrade outdated #updates all outdated packages

http://guide.macports.org/chunked/using.html

DJango Model Inheritance with subclasses

2009-10-18T14:09:00.002-05:00

http://www.djangosnippets.org/snippets/1034/
http://www.djangosnippets.org/snippets/1031/
http://adam.gomaa.us/blog/2009/feb/16/subclassing-django-querysets/
http://docs.djangoproject.com/en/dev/ref/contrib/contenttypes/

Python List Operations (map, for comprehensions)

2009-10-09T11:37:00.006-05:00

EXAMPLE 1
# Create a list
>>> l = [1,2,3,4,5,6]

# iterate through the list using a for comprehension
>>> [i for i in l]
[1, 2, 3, 4, 5, 6]

# square elements in the list using a for comprehension
>>> [i**2 for i in l]
[1, 4, 9, 16, 25, 36]

# square only even elements in the list using a for comprehension
>>> [i**2 for i in l if i % 2 == 0]
[4, 16, 36]

# iterate through the list using map
>>> map(lambda w: w, l)
[1, 2, 3, 4, 5, 6]

# square each element of the list using map
>>> map(lambda w: w**2, l)
[1, 4, 9, 16, 25, 36]

# or use map to call a separately defined function to# iterate through the list
>>> def squa(x):
...... return x**2

>>> map(squa, l)
[1, 4, 9, 16, 25, 36]

# use map function to call a separately defined function that takes 2 args
>>> l2 = map(lambda w: (w,2),l)
>>> l2
[(1, 2), (2, 2), (3, 2), (4, 2), (5, 2), (6, 2)]

>>> def pow(base, expo):
...... return base**expo

>>> map(lambda (x,y): pow(x,y), l2)
[1, 4, 9, 16, 25, 36]

# filter a list
>>> filter(lambda w: w > 2, l)
[3, 4, 5, 6]

EXAMPLE 2
# define a simple class to play with
>>> class A(object):
...... def __init__(self, x):
........ self.x = x

# create simple list of object instances of the class
>>> l = [A(1), A(2), A(3)]

# use a for (list) comprehension to iterate through the list
>>> [i.x for i in l]
[1, 2, 3]

# use a for comprehension to iterate with a condition
>>> [i.x for i in l if i.x > 1]
[2, 3]

# use map to apply a function to every element in the list
>>> map(lambda w: w.x, l)
[1, 2, 3]

>>> map(lambda w: w.x * w.x, l)
[1, 4, 9]

EXAMPLE 1

# Create a list

>>> l = [1,2,3,4,5,6]

# iterate through the list using a for comprehension

>>> [i for i in l]

[1, 2, 3, 4, 5, 6]

# square elements in the list using a for comprehension

>>> [i**2 for i in l]

[1, 4, 9, 16, 25, 36]

# square only even elements in the list using a for comprehension

>>> [i**2 for i in l if i % 2 == 0]

[4, 16, 36]

# iterate through the list using map

>>> map(lambda w: w, l)

[1, 2, 3, 4, 5, 6]

# square each element of the list using map

>>> map(lambda w: w**2, l)

[1, 4, 9, 16, 25, 36]

# or use map to call a separately defined function to# iterate through the list

>>> def squa(x):

...... return x**2

>>> map(squa, l)

[1, 4, 9, 16, 25, 36]

# use map function to call a separately defined function that takes 2 args

>>> l2 = map(lambda w: (w,2),l)

>>> l2

[(1, 2), (2, 2), (3, 2), (4, 2), (5, 2), (6, 2)]

>>> def pow(base, expo):

...... return base**expo

>>> map(lambda (x,y): pow(x,y), l2)

[1, 4, 9, 16, 25, 36]

# filter a list

>>> filter(lambda w: w > 2, l)

[3, 4, 5, 6]

EXAMPLE 3

# split a list into chunks

>>> map(None, *(iter(range(10)),) * 3)

[(0, 1, 2), (3, 4, 5), (6, 7, 8), (9, None, None)]

http://stackoverflow.com/questions/1335392/iteration-over-list-slices

Django South Migrations

2009-10-07T21:04:00.016-05:00

Install Django South

- Place the untared 'south' folder into your Django applicaion

- Modify the project's settings.py and add 'south' to the INSTALLED_APPS list

- For new Django projects:

---1) initialize any Django apps or models to monitor with South.

----a) See Initializing Applications below

---2) run ./manage.py syncdb

---3) run ./manage.py migrate

~~To view help on startmigration:~~ # South 7.x+ uses the schemamigration command instead

~~./manage.py startmigration~~

~~Usage: ./manage.py startmigration appname migrationname [--initial] [--auto]~~

~~[--model ModelName] [--add-field ModelName.field_name] [--freeze] [--stdout]~~

Initializing Applications - needed whenever you want to track more Django apps

# this creates a 'migrations' folder in your Django app directory

~~./manage startmigration app_name comment --initial~~ # updates for South 7.x+
./manage schemamigration app_name comment --initial # Init's all models in application
./manage startmigration app_name ~~--initial~~ # updates for South 7.x+

./manage schemamigration app_name --initial # comment is optional here

Adding South with no data and no existing models.py objects:
# 1) syncdb as normal
python manage.py syncdb
# 2) Start an application if needed
python manage.py startapp appname
# 3) create a migration file with a blueprint to create tables
~~python manage.py startmigration appname --initial~~ # updates for South 7.x+
python manage.py schemamigration appname --initial
# 4) Apply the blueprint and create the tables
python manage.py migrate appname

# 5) Repeat steps 3 and 4 for each additional app if needed

Adding south with no data and existing models.py objects:
# 1) syncdb as normal if needed
python manage.py syncdb
# 2) create a migration file with a blueprint to create tables
~~python manage.py startmigration appname --initial~~ # updates for South 7.x+
python manage.py schemamigration appname --initial
# 3) Apply the blueprint but it doesn't actually create the tables
python manage.py migrate appname --fake
# 4) Repeat steps 2 and 3 for each additional app if needed

Applying changes after making changes to a model (repeat for each app):
# Creates a migration 'blueprint' file and guesses what should take place
~~python manage.py startmigration appname migrationname --auto~~ # updates for South 7.x+
python manage.py schemamigration appname migrationname --auto
# OR create the blueprint file but it will be empty
~~python manage.py startmigration appname~~ ~~migrationname~~ # updates for South 7.x+
python manage.py schemamigration appname migrationname
# make any modifications needed
# migrate the tables in the app
python manage.py migrate appname
# OR migrate all tables in all apps
python manage.py migrate

Adding a new model to an app already managed with south:
~~./manage startmigration app_name comment --model m1 --model m2~~ # updates for South 7.x+
./manage schemamigration app_name comment --model m1 --model m2 # inits specific models

List available migrations
python manage.py migrate --list

http://south.aeracode.org/wiki/ConvertingAnApp

Oracle Space queries

2009-09-22T10:41:00.008-05:00

Get Used space for current user
SELECT sum(bytes)/(1024 * 1024) as "Used (MB)"
FROM user_SEGMENTS
ORDER BY 1 desc

Get tablespace usage for all users using that tablespace
select owner, sum(bytes)/power(2,20)mb
from dba_extents
where tablespace_name = 'TABLESPACE'
group by owner
order by 2 desc;

Get Total tablespace available for current user
select tablespace_name,round(sum(bytes) / (1024 * 1024),2) "Tablespace SIZE (MB)"
from user_free_space
group by tablespace_name

Get space of individual tables/indexes (objects) in a schema
select sum(bytes) / (1024 * 1024) as MB,owner,segment_type, segment_name
from dba_segments s
where owner = 'SCHEMA'
group by owner,segment_type, segment_name
order by MB desc

Get USER_ dictornary Tables
SELECT table_name, comments
FROM dictionary
WHERE table_name LIKE 'USER_%'
ORDER BY table_name;

http://snipplr.com/view/4748/get-a-list-of-all-the-user-tables-oracle/
http://www.freelists.org/post/oracle-l/dba-extents-vs-dba-segments,8

CSS Pre Tag formatting

2009-09-12T11:12:00.002-05:00

Make pre tags wrap as expected

pre {
white-space: pre-wrap;       /* css-3 should we be so lucky... */
white-space: -moz-pre-wrap;  /* Mozilla, since 1999 */
white-space: -pre-wrap;      /* Opera 4-6  */
white-space: -o-pre-wrap;    /* Opera 7  */
word-wrap: break-word;       /* Internet Explorer 5.5+ */
_white-space: pre;   /* IE only hack to re-specify in addition to
word-wrap  */
}

http://archivist.incutio.com/viewlist/css-discuss/55677
http://bavotasan.com/tutorials/how-to-wrap-text-within-the-pre-tag-using-css/

Relative to Absolute Path in Shell Script

2009-08-21T15:27:00.002-05:00

Simple bash function to convert relative paths to absolute pats
fun_abs
{ echo “`cd \`dirname $1\`; pwd`/`basename $1`” }

http://www.robertpeaslee.com/index.php/converting-a-relative-path-to-an-absolute-path-in-bash/

UPDATE: unfortuately the above only works for directories that exist - the perl hack will get around this

fun_abs
{FILE=`$PERL -e "use File::Spec::Functions qw[rel2abs];print rel2abs('$1');"`; echo "$FILE"; }

Get File Extension in Shell Script

2009-08-20T10:23:00.004-05:00

Returns the last file extension in the file name
echo "thisfile.txt.log"awk -F . '{print $NF}' # returns "log"

http://liquidat.wordpress.com/2007/09/29/short-tip-get-file-extension-in-shell-script/

Correct CSS PNG Color Mismatch

2009-07-25T20:08:00.003-05:00

Copied from link below
Mac OS X will render the images using the color profile they got stored, actually png gamma correction would be more exact. This is not immediately apparent because at a glance the files may look identical but using the pngs over a CSS background color often reveals an unpleasant sight on the mac, especially on Safari.
To prevent such a color mismatch either use a different image format or strip the png gamma correction. In the case of the latter, pngcrush does a pretty good job.
I was pleasantly surprised that I could install it with ease on Fedora ("yum install -y pngcrush"). To use it run:

pngcrush -rem gAMA -rem cHRM -rem iCCP -rem sRGB png-file-name optimized-png-file-name

http://www.viseztrance.com/2009/02/fixing-the-png-color-mismatch-on-mac-os-x.html

vim settings

2009-05-22T12:50:00.020-05:00

# start vim with an alternate vimrc location
vim -u filename

# Turn compatibility mode off via set command
:set nocompatible

# view special characters in a vim session
:set list
:set nolist # disable

# set search result hilighting
:set hlsearch

# use different visible characters to see special characters in vim
:set listchars=eol:$,tab:>-,trail:~,extends:>,precedes:< :set list # view line numbers
:set nu
:set nonu # disable

# enable linewrap (on by default)
:set wrap
:set nowrap #disable

# indenting source code (setup)
:set et # expand tabs to spaces
:set sw=4
:set smarttab

# Manually choose the language for syntax highlighting

:setf language
:setf html # example

# The available languages for syntax highlighting

# can be found in the vim install directory.

# In my system, it was /usr/share/vim/vim72/syntax

# indenting source code (usage)
1) in command mode press v to and arrows to select a range of lines
2) after selecting, press ==

http://vim.wikia.com/wiki/Indenting_source_code

# indent multiple lines (not necessarily source code)
1) in command mode press v and arrows to select a range of lines
2) type shift >
shift < will shift a block back

# compatibility mode
# by default, vim will enter into compatibility mode when loading.
# Disable this behavior one of two ways
# 1) Create a .vimrc in your home directory
# 2) Within the vim command line, type :set nocompatible

# trim whitespace at end of line
:1,$s/[ <tab>]*$//

# get vim to return the previous position when editing files

" Tell vim to remember certain things when we exit
"  '10 : marks will be remembered for up to 10 previously edited files
"  "100 : will save up to 100 lines for each register
"  :20 : up to 20 lines of command-line history will be remembered
"  % : saves and restores the buffer list
"  n... : where to save the viminfo files
set viminfo='10,\"100,:20,%,n~/.viminfo

" when we reload, tell vim to restore the cursor to the saved position
augroup JumpCursorOnEdit
au!
autocmd BufReadPost *
\ if expand(":p:h") !=? $TEMP |
\ if line("'\"") > 1 && line("'\"") <= line("$") |
\ let JumpCursorOnEdit_foo = line("'\"") |
\ let b:doopenfold = 1 |
\ if (foldlevel(JumpCursorOnEdit_foo) > foldlevel(JumpCursorOnEdit_foo - 1)) |
\ let JumpCursorOnEdit_foo = JumpCursorOnEdit_foo - 1 |
\ let b:doopenfold = 2 |
\ endif |
\ exe JumpCursorOnEdit_foo |
\ endif |
\ endif
" Need to postpone using "zv" until after reading the modelines.
autocmd BufWinEnter *
\ if exists("b:doopenfold") |
\ exe "normal zv" |
\ if(b:doopenfold > 1) |
\ exe "+".1 |
\ endif |
\ unlet b:doopenfold |
\ endif
augroup END

http://vim.wikia.com/wiki/Restore_cursor_to_file_position_in_previous_editing_session

http://ubuntuforums.org/showthread.php?t=789327
http://www.oualline.com/vim-cook.html
http://stackoverflow.com/questions/1675688/make-vim-show-all-white-spaces-as-a-character
http://jamesreubenknowles.com/set-vim-syntax-language-270

svn+ssh on alternate port

2009-02-01T19:29:00.004-06:00

Setup the config
# Add the following lines to ~/.subversion/config
# must be put in the [tunnels] section of the config file
sshnew=ssh -l user -p 2222

Execute the svn:
svn co svn+sshnew://servername.com/path/to/repository/

Djanog show model query

2009-02-01T13:15:00.004-06:00

Show the raw SQL generated by a model access:
# DEBUG needs to be set True
>>> from django.db import connection
>>> connection.queries
[{'sql': 'SELECT polls_polls.id,polls_polls.question,polls_polls.pub_date FROM polls_polls',
'time': '0.002'}]

``sql``  # The raw SQL statement
``time`` # How long the statement took to execute, in seconds.

Reset the queries list returned above:

>>> from django import db
>>> db.reset_queries()

http://docs.djangoproject.com/en/dev/faq/models/

http://docs.djangoproject.com/en/dev/faq/models/#how-can-i-see-the-raw-sql-queries-django-is-running

Secure Django Location with htaccess file (webfaction)

2009-01-17T13:07:00.004-06:00

Basic steps to secure a Django account using htaccess file
note, this examples assumes apache 2.2, in which the basic auth apache module changed.
note, this example uses a WebFaction account but can be applied to any Django Apache install.

1) Execute this command to create an htpassword file
htpasswd -c /home/my_account/webapps/evesch/apache2/conf/.mypasswds my_user

2) Execute this command to create a htgroups file
echo "managers: my_user" > /home/my_account/webapps/evesch/apache2/conf/.mygroups

3) Modify your httpd.conf file to look like this.
############# contents of /home/my_account/webapps/evesch/apache2 #####
ServerRoot "/home/my_account/webapps/evesch/apache2"

LoadModule dir_module modules/mod_dir.so
LoadModule env_module modules/mod_env.so
LoadModule log_config_module modules/mod_log_config.so
LoadModule mime_module modules/mod_mime.so
LoadModule python_module modules/mod_python.so
LoadModule rewrite_module modules/mod_rewrite.so

# added by me (joe)
LoadModule auth_basic_module modules/mod_auth_basic.so
LoadModule authn_file_module modules/mod_authn_file.so
LoadModule authz_user_module modules/mod_authz_user.so
LoadModule authz_groupfile_module modules/mod_authz_groupfile.so

KeepAlive Off
Listen 7637
LogFormat "%{X-Forwarded-For}i %l %u %t \"%r\" %>s %b \"%{Referer}i\" \"%{User-Agent}i\"" combined
CustomLog logs/access_log combined
ServerLimit 2

<Location "/">
PythonHandler django.core.handlers.modpython
PythonPath "['/home/my_account/webapps/evesch', '/home/my_account/webapps/evesch/lib/python2.5'] + sys.path"
SetEnv DJANGO_SETTINGS_MODULE myproject.settings
SetHandler python-program

# added by me (joe)
AuthType Basic
AuthName "Under Construction"
AuthUserFile /home/my_account/webapps/evesch/apache2/conf/.mypasswds
AuthGroupFile /home/my_account/webapps/evesch/apache2/conf/.mygroups
Require group managers
</Location>

##############################################

4) restart your apache server and web browser
/home/my_account/webapps/evesch/apache2/bin/stop
/home/my_account/webapps/evesch/apache2/bin/start

http://forum.webfaction.com/viewtopic.php?id=2363

Show RAM Memory Usage (Linux)

2009-01-17T12:43:00.003-06:00

Show RAM used by given user (in MB)
ps -u your_username -o rss,command | grep -v peruser | awk '{sum+=$1} END {print sum/1024}'

Show RAM used by individual process (in KB)
ps -u your_username -o rss,pid,command


http://forum.webfaction.com/viewtopic.php?id=2356

Compile httpd (Apache)

2009-01-08T18:50:00.001-06:00

Http Configure options

./configure --enable-so --with-mpm=worker --prefix=/opt/httpd/http2.2/ --enable-rewrite --enable-alias --with-port=8080 --enable-cgi --enable-spelling | tee jjj.configure | tee jjj.configure
make | tee jjj.make
make install | tee jjj.make

mod_python Configure options

./configure --with-apxs=/opt/httpd/http2.2/bin/apxs --with-python=/opt/python2.6/bin/python
make
make install

http://cavedoni.com/2005/django-osx

Change keyboard in X

2009-01-06T18:11:00.002-06:00

http://ubuntu-tutorials.com/2008/01/31/changing-the-system-keyboard-mapping-on-ubuntu-dvorak-vs-qwerty/
http://xorg.freedesktop.org/archive/X11R7.0/doc/html/setxkbmap.1.html

Find Usage

2008-12-16T11:26:00.009-06:00

"Grep" through each file in a directory:
ls xargs grep -i 'STRING'

List all directories and subdirectories in a directory
find . -type d

List all files in a directory no recursion
# workaround for "parameter list is too long".
find . $ ! -name . -prune $ -name 'STRING*' -print
find . -maxdepth 1 -name 'STRING*' -print # GNU find only

Remove all files in a directory no recursion
# workaround for "parameter list is too long".
find . $ ! -name . -prune $ -name 'STRING*' -print -exec rm {} \;

Find a file with a given inode and delete it
ls -lai # lists the inodes next to the files
find . -inum 12345 -exec rm {} \; # finds and removes by inode

http://www.faqs.org/qa/qa-1381.html
http://sial.org/howto/shell/

Django Model Many-to-One

2008-10-13T23:31:00.019-05:00

Define the models

from django.db import models

class Reporter(models.Model):
.. first_name = models.CharField(max_length=30)
.. last_name = models.CharField(max_length=30)
.. email = models.EmailField(blank=True, null=True)

def __unicode__(self):
.. return u"%s %s" % (self.first_name, self.last_name)

class Article(models.Model):
.. headline = models.CharField(max_length=100)
.. pub_date = models.DateField()
.. reporter = models.ForeignKey(Reporter, blank=True, null=True)

def __unicode__(self):
.. return self.headline

.. class Meta:
.... ordering = ('headline',)

------------------------------------------------
Add some objects
# import models from core.models
>>> from core.models import Reporter, Article
>>> from datetime import datetime # needed for inserting current time
# add some reporters
>>> r1 = Reporter(first_name="Joe", last_name="Jaz")
>>> r1.save()
>>> r2 = Reporter(first_name="Jane", last_name="Smith")
>>> r2.save()
# add some articles
>>> a1 = Article(headline="Article1", pub_date=datetime(2008,10,14))
>>> a1.reporter = r1 # can add separately
>>> a1.save()
>>> a2 = Article(headline="Article2", pub_date=datetime(2008,10,15), reporter=r2)
>>> a2.save()
# show an article's reporter
>>> a1.reporter
<Reporter: Joe Jaz>
>>> a2.reporter.first_name
'Jane'
# create an article "under" a Reporter via the Reporter object
# the 'article_set' method is generated on model creation using the lower-case model name
# the create method automatically does a save()
>>> a3 = r1.article_set.create(headline="Article3", pub_date=datetime.now())
>>> a3.headline
'Article3'
>>> a3
<Article: Article1> # returns object with Article type
>>> a3.reporter
<Reporter: Joe Jaz> # returns object with Reporter type - the article reporter
# create an article with no reporter and then add it to a particular reporters set of articles
# the add method automatically does a save()
>>> a4 = Article(headline="Article4", pub_date=datetime.now())
>>> r2.article_set.add(a4)
# show all of the given reporter's articles
>>> r2.article_set.all()
[<Article: Article2>, <Article: Article4>]
# count the number of articles that belong to a reporter
>>> r2.record_set.count()
2
>>> r1.record_set.count()
2
# associate article 4 with reporter 1 (instead of reporter 2)
>>> r1.article_set.add(a4)
# we can assign it back and use different syntax this time
>>> a4.reporter = r2
>>> a4.save()
# we can assign multiple articles to a reporter at a time
# this syntax calls save automatically
>>> r1.article_set = [a2, a4]
# we can create an article and assign it with a reporter_id instead of the reporter object
# the reporter_id field is generated with the model
>>> a5 = Article(headline="Article5",pub_date=datetime.now(),reporter_id=r1.id)
>>> a5.save()
# or reporter_id can be a string
>>> a6 = Article(headline="Article6",pub_date=datetime.now(), reporter_id="1")
>>> a6.save()
# Do a query to find reporters of article 1 and article 3
>>> Reporter.objects.filter(article__in=[a1,a3])

http://www.djangoproject.com/documentation/models/many_to_one/

Django OneToMany & ManyToMany Recursive Models

2008-10-12T16:46:00.007-05:00

Define the models in core.models.

from django.db import models

class Place(models.Model):
.. name = models.CharField(max_length=50);
.. parent = models.ForeignKey('self', null=True);

.. def __unicode__(self):
.... return "%s > %s" % (self.name,self.parent )

class Fan(models.Model):
.. name = models.CharField(max_length=8)
.. idol = models.ManyToManyField('self', null=True)

.. def __unicode__(self):
.... return self.name

----------------------------------------------------------
Interact with a Foreign Key (One to Many)

# import Place model
from core.models import Place
# add a Place and save
world = Place(name="Earth", parent=None)
world.save()
# add two Places with the world as a parent
country1 = Place(name="US", parent=world)
country1.save()
country2 = Place(name="UK", parent=world)
country2.save()
# add two Places with the US as a parent
city1 = Place(name="Chicago")
city1.parent = country1 # make the association through an update
city1.save()
city2 = Place(name="New York", parent=country1) # associate through insert
city2.save()
# select all the places
Place.objects.all()

------------------------------------------------------
Interact with a Many to Many model

from core.models import Fan

# Define 2 people
person1 = Fan(name="Joe")
person1.save()
person2 = Fan(name="Jane")
person2.save()
# associate the two people
person1.idol.add(person2)

------------------------------------------------------

Java Jar Commands

2008-09-22T14:18:00.004-05:00

Creating the File structure
# simple suggested structure
project_directory/
\
_ bin/ # binary files
\_ byte_file.class # compiled java bytecode
_src/ # source files (optional)
_META-INF/ # standard location for manifest file
\_MANIFEST.MF # manifest file containing jar variables

Editing the MANIFEST.MF
The manifest can contain variables such as the following:
Main-Class: class_name # name of a class with a "main" method that will run when jar is run.
Class-path: path/to/classes/in/package/ # path start at root of package

Create the Jar
jar -cvfm jar_name.jar path/to/manifest.mf dir/to/class/files/
jar -cvfm project.jar META-INF/MANIFEST.MF bin/
-c # compress into a jar
-v # verbose output
-f # compress to a file specified on the command line (versus to stdout)
-m # specify a path to a manifest file.

Execute the Jar
java -jar jar_name.jar

Extract a Jar file
jar -xvf jar_name.jar

SSH Client Persistance

2008-08-19T21:02:00.003-05:00

Keep SSH connections from timing out
On your client, add this line to your /etc/ssh/ssh_config:
ServerAliveInterval 30

Sound Card Info

2008-08-09T12:30:00.008-05:00

Check what ALSA soundcards are available on a Linux system:
cat /proc/asound/cards

Check what features each ALSA card has:
cat /proc/asound/devices
# note that the left number column corresponds to a soundcard in the above list

List sound cards detected
aplay -l

Set the default sound card:
# Enter this into /etc/asound.conf or .asoundrc

pcm.!default {
type hw
card 1
}
ctl.!default {
type hw
card 1
}


Using alsamixer to select which sound card to affect: 
alsamixer -c <sound_card_number_from_proc>

Using mplayer with a non-default sound device
mplayer -ao oss:/dev/dsp *.mp3 # device 0
mplayer -ao alsa:device=hw=0.0 *.mp3 # device 0
mplayer -ao oss:/dev/dsp1 *.mp3 # device 1
mplayer -ao alsa:device=how=1.0 *.mp3 # device 1

http://seehuhn.de/pages/alsa
http://ubuntuforums.org/showthread.php?t=747054

Sendmail/Postfix Info

2008-07-17T16:58:00.001-05:00

Sendmail view mailq:
mailq
sendmail -bp

Sendmail view verbose info:
sendmail -v

http://www.unix.com.ua/orelly/networking/tcpip/ch10_08.htm

Postfix Info
http://www.akadia.com/services/postfix_mta.html

Join UNIX Command

2008-07-09T15:57:00.004-05:00

Join takes two sorted text files and joins them together on common keys, similar to the Join SQL statements.

Basic join syntax:
join file1 file2

Specify a delimiter other than the default of whitespace (for input and output):
join -t: file1 file2 # sets the delimiter to be a colon

Join on specific columns in each file:
join -1 1 -2 1 file1 file2 # this will make file1's first column join with the file2's first column.

Join on a specific column and output specific columns:
join -11 -21 -o 1.1 1.2 2.1 2.2 file1 file2 # the -o options takes space-seperated arguments that take the form of file.column to specify only particular columns.

Specify filler text in a result column when two keys do not match between files:
join -e "none" file1 file2 -o 1.1 1.2 2.1 2.2 # the -e option needs to be used with -o

http://www.softpanorama.org/Tools/join.shtml
http://www.computerhope.com/unix/ujoin.htm

Compile Python 2.5 From Source

2008-06-02T23:46:00.007-05:00

Download from Python web page
http://www.python.org/

UnTAR and enter the directory
tar -jxvf Python-2.5.2.tar.bz2
cd Python-2.5.2

Configure using an alternate install dir
(use ./configure -h for config options)
./configure --prefix=/opt/python/

Build and install the software
make | tee makelog
make test | tee maketestlog
make install | tee makeinstalllog
This will create a python binary. If you specified the prefix, it will put it in the bin/ directory under the prefix directory. However, this compiles the command line python version. To get it working with Apache, preform the following.

Download mod_python
http://www.modpython.org/

Untar and enter into the directory
tar -zxvf mod_python-3.3.1.tgz
cd mod_python-3.3.1

Configure mod_python.
It needs to know where python binary to use, and the apache build tool apxs is located during the configure.

./configure \
--with-apxs=/usr/sbin/apxs \ # the apache build tool
--with-python=/opt/python25/bin/python2.5 \ # path to python binary
| tee output.config
make | tee output.make
make install | tee output.makeinstall

make install should place a binary module in /etc/httpd/modules
(or /usr/lib/httpd/modules)
The module will be called mod_python.so

Configure Apache
You will need to tell Apache to load the module by adding the following line in the Apache configuration file (httpd.conf)

LoadModule python_module modules/mod_python.so
<Directory /path/to/somewhere >
  AddHandler mod_python .py
  PythonHandler mptest
  PythonDebug On
</Directory>

Imaging Support
To be able to manipulate images with python, you need the PIL library. Download it from http://www.pythonware.com/products/pil/
After untaring and entering the directory, install by: python setup.py install
It will copy files into your default (`which python`) python install.

http://www.modpython.org/live/current/doc-html/inst-apacheconfig.html

Basic apache .htaccess authentication

2008-04-20T12:01:00.010-05:00

To restrict access to a specific directory
Create a .htaccess file in the directory that you want to restrict
AuthUserFile /path/to/.mypasswds
AuthGroupFile /dev/null
AuthName "Title for password box"
AuthType Basic
require user username
require user user2 # you can add more users by setting multiple require statements

To generate an encrypted password
Execute this shell command which will create the file .mypasswds with a hashed password entry
htpasswd -c /path/to/.mypasswds username
New password: password
Re-type new password: password

Enable the configuration
The module mod_auth must be loaded in Apache2 config
LoadModule auth_module modules/mod_auth.so

Edit the Apache2 config files to set the AllowOverride for the directory you want to protect
<Directory "/var/www/html">
Options Indexes FollowSymLinks
AllowOverride AuthConfig
</Directory>

http://www.yolinux.com/TUTORIALS/LinuxTutorialApacheAddingLoginSiteProtection.html
http://engr.oregonstate.edu/computing/web/43
http://httpd.apache.org/docs/2.0/howto/auth.html

Compile PHP 5 From Source

2008-04-17T21:47:00.009-05:00

Before configuring, you must have development versions of the given packages installed. For example, for mcrypt support, you must install mcrypt-devel. In the list below, this applies to zlib-devel, libxml2-devel, mysql-devel, bzip2-devel, httpd-devel, and mhash-devel. Some configure options below didn't require a development version.

View configure options
./configure --help
Many configure options allow an optional =/path syntax to allow you to specify a non-default location for the development libraries.

Special configure notes
The --prefix config option allows you to specify a non-default location to install the php distribution (The root of the install, if you will)
The --exec-prefix directory allows you to specify an alternate location to install the compiled PHP binary files. In this example, they are set to the same location in the /opt directory.
Highly recommended to install --with-apxs (Apache 1.x) or --with-apxs2 (Apache 2.x). This creates the Apache module called libphp5.so that is normally installed in the Apache modules directory. This module is what Apache uses to run PHP web sites (and not the command line PHP program from the --prefix directory).

Configure command
./configure \
--prefix=/opt/php5/dist \
--exec-prefix=/opt/php5/dist \
--with-mysql \
--with-pdo-mysql \
--with-zlib \
--with-bz2 \
--enable-zip \
--with-openssl \
--with-mcrypt \
--with-mhash \
--with-curl \
--with-gd \
--with-jpeg-dir \
--with-png-dir \
--with-ttf \
--with-apxs2 \
--enable-ftp \
--enable-exif \
--with-freetype-dir \
--enable-calendar \
--enable-soap \
--enable-mbstring \
--with-libxml-dir=/usr/lib

Brief explanation of given config options
--prefix=/opt/php5/dist # sets where the root of the install will be
--exec-prefix=/opt/php5/dist # sets where the root of the binary install will be
--with-mysql # add mysql support
--with-pdo-mysql # add support for php mysql database abstraction layer
--with-zlib # add support for compressing using zlib
--with-bz2 # adds support for bunzip2
--enable-zip # adds zip compression support
--with-openssl # adds openssl functionality
--with-mcrypt # adds support for mycrypt style functions (encryption)
--with-mhash # adds special hashing functions
--with-curl # adds support for cURL which
--with-gd # adds support for manipulating images with php
--with-jpeg-dir # adds support for manipulating jpegs with php
--with-png-dir # adds support for manipulating png files with php
--with-ttf # adds support for manipulating true type fonts
--with-apxs2 # adds support for apache2 module (important)
--enable-ftp # adds support for php ftp
--enable-exif # adds support for manipulating Image metadata
--with-freetype-dir # adds support for FreeType2
--enable-calendar # don't know
--enable-soap # adds support for SOAP XML stuff
--enable-mbstring # multi-byte string support
--with-libxml-dir=/usr/lib # adds XML support (required)

Compile and install PHP
make | tee output.make
make test | tee output.maketest
make install | tee output.makeinstall

Special notes about the compile
When using --with-apxs or --with-apxs2, the libphp5.so will be compiled in the "libs/" directory of the PHP source code. The make install will try to copy this automatically to your Apache modules directory. (In the case of the above example, make install placed the module in the /etc/httpd/modules directory - this is really a simlink to /usr/lib/httpd/modules/ in this case.)
The command line PHP binary will be located in the bin/ directory of the path specified by the --prefix option.

Install the mod_php shared library into apache
After a compile of php with the --with-apxs or --with-apxs2, and placement in the /etc/httpd/modules directory (as mentioned above), the module needs to be referenced in the httpd.conf
LoadModule php5_module /usr/lib/httpd/modules/libphp5.so

# Also, the following needs to be added to the httpd.conf

# Cause the PHP interpreter to handle files with a .php extension
AddHandler php5-script php
AddType text/html php

# PHP Syntax Coloring
AddType application/x-httpd-php-source phps

# Add index.php to the list of files that will be served as directory
DirectoryIndex index.php

http://dan.drydog.com/apache2php.html
http://us.php.net/manual/en/configure.php#configure.options.servers

Grub Config

2008-02-07T20:56:00.006-06:00

Example of menu.lst booting a kernel

title Item Title Goes here

# this first root option selects which partition to set GRUB root
root (hd0,0) # name of the partition that grub should look for the kernel image

# the second root option in the kernel statement is the root partition that Linux will mount to
kernel (hd0,4)/boot/vmlinuz ro root=/dev/sda5 # location of read only image.

initrd (hd0,4)/boot/initrd.img # location of initrd image
boot # actually boots the system with the given commands

Example of menu.lst chainloading an OS

title Item Title Goes here
root (hd0,1)
chainloader +1

Installing grub into a boot sector (within the os)

sudo grub
> root (hd0,0)
> setup (hd0)
exit

Create MySQL User

2007-11-22T13:34:00.003-06:00

To create a user in mysql, issue the following:
CREATE USER 'user'@'host' IDENTIFIED BY 'password';

To create a user in mysql and assign grants, issue the following:
grant privilege on database.table to 'user'@'host' [with grant option] [identified by "password"]

Available privileges:
ALL [PRIVILEGES]
ALTER
CREATE
DELETE
DROP
FILE
INDEX
INSERT
PROCESS
USAGE
RELOAD
SELECT
SHUTDOWN
UPDATE

To revoke privileges:
revoke privilege on database.table from user

To revoke grant option:
revoke grant option on database.table from user

To change a password:
UPDATE mysql.user SET Password=PASSWORD('password')
WHERE User='someuser';

http://www.tech-faq.com/reset-mysql-password.shtml

Reset MYSQL root password

2007-11-22T13:32:00.007-06:00

Reset the mysql "root" password:
1) Log in as UNIX root
2) Stop the MySQL Deamon (may vary depending on OS)
/etc/rc.d/init.d/mysqld stop
3) Start mysql in safe mode
/usr/local/mysql/bin/mysqld_safe --skip-grant-tables
4) Log into the mysql schema as root in safe mode
/usr/local/mysql/bin/mysql --user=root mysql
5) Run the update to change the password
UPDATE user SET Password=PASSWORD('YOUR_PASSWORD')
WHERE Host='localhost' AND User='root';
flush privileges;
6) Restart the MySQL server deamon
/etc/rc.d/init.d/mysqld start

Method set a root password
mysqladmin -u root password 'new-password'

http://help.hardhathosting.com/question.php/200

Terminal Information

2007-09-14T15:18:00.002-05:00

The $TERM environment variable sets the terminal type. The terminal type environment variable should specify a terminal type definition file that is usually located in the /usr/share/lib/terminfo/ directory. This terminal type in the shell environment should match the terminal type setting in your terminal emulator program (xterm, putty, gterm, secureCRT, etc. ).

Set this environment variable to override default terminfo DB location
TERMINFO

Use this to query a terminfo DB in a nondefault location
infocmp -A /nonstandard/terminfo/DB/location/path

Terminfo default DB location
/usr/share/lib/terminfo

Show differences between terms
infocmp -d vt100 vt220

Show similarities between terms
infocmp -c vt100 vt220

Show term capabilities
infocmp # no args defaults to $TERM var
infocmp -I # no args defaults to $TERM var
infocmp term_name # term_name is a filename in the terminfo DB location
infocmp -1 # one item per line

Print ansi escape sequences for a given terminal capability
tput option # option listed in infocmp output

Compile a terminfo source file to be stored in the TERMINFO dir
tic filename

Create Windows style newline characters from the terminal
type: Ctrl-v Ctrl-m
printf '\r\n'

Display Windows style newline characters
cat -ve filename

Enable color in vim for terminals that support it (xterm, vt100, linux)
Add the following to the ~/.vimrc
set t_Co=8
set t_Sf=^[[3%p1%dm
set t_Sb=^[[4%p1%dm
syntax on

Replace the ^[ with a real key (type CTRL-V ESC)

http://vimdoc.sourceforge.net/htmldoc/syntax.html#xterm-color
http://vim.wikia.com/wiki/Color_highlighting_on_telnet
http://en.wikipedia.org/wiki/Newline

Subversion

2007-08-22T19:25:00.011-05:00

Setting up a repository:
svnadmin create /usr/local/svn/newrepos

Import files into repository:
svn import /import/dir file:///usr/local/svn/newrepos/projectname -m "comment"

List files in a repository tree:
svn list file:///usr/local/svn/newrepos/projectname
svn ls #while in sandbox
svn ls -v # verbose

Checkout a directory:
svn checkout "http://host:port/path"
svn co http://host:port/path
svn checkout file:///path/to/repos
svn checkout file:///localhost/path/to/repos
svn checkout svn+ssh://host/path/to/repos

Check a specific revision:
svn checkout -r revision_num
svn checkout -r {2006-02-17}
svn checkout -r {"2006-02-17 15:30"}

Checkin a file:
svn commit filename.txt -m "some comment here"

Update working copy with committed changes from other users:
svn update

Manage local copy (sandbox):
svn add filename
svn delete filename
svn move file1 file2
svn copy file1 file2
svn mkdir dirname

View files changed since last commit:
svn stat
svn stat filename
svn status -v # verbose view of all files in sandbox
svn status -v -u # contacts the repository to show more info

View changes since last commit:
svn diff # show diffs within files

Revert a file (working copy):
svn revert file

Restore a previously committed file back to working copy:

svn merge -r curr_version:prev_version working_file.txt

Show revision history:
svn log
svn log -r 5:19 # shows revisions 5 through 19
svn log -r 9 # shows revision 9
svn log filename # show revisions on a given file
svn log -r {2006-11-20}:{2006-11-22} # show versions between two dates
svn log --verbose # show all files in all revisions
svn log --verbose | grep "/path/to/file/" -B10 # grep history and show last 10 lines

Show file info:
svn info filename

Backup a repository:
svnadmin dump /path/to/repository > dumpfile

Restore a repository:
svnadmin load /path/to/repository/ < dumpfile.dump

Export a repository without all the metadata:
svn export file:///path/to/repository export_dir

Ignore files in the svn working copy:
create a file in home directory called .svnignore
While in working copy, run command
svn -R propset svn:ignore -F /path/to/home/.svnignore .

http://pandemoniumillusion.wordpress.com/2008/05/07/ignore-pyc-files-in-subversion/

http://svnbook.red-bean.com/en/1.0/re10.html
http://svnbook.red-bean.com/en/1.0/re36.html
http://aralbalkan.com/1381
http://svnbook.red-bean.com/en/1.1/svn-book.html#svn-ch-3-sect-6.2.2

Serial to USB converter

2007-04-22T18:30:00.000-05:00

Prerequisites
minicom

kernel configure options:
USB=m # USB subsystem
USB_SERIAL=m # USB serial port support
USB_SERIAL_GENERIC=m # USB serial port generic driver
USB_SERIAL_PL2303=m # USB serial port driver

menuconfig path:
- Device Drivers ->
-- USB Support ->
--- USB Serial Converter support ->
---- USB Generic Serial Driver
---- (hardware specific driver)
---- e.x. USB Prolific 2303 Single Port Serial Driver

lsmod Listing (modules loaded):
usbcore pl2303,usbserial,...
- usbserial pl2303

Minicom serial port settings:
Serial Device: /dev/ttyUSB0
Bps/Par/Bits: 9600 8N1
Hardware Flow Control: No
Software Flow Control: No

http://www.gentoo.org/doc/en/usb-guide.xml

iproute commands

2007-04-14T11:48:00.000-05:00

Prerequisites
iproute2 package

kernel configure options:
CONFIG_IP_ADVANCED_ROUTER=y
CONFIG_IP_MULTIPLE_TABLES=y
CONFIG_IP_ROUTE_MULTIPATH=y

menuconfig path:
- Networking -->
-- Networking Support -->
--- Networking options -->
---- TCP/IP networking
----- IP: Advanced router
----- IP: policy routing
----- IP: equal cost multipath

Enable IP Forwarding
echo "1" > /proc/sys/net/ipv4/ip_forward
echo 0 > /proc/sys/net/ipv4/conf/all/rp_filter

IP LINK (cmd subset)

Show hardware address
ip link
ip link [show|ls|list|sh]

Take an interface up/down
ip link set dev (device) up
ip link set (device) up
ip link set dev (device) down
ip link set (device) up

Change the MAC address of the interface
ip link set dev (device) address (arp address)

IP ADDRESS (cmd subset)

Show ip address
ip addr
ip addr [show|sh|list|ls]
ip addr show dev eth0 # show specific device

Show ip addresses (the old "ifconfig" way)
ifconfig -a
ifconfig

Assign IP address to interface
ip addr [add|a] (address)/(mask) dev (device)
ex: ip addr add 192.168.1.2/24 dev eth0

Assign IP address to interface (the old "ifconfig" way)
ifconfig eth0 192.168.1.2 netmask 255.255.255.0

Remove IP address from interface
ip addr [delete|del|d] (address)/(mask) dev (device)
ex: ip addr del 192.168.1.2/24 dev eth0

IP NEIGHBOR (cmd subset)

Show ip Layer 2 neighbors (arp table)
ip neigh

Show ip Layer 2 netighbors (the old "arp" way)
arp -a

Add new ARP mapping
ip neigh add 192.168.1.100 dev eth0 lladdr (mac address)
ip neigh add 192.168.1.100 dev eth0 lladdr (mac address) nud reachable

NUD Statuses
--permanent (perm) # administrative mapping
--noarp # neighbor valid but no attempt to rearp will be made
--reachable # neighbor entry valid until timeout
--stale # old arp entry

Remove ARP mapping
ip neigh del 192.168.1.100 dev eth0

Flush arp table
ip neigh flush

IP ROUTE (cmd subset)

Show ip routing tables
ip route

Show ip routing tabes (the old "route" way)
route

Add Static Route
ip route add (network)/(mask) via (ip to route through)
e.x. ip route add 10.0.0.0/24 via 192.244.7.65
ip route add (network)/(mask) src INTERFACE_IP dev (device)
ip route add (network)/(mask) dev (device) protocol static

Add Static Route (old "route" method)
route add -net 10.0.0.0 netmask 255.255.255.0 dev eth0

Add Default Route
ip route add default via GATEWAY_IP

Add Default Route (old "route" method)
route add default gw 192.168.1.1

Del Route
ip route del 10.0.0.0/24

Delete all routes on an interface
ip route flush dev eth0

http://www.policyrouting.org/iproute2.doc.html
http://gentoo-wiki.com/Dual_internet_connections

Date Time Clocks

2007-04-07T10:13:00.000-05:00

Setting the timezone:
ln -sf /usr/share/zoneinfo/your/zone /etc/localtime

Show time in a given timezone:
zdump Japan
zdump America/Chicago
zdump US/Central

View date and time:
date
date --utc #universal time

Set the date and time:
date -s "16:15:00" # just the time
date -s "16:15:00 April 7, 2007" # date and time
date 040716552007.00 # the fields being MMDDhhmmCCYY.ss

Verify connectivity with NTP server:
ntpdate -q time.nist.gov

Several common time servers:
clock.redhat.com
clock2.redhat.com
ns1.tuxfamily.org
time.nist.gov
time.apple.com

Manually use NTP to set time:
ntpdate time.nist.gov
ntpdate -v time.nist.gov # verbose output

Use NTPd to automatically set date and time:
/etc/init.d/ntpd start #ntpd must be installed, of course

Monitor NTPd:
ntpdc -p
ntpdc -p -n

http://www.linuxsa.org.au/tips/time.html
http://www.vanemery.com/Linux/RH-Linux-Time.html

Fake RAID links

2007-04-01T14:13:00.000-05:00

Wikipedia FakeRAID Def
http://en.wikipedia.org/wiki/Fakeraid
Not transparent to the OS like real RAID. Not controlled entirely by the operating system either like softRAID. However, it uses hardware and needs an OS driver.

LINUX FakeRAID drivers
http://linux-ata.org/driver-status.html

List of FaikRAID hardware devices/controllers
http://linuxmafia.com/faq/Hardware/sata.html

My MoBo
http://www.pcstats.com/articleview.cfm?articleid=1770&page=1

This Mobo uses the following kernel modules:
sata_nv: for the 4 nForce4 controllers

kernel configure option:
CONFIG_SATA_NV=y

menconfig path:
- Device Drivers -->
-- Serial ATA (prod) and ... -->
--- ATA device support -->
---- NVIDIA SATA support
----- sata_sil: for the 4 Sillicon Image 3314 controllers

kernel configure option:
CONFIG_SATA_SLI = y

menuconfig path:
- Device Drivers -->
-- Serial ATA (prod) and ... -->
--- ATA device support -->
---- Silicon Image SATA support
----- ahci: generic Open standard (I don't have this, but worth a mention)

kernel configure option:
CONFIG_SATA_AHCI=y

menuconfig path:
- Device Drivers -->
-- Serial ATA (prod) and ... -->
--- ATA device support -->
---- AHCI SATA support

Both of these modules depend on the module:
libata

Libata depends on the following module:
scsi_mod

kernel configure option:
CONFIG_SCSI=y

menuconfig path:
- Device Drivers -->
-- SCSI device support -->
--- SCSI device support

Linux Quota Support

2007-03-20T01:25:00.000-05:00

http://www.yolinux.com/TUTORIALS/LinuxTutorialQuotas.html

mdadm software raid usage

2007-03-20T00:46:00.003-05:00

Install
mdadm package needed (aptitude install mdadm)
http://neil.brown.name/blog/mdadm

Create array:
mdadm --create --verbose /dev/md0 --level=0 --raid-devices=2 /dev/sdb1 /dev/sdc1
or
mdadm -Cv /dev/md0 -l0 -n2 -c128 /dev/sdb1 /dev/sdc1

/etc/mdadm.conf
# optional, but used for easier interaction
DEVICE /dev/sdb1 /dev/sdc1
ARRAY /dev/md0 devices=/dev/sda1,/dev/sdb1

Show md info:
# produces output that you would put in the mdadm.conf
mdadm --detail --scan

Show device info:
mdadm -E /dev/sdc1
# shows info about the component disks

Start (assemble) Existing Array:
mdadm -As /dev/md0
# -A starts assemble mode to assemble existing array
# -s indicates the use of /etc/mdadm.conf
or
mdadm -A /dev/md0 /dev/sdb1 /dev/sdc1
# starts array without existing /etc/mdadm.conf

Assemble an existing array automatically:
mdadm --assemble --scan
or
mdadm -As

Stop the array:
mdadm -S /dev/md0
or
mdadm --stop /dev/md0

Add a disk:
mdadm /dev/md0 --add /dev/sdc1

Remove a disk:
mdadm /dev/md0 --fail /dev/sdc1 --remove /dev/sdc1
# "fails" and removes a disk

fstab entry for the array:
# this assumes the array was formated with ext3
/dev/md0 /export/ ext3 suid,dev,defaults,exec 0 0

http://www.linuxdevcenter.com/pub/a/linux/2002/12/05/RAID.html

SUDO

2007-03-03T22:23:00.002-06:00

Generic syntax of /etc/sudoers
<users_to_allow> <host> = (run_as_user) <command_to_run>
<users_to_allow> <host> = (run_as_user) <file_to_grant>

No password for users in admin group
%admin ALL= (ALL) NOPASSWD: ALL

No password for joeuser user
joeuser ALL=(ALL) NOPASSWD: ALL

Allow joeuser to run only certain privileged commands as root
joeuser ALL= /bin/kill, /usr/local/bin/

Allow joeuser to run certain commands as given users
joeuser ALL=(janeuser,johnuser) /bin/kill, /usr/local/bin

Run sudo command as another user using
sudo -u janeuser /bin/kill

http://aplawrence.com/Basics/sudo.html

Screen Command Usage

2007-02-26T13:17:00.010-06:00

Create a new Screen session
screen # creates a default session
screen -S name # creates a session with a name

# Screen has two similar concepts: screens and window sessions.
# screens are seperate processes that show up when doing
# a "screen -ls" listing. Windows are "sub-screens" of a given
# screen process used for split screen modes and such.

Rename an already created window
CTRL-a A # then provide name

Detach from a screen session
screen -d # detach from a command prompt
Ctrl-a d # detach from anywhere in the session

List screen sessionsscreen -ls

Reattach to a screen session
screen -r # attaches to default session
screen -r name # attaches to specific session (unique string in the name)
Toggle between several attached windowsCtrl-a Ctrl-a

Attach to an already attached screen (mirror)
screen -x # attaches to default session
screen -x name # attaches to named session

Share a screen sessionowner:
screen -S name # start and name the screen
Ctrl-a :multiuser on # add multiuser support
Ctrl-a :acladd userids # share the screen to userids (comma sep)

client:
screen -x username/session # attach to the screen (in mirror)Change permissions on a shared session
owner:
Ctrl-a :aclchg userids [permbits] [list]
# permbits = rwx and prefixed by "-" or "+"
# list of commands - #(all windows), ?(all commands)

Remove user access
owner:
acldel userids

Lock a Screen
ctrl-a ctrl-x # requires user to enter password to unlock

Copy and Paste from the screen scrollback buffer
Ctrl-a [Esc] # enters scrollback editor (movement like vi)
[spacebar] # starts and stops a copy selection range
Ctrl-a :paste . # pastes the copy buffer to the termScreenrc options:
/home/> vi .screenrc
#shell -bash
shell -ksh # Make Screenrc exe .profile file on login

# Set the scrollback buffer
defscrollback 5000

# Turn off the startup banner
startup_message off

# displays a status line at the bottom of the terminal window.
hardstatus alwayslastline "Screen: %w %c:%s %D, %M/%d/%Y "

# detach on hangup - if my dial-up session fails, screen will simply
# detach and let me re re-attach later
autodetach on

Split Screen
# Create two screens
screen # create first screen
CTRL-a c # create second screen session (1 screen instance, but 2 window sessions)
CTRL-a S # split the screen
CTRL-a TAB # move to the bottom part of the split
CTRL-a " # will prompt for a screen number. Choose 0 or 1 indicating the window session
CTRL-a Q # quit all splits except the current

Scrollback / Copy and Paste
CTRL-a [ or CTRL-a ESC # frees the cursor to move into scrollback buffer (Copy mode)
# ESC to exit copy mode without copying
# use arrow keys or h,j,k,l to navigate in copy mode
# use CTRL-F and CTRL-B to page up and down in copy mode
ENTER # indicates a start point for text copy
ENTER # (second) indicates an end point for text copy
CTRL-a ] # pastes copied text range

http://aperiodic.net/screen/quick_reference
http://news.softpedia.com/news/GNU-Screen-Tutorial-44274.shtml