Django/Web Modules

Modernizer - Feature Detection for JS
http://modernizr.com/

Django-flash - Rails-like django flash messages:
https://github.com/danielfm/django-flash

Facebook-Graph - Django facebook graph API
https://github.com/feinheit/django-facebook-graph

django-cumulus - Rackspace manager

http://packages.python.org/django-cumulus/

Django-ittybitty - Django based URL Shortener:

http://code.google.com/p/django-ittybitty/

django-ittybitty

Django Storages

http://django-storages.readthedocs.org/en/latest/
django-storages

Python-Postmark - API for http://postmarkapp.com/

https://github.com/themartorana/python-postmark

Django Cache Utils

http://pypi.python.org/pypi/django-cache-utils

stripe-python - API for Stripe.com (like Braintree)

https://github.com/stripe/stripe-python

FrogBugz - online ticket tracking system

http://www.fogcreek.com/fogbugz/pricing.html

Form Builder
http://www.wufoo.com

Factual - service for data information and places. GeoData
http://v2.factual.com/

CartoDB - open source data browser db
https://github.com/Vizzuality/cartodb

GeoCoda - Online Geocoding tool
https://geocoda.com/

Census Tiger Dataset
http://www.census.gov

SendGrid - Email Web applications
http://sendgrid.com/

RapidSSL - Cheap SSL Host

django-tables2 - manipulate html tables programmatically similar to Django forms.
http://django-tables2.readthedocs.org/en/latest/index.html

django-message-extends - extensions to Django messages such as sticky messages and assigned messages
https://github.com/AliLozano/django-messages-extends

Simple Logging

Simple Logging

In someplace like python shell, __init__.py, or settings.py (pre Django 1.3):

import logging
logger = logging.getLogger('mymodule.test')
logger.setLevel(logging.DEBUG)
formatter = logging.Formatter("%(asctime)s - %(name)s - %(lineno)s - %(levelname)s - %(message)s")

# Configure log to file
filelog = logging.FileHandler("mylogfile.log", 'a')
filelog.setLevel(logging.INFO)
filelog.setFormatter(formatter)
logger.addHandler(filelog)

# Configure log to stdout
conlog = logging.StreamHandler()
conlog.setLevel(logging.DEBUG)
conlog.setFormatter(formatter)
logger.addHandler(conlog)

In place you want to log in, like views.py:

import logging
logger = logging.getLogger('mymodule.test')
logger.debug("My Log text")
logger.info("My info text")

Another example of simple logging:

import logging
logger = logging.getLogger('mymodulename')
logger.setLevel(logging.DEBUG)
ch = logging.StreamHandler()
ch.setLevel(logging.DEBUG)
logger.addHandler(ch)

---------------------------------------------------

A simple example:

import logging
logging.basicConfig(filename='test.log', level=logging.DEBUG)
logging.info("INFO")
logging.debug("DEBUG")
logging.error("ERROR")


Sources:
http://dancingpenguinsoflight.com/2009/03/simple-and-effective-python-logging/
http://www.blog.pythonlibrary.org/2012/08/02/python-101-an-intro-to-logging/

Dulwich Command Notes

# Load Git repo into object

from dulwich.repo import Repo

repo = Repo("/home/jjasinsk/Sites/env/proj/gitdjango/")

# Find sha hash of Git repo head

repo.head()

# 'e80b3ada92d2a02ce353c83683a6533e25240a1a'

# Find the Commit object of the repo head

repo[repo.head()]

# Commit e80b3ada92d2a02ce353c83683a6533e25240a1a

# Find all the repo tags

repo.refs.subkeys('refs/tags')

# set(['v0.0.1'])

# Find all the repo branches

repo.refs.subkeys('refs/heads')

# set(['master', 'develop'])

# Find all the repo refs and their sha hashes

repo.get_refs()

# {'HEAD': 'e80b3ada92d2a02ce353c83683a6533e25240a1a', 

   'refs/heads/develop': 'b53fe83f58f293decae39a70106a5ae78886ba5e', 

   'refs/tags/v0.0.1': '5554750c44f8dc17ab9dc5926129cf409d06d0bb', 

   'refs/heads/master': 'e80b3ada92d2a02ce353c83683a6533e25240a1a'}

# Find all the commits tracing back to head Commit 

repo.revision_history(repo.head())

# [Commit fcc1e1eb645f6731c4a8701b5facf852738850a9, 

    Commit 39eaa48156e3df2fd2cbe9922e33787cbafdad90, 

    Commit e80b3ada92d2a02ce353c83683a6533e25240a1a,

   ...

TAG OBJECTS

# Get the Tag object by looking up it's ref path
repo['refs/tags/v0.0.1']

# Tag 5554750c44f8dc17ab9dc5926129cf409d06d0b;

# Find Tag by sha hash

repo.get_object('5554750c44f8dc17ab9dc5926129cf409d06d0bb')

# Tag 5554750c44f8dc17ab9dc5926129cf409d06d0bb

repo.tag("5554750c44f8dc17ab9dc5926129cf409d06d0bb")

# Tag 5554750c44f8dc17ab9dc5926129cf409d06d0bb

repo['5554750c44f8dc17ab9dc5926129cf409d06d0bb']

# Tag 5554750c44f8dc17ab9dc5926129cf409d06d0bb

# Find information about a tag

t = repo['5554750c44f8dc17ab9dc5926129cf409d06d0bb']

# Find tagger 
t.tagger

# 'Joe Jasinski joe.jasinski@example.com'

# Find tag date

from datetime import datetime

datetime.utcfromtimestamp(t.tag_time)

datetime.datetime(2012, 3, 30, 6, 58, 53)

# Print tag contents

print t.as_pretty_string()

# ...

# Find Tag sha hash (id)

t.id

# '5554750c44f8dc17ab9dc5926129cf409d06d0bb'

   OR
s = t.sha()

s.hexdigest()

# '5554750c44f8dc17ab9dc5926129cf409d06d0bb'

# Find Tag message

t.message

# 'test tag\n'

# Find Tag Type

t.get_type()

# 4

COMMIT OBJECTS

# Find Commit object from ref spec
repo['refs/heads/master']

# Commit e80b3ada92d2a02ce353c83683a6533e25240a1a

# Look up commit several different ways

repo.get_object('b53fe83f58f293decae39a70106a5ae78886ba5e')

# Commit b53fe83f58f293decae39a70106a5ae78886ba5e

repo.commit("b53fe83f58f293decae39a70106a5ae78886ba5e")

# Commit b53fe83f58f293decae39a70106a5ae78886ba5e

repo['b53fe83f58f293decae39a70106a5ae78886ba5e']

# Commit b53fe83f58f293decae39a70106a5ae78886ba5e

# Find Commit hash 

repo.ref('refs/heads/master')

# 'e80b3ada92d2a02ce353c83683a6533e25240a1a'

# Find commit author 

repo['refs/heads/master'].author

# 'Joe Jasinski joe.jasinski@example.com'

# Find committer 

repo['refs/heads/master'].committer

# 'Joe Jasinski joe.jasinski@example.com'

# Find Commit message

repo['refs/heads/master'].message

# 'updated style\n'

# Find commit time

from datetime import datetime

datetime.utcfromtimestamp(repo['refs/heads/master'].commit_time)

# Find Trees associated with commit

repo['b53fe83f58f293decae39a70106a5ae78886ba5e'].tree

'b3437c8329fc3f6306936c7ca1986cefc1629862'

repo[c.tree].items()

[TreeEntry(path='.gitignore', mode=33188, sha='13609a506289eba22b61df5eab0f89b7aede323d'), 

TreeEntry(path='example', mode=16384, sha='f8e95712570c4316868ee9af32088324e8d3e498'), 

TreeEntry(path='gitdjango', mode=16384, sha='32b164fac85e5b40dd8d4265a0a8df765bf57300'), 

TreeEntry(path='manage.py', mode=33261, sha='2605e3768e2bab92ef0550d452036472e9a6a210'), 

...

 repo[repo[c.tree].items()[0].sha]

TREE OBJECTS

# Find Tree associated with hash 
repo.get_object('b3437c8329fc3f6306936c7ca1986cefc1629862')

# Tree b3437c8329fc3f6306936c7ca1986cefc1629862

# Find all items under a Tree

repo[b3437c8329fc3f6306936c7ca1986cefc1629862].items()

BLOB OBJECTS

# Find Blob associated with hash 

repo.get_blob("2605e3768e2bab92ef0550d452036472e9a6a210")

# Blob 2605e3768e2bab92ef0550d452036472e9a6a210

repo['2605e3768e2bab92ef0550d452036472e9a6a210']

# Blob 2605e3768e2bab92ef0550d452036472e9a6a210

MISC

# Iterate through all objects under a gree
for entry in repo.object_store.iter_tree_contents(repo['refs/heads/master'].tree):

    print entry

# TreeEntry(path='.gitignore', mode=33188, sha='13609a506289eba22b61df5eab0f89b7aede323d')

# TreeEntry(path='example/__init__.py', mode=33188, sha='e69de29bb2d1d6434b8b29ae775ad8c2e48c5391')

# TreeEntry(path='example/__init__.pyc', mode=33188, sha='ced988bb5f3f22f4f1956bf852f106b23eea5c09')

# ...

# Find a the hash of an object by specifying a filesystem path

from dulwich.object_store import tree_lookup_path  

(mode, hash) = tree_lookup_path(repo.get_object, repo['refs/heads/master'].tree, 'gitdjango/models.py')

# Find an object filehandle relative to inside the .git directory

repo.get_named_file('../requirements.pip')

# open file '/home/jjasinsk/Sites/env/proj/gitdjango/.git/../requirements.pip', mode 'rb' at 0x101cdf938

f.read()

# 'Django==1.4\nGitPython==0.3.2.RC1\nasync==0.6.1\ngitdb==0.5.4\ninclude-server==3.1-toolwhip.1\nsmmap==0.8.2\nwsgiref==0.1.2\n'

Monitoring Connections

Find IPs that have the most connections
netstat -plan|grep :80|awk {'print $5'}|cut -d: -f 1|sort|uniq -c|sort -nk 1
http://www.webhostingtalk.com/showthread.php?t=673467

Find Listening Connections (open ports)
netstat -an | grep -i listening

Other Netstat Commands:

netstat -ntl
netstat -nputw
netstat -a -n -p
netstat -tulpn
netstat -tunapl

http://unix.stackexchange.com/questions/56453/how-can-i-monitor-all-outgoing-requests-connections-from-my-machine

http://www.webhostingtalk.com/showthread.php?t=991911

Basic Vagrant Usage

Doing a 'vagrant box add boxname.box' creates a .vagrant.d/ directory in your home directory. This directory stores base boxes installed. Doing a 'vagrant init' in some other directory creates a Vagrantfile inside of that directory which specifies which base box to use and provides any other provisioning information. 

List vagrant base boxes installed:
vagrant box list
# returns list of boxnames and type
mybox (virtualbox)

Add a new vagrant base box named "my_box" based on box image:
vagrant box add my_box http://files.vagrantup.com/lucid32.box

Remove a base box:
vagrant box remove my_box type
# for a virtualbox use virtualbox as the type

Location of vagrant base boxes:
cd ~/.vagrant.d/

Create a directory where customizations off the of the base box should take place.  then run the following: 
cd ~/Sites/VM/test/
vagrant init
vagrant init lucid32  # as an alternative to explicitly indicate which base box to use

Example Vagrant file in ~/Sites/VM/test/Vagrantfile

Vagrant::Config.run do |config|

     config.vm.box = "lucid32"  # specify which base box to use

end

Run a virtual machine.  Builds a virtual machine using box and Vagrantfile and runs it.  You must be inside the given vagrant directory (~/Sites/VM/test) in order for this to work.

vagrant up

Destroy a virtual machine. Destroy's the virtual machine:
vagrant destroy

SSH into a virtual machine:
vagrant ssh

Running a packaged virtual machine
vagrant box add my_box /path/to/the/package.box
vagrant init my_box
vagrant up

Suspend a virtual machine (save state to disk):
vagrant suspend

Resume a suspended virtual machine:
vagrant resume

Some Customizations to the default config (Pre-vagrant 1.1): 
Vagrant::Config.run do |config|
    # this is basically a way to programmatically invoke the
    # 'VBoxManage modifyvm' command from within Vagrant 

    # NOTE: this requires a fairly new version of vagrant 
    # to use this syntax. 
    config.vm.customize ["modifyvm", :id, "--memory", 1024]

    # By default vagrant boots boxes headless.  This enables the 
    # Virtualbox GUI
    config.vm.boot_mode = :gui

    # Path to ssh private key used to log into box with 
    # vagrant ssh. If it's a relative path, it will be relative 
    # to the dir holding the Vagrantfile

    config.ssh.private_key_path = 'vagrant_id_dsa'

    # Creates a shared folder on host called myshared in the 
    # same directory as the Vagrantfile.  On the guest, it 
    # mounts the shared folder as /my/shared.  The :create 
    # setting creates the directory on the host if it doesn't
    # exist.   

    config.vm.share_folder 'my-shared', '/my/shared/', 'myshared', {:create, true}

end

Some Customizations to the default config (Post-vagrant 1.1): 

Vagrant.configure("2") do |config|
   # setup port forwarding 
  config.vm.network :forwarded_port, guest: 8000, host: 18000, auto_correct: true
  config.vm.network :forwarded_port, guest: 5432, host: 15432, auto_correct: true

  config.vm.provider :virtualbox do |vb|
      # Don't boot with headless mode
      vb.gui = true

      # Use VBoxManage to customize the VM. For example to change memory:
      vb.customize ["modifyvm", :id, "--memory", "1024"]

     # Create a shared folder   (host path, guest path)
     config.vm.synced_folder "../data", "/vagrant_data"
  end


Turn on Debugging
export VAGRANT_LOG=debug


List of vagrant base boxes:
http://vagrantbox.es/

Get SSH configuration for machines:

vagrant ssh-config
Host swarm_manager
  HostName 127.0.0.1
  User vagrant
  Port 2222
  UserKnownHostsFile /dev/null
  StrictHostKeyChecking no
  PasswordAuthentication no
  IdentityFile /Users/jjasinski/Sites/test_docker_swarm/.vagrant/machines/swarm_manager/virtualbox/private_key
  IdentitiesOnly yes
  LogLevel FATAL

Host swarm_node1
  HostName 127.0.0.1
  User vagrant
  Port 2200
  UserKnownHostsFile /dev/null
  StrictHostKeyChecking no
  PasswordAuthentication no
  IdentityFile /Users/jjasinski/Sites/test_docker_swarm/.vagrant/machines/swarm_node1/virtualbox/private_key
  IdentitiesOnly yes
  LogLevel FATAL

Email Web Applications 

http://postmarkapp.com/
http://cloudmailin.com/
http://mailchimp.com
http://myemma.com/

Finding table sizes in postgres

Finding table sizes in postgres

Taken from: http://wiki.postgresql.org/wiki/Disk_Usage

SELECT nspname || '.' || relname AS "relation",
    pg_size_pretty(pg_total_relation_size(C.oid)) AS "total_size"
  FROM pg_class C
  LEFT JOIN pg_namespace N ON (N.oid = C.relnamespace)
  WHERE nspname NOT IN ('pg_catalog', 'information_schema')
    AND C.relkind <> 'i'
    AND nspname !~ '^pg_toast'
  ORDER BY pg_total_relation_size(C.oid) DESC
  LIMIT 20;

Copy File through Compressed Tar

Copy and Compress File through an ssh connection
tar jcvf - myfile.txt | ssh myhost.com "cat > myfile.tar.bz2"

Django Session Introspection

Get Data Stored in a Session
from django.contrib.sessions import models
s = models.Session.objects.get(session_key='05adfa345a54525cad912')
s.get_decoded()

ImageMatik


Convert all TIFs in a directory to JPEGs
mogrify -format jpg -quality 50 *.tiff

http://linuxuser32.wordpress.com/2007/06/16/batch-image-convert-scale-thumbnail-jpegs-pdf/

Python: lxml insert element before and after given element

Python: lxml insert element before and after given element

Based on: http://stackoverflow.com/questions/7474972/python-lxml-append-element-after-another-element

from lxml import etree

xml = etree.parse('old_xml.xml')

count = 1

for pp in xml.xpath('/response/items/item'):

   prod_id = etree.Element('new_element')  # this element will be inserted

   prod_id.text = "%s" % count

   nav = pp.find('name')  # we will insert before/after this element

   div = nav.getparent()

   pp.insert(pp.index(nav), prod_id) # insert before 'name'

   pp.insert(pp.index(nav) + 1, prod_id) # insert after 'name'

   count += 1

out = open('new_xml.xml','w')

out.write(etree.tostring(xml, pretty_print=True))

out.close()

Python: Detect Charset and convert to Unicode


Converting data to Unicode via chardet 
http://stackoverflow.com/questions/2686709/encoding-in-python-with-lxml-complex-solution

import chardet
from lxml import html

f = open('my.xml')

content = f.read()

f.close()

encoding = chardet.detect(content)['encoding']
if encoding != 'utf-8':
    content = content.decode(encoding, 'replace').encode('utf-8')

Converting Data to Unicode via UnicodeDammit 

http://lxml.de/elementsoup.html

from BeautifulSoup import UnicodeDammit

converted = UnicodeDammit(content)

   if not converted.unicode: 

    raise UnicodeDecodeError( 

       "Failed to detect encoding, tried [%s]", 

        ', '.join(converted.triedEncodings))  

        # print converted.originalEncoding

         return converted.unicode

Credit goes to Ian Bicking and others on the lxml team

Using lxml xpath to get elements with a default namespace

xml_string="""

<a xmlns="http://www.w3.org/1999/xhtml">

  <b>
    <r>
      <Value>string</Value>

    </r>

    <r>

      <Value>string</Value>

    </r>

  </b>

  <e>

    <string>string1</string>

    <string>string2</string>

  </e>

</a>

"""

from lxml import etree

xml = etree.fromstring(xml_string)

xml.xpath('/w3:a/w3:b/w3:r/w3:Value/text()', namespaces={'w3':'http://www.w3.org/1999/xhtml'})

SQLAlchemy Notes

Summarized From 
http://www.sqlalchemy.org/docs/core/tutorial.html


This covers the basics of the Non-orm usage of SQLAlchemy

# Define connection engine
from sqlalchemy import create_engine
engine = create_engine('sqlite:///:memory:', echo=True)

# Create some database tables 
from sqlalchemy import Table, Column, Integer, String, MetaData, ForeignKey
metadata = MetaData()
users = Table('users', metadata,
    Column('id', Integer, primary_key=True),
    Column('name', String),
    Column('fullname', String),
)

addresses = Table('addresses', metadata,
  Column('id', Integer, primary_key=True),
  Column('user_id', None, ForeignKey('users.id')),
  Column('email_address', String, nullable=False)
)

metadata.create_all(engine)

# Insert into the tables
ins = users.insert()
ins.values(name='jack', fullname='Jack the Pumpkin king')

conn = engine.connect()
result = conn.execute(ins)
result.inserted_primary_key

# Multiple insert into the tables
conn.execute(addresses.insert(), [
   {'user_id': 1, 'email_address' : 'jack@yahoo.com'},
   {'user_id': 1, 'email_address' : 'jack@msn.com'},
   {'user_id': 2, 'email_address' : 'www@www.org'},
   {'user_id': 2, 'email_address' : 'wendy@aol.com'},
])

result = engine.execute(users.insert(), name='fred', fullname="Fred Flintstone")

# Select from the tables
from sqlalchemy.sql import select

# Select all fields in user table
s = select([users])
result = conn.execute(s)
list(result)

# Select name and fullname columns from user table
s = select([users.c.name, users.c.fullname])

# Join two user and address tables
#  Cartesian product 
list(conn.execute(select([users, addresses])))
# Join on id columns
list(conn.execute(select([users, addresses], users.c.id==addresses.c.user_id)))

# create conditions for the where clause
from sqlalchemy.sql import and_, or_, not_
print and_(users.c.name.like('j%'), users.c.id==addresses.c.user_id, )


s = select([(users.c.fullname + "--" + addresses.c.email_address).label('title')])
print conn.execute(s).fetchall()

print users.join(addresses)
print users.join(addresses, addresses.c.email_address.like(users.c.name + '%'))


s = select([users.c.fullname], from_obj=[
   users.join(addresses, addresses.c.email_address.like(users.c.name + '%'))
   ])
print conn.execute(s).fetchall()

# Delete records based on a condition
conn.execute(users.delete().where(users.c.name > 'm'))

Django: Quickly testing an authenticated ajax

Make a authenticated post with the django test client and mark simulate an ajax post:
from django.test import Client

c = Client()

c.login(username='jjasinski', password='xxxx')

print c.post('/user/posts/1/',{},HTTP_X_REQUESTED_WITH='XMLHttpRequest')

Self-signed SSL Certificate

Generate self-signed certificate
# Generate Encrypted RSA Private Key with passphrase
openssl genrsa -des3 -out myssl.key 2048

# Generate Unencrypted RSA Private Key 
openssl genrsa -out myssl.key 2048

# Remove Passphrase from key

mv myssl.key myssl.key.org
openssl rsa -in myssl.key.org -out myssl.key

# Generate Certificate Signing Request with an existing Private Key

openssl req -sha256 -new -key myssl.key -out myssl.csr

# Create SSL certificate
openssl x509 -req -days 365 -in myssl.csr -signkey myssl.key -out myssl.crt

Generating a key and csr in one command

# for providing to your certificate provider

openssl req -sha256 -new -newkey rsa:2048 -nodes -keyout yourdomain.key -out yourdomain.csr

   req = create PKCS#10 X.509 Certificate Signing Request
   -sha256 = adds support for SHA-2

Decode PEM encoded SSL certificate meta information
openssl x509 -in certificate.crt -text -noout
OR
openssl x509 -noout -text -modulus -in https://example.com.crt

Decode DER encoded SSL certificate meta information
openssl x509 -in certificate.crt -inform der -text -noout

Decode SSL key meta information
openssl rsa -noout -text -modulus -in https://example.com.key

Convert PEM encoded SSL certificate to DER encoded:
openssl x509 -in certificate.crt -outform der -out certificate.der

Convert DER encoded SSL certificate to PEM encoded:
openssl x509 -in certfile.crt -inform der -outform pem -out certificate.pem


Note: in the meta info, the Modulus should be the same for the key and cert.


Sources
http://articles.slicehost.com/2007/12/19/ubuntu-gutsy-self-signed-ssl-certificates-and-nginx
http://support.godaddy.com/help/article/3601/generating-a-certificate-signing-request-nginx
http://www.sslshopper.com/certificate-decoder.html
https://www.digitalocean.com/community/tutorials/openssl-essentials-working-with-ssl-certificates-private-keys-and-csrs
https://support.ssl.com/Knowledgebase/Article/View/19/0/der-vs-crt-vs-cer-vs-pem-certificates-and-how-to-convert-them

Using Rsync to copy file Structure

Simple rsync copy on local filesystem:
rsync -r -t -v -l source/ dest

rsync over ssh:
rsync -r -t -v -l username@source.com:/path/to/src/ /path/to/dest

rsync copy shortcut to archive:
rsync -avz username@source.com:/path/to/src/ /path/to/dest
rsync -avz --progress username@source.com:/path/to/src/ /path/to/dest

rsync copy archive without caring about owner and group:
rsync -rlptD  username@source.com:/path/to/src/ /path/to/dest
rsync -a --no-o --no-g username@source.com:/path/to/src/ /path/to/dest

Common Flags:
 -a = archive copy.  Equivalent to flags -rlptgoD
 -r = recursive copy
 -t = save timestamp
 -v = verbose copy
 -vv = more verbose
 -u = don't copy over existing files newer than source
 -l = preserve symlinks
 -p = preserve premissions
 -g = preserve group
 -o = preserve owner
 -D = preserve devices (superuser only)
 -z = compress transfer
 --progress = show copy progress
--delete = delete stuff on the receiving side that doesn't exist on the sending side

http://www.cyberciti.biz/tips/linux-use-rsync-transfer-mirror-files-directories.html
http://en.wikipedia.org/wiki/Rsync
http://serverfault.com/questions/364709/how-to-keep-rsync-from-chowning-transfered-files

Git INFO Common Commands

Configure Git client options:
# show config options
git config --list # show all
git config --global --list # show only global
git config --local --list # show only local

# --global option impacts options at a global level
# --local option impacts options at a local level (.git/config)
# set username/email associated with client
git config --global user.email "joe@example.com"
git config --global user.name "Joe Jaz"

# add color syntax highlighting
git config --global color.status auto
git config --global color.branch auto

# set default pager and editor
git config --global core.editor vim # or set the GIT_EDITOR environment variable
git config --global core.pager less # or set the GIT_PAGER environment variable

# display an option value
git config group.value
git config user.name

Start a new repository and import:
# In the root folder to add to version control:
git init
git init --bare  # creates a new repository (with no working copy)
git add . # add everything in directory to repo
git commit -m 'initial commit'

Making commits:
# commit specific files
git commit file1 file2 -m 'comment'

# commit all 'added' files
git commit -a -m 'comment'

# change a commit message
git commit --amend -m 'New Message'

Create new repository by cloning existing one:
git clone --bare . path_or_url_to_git_dir.git

Setting the remote repository (to push and pull from):
git remote add short_name path_or_url_to_git_dir.git
git remote add origin git://github.com/imagescape/iscape-authlog.git

Show remote repositories available:
git remote
git remote -v # show the url
git remote show origin  # show detail information about origin

Remove a remote repository: 
git remove repo_name

Rename a remote repository: 
git rename old_repo_name new_repo_name

Push and Pull from remote repository:
git push origin # push recent commits to remote repository called "origin"
git pull origin # pull updates from origin

Revert changes in working copy:
git clean -n
git clean --dry-run

Show file differences:
git diff   # show differences between working copy and any files staged for commit
git diff --staged   # compares staged changes with last commit
git diff --cached  # same as above for git older than 1.6.1

git difftool -t [meld|kdiff3|vimdiff|etc.] [diff criteria]
git difftool -t meld --staged
git difftool --tool=meld --staged
git difftool origin/master  --dir-diff # use difftool to diff entire dir tree

git diff -U999999999 master.. -- file.txt # shows the full file.txt, with diff markup, in the master/branch diff results

git diff --name-status brancha..branchb # shows list of changed files

Show File/s History:
git log filename # Show single file versions

# Show number of lines changed in file/s
git log -stat filename # single file
git log -stat # multiple files

# show log summaries on one line
git log --pretty=oneline # Pre-defined format
git log --pretty=format:"%h - %cD (%an) %s" # custom format (see below)
git log --pretty=format:"%h %s" --graph
git log --graph --full-history --all --color \

           --pretty=format:"%x1b[31m%h%x09%x1b[32m%d%x1b[0m%x20%s"
    Pre-Defined Pretty formats:
    oneline
    short
    medium
    full
    fuller
    email
    raw
    format:string # where string is

        Format Strings
        %H: commit hash
        %h: abbreviated commit hash
        %T: tree hash
        %t: abbreviated tree hash
        %P: parent hashes
        %p: abbreviated parent hashes
        %an: author name
        %aN: author name (respecting .mailmap, see git-shortlog(1) or git-blame(1))
        %ae: author email
        %aE: author email (respecting .mailmap, see git-shortlog(1) or git-blame(1))
        %ad: author date (format respects --date= option)
        %aD: author date, RFC2822 style
        %ar: author date, relative
        %at: author date, UNIX timestamp
        %ai: author date, ISO 8601 format
        %cn: committer name
        %cN: committer name (respecting .mailmap, see git-shortlog(1) or git-blame(1))
        %ce: committer email
        %cE: committer email (respecting .mailmap, see git-shortlog(1) or git-blame(1))
        %cd: committer date
        %cD: committer date, RFC2822 style
        %cr: committer date, relative
        %ct: committer date, UNIX timestamp
        %ci: committer date, ISO 8601 format
        %d: ref names, like the --decorate option of git-log(1)
        %e: encoding
        %s: subject
        %f: sanitized subject line, suitable for a filename
        %b: body
        %B: raw body (unwrapped subject and body)
        %N: commit notes
        %gD: reflog selector, e.g., refs/stash@{1}
        %gd: shortened reflog selector, e.g., stash@{1}
        %gs: reflog subject
        %Cred: switch color to red
        %Cgreen: switch color to green
        %Cblue: switch color to blue
        %Creset: reset color
        %C(…): color specification, as described in color.branch.* config option
        %m: left, right or boundary mark
        %n: newline
        %%: a raw %
        %x00: print a byte from a hex code
        %w([[,[,]]]): switch line wrapping, like the -w option of git-shortlog(1).

Show git branches available:
git branch

Show all branches including remote branches:
git branch -a

Create new branch:
git branch new_branch

Change branches:
git checkout new_branch

Delete branch:
git branch -d new_branch # delete only if all changes are merged
git branch -D new_branch  # delete even if there are unmerged changes

Merge branch into current copy:
git merge --no-ff branch_to_merge_in

Resolve a Merge:
1) Edit file to resolve
2) git add conflict_file
3) git commit

Using Mergetool
git mergetool
git mergetool --tool=meld
git mergetool -t meld

Abort a merge conflict before committing:
git merge --abort

Undo a Merge:  # restores state to pre-merge
git reset --heard HEAD

Undo a local commit: 
git reset --soft HEAD^

Find the commit where two branches diverge:
git merge-base branch1 branch2

    http://stackoverflow.com/questions/1549146/find-common-ancestor-of-two-branches

Find what branches were merged into a given branch:
git branch --merged master  # local branches only
git branch -a --merged master   # local and remote branches

Find what branches were NOT merged into a given branch:
git branch --no-merged master  # local branches only
git branch -a --no-merged master   # local and remote branches

Find what branches are merged into develop, but not master:
comm -12 <(sort <(git branch -a --no-merged origin/master)) <(sort <(git branch -a --merged origin/develop))

    https://stackoverflow.com/questions/8071079/git-list-branches-merged-into-a-branch-but-not-into-another

Copy all branches from origin to new_remote: 
git push new_remote refs/remotes/origin/*:refs/heads/*

Get number of commits per author
apt-get install git-extras

git summary -n

References:
http://jonas.nitro.dk/git/quick-reference.html
http://www.vogella.de/articles/Git/article.html#git_stagingindex
http://book.git-scm.com/3_basic_branching_and_merging.html
http://stackoverflow.com/questions/1838873/visualizing-branch-topology-in-git
http://stackoverflow.com/questions/226976/how-can-i-know-in-git-if-a-branch-has-been-already-merged-into-master
http://stackoverflow.com/questions/7818927/git-push-branch-from-one-remote-to-another

Python Process Info

Pipe the output of one process into the input of another via python:
import subprocess
p1 = subprocess.Popen(['echo','joe'], shell=False, stdout=subprocess.PIPE)
p2 = subprocess.Popen(['cat','-n'],
    stdin=p1.stdout, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
p2.stdout.read()

# Or 
out,err=p2.communicate()

Pipe standard input to a program
import subprocess
p1 = subprocess.Popen(['node', 'index.js'], shell=False,
    stdin=subprocess.PIPE, stdout=subprocess.PIPE, stderr=subprocess.PIPE)
stdout, stderr = p1.communicate('standard input here')
p1.returncode

Convenience method for splitting apart shell commands for use in the above:
>>> import shlex
>>> shlex.split('ps -ef')
['ps', '-ef']

Common Open ID Urls

Taken From http://stackoverflow.com/questions/1116743/where-can-i-find-a-list-of-openid-provider-urls

Google https://www.google.com/accounts/o8/id
Yahoo https://me.yahoo.com
Flickr http://www.flickr.com/username
AOL http://openid.aol.com/username
Blogspot https://www.blogspot.com/
LiveJournal http://username.livejournal.com/
Wordpress https://username.wordpress.com/
VerisignLabs https://pip.verisignlabs.com/
MyOpenID https://www.myopenid.com/
MyVidoop https://myvidoop.com/
ClaimID https://claimid.com/username
Technorati https://technorati.com/people/technorati/username/