Wednesday, August 07, 2024

systemd advanced topics

Basic Commands

# start / stop
systemctl start myservice
systemctl stop service

# reload after making changes to systemd unit files
# note: a reboot also does this
systemctl daemon-reload

# list dependencies of everything
systemctl  list-dependencies

# list dependencies for a given unit file
systemctl list-dependencies myservice.service

# list reverse dependencies  for a given unit file
systemctl list-dependencies myservice.service --reverse --all

Locations

Location of unit files
/etc/systemd/system/

Location of OS installed unit files (so users don't break the system)
/usr/lib/systemd/system/

Location of  runtime systemd unit files
/run/systemd/system

Various types of unit files ".target", ".service", ".path", ".mount", etc

.target = is like a "runlevel" - it lets you tie many unit files under it so you can enable/disable entire groups of things at once

.service = is a running process.

.path = lets you watch for changes to a path using inotify, and then trigger another unit file

.mount = lets you mount filesystems using systemd. /etc/fstab is automatically turned into unit files

and more...

Sections of a unit file

[Unit]  Metadata and trigger ordering dependencies

Description = description about the unit file

Before = ensures that units specified here are triggered before this unit

After = units listed here are triggered after this unit

Requires = list of units that must be running/completed before this one

Wants = similar to requires but ignores units that are not found or fail

[Install]  Metadata and trigger ordering dependencies

WantedBy = reverse dependency for "wants". units listed here must wait for this unit to run/complete before they can run

RequriedBy = reverse dependency for "requires".

Activating the Install section

Items listed in the [Install] section do not take affect until you run "systemctl enable myservice.service". Or conversely "systemctl disable myservice.service".  For example, say you have a unit file as such:

[Unit]
Description = myservice.service

[Install]
WantedBy = multi-user.target
RequiredBy = cloud-final.target


When you run "systemctl enable myservice.service", a symlink to this unit file will be created in /etc/systemd/system/multi-user.target.wants/ and in /etc/systemd/system/cloud-final.target/. This will make the dependency take effect.

Note: if the /etc/systemd/system/multi-user.target.wants/ dir does not exist, it will be created
Note: this also works for all types unit files (e.g, you can run 'systemctl enable myservice.path')

[Target | Service | Path | Mount | etc] Type specific unit file configuration

Services
 
[Service]
# Types:
# simple = "will consider the unit started immediately after the main service process has been forked off"
# oneshot = "is similar to simple; however, the service manager will consider the unit up after the main process exits."
Type = simple
 
# If True, service shall be considered active even when all its processes exited.
RemainAfterExit =
true

# command to run
ExecStart = /bin/bash -c 'echo hi'
 
ExecStartPre=...
ExecStartPost=...

Path
Waits for a path to create or update
https://www.freedesktop.org/software/systemd/man/latest/systemd.path.html


Conditionally starting a service

Two types: 
- Conditions fail (quietly) if a unit is run and a condition is not met.  
- Asserts fail and return an error

Run if file exists and not empty 

[Unit]
ConditionPathExists = /opt/empty.txt
ConditionFileNotEmpty = /opt/empty.txt

Run if file exists and not empty 

[Unit] ConditionPathExists =! /opt/empty.txt 

Run if either abc.txt or def.txt exists and user is root

[Unit] ConditionPathExists=| /tmp/abc.txt ConditionPathExists=| /tmp/def.txt ConditionUser = root

 
List of conditions
https://www.freedesktop.org/software/systemd/man/latest/systemd.unit.html#Conditions%20and%20Asserts 

Path unit: Running unit based on path

Example unit file

[Unit]
Description="Trigger unit based on /tmp/test path"

[Path]
PathModified=/tmp/test
# other options PathExists=, PathExistsGlob=, PathChanged=, PathModified=, DirectoryNotEmpty=

Unit=unit-to-trigger.service

[Install]
WantedBy=multi-user.target 
 

Friday, November 03, 2017

ssh-copy-id: add authorized_keys entry automatically


Copy authorized key to remote server:
ssh-copy-id -i ~/.ssh/id_rsa example.com

-i = private key associated with the public key that you would like to copy

Saturday, June 17, 2017

Postgresql: Read-Only User



create user xxxxx with password 'yyyy';
GRANT connect ON DATABASE mydb1 TO xxxxx;
GRANT connect ON DATABASE mydb2 TO xxxxx;

\c mydb1
GRANT usage ON SCHEMA public TO xxxxx;
GRANT select ON ALL TABLES IN SCHEMA public TO xxxxx;
GRANT select ON ALL SEQUENCES IN SCHEMA public TO xxxxx;

\c mydb2
GRANT usage ON SCHEMA public TO xxxxx;
GRANT select ON ALL TABLES IN SCHEMA public TO xxxxx;
GRANT select ON ALL SEQUENCES IN SCHEMA public TO xxxxx;

Sunday, June 12, 2016

Open Source Webservices


Version Control
Gitlab - https://about.gitlab.com/
Phabricator - http://phabricator.org/

Ticket Management (kanban)
Gitlab Kanban Board - http://kanban.leanlabs.io/
Taiga - https://taiga.io/
Kanboard - https://kanboard.net/

Group Chat
Mattermost - mattermost.org/ - also bundled with gitlab
Rocket.Chat - https://rocket.chat/
Let's Chat - http://sdelements.github.io/lets-chat/
Friends - http://moose-team.github.io/friends/
Zulip - https://www.zulip.org/

Compare: https://blog.okturtles.com/2015/11/five-open-source-slack-alternatives/
Amazon S3 Compatible Alternatives
Minio - https://www.minio.io/
Red Hat Ceph - http://ceph.com/

Password Generator
Diceware - https://www.rempe.us/diceware/#eff

Make Image out of code:
Cabon - https://carbon.now.sh

PaaS
openstack
Kubernetes
Kel - http://docs.kelproject.com/
Doku - http://dokku.viewdocs.io/dokku/
Flynn - https://flynn.io/
   More: https://www.quora.com/Is-there-open-source-software-that-implements-Amazon-S3-plug-compatible-storage

Malicatcher
Send email to here for debugging
https://hub.docker.com/r/schickling/mailcatcher/
https://mailcatcher.me/



Friday, March 04, 2016

Make Unix Password Hash

mkpasswd --method=SHA-512

Webservices


Email Spam Checker
http://www.mail-tester.com/

Git Hosting
Gitlab - https://about.gitlab.com/ (open source)
Phabricator - http://phabricator.org/ (open source)

Group Chat
Mattermost - http://www.mattermost.org/ (open source)
Slack - http://www.json-generator.com/
HipChat - https://www.hipchat.com/
Rocket - https://rocket.chat/features (open source)

Javascript Editor
jsfiddle https://jsfiddle.net/
Codepen - http://codepen.io/
jsbin - https://jsbin.com/ (open source)

JSON Generator
http://www.json-generator.com/

Online IDE
Cloudnine - https://c9.io/ (open source)

HTML/CS/JS Cleanup
http://www.dirtymarkup.com/

SSL Certificate Checker
https://www.ssllabs.com/ssltest

SSL Check Intermediate Certificate Chain
https://www.sslshopper.com/ssl-checker.html

Fake Email Tester (Send emails to here on dev and view them)
https://mailtrap.io/

Terminal Emulator:
https://hyperterm.org/ (open source)

Note Taking Desktop App
Boostnote - https://b00st.io/ (open source)

Python/Django Storage Backend Notes

Inspired by:
https://django-storages.readthedocs.org/en/latest/backends/amazon-S3.html#storage

Interacting directly with storage backend
from django.conf import settings
from django.core.files.storage import get_storage_class
STORAGE_CLASS_STRING = getattr(settings, "MY_STORAGE_CLASS",  \
        settings.DEFAULT_FILE_STORAGE)
sc = get_storage_class(STORAGE_CLASS_STRING)
s = sc()
s.url("jjj-test/JOE.jpg")
# '/media/jjj-test/JOE.jpg'
s.path("jjj-test/JOE.jpg")
# u'/home/jjasinski/Sites/mysite/htdocs/media/jjj-test/JOE.jpg'
s.exists("jjj-test/JOE.jpg")
# False
f = s.open("jjj-test/JOE.jpg", 'w')
f.write("joe test")
f.close()
s.exists("jjj-test/JOE.jpg")
# True
s.delete("jjj-test/JOE.jpg")
s.exists("jjj-test/JOE.jpg")
# False

Interacting with a Model's storage
from django.core.files.base import ContentFile
obj = MyModel()
obj.photo.save('django_test.txt', ContentFile('content'))
obj.photo.size
obj.photo.read()
obj.delete()

Wednesday, December 09, 2015

Django: staticfiles

Locate static file location in codebase:

>>> from django.contrib.staticfiles.finders import find

>>> find("img/logo.png", all=True)
[u'/sites/static_in_pro/our_static/img/logo.png']

>>> find("img/logo.png",)
u'/sites/static_in_pro/our_static/img/logo.png'

Monday, December 07, 2015

LetsEncrypt with HAProxy or Nginx


At this time, LetsEncrypt is in public beta, but I suspect that it will continue to evolve. 

# all commands must be done as root
sudo su

# Download the letsencrypt repo 
git clone https://github.com/letsencrypt/letsencrypt.git /top/letsencrypt/

# change to the desired keys directory.  All commands following are relative to this dir.
cd /jaz/sites/common/etc/keys/

# Generate a 4096 bit ssl private key
openssl genrsa 4096 > jazstudios.com.key

# Generate the certificate signing request.  The following allows lets you specify a SAN (Subject Alternative Name) which allows www and non-www versions of the same domain.  The output needs to be in "der" format. 

openssl req -new -sha256 \
    -key joejasinski.com.key \
    -subj "/C=US/ST=IL/L=Chicago/O=Jazstudios/OU=Information Technology/CN=www.joejasinski.com" \
    -reqexts SAN \
    -outform der \
    -config <(cat /etc/ssl/openssl.cnf \
        <(printf "[SAN]\nsubjectAltName=DNS:joejasinski.com,DNS:www.joejasinski.com")) \
    -out joejasinski.com.csr

# execute the letsencrypt command.  This will prompt you through a few actions.  The most important is that you will need to stop any server running on port 80 and run the python script that they provide in the output.  This will serve up a specific secret file at a specific location, allowing letsencrypt to authenticate the server.  (You could also host the secret file with your webserver)

/opt/letsencrypt/letsencrypt-auto --email example@gmail.com --text \
    --authenticator manual \
    --work-dir /tmp/work/ \
    --config-dir /tmp/config/ \
    --logs-dir /tmp/logs/ auth \
    --cert-path /tmp/certs/ \
    --chain-path /tmp/chains/ \
    --csr joejasinski.com.csr

# --text = use the text based 'wizard' installer instead of an ncurses one
# --authenticator manual = the admin must manually host the verification file
# --csr = path to the previously generated csr file


# The command that it will have you run looks something like this:

mkdir -p /tmp/letsencrypt/public_html/.well-known/acme-challenge
cd /tmp/letsencrypt/public_html
printf "%s" asdfkjasfdasfdasfdasfdasdf > .well-known/acme-challenge/asdfasdfasfd
# run only once per server:
$(command -v python2 || command -v python2.7 || command -v python2.6) -c \
"import BaseHTTPServer, SimpleHTTPServer; \
s = BaseHTTPServer.HTTPServer(('', 80), SimpleHTTPServer.SimpleHTTPRequestHandler); \
s.serve_forever()"

# The output of the of the letsencrypt-auto command will be a file called 0000_chain.pem.  This file contains the host certificate and the intermediate certificate.  It will look something like this.  

-----BEGIN CERTIFICATE-----
       Host certificate contents
-----END CERTIFICATE-----
-----BEGIN CERTIFICATE-----
       letsencrypt intermediate certificate
-----END CERTIFICATE-----


# Unrelated, but a good idea: generate a dhparam used for perfect forward security 

openssl dhparam -out dhparam.pem 4096

For hosting with Nginx, this file can be set to the ssl_certificate parameter. The ssl_certificate_key setting would be set to the location of the key file.

For hosting with HAProxy, you want to modify the file so it looks something like this:

-----BEGIN CERTIFICATE-----
       Host certificate contents
-----END CERTIFICATE-----
-----BEGIN RSA PRIVATE KEY-----
       Private key contents
-----END RSA PRIVATE KEY-----
-----BEGIN CERTIFICATE-----
       letsencrypt intermediate certificate
-----END CERTIFICATE-----
-----BEGIN DH PARAMETERS-----
       contents of dhparam.pem
-----END DH PARAMETERS-----