Setting up Gallery2+PostgresQL+PHP5 on Debian Etch

The Gallery2 package for Debian Etch seems to be very well done, except one thing - it's geared towards MySQL usage (the Gallery makers recommend MySQL because they say that they use some special MySQL constructs which have to be emulated on other databases).
My view on this is:

• I prefer PostgresQL - it's a more mature database.


• I prefer Apache2


• I prefer PHP5 - I hate this language and think it has tendency to push poor programmers to shoot themselves (and their users) in their feet but if I have to install it on my machine then I'd rather use the latest and greatest.


• I prefer mod-php5 as opposed to the other ways to integrate Apache2 with PHP.

In addition, the Debian packages state their dependencies in a way that seems to encourage Aptitude to choose Apache (1), MySQL and PHP4.

So first thing first: After picking gallery2 for installation in Aptitude, go through all the packages it depends on and remove the ones you don't want and which have other alternatives. Get rid of the PHP4 and MySQL dependencies - Aptitude makes it pretty easy by simply making sure that you choose other packages to satisfy the dependencies and remove the rest. Just follow the suggestions made by Aptitude when you ask to remove a package with dependencies on it, and make sure that Aptitude doesn't signal "Broken Packages" before going ahead to execute the changes.

Setup a separate user for gallery:
$ sudo adduser --system --group gallery

Does the trick - add a system user (with a shell of /bin/false), create a home directory for it and a group especially for it.

The following should be pretty self-explanatory but the summary is: Create a new role ("user"), create a database, set a password, allow role to be logged in:
$ sudo -u postgres psql
postgres=# create role gallery;
postgres=# create database gallery owner gallery;
postgres=# alter role gallery encrypted password 'password';
postgres=# alter role gallery login;
postgres=# \q
Allow logging in to role "gallery" from another user without having to run psql as the UNIX user "gallery" by adding the following lines to the end of /etc/postgresql/8.1/main/pg_hba.conf:

local gallery gallery md5

And I also had to enable access to Gallery2's index.php by adding "Indexes" to the Options line in /etc/apache2/conf.d/gallery2 (link to /etc/gallery2/apache.conf

After that I could access my web server's "/photos/" and start the automatic configuration.

How to block Skype traffic with one iptables rule

There is an interesting lecture (PDF) by Philippe Biondi and Fabrice Desclaux about how and what they learned about the Skype client code and protocol. On page 75 they give the following secret incantation for blocking Skype traffic:
iptables -I FORWARD −p udp −m length −−length 39 −m u32 −−u32 ’27&0x8f=7’ −−u32 ’31=0x527c4833’ −j DROP

xdns - archeological diggings

Finally I re-found my own DNS Library source still almost at the same place where I left it almost 15 years ago.

Granted - the "xdns" program around it is pretty useless today, and the dns packet creation and parsing library is a classic candidate for C++ -style Object-based interface, but the basics are still there and might be useful for others who still have to deal with raw DNS packets...

SSH breaks when the loopback device isn't configured

Helping others troubleshooting is great fun and very rewarding - you get to learn some new things yourself.

In this case, this chap had a trouble getting X11 forwarded over an SSH tunnel. Everything looked fine, the same setup on another machine worked just as it should, all the flags (X11Forwarding) were set correctly but still the $DISPLAY envariable wasn't set on the remote side.

The bottom line - the loopback device ("lo") wasn't configured. This can be checked with a simple "ifconfig".

The break-through in the investigation came when he executed sshd in debug mode and received the following errors:

debug2: bind port 6999: Cannot assign requested address
...
Failed to allocate internet-domain X11 display socket.

Some googling revealed the following post, which actually gave the answer (so at least some of the credit should go to Jim Prewett, who bothered to document his findings four years ago): http://marc.theaimsgroup.com/?l=openssh-unix-dev&m=104336969724537

WARNING: You have to run sshd in debug mode on an alternate port because sshd debug mode will only serve one connection then exit, so you have to keep your existing non-debug sshd running or else you'll lock yourself out.

Parallel xargs and faster ssh connection initiation

You learn something new every day, even with old friends such as ssh and xargs.

SSH: It turns out that it's possible to create a live "Master" connection to a specific remote host which then can be used by other ssh command executions to quickly open sub-channels to that host without going through the authentication process every time and without compromising on security.

Example:

$ ssh -fMN -o ControlPath=~/.ssh-control-sock remote-host
$ ssh -o ControlPath=~/.ssh-control-sock remote-host date

more about this in ssh(1) and ssh_config(5)

XARGS: In response to some Digg pointer to xjobs, which seems to be similar to xargs only it can run multiple jobs in parallel, someone pointed out to xargs' own -P (--max-procs) argument which does exactly the same. Just have to remember to limit the number of file names passed to each job using -n (--max-args), otherwise all the file names will be passed to a single job.

The final IPTables setup and some initial results

Hi and welcome back. I've just realized that I haven't published the final solution for data gathering. So in this post I'll:

1. Describe the IPTables setup
2. Show the initial script I wrote to read the results
3. Show some preliminary raw data

IPTables Setup
I've changed the setup described in the first post about the subject to also add the "foreign" side of each identified Skype connection to the "recent" list (using the "recent" module). That way when an ICMP packet arrives from a host in that list I assume that it's related to Skype (I guess there are very slim chances to have Skype traffic with a server such as web hosts).

Here is the updated setup:

# match all outgoing packets from gid skype, mark their connection
# and add their destination to the "recent list" so we can count ICMP packets to/from them
iptables -A OUTPUT -m owner --gid-owner skype --out-interface eth0   --protocol tcp -m recent --rdest --set --name Skype -j CONNMARK --set-mark 1
iptables -A OUTPUT -m owner --gid-owner skype --out-interface eth0   --protocol udp -m recent --rdest --set --name Skype -j CONNMARK --set-mark 2

# count ICMP packets going to hosts which appear in our "recent" list
iptables -A OUTPUT --out-interface eth0 --protocol icmp -m recent --rdest   --name Skype --update -j ACCEPT -m comment --comment skype-out-icmp

# all packets which match the connection should go through the skype rule
iptables -A OUTPUT -m connmark --mark 1 -m comment --comment skype-out-tcp
iptables -A OUTPUT -m connmark --mark 2 -m comment --comment skype-out-udp

# match all packets on Skype's public TCP port and mark their connection
iptables -A INPUT -p tcp -m tcp --dport 21212 --in-interface eth0 -j CONNMARK --set-mark 1
iptables -A INPUT -p udp -m udp --dport 21212 --in-interface eth0 -j CONNMARK --set-mark 2
# count ICMP packets coming from hosts which appear in our "recent" list
iptables -A INPUT -p icmp --in-interface eth0 -m recent --name Skype   --update -j ACCEPT -m comment --comment skype-in-icmp

# all packets which match the connection
iptables -A INPUT -m connmark --mark 1 -m comment --comment skype-in-tcp
iptables -A INPUT -m connmark --mark 2 -m comment --comment skype-in-udp

The counter reading script was modified to add "icmp" to the list of protocols it looks for and to total bytes/packets over the various planes: direction (total in vs. total out) and protocol (tcp vs. udp vs. icmp)

Here is what the counters look like after about 25 days of data gathering:

$ sudo ./getcounts.pl
tcp_out_bytes   35838386
icmp_out_bytes  155016
tcp_in_pkts     441671
icmp_out_pkts   1023
icmp_in_pkts    4526
udp_out_bytes   242046393
tcp_out_pkts    540629
icmp_in_bytes   505522
udp_out_pkts    1799468
udp_in_pkts     1607313
udp_in_bytes    204286584
tcp_in_bytes    42500198
====== totals =====
tcp_pkts        982300
tcp_bytes       78338584
icmp_pkts       5549
out_pkts        2341120
icmp_bytes      660538
udp_bytes       446332977
in_bytes        247292304
udp_pkts        3406781
out_bytes       278039795
in_pkts         2053510

I'm still looking for time to add writing of these numbers into an RRD file so it'll be possible to graph them across different periods, but for now my simple conslusion is about the "in_bytes" numbers (and to a lesser degree, the "out_bytes"): they are 235.8 incoming mega bytes and 265.1 outgoing mega bytes.

Over a period of 25 days this puts it at around 9.4 incoming mega bytes per day and 10.6 outgoing megabytes per day. Over a month (let's say it's 30 days) it's 283.0 mega bytes per month of incoming traffic and 318.2 mega bytes of outgoing traffic. This includes my own Skype conversations (admittedly, not much this month).

Whether this proves Cringley's point or not? I'm not sure. I didn't believe that there is that much traffic involved until I startted this experiment, but I'm still not completly convinced it's "too much to handle".

In my personal context it's still less than 2% (1.38%, to be precise) of my download quota of 20Gb per month.

So for now I'm not going to give up on the advantages of a smoother connection (which is the reason I configured my desktop as a "Super-node" in the first place).

I'd be glad to learn from you what you think about the experiment (have I missed some packets?) and the result - do you agree with my conclusion so far or not?

Work journal - u32 isn't going to make it(?)

I was planning to use the IPTables U32 module to look at the "original packet" which is included in the ICMP packets destined to Skype-related connection attempts but this might not cut it because there are other ICMP messages involved, not just the "port unreachable" I saw at first.

Instead, now I plan to use the "recent" module.

(Note: try running "iptables -m recent -h" to list options not mentioned in the manual. Apparently it's a good thing to do with any IPTables module)

The "recent" module is designed with "keep the bad guys out" situation in mind (by adding attacker's source address to a tempoary list) but with the "secret" --rdest option, which allows me to add destination addresses of outgoing packets, it might be possible to add the IP of any host with which Skype have just attempted to converse to a temporary cache which will allow this host to send back errors and which will automatically expire in a preset time.