Blog

MySQL has many odd quirks. One that bit us recently was this: regression: >=mysql-5.4 utf8 collations are marked as not ascii compatible When using the utf8_unicode_ci collation, datetime column comparisons against strings would ignore any indexes. The lack of working indexes obviously caused huge performance degradation. Our bug report was ignored in Februari. Apparently a new bug was opened in March: Datetime field comparisons do not work properly with utf8_unicode_ci collation

Mozilla Thunderbird uses an odd max line length of 999 + CRLF: 1001 characters. When using DKIM preprocessing, this can result in DKIM validation failure. To reproduce, we would send a mail that didn’t wrap well with line lengths in excess of 999 characters. Like this mail with 1000 'x' characters: Thunderbird splits that into 999 times 'x', and CRLF and another line with a single 'x'. However, that first line gets split again.

Numerous articles have been written about why you want to install Pillow instead of PIL to get the Python Imaging tools. Like Problems with PIL? Use Pillow instead! (Find more by searching for “IOError: decoder zip not available”.) This note concerns something more insidious: a seemingly broken Pillow installation after the removal of PIL. ~$ mkvirtualenv piltest (piltest)~$ pip install PIL (piltest)~$ pip freeze | grep -i pil PIL==1.1.7 Now this should work:

Caveats arbiter - 10.10.10.1 node1 - 10.10.10.10 node1 - 10.10.10.11 Installing the nodes update /etc/hosts to make sure all the machines are resolvable, perform this on all 3 hosts (2 active mysql nodes and the machine the arbiter resides on). Update the hostnames and addresses accordingly. $ cat << EOF >> /etc/hosts 10.10.10.1 arbiter 10.10.10.10 node1 10.10.10.11 node2 EOF Setup APT to use the MariaDB repositories

Just a quick post with the most straight forward way to configure http webservers/proxies in regard to ssl certificate chains. I’m open for improvements if it improves readability or reduces linecount. :-) pound In /etc/pound/pound.cfg you only need one line in the HTTPS listener: ListenHTTPS Cert "/etc/pound/file-with-key-and-cert-chain.pem" apache2 SSLCertificateFile /path/to/file-with-key-and-cert-chain.pem SSLCertificateChainFile /path/to/file-with-key-and-cert-chain.pem

Most browser trusted SSL certificates work with intermediate certificates nowadays. The CA only ships the root certificates for inclusion with the browsers and because they do this your certificate can’t be verified if you don’t include the intermediate certificates in your webserver configuration. chain order So you’re in the bottom in a chain of trust. Your certificate is signed by a certificate which is signed by another certificate and so on.

Of course you do daily backups of your data. For your databases, you generally need a dump of the data for it to be useful. For your and my reference, here are a few database dump scripts. Make sure /var/backups is backed up daily. Observe that we keep extra backups around. I’ve found that the need to access an old database is far more common than accessing old files (from the backup storage).

So, there have been numerous reports around the internet that the HP ProBook 4510s has fan speed issues. In my case, the following happens: Turn on the laptop: no problem. Using the laptop while on AC: no problem. Switch on the laptop from suspend mode without AC plugged in: problem! The fan goes into overdrive and stays there. It's related to the temperature measurements, somehow. According to the sensors(1) tool, it's the temp6 value that's off.

If you add new disks to a virtual machine you don’t have to reboot to be able to use them. I assume this works the same for normal hotswappable disks as well but that just doesn’t happen too often these days. Didn’t test for all drivers/use cases but the example below is in a VMWARE environment. It has to be handled as scsi by the kernel obviously. check for current disks:

How to I grant myself permissions to show MySQL FUNCTION and PROCEDURE bodies? It isn’t this: mysql> grant all privileges on procedure *.* to walter; ERROR 1144 (42000): Illegal GRANT/REVOKE command; please consult the manual to see which privileges can be used But it is this: mysql> grant select on mysql.proc to walter; Query OK, 0 rows affected (0.00 sec)

Does Python Twisted pass the parent environment to child processes? By default no, but if you pass env=None then it does. Ergo, default is env={}. Let’s build a quick example. For those unfamiliar with twisted this may provide a quick intro. import os from twisted.internet import protocol, reactor, utils This is what we’re going to “run”: proc = ['/bin/sh', '-c', 'export'] #kwargs = {'env': None} kwargs = {} A quick way to show output and end after the 2 runs.

There is no way to “block” signals temporarily from critical sections (since this is not supported by all Unix flavors). Says the python signal module manual. But I’m using Linux, where it is possible to block signals, so I don’t think that limitation applies. And it doesn’t. pysigset takes the burden off calling sigprocmask(2) through ctypes and provides a “pythonic” interface to temporarily blocking signals in python. from signal import SIGINT, SIGTERM from pysigset import suspended_signals with suspended_signals(SIGINT, SIGTERM): # Signals are blocked here.

For someone who is used to using only GNU sed(1) it may come as a surprise that some of the metacharacters don’t work with sed on other OS’es. Specifically: sed on Darwin (BSD) does not grok \+, \| and \? when in basic regular expression mode (the default). Switching to extended (modern) regular expression isn’t a good idea if you’re aiming for compatibility, because the option -E differs from the GNU sed option -r.

For reuse, an OpenSIPS/Kamailio snippet to translate commonly used SIP (VoIP) phone Caller-ID (CLI) headers into a single one (Remote-Party-ID). It tries these headers in order, to do a best guess of what the caller wants: P-Asserted-Identity (with Privacy) P-Preferred-Identity (with Privacy) Remote-Party-ID From Of course you’ll have to run the found CLI against an allow list, but this code expects that to be done on the next hop. route[sub_cli_as_rpid] { $var(tmp_name) = ""; # (nothing) $var(tmp_num) = "Anonymous"; # (unknown) $var(tmp_clir) = 0; # PAI/PPI-privacy if (is_present_hf("Privacy")) { if (!

More or less useless/useful tips and tricks, bundled together. They weren’t worthy of a box div on their own. I gave them only a li each. kill -WINCH $$ — when your terminal is messed up where the row moves up one line before you’ve reached the line-length ($COLUMNS): a SIGWINCH signal to the current shell will make everything alright again. hash -r — you moved applications around in your $PATH and bash claims that some applications don’t exist in your $PATH even though you (and ls) know that they do:

When upgrading Ubuntu command line, many people probably still change the release name in the sources.list and update && dist-upgrade. Although is still works fine, Ubuntu also provides extras on this front. Upgrade the new way: sudo do-release-upgrade Whats the benefit? It provides things like a temporary SSH server and probably does other sanity checks to increase the chances of an successful upgrade and eases the troubleshooting if things go wrong.

One useful tool we recently started using is etckeeper. It provides version control over your /etc/ directory, which may prove quite useful when you maintain system in co-op way with your customers. Etckeeper also comes with hooks for apt, so even if you use it terribly it does give you an insightful history of when which package was installed. Installation and setup in Ubuntu: apt-get install git etckeeper # change VCS in /etc/etckeeper/etckeeper.

Building Canon MF8350Cdn (and other) CUPS drivers for Linux Debian and/or Ubuntu on amd64 is still a pain in the behind. Problems encountered during the installation, include: Regular make installed stuff in different places but forgot many parts. Solution: use dpkg-buildpackage The libtool copied wasn’t able to build for shared libs. Solution: remove --enable-shared/--disable-shared command line options. amd64 had to be added to the architecture targets. A shell script had to lose a bashism.

If I’m going to be requesting SSL certificates more often, I’d better automate the process a bit. The result: easycert.sh (view) Possible invocation styles: $ easycert.sh -h Usage: easycert.sh -c NL -l Groningen -o OSSO\ B.V. -e info@osso.nl osso.nl Usage: easycert.sh osso.nl "/C=NL/L=Groningen/O=OSSO B.V./CN=osso.nl/" Usage: easycert.sh -T www.osso.nl 443 Generating a key and certificate: $ easycert.sh -o "My Company" mycompany.com Subject: /C=NL/L=Groningen/O=My Company/CN=mycompany.com/emailAddress=info@osso.nl Enter to proceed... Generating RSA private key, 4096 bit long modulus .

So, what is the difference between uid and euid and the setuid and seteuid calls? Hao Chen, David Wagner and Drew Dean wrote an excellent paper called Setuid Demystified. It explains all the ins and outs. To answer the question, we need only parts of the article. Let the quoting begin. Each process has three user IDs: the real user ID (real uid, or ruid), the effective user ID (effective uid, or euid), and the saved user ID (saved uid, or suid).

After tweaking my postfix configuration, I apparently broke submission on port 587. Every time I connected, I immediately got: 554 5.7.1 <my.host.name[1.2.3.4]>: Client host rejected: Access denied That’s strange. Postfix is supposed to reject unauthenticated clients only in master.cf: submission inet n - n - - smtpd -o smtpd_tls_security_level=encrypt -o smtpd_tls_auth_only=yes -o smtpd_client_restrictions=permit_sasl_authenticated,reject But if it rejects me at connect time, I don’t have a chance to identify myself.

IPython after version 0.10 is not friends with older Django (e.g. 1.1.x) versions anymore. shell = IPython.Shell.IPShell(argv=[]) AttributeError: 'module' object has no attribute 'Shell' This is fixed in newer Django’s, but this isn’t backported. Here, a patch. (Nothing more than a diff between the old and the new Django version.) --- django/core/management/commands/shell.py 2012-03-28 16:10:28.000000000 +0200 +++ django/core/management/commands/shell.py 2012-01-24 10:27:50.405338739 +0100 @@ -8,9 +8,38 @@ help='Tells Django to use plain Python, not IPython.

Bertrand Le Roy describes how Microsoft added a Date object extension to JSON in a compatible fashion to implement serialization and serialization of timezone agnostic datetimes. Our current approach is using a small loophole in the JSON specs. In a JSON string literal, you may (or may not) escape some characters. Among those characters, weirdly enough, there is the slash character ('/'). This is weird because there actually is no reason that I can think of why you’d want to do that.

Does the Siemens Gigaset N300A handle SRV records? Yes it does.. but.. Let’s look at a bit of Gigaset DNS traffic: 09:24:26.059850 IP gigaset.local.32978 > nameserver.local.53: 50512+ NAPTR? gigaset.voip.example.com. (28) 09:24:26.061552 IP nameserver.local.53 > gigaset.local.32978: 50512 0/1/0 (87) 09:24:26.063894 IP gigaset.local.32978 > nameserver.local.53: 25738+ SRV? _sip._udp.gigaset.voip.example.com. (38) 09:24:26.064445 IP nameserver.local.53 > gigaset.local.32978: 25738 2/3/2 SRV proxy1.voip.example.com.:5060 10 0, SRV proxy2.voip.example.com.:5060 20 0 (231) 09:24:26.066939 IP gigaset.local.32978 > nameserver.local.53: 22676+ SRV? _sip.

How do you reply to a mailing list post when you do not have the mail in your INBOX? With Thunderbird it is easy enough, as long as you know how. How it works Mail threads are matched by comparing the In-Reply-To header with the Message-ID. Here’s an example from an Asterisk project reviewboard mailing: Date: Fri, 08 Jun 2012 08:08:42 -0000 Message-ID: <20120608080842.8103.32910@hotblack.digium.com> In-Reply-To: <20120607143847.27705.11556@hotblack.digium.com> References: <20120607143847.27705.11556@hotblack.digium.com> Subject: Re: [asterisk-dev] [Code Review] Fix issue of unrecognized inbound ACK when Asterisk responds to an INVITE with a 481 Every e-mail message has a globally unique message identifier.

So python’s base64 does not have a b85decode function? Adobe uses it the ASCII-85 encoding in PDF and PostScript files. Here is a quick and dirty one hacked together. See the wikipedia article for the ASCII-85 (base85) specs. Prologue; we only need sys to print a warning. # vim: set ts=8 sw=4 sts=4 et ai: # Example base85 decoder, Walter Doekes 2012 import sys Split the data up into 5-character chunks; 5 characters encode 4 octets.

For LibreOffice’s oocalc on latest Ubuntu (libreoffice-base 1:3.5.2-2ubuntu1 to take the locale settings into account for the date types, the LC_CTYPE needs to be set. $ LC_CTYPE=en_US.UTF-8 oocalc This causes a date input of 31-01-2012 to not get parsed as a date. $ LC_CTYPE=nl_NL.UTF-8 oocalc This causes the same input of 31-01-2012 to get properly understood as the DD-MM-YYYY format. That does not make sense. LC_CTYPE should be used for character classification, collation and case conversion.

So, I wanted to test video support with Asterisk. That was easier said than done, because the SIP softphones that ship with Ubuntu don’t all do what they promise. This was done on a setup that works for numerous hardphones and PBXs out there. Looking through the registration list at any given time reveals at least 40+ different user agents and a large multiple of that if you take the different versions into account.

Look at this snippet of Django code in models.py, and in particular the help_text bit: from django.db import models from django.utils.translation import ugettext_lazy as _ from django.utils.safestring import mark_safe class MyModel(models.Model): my_field = models.CharField(max_length=123, help_text=mark_safe(_('Some <b>help</b> text.'))) For those unfamiliar with Django. A quick run-down: The definition of MyModel creates a mapping between the MyModel class and a underlying app_mymodel table in a database. That table will consist of two columns: id, an automatic integer as primary key (created by default), and my_field, a varchar/text field of at most 123 characters.

The Ubuntu do-release-upgrade broke my ipython classic mode. The ipython package was upgraded, and apparently the configuration parser was changed. In bash, I want colors to help me find the beginning and end of output — see this bug report for others agreeing with me that the derogatory comment about “focus should be on the output, not on the prompt” in the skeleton .bashrc is is retared, but I diverge — in ipython, I just want to see the nice >>> blocks that I’m used to and no extra spaces.

An init.d script to stop and start safe_asterisk started asterisk. If asterisk is not stopped in 5 seconds, it is forcibly killed. safe_asterisk-init.d (view) # wget http://wjd.nu/files/2012/04/safe_asterisk-init.d -O/etc/init.d/asterisk ; chmod 755 /etc/init.d/asterisk Also possibly useful, the changes I made to safe_asterisk on a machine where: there wasn’t a tty left to spam output on, root is configured in /etc/aliases to a sane destination, /var/spool/asterisk is the asterisk user homedir anyway, and, attempting to set maxfiles to the highest value possible, wasn’t allowed.

Every one in a while, I see an unexpected 403 response to a SIP client’s REGISTER request. Thusfar the digest response calculation has never been wrong, but it feels good to get that check out of the way and move on to other possible causes. For your enjoyment and mine, a Bourne-shell compatible shell script that calculates (qop-less) Digest authentication responses. Download: hahacalc.sh (view) $ hahacalc Usage: hahacalc.sh USERNAME REALM METHOD DIGESTURI NONCE [PASSWORD] [COMPARE] Or: hahacalc.

If you’re switching from Ubuntu Oneiric to Ubuntu Precise and you’re using python-virtualenv, you might be in for a surprise: The default access to the global site-packages modules is reversed between virtualenv 1.6.x and 1.7. When you were used to finding your apt-get installed python modules like python-mysqldb and python-psycopg2 in your new virtualenv environment, now they’re suddenly unavailable. The culprit: --no-site-packages Ignored (the default). Don´t give access to the global site-packages modules to the virtual environment.

From the MySQL 5.1 manual: 15.4.1.16. Replication and REPAIR TABLE When used on a corrupted or otherwise damaged table, it is possible for the REPAIR TABLE statement to delete rows that cannot be recovered. However, any such modifications of table data performed by this statement are not replicated, which can cause master and slave to lose synchronization. For this reason, in the event that a table on the master becomes damaged and you use REPAIR TABLE to repair it, you should first stop replication (if it is still running) before using REPAIR TABLE, then afterward compare the master’s and slave’s copies of the table and be prepared to correct any discrepancies manually, before restarting replication.

Suppose I’m on machine DESKTOP and I want to copy files from server APPLE to server BANANA. DESKTOP has access to both, but firewalls and/or missing ssh keys prevent direct access between APPLE and BANANA. Regular scp(1) will now fail. It will attempt to do a direct copy and then give up. This is where this indirect scp wrapper (view) comes in: First, it tries to do the direct copy.

So, hardware trouble caused a VPS to go down. This VPS was running a MySQL server in a slave setup. Not surprisingly, the unclean shutdown broke succesful slaving. There are several possibly causes for slave setup breakage. This time it was the local relay log file (mysqld-relay-bin.xxxx) that was out of sync. SHOW SLAVE STATUS\G looked like this: ... Master_Log_File: mysql-bin.001814 <-- remote/master file (IO thread) Read_Master_Log_Pos: 33453535 <-- remote/master pos (IO thread) Relay_Log_File: mysqld-relay-bin.

Sometimes you’re in a situation where you know that a database is more heavily loaded than it should be. Time to figure out which queries are stressing it the most. The standard thing to do with a MySQL database would be to enable query logging with general_log_file. Or, to get only slow queries and those not using indexes, the log_slow_queries. But, if this is a mission critical and heavily loaded database, adding expensive logging may be just enough to give it that final push to become overloaded.

Just now, I tried to convert an integer column in a PostgreSQL database to one of type VARCHAR. I knew you had to do an explicit cast, so I was a bit stumped that I still wasn’t allowed to perform the ALTER TABLE. mydb=> ALTER TABLE mytable ALTER COLUMN mycolumn TYPE VARCHAR(31) USING mycolumn::text; ERROR: operator does not exist: character varying >= integer HINT: No operator matches the given name and argument type(s).

Some people seem to think that fixing the symptom is fixing the problem. import random def return_one_of(list): return list[random.randint(0, len(list))] def say_something(): try: print return_one_of(["Hello World!", "Hi!", "How you doin'?"]) except: return say_something() say_something() Gah! This is obviously an example, but there are people who do this and claim to have “fixed the problem”. Let me reiterate: the fact that your code does not raise any exceptions does NOT mean that it is not broken code!

The current django-mongodb-engine doesn’t seem to ship with a working manage dbshell command yet. Right now it returns this: $ ./manage.py dbshell ... File "/home/walter/.virtualenvs/myproject/lib/python2.6/site-packages/django/core/management/commands/dbshell.py", line 21, in handle connection.client.runshell() File "/home/walter/.virtualenvs/myproject/lib/python2.6/site-packages/django_mongodb_engine/base.py", line 108, in __getattr__ raise AttributeError(attr) AttributeError: client The fix is simple, patch your django_mongodb_engine with this: --- django_mongodb_engine/base.py.orig 2011-11-15 11:53:47.000000000 +0100 +++ django_mongodb_engine/base.py 2011-11-15 11:54:07.000000000 +0100 @@ -7,6 +7,7 @@ from pymongo.connection import Connection from pymongo.collection import Collection +from .

So. SSL certificates are still black magic to me. Especially when they cause trouble. Like when one of the sysadmins has forgotten to add the certificate bundle to the apache2 config. Then you get stuff like this: $ hg pull -u abort: error: _ssl.c:503: error:14090086:SSL routines:SSL3_GET_SERVER_CERTIFICATE:certificate verify failed Most web browsers do not notice this as they already have the intermediate CA files, but /etc/ssl/certs/ca-certificates.crt seemingly doesn’t. The problem in this case was not that I was missing any certificates locally.

You may not always have gdb(1) at hand. Here are a couple of other options at your disposal. #1 Use addr2line to get the crash location $ cat badmem.c void function_c() { int *i = (int*)0xdeadbeef; *i = 123; } // <-- line 1 void function_b() { function_c(); } void function_a() { function_b(); } int main() { function_a(); return 0; } $ gcc -g badmem.c -o badmem $ ./badmem Segmentation fault No core dump?

Sometimes you want a backtrace or a core dump from a process that you do not want to stall. This could concern a multithreaded application of which some threads are still doing important work (like handling customer calls). Firing up gdb would halt the process for as long as you’re getting info, and raising a SIGABRT to get a core dump has the negative side-effect of killing the process. Neither is acceptable in a production environment.

While looking through opensips logs of a customer, sometimes we would see the following: ERROR:core:parse_via: invalid port number <110900> ERROR:core:parse_via: <SIP/2.0/UDP 1.2.3.4:110900;branch=z9hG4bKabcdef... ERROR:core:parse_via: parsed so far:<SIP/2.0/UDP 1.2.3.4:110900;branch=z9hG4bKabcdef... ERROR:core:get_hdr_field: bad via As you can see, that 6-digit port number is invalid. Furthermore, when sniffing this traffic, we could see that the port number is almost right. The traffic came from port 11090 (one less zero at the end). Not only the Via header, the Contact header too had the extra appended zero.

If you want to be able to sniff your IPsec traffic with OpenSwan, you’ll need to get KLIPS instead of the default NETKEY IPsec protocol stack. Installing that on Ubuntu/Karmic should be a matter of: ~# apt-get install openswan-modules-source ~# cd /usr/src /usr/src# tar jxvf openswan-modules.tar.bz2 /usr/src# cd modules/openswan /usr/src/modules/openswan# make KERNELSRC=/lib/modules/`uname -r`/build module module_install But it’s not. Right now, we’re running the default Linux kernel 2.6.31-23-server on this Karmic machine.

After disk failure on our company music server, I lost my enqueue-some-random-music-script. That shan’t happen again. So here, for my own enjoyment: autoenq.sh #!/bin/sh enqueue_app="mocp -a" music_glob="*.mp3" music_path="`dirname "$0"`" list_path="$music_path/.autoenq.list" if [ "$*" = "-c" ]; then # Create list of all files find . -type f -iname "$music_glob" > "$list_path.tmp" 2>/dev/null # no lost+found # Create list of all dirs that have files cat "$list_path.tmp" | sed -e 's/\/[^\/]*$//' | sort | uniq > "$list_path" exit 0 fi args="`echo "$*" | sed -e "s/['\\\\]//g"`" # no backslashes and single quotes please args="`echo "$args" | sed -e 's/[[:blank:]]\+/.

In the Asterisk PBX dialplan, expressions can be formed using the $[...] syntax. Addition, subtraction, comparison and so on are defined. As is a regex operator: =~ Unfortunately, the documentation about the details of the implementation is hard to find. Here, a breakdown of my findings: static struct val * op_eqtilde is defined in main/ast_expr2.y It uses the REG_EXTENDED flag when calling regcomp: so extended regular expression syntax is used.

If you use ssh to run commands remotely, you may have run into the problem that you need an extra layer of escaping. Let’s say you have application myapp that for some reason only runs on host myserver. If you have functional ssh keys to log onto myserver it can be helpful to create a myapp wrapper on your desktop. After all, this: $ myapp myargs … is far more convenient than doing this:

Suppose you have an is_enabled boolean in your Django model. class Rule(models.Model): is_enabled = models.BooleanField(blank=True) # other exciting fields here And now imagine you want to negate the is_enabled values. Something you would easily do in SQL, with: UPDATE myapp_rule SET is_enabled = NOT is_enabled; The Django F-syntax is nice, and looks like it should be up for the task. Let’s sum up a couple of attempts: Rule.objects.update(is_enabled=(not F('is_enabled'))) No! You get this:

I’ve previously written about MySQL pain in the behind issues involving views with SECURITY DEFINER and bad client collation selection. For the former problem, I wrote a script that you could call periodically to warn you of potential problems with your views. Now I’ve extended it to warn you about collation issues as well. Put warn-mysql-issues.sh (view) in your cron tab and run it periodically. It’ll save you from production-time errors that you get when attempting to compare a string of one collation with another.