A secure way to sign HTML blocks


A couple of years ago I was talking with my colleages in those years about security in some websites. We were not talking about SSL (which is, by the way more popuplar now), because SSL only works at connection level. With SSL you can guarantee that the communication is reliable (in terms of authenticity) and that the endpoint server is actually who pretend to be.

But SSL hides a shameful secret, a flaw in the design which can provoke, eventually, a big security problem. This neglected detail is too evident that no one think very mucho about it: “SSL doesn’t guarantee you anything about the content that you are viewing”.

We can build an imaginary experiment. Let’s suppose that a big e-commerce web site which has payments enabled for their customers wants to fire an employment. That employment is a good qualified programmer with access to the site source code. Before they fired the worker, he modify the source code to add a very small piece of code (buried in a millions of lines of e-commerce code) which just change a little bit thing. The action of the payment HTML form now send credit card data to an anonymous web service running in some weird country.

Now, let’s do another exercise in imagination. Suppose you are an unsuspecting user who loves products of our company. You buy a couple of goods, and probably you pay with your credit card… Ops! Back a moment… Now your credit card data is stored in a probably not very safe database in one server located in our Weird Country, ready to be sold to anyone who can pay for that kind of information (and I can assure you that they aren’t good people).

In this case SSL is green. Is the real server with a trust communication. But in this case SSL doesn’t help us to avoid the crime. That’s the reason why we need content signing eventually.

Thinking about this problem I create a way to facilitate this implementation. The core of the idea is the attribute data-signature. This attribute can be used in any HTML5 block, and it’s a signature of the HTML representation of all childs of the block which has the attribute. So, for example in the following code:

<div id="content" class="myclass_for_stylish" data-signature="eWVzIG1hcnRoYSwgdGhpcyBpcyBub3QgYSByZWFsIHNpZ25hdHVyZQo=">
  <!-- This is a normal comment -->
  <p>Some paragraph here</p>

The signature is valid for the HTML <p>Some paragraph</p>. We don’t need to sign the comment (nothing important could be saved there). The signature algorithm is, right now irrelevant. We work on that point some paragraphs below.

Of course, nested blocks can be signed also.

With this approximation, we are sure that the content of the div block is genuine, because we assume that the developer has no access to master keys to sign critical data. In out store example, the critical data is just the form block, and needs to be hard coded, but, anyway, this is usually a fixed string in a template.

Finally we need to talk a little bit about the algorithm to sign. We can use any public key based algorithm, and the only problem is how can we check that the signature is right. Well, there are a lot of solutions for that problem.

One solution could be that the browser (or browser extension ;)) validate the signature looking for the public key associated with the domain in a public CA (or web of trust model).

So, this is a simple way to validate HTML blocks and put more security in web sites. Do you think that this kind of systems are necessary? or convenient? Do you know any other way to sign content in web sites?

Let’s think about this when click the “Payment” button ;)

Simple way to manage lots of system users in distributed environments.

Few years ago I was working in design of a large cluster of systems to perform some actions (solving some mathematical models, sharding database…). From the point of view of the systems, I had to deal with a number of pesky troubles. One of them was the user management.

Since I had more than one hundred of hosts, and this number could be grow up in short, and I’ve a number of users which need to access to all hosts, I need to think in a way to easy user management. Actually user management is, in my opinion a pain in the butt. If you ve a central user directory, you need to deal with a big and fat single point of failure, so you need to create some kind of HA service for this directory. And if you ve systems around the world, then you need to replicate the user data in different directories and keep them synchronized. If it is not the hell, it must be very similar.

Dealing with user managemet is a royal hassle for system administrators in every place, but in the cloud (i.e. a number of hosts distributed around the world), it’s also a punishment. So, I need to solve (almost in part) this problem before moving forward in my deployment. I do not need a full user management really, just a basic UID mapping and a way to authenticate users (for which I could use the old-and-friendly authorized_keys).

So, How can I manage a big number of users in a single way, and to be effective in a distributed environment? That’s not a simple questions, and of course each implementation has its own solution, from authentication services to suites of scripts. Anyway, I was looking for a simple to manage ones, cause of I was the responsible to manage the entire environment and I’m too lazy too ;)

Thinking about the problem, I imagine a system without any user, let’s imaging that there are just one user, and any other user is just an alias for the first one. It could be easy to manage, because we only need one UID, but we need to solve the alias mapping.
Here is when libnss_map join into the game. The libnss_map is a library designed to be used with GNU NSS service. The NSS allows the system to get user credentials from many sources, which can be configured easy from the /etc/nsswitch.conf file.

For example, we can configure our system in the following way:

passwd:      files map
shadow:      files map
group:       files map

So, for each user to get credentials NSS will lookup in standard files first, and then using the map module (libnss_map).The map module works as the flow diagram shows.

Flow diagram of how get credentials works with libnss_map

As you can see in the diagram there are two major steps in lookup. The first one is the responsible to map an user to a virtual one. The virtual user is static, and it’s defined in /etc/nssmap.conf. This file has the same syntax like passwd does. For example:


Which means that any user who does not exists in /etc/passwd will be mapped into this one, with UID 10000.

Okay, sounds good, but there are a lot questions yet. What about the password? What about the home dir?

Well, I do not find a good solution for password, so nssmap will return a masked password (account is enabled, but password will be unpredictable), and I authenticate the user using other methods via PAM, or public keys via SSH.

Home directory is easier. The home directory field in the user definition (inside /etc/nssmap.conf file) is used as prefix, and it will be completed with the user name (the name of the user which is intended to login, not the virtual one). So, for example, for the hypothetical user “sample”, the effective home directory will be “/home/sample”, because “/home/” is the prefix. Please note that the end slash is mandatory in current implementation.

Finally I need to solve another big problem: if two users has the same UID then both can change the same files, or delete the files of other “virtual” user. How can we solve it? There are not single answer, not easy afterall. In my case I use special shell, which
ensure that the user cannot remove, touch or even read files in any path into /home except his own home directory, but it’s not a full solution yet.

Here is an example using nss map:

host ~ # sudo su - test
No directory, logging in with HOME=/
test@host / $ id
uid=10000(virtual) gid=10000(virtual) groups=10000(virtual)

In the meanwhile, a basic code is available in my github, and I still researching in this kind of authorization. Keep in touch and enjoy! An of course, feedback is welcome :D

Hands on dreamplug

A DreamPlug device

Update: Óscar García gently published a similar spanish version of this article.

Couple of weeks ago I received my dreamplug, from NewIT. Though I knew some software troubles in device, I remains hopeful. However when I unpacked and powered on the plug I got a number of problems. This is the history about these problems and their solutions. I hope that my experience can be useful for anyone who take the same model and have the same problems. Afterall, the NewIT forum has a number of posts about these and other problems, so reading it is recommended.

The first problem that I found was a bad partition table, which is also a well-known bug in NewIT forums; and the second one is the Ubuntu installation which is not very tiny by default :) Fortunately, both problems are easy to solve.

These are the steps that I followed to “upgrade” my dreamplug software to a cooler one. Please note that the Globalscale guys do not support other system images and installations AFAIK, so perform this changes under your own responsibility. You are warned!.
Continue reading

New dtools and a bit more

Last week was crazy. I published a new release of dtools, the 4.2, a new web site for the dtools project and a couple of patches for version 5.0 of collectd.

In the last months, dtools becomes in an useful tool for me. I use dtools everyday for system administration in large distributed networks. So I decided to improve some functionalities and also test and retest current features, so in a couple of months I expect to launch a new release of dtools.

In the meantime, I still working in whistler, a XMPP bot for MUC rooms, I hope that SleekXMPP library, which is the XMPP engine used by whistler become to release early. At that point we must remove any dependency with the old xmpppy in the code and we will ready to release the version 2.0 of whistler.

Integer conversions in bash

Since version 2, bash support a single aritmethic operations. Altough bash is not a mathematical shell (use bc instead), you can perform certain conversions using the bash arithmetic logic.

For example you can remove the left zeroes in a decimal number without require any external utility or print formats, let’s suppose that you want to strip zeroes from the number 007, which is stored in bond variable.

$ echo $bond
$ let nozeros=10#$bond
$ echo $nozeros

In many forums and mailing list, people need to use ugly sed expressions, or awk invokation, but (with bash) it’s just simply :)

Using the same trick, you can perform a base conversions, for example:

$ let i=0x10
$ echo $i
$ let i=2#10000
$ echo $i

Or create an easy number checking:

$ is_decimal () { let i=10#$1 2>/dev/null; }
$ is_decimal 'a' || echo Nop
$ is_decimal 56 && echo 'Yep'


Whistler: a new Jabber MUC bot.

Few days ago, I start a new project called whistler. Whistler is a bot written in python using the greatest xmppy library, designed to work in XMPP networks (like jabber or GTalk. In first time I tried to use the quinoa framework, and it is very usefull, but have some issues for me, for example you cannot set another server configuration, which is a problem for GTalk accounts. So, after tried a number of frameworks, I decided to create my own one. Probably not the best, but mine :)

Whistler is intended to manage the connectical MUC room, and only basic functionalities are provided. Obviously it is under heavy development yet.

The code is publicy available on github whistler repository, and you can clone as usually:

$ git clone git://github.com/ajdiaz/whistler

You require xmppy to work with whistler and python >= 2.5. In few days I will publish the projecti into pypi too.

Enjoy and remember, any feedback is welcome ;)

My exheres repository

A couple of months ago, I reinstall my PC with exherbo and now I can say that I’m a happy user (well, a happy “developer” since Exherbo do not have users). So, I decided to port a number of ebuild than I use frequently to exheres, and create my own exheres repository in github, called ajdiaz-exheres.

Now I will try to include my repo in unavailable-unofficial list… In the meanwhile, here is the configuration that you need for paludis:

location = ${ROOT}/var/db/paludis/repositories/ajdiaz-exheres
sync = git://github.com/ajdiaz/ajdiaz-exheres
format = e


Update dot files with git

For a years I was using a custom created scripts to keep my dot files updated. I had a local repository in bazaar and a script which check differences between home dot files and files stored in the repository. This solutions works fine for years, but now I want to do some changes…

The first one is moving my dot files to git (and probably pushed them to github), and the second one is to create a hook for git to update my dot files. I known that there are a lot of similar solutions, one more complex, other more easy, but this is mine :)

So, I created a post-commit hook script for git, which perform the modifications that I need. Now I just only do this steps:

1. Create a new git repo:

mkdir mydots_repo
cd my_dots_repo &amp;&amp; git init

2. Put the hook:

wget -O .git/hooks/pre-commit  http://2tu.us/2scm
chmod 755 .git/hooks/pre-commit

Or just put this content to pre-commit hook:

#! /bin/bash
# (c) 2010 Andres J. Diaz <ajdiaz@connectical.com>
# A hook to git-commit(1) to update the home dot files link using this
# repository as based.
# To enable this hook, rename this file to "post-commit".

for dot in $PWD/*; do

 if [ -L "${home_dot}" ]; then
 if [ "${home_dot}" -ef "$dot" ]; then
 echo "[skip] ${home_dot}: is already updated"
 rm -f "${home_dot}" && \
 ln -s "$dot" "${home_dot}" && \
 echo "[done] updated link: ${home_dot}"
 if [ -r "${home_dot}" ]; then
 echo "[keep] ${home_dot}: is regular file"
 ln -s "$dot" "${home_dot}" && \
 echo "[done] updated link: ${home_dot}"

3. Copy old files:

cp ~/old/bzr_repo/* .
git add *

4. Commit and recreate links:

git commit -a -m'initial import'

And it’s works :)

Moving to github

Since one week ago, we are moving the Connectical servers from old location in Virpus datacenter on Texas to our own managed infraestructure, build on the top of a GuruPlugs cluster.

We are discussing now about how distribute the infraestructure and how to keep a number of copies in remote locations up-to-date, we are exploring solutions like elliptics or some similar.

In the meanwhile I created my github account to still my projects under development, and also to have a backup of some projects that I really use everyday.