Moving to GitHub pages

Moved to https://lucabrunox.github.io/ let's see how it goes :)

OpenVPN configuration for server and multiple clients

This is a simple post showing a basic configuration for setting up OpenVPN server accepting multiple clients with TLS.

First of all generate self-signed server and client private key and certificates, and dh params. Make sure to write the Organization Name AND Common Name (CN) when asked, otherwise openvpn will fail to verify the certificates.

openssl req -newkey rsa:2048 -nodes -keyout serverkey.pem -x509 -days 365000 -out servercert.pem
openssl req -newkey rsa:2048 -nodes -keyout clientkey.pem -out client.csr
openssl x509 -req -days 365000 -in client.csr -CA servercert.pem -CAkey serverkey.pem -set_serial 01 -out clientcert.pem
openssl dhparam -outform PEM -out dh.pem 1024


Server configuration:

dev tun
mode server
tls-server
server 10.8.0.0 255.255.255.0
ca servercert.pem
cert servercert.pem
key serverkey.pem
dh dh.pem
duplicate-cn
topology subnet
keepalive 10 60
ping-timer-rem
persist-tun
persist-key


If you plan to use the server as gateway like configured below, don't forget to enable IP forwarding and masquerading:

sysctl net.ipv4.ip_forward=1
iptables -t nat -A POSTROUTING -s 10.8.0.0/24 -j MASQUERADE

Client configuration:

client
redirect-gateway
tls-client
key clientkey.pem
cert clientcert.pem
ca servercert.pem
keepalive 10 60
dev tun


Have fun, tune as needed.

Cheap Docker images with Nix

Let's talk about Docker and Nix today. Before explaining what Nix is, if you don't know yet, and before going into the details, I will show you a snippet similar to a Dockerfile for creating a Redis image equivalent to the one in docker hub.

The final image will be around 42mb (or 25mb) in size, compared to 177mb.

EDIT: as mentioned on HN, alpine-based images can even go around 15mb in size.

If you want to try this, the first step is to install Nix.

Here's the redis.nix snippet:


Build it with: nix-build redis.nix
Load it with: docker load < result

Once loaded, you can see with docker images that it takes about 42mb of space.

Fundamental differences with classic docker builds

• We do not use any base image, like it's done for most docker images including redis from the hub. It starts from scratch. In fact, we set up some basic shadow-related files with the shadowSetup utility, enough to add the redis user and make gosu work.
• The Redis package is not being compiled inside Docker. It's being done by Nix, just like any other package.
• The built image has only one layer, compared to dozens usually spitted by a readable Dockerfile. In our case, having multiple layers is useless because caching is handled by Nix, and not by Docker.

A smaller image

We can cut the size down to 25mb by avoid using id from coreutils. As an example we'll always launch redis without the entrypoint:


You might ask: but coreutils is still needed for the chown, mkdir and other commands like that!

The secret is that those commands are only used at build time and are not required at runtime in the container. Nix is able to detect that automatically for us.

It means we don't need to manually remove packages after the container is built, like with other package managers! See this line in Redis Dockerfile for example.

Using a different redis version

Let's say we want to build a Docker image with Redis 2.8.23. First we want to write a package (or derivation in Nix land) for it, and then use that inside the image:


Note we also added the tag 2.8.23 to the resulting image. And that's it. The beauty is that we reuse the same redis expression from nixpkgs, but we override only the version to build.

A generic build

There's more you can do with Nix. Being a language, it's possible to create a generic function for building Redis images given a specific package:


We created a "redisImage" function that takes a "redis" parameter as input, and returns a Docker image as output.

Build it with:

• nix-build redis-generic.nix -A redisDocker_3_0_7 
• nix-build redis-generic.nix -A redisDocker_2_8_23

Building off a base image

One of the selling points of Docker is reusing an existing image to add more stuff on top of it.

Nix comes with a completely different set of packages compared to other distros, with its own toolchain and glibc version. This doesn't mean it's not possible to base a new image off an existing Debian image for instance.

By using dockerTools.pullImage it's also possible to pull images from the Docker hub.


Build it with: nix-build redis-generic.nix -A redisOnDebian.

Note that we added a couple of things. We pass the base image (debianImage), to our generic redisImage function, and that we only initialize shadow-utils if the base image is null.

The result is a Docker image based off latest Debian but running Redis compiled with nixpkgs toolchain and using nixpkgs glibc. It's about 150mb. It has all the layers from the base image, plus the new single layer for Redis.

That said, it's as well possible to use one of the previously defined Redis images as base image. The result of `pullImage` and `buildImage` is a .tar.gz docker image in both cases.

You realize it's possible to build something quite similar to docker-library using only Nix expressions. It might be an interesting project.

Be aware that things like PAM configurations, or other stuff, created to be suitable for Debian may not work with Nix programs that use a different glibc.

Other random details

The code above has been made possible by using nixpkgs commit 3ae4d2afe (2016-04-14) onwards, commit at which I've finally packaged gosu and since the size of the derivations have been notably reduced.

Building the image is done without using any of the Docker commands. The way it works is as follows:

1. Create a layer directory with all the produced contents inside. This includes the filesystem as well as the json metadata. This process will use certain build dependencies (like coreutils, shadow-utils, bash, redis, gosu, ...).
2. Ask Nix what are the runtime dependencies of the layer directory (like redis, gosu). Such dependencies will be always a subset of the build dependencies.
3. Add such runtime dependencies to the layer directory.
4. Pack the layer in a .tar.gz by following the Docker specification.

I'd like to state that Nix has a safer and easier caching of operations while building the image.
As for Docker, great care has to be taken in order to use the layer cache correctly, because such caching is solely based on the RUN command string. This blog post explains it well.
This is not the case for Nix, because every output depends on a set of exact inputs. If any of the inputs change, the output will be rebuilt.

So what is Nix?

Nix is a language and deployment tool, often used as package manager or configuration builder and system provisioning. The operating system NixOS is based on it.

The code shown above is Nix. We have used the nixpkgs repository which provides several reusable Nix expressions like redis and dockerTools.

The Nix concept is simple: write a Nix expression, build it. This is how the building process works at a high-level:

1. Read a Nix expression
2. Evaluate it and determine the thing (called derivation) to be built.
3. By evaluating the code, Nix is able to determine exactly the build inputs needed for such derivation.
4. Build (or fetch from cache) all the needed inputs.
5. Build (or fetch from the cache) the final derivation.

Nix stores all such derivations in a common nix store (usually /nix/store), identified by an hash. Each derivation may have dependencies to other paths in the same store. Each derivation is stored in a separate directory from other derivations.

Won't go deeper as there's plenty of documentation about how Nix works and how its storage works.

Hope you enjoyed the reading, and that you may give Nix a shot.

TypeScript and NodeJS, I'm sold

TypeScript is a typed superset of JavaScript that compiles to plain JavaScript, the way you expect it to be.

I’ve heard of it a long time ago, but recently with TypeScript 1.7 it got async functions, which means you can awaitasynchronous function calls, similarly to C#, Vala, Go and other languages with syntax support for concurrency. That makes coroutines a pleasant experience compared to plain JavaScript. That’s also the main reason why I didn’t choose Dart.

I’m writing a NodeJS application so I decided to give it a go. Here’s my personal tour of TypeScript and why I’m sold to using it.

Does it complain when using undeclared variables?

console.log(foo);

Cannot find name 'foo'.

Sold!

Does it infer types?

var foo = 123;
foo = "bar";

Type 'string' is not assignable to type 'number'.

Sold!

Does it support async arrow functions?

async function foo() {
}
var bar = async () => { await (foo); };

Sold!

Does it support sum types?

var foo: number | string;
foo = 123;
foo = "bar";

Sold!

Does it play nice with CommonJS/AMD imports and external libraries?

Yes, it does very well. Sold!

Is it easy to migrate from and to JavaScript?

Yes. TypeScript makes use of latest ECMAScript features in its syntax when possible, so that JavaScript -> TypeScript is as painless as possible. Sold!

Also to go back from TypeScript to JavaScript, either use the generated code, or remove all the type annotations in the code by yourself.

Does it have non-nullable variables?

No. This is mostly due to the JavaScript nature though. But I’m sure the TypeScript community will come up with a nice solution throughout this topic.

---

I’m going to use TypeScript wherever I can in this new year instead of plain JavaScript. In particular, I’m rewriting my latest NodeJS application in TypeScript right now.

I hope it will be a great year for this language. The project is exceptionally active, and I hope to contribute back to it.

Nix pill 19: fundamentals of stdenv

Welcome to the 19th Nix pill. In the previous 18th pill we did dive into the algorithm used by Nix to compute the store paths, and also introduced fixed-output store paths.
This time we will instead look into nixpkgs, in particular one of its core derivation: stdenv .

The stdenv is not a special derivation, but it's very important for the nixpkgs repository. It serves as base for packaging software. It is used to pull in dependencies such as the GCC toolchain, GNU make, core utilities, patch and diff utilities, and so on. Basic tools needed to compile a huge pile of software currently present in nixpkgs.

What is stdenv

First of all stdenv is a derivation. And it's a very simple one:

$ nix-build '<nixpkgs>' -A stdenv
/nix/store/k4jklkcag4zq4xkqhkpy156mgfm34ipn-stdenv
$ ls -R result/
result/:
nix-support/  setup

result/nix-support:
propagated-user-env-packages

It has just two files: /setup and /nix-support/propagated-user-env-packages. Don't care about the latter, it's even empty. The important file is /setup.
How can this simple derivation pull in all the toolchain and basic tools needed to compile packages? Let's look at the runtime dependencies:

$ nix-store -q --references result
/nix/store/3a45nb37s0ndljp68228snsqr3qsyp96-bzip2-1.0.6
/nix/store/a457ywa1haa0sgr9g7a1pgldrg3s798d-coreutils-8.24
/nix/store/zmd4jk4db5lgxb8l93mhkvr3x92g2sx2-bash-4.3-p39
/nix/store/47sfpm2qclpqvrzijizimk4md1739b1b-gcc-wrapper-4.9.3
...

How can it be? The package must be referring to those package somehow. In fact, they are hardcoded in the /setup file:

$ head result/setup
export SHELL=/nix/store/zmd4jk4db5lgxb8l93mhkvr3x92g2sx2-bash-4.3-p39/bin/bash
initialPath="/nix/store/a457ywa1haa0sgr9g7a1pgldrg3s798d-coreutils-8.24 ..."
defaultNativeBuildInputs="/nix/store/sgwq15xg00xnm435gjicspm048rqg9y6-patchelf-0.8 ..."

The setup file

Remember our generic builder.sh in Pill 8? It sets up a basic PATH, unpacks the source and runs the usual autotools commands for us.
The stdenv setup file is exactly that. It sets up several environment variables like PATH and creates some helper bash functions to build a package. I invite you to read it, it's only 860 lines at the time of this writing.

The hardcoded toolchain and utilities are used to initially fill up the environment variables so that it's more pleasant to run common commands, similarly but not equal like we did with our builder with baseInputs and buildInputs.

The build with stdenv works in phases. Phases are like unpackPhase, configurePhase, buildPhase, checkPhase, installPhase, fixupPhase. You can see the default list in the genericBuild function.
What genericBuild does is just run these phases. Default phases are just bash functions, you can easily read them.

Every phase has hooks to run commands before and after the phase has been executed. Phases can be overwritten, reordered, whatever, it's just bash code.

How to use this file? Like our old builder. To test it, we enter a fake empty derivation, source the stdenv setup, unpack the hello sources and build it:

$ nix-shell -E 'derivation { name = "fake"; builder = "fake"; system = "x86_64-linux"; }'
nix-shell$ unset PATH
nix-shell$ source /nix/store/k4jklkcag4zq4xkqhkpy156mgfm34ipn-stdenv/setup
nix-shell$ tar -xf hello-2.9.tar.gz
nix-shell$ cd hello-2.9
nix-shell$ configurePhase
...
nix-shell$ buildPhase
...

I unset PATH to further show that the stdenv is enough self-contained to build autotools packages that have no other dependencies.

So we ran the configurePhase function and buildPhase function and they worked. These bash functions should be self-explanatory, you can read the code in the setup file.

How is the setup file built

Very little digression for completeness. The stdenv derivation is just that setup file. That setup file is just this setup.sh in nixpkgs plus some lines on top of it, put by this simple builder:

...
echo "export SHELL=$shell" > $out/setup
echo "initialPath=\"$initialPath\"" >> $out/setup
echo "defaultNativeBuildInputs=\"$defaultNativeBuildInputs\"" >> $out/setup
echo "$preHook" >> $out/setup
cat "$setup" >> $out/setup
...

Nothing much to say, but you can read the Nix code that pass $initialPath and $defaultNativeBuildInputs. Not much interesting to continue further in this pill.

The stdenv.mkDerivation function

Until now we worked with plain bash scripts. What about the Nix side? The nixpkgs repository offers a useful function, like we did with our old builder. It is a wrapper around the raw derivation function which pulls in the stdenv for us, and runs genericBuild. It's stdenv.mkDerivation.

Note how stdenv is a derivation but it's also an attribute set which contains some other attributes, like mkDerivation. Nothing fancy here, just convenience.

Let's write a hello.nix expression using this new discovered stdenv:

with import <nixpkgs> {};
stdenv.mkDerivation {
  name = "hello";
  src = ./hello-2.9.tar.gz;
}

Don't be scared by the with expression. It pulls the nixpkgs repository into scope, so we can directly use stdenv. It looks very similar to the hello expression in Pill 8.
It builds, and runs fine:

$ nix-build hello.nix
...
/nix/store/6flbdbpq6sc1dc79xjx01bz43zwgj3wc-hello
$ result/bin/hello
Hello, world!

The stdenv.mkDerivation builder

Let's take a look at the builder used by mkDerivation. You can read the code here in nixpkgs:

{
  ...
  builder = attrs.realBuilder or shell;
  args = attrs.args or ["-e" (attrs.builder or ./default-builder.sh)];
  stdenv = result;
  ...
}

Also take a look at our old derivation wrapper in previous pills! The builder is bash (that shell variable), the argument to the builder (bash) is default-builder.sh, and then we add the environment variable $stdenv in the derivation which is the stdenv derivation.

You can open default-builder.sh and see what it does:

source $stdenv/setup
genericBuild

It's what we did in Pill 10 to make the derivations nix-shell friendly. When entering the shell, the setup file only sets up the environment without building anything. When doing nix-build, it actually runs the build process.

To get a clear understanding of the environment variables, look at the .drv of the hello derivation:

$ pp-aterm -i $(nix-instantiate hello.nix)
Derive(
  [("out", "/nix/store/6flbdbpq6sc1dc79xjx01bz43zwgj3wc-hello", "", "")]
, [("/nix/store/8z4xw8a0ax1csa0l83zflsm4jw9c94w2-bash-4.3-p39.drv", ["out"]), ("/nix/store/j0905apmxw2qb4ng5j40d4ghpiwa3mi1-stdenv.drv", ["out"])]
, ["/nix/store/0q6pfasdma4as22kyaknk4kwx4h58480-hello-2.9.tar.gz", "/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"]
, "x86_64-linux"
, "/nix/store/zmd4jk4db5lgxb8l93mhkvr3x92g2sx2-bash-4.3-p39/bin/bash"
, ["-e", "/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"]
, [ ("buildInputs", "")
  , ("builder", "/nix/store/zmd4jk4db5lgxb8l93mhkvr3x92g2sx2-bash-4.3-p39/bin/bash")
  , ("name", "hello")
  , ("nativeBuildInputs", "")
  , ("out", "/nix/store/6flbdbpq6sc1dc79xjx01bz43zwgj3wc-hello")
  , ("propagatedBuildInputs", "")
  , ("propagatedNativeBuildInputs", "")
  , ("src", "/nix/store/0q6pfasdma4as22kyaknk4kwx4h58480-hello-2.9.tar.gz")
  , ("stdenv", "/nix/store/k4jklkcag4zq4xkqhkpy156mgfm34ipn-stdenv")
  , ("system", "x86_64-linux")
  ]
)

So short I decided to paste it entirely above. The builder is bash, with -e default-builder.sh arguments. Then you can see the src and stdenv environment variables.

Last bit, the unpackPhase in the setup is used to unpack the sources and enter the directory, again like we did in our old builder.

Conclusion

The stdenv is the core of the nixpkgs repository. All packages use the stdenv.mkDerivation wrapper instead of the raw derivation. It does a bunch of operations for us and also sets up a pleasant build environment.

The overall process is simple:

• nix-build
• bash -e default-builder.sh
• source $stdenv/setup
• genericBuild

That's it, everything you need to know about the stdenv phases is in the setup file.

Really, take your time to read that file. Don't forget that juicy docs are also available in the nixpkgs manual.

Next pill...

...we will talk about how to add dependencies to our packages, buildInputs, propagatedBuildInputs and setup hooks. These three concepts are at the base of the current nixpkgs packages composition.

To be notified about the new pill, stay tuned on #NixPills, follow @lethalman or subscribe to the nixpills rss.

NixOS, Consul, Nginx and containers

This is a follow up post on https://medium.com/@dan.ellis/you-dont-need-1mm-for-a-distributed-system-70901d4741e1 . I think the post was well written, so I decided to write a variant using NixOS.

We'll be using declarative nixos containers, which do not use docker but systemd-nspawn. Also systemd is started as init system inside containers.

Please note that this configuration can be applied to any nixos machine, and also the containers configuration could be applied to real servers or other kinds of virtualization, e.g. via nixops. That is, the same syntax and configuration can be reused anywhere else within the nix world.

For example, you could create docker containers with nixos, and keep running the host with another distribution.

However for simplicity we'll use a NixOS system.

Architecture: the host runs nginx and a consul server, then spawns several containers with a python service and a consul client. On the host, consul-template will rewrite the nginx configuration when the health check status of container services change.

Please use a recent unstable release of nixos at the time of this writing (19 Feb 2015, at least commit aec96d4), as it contains the recently packaged consul-template.

Step 1: write the service

Let's write our python service in /root/work.py:

#!/usr/bin/env python

import random

from flask import Flask
app = Flask(__name__)

def find_prime(start):
    """
    Find a prime greater than `start`.
    """
    current = start
    while True:
        for p in xrange(2, current):
            if current % p == 0:
                break
        else:
            return current
        current += 1

@app.route("/")
def home():
    return str(find_prime(random.randint(2 ** 25, 2 ** 26)))

if __name__ == "__main__":
    app.run(host='0.0.0.0', port=8080, debug=True)

The only difference with the original post is that we explicitly set the port to 8080 (who knows if a day flask changes the default port).

Step 2: write the nixos config

Write the following in /etc/nixos/prime.nix:

{ lib, pkgs, config, ... }:

let

  pypkgs = pkgs.python27Packages;

  # Create a self-contained package for our service
  work = pkgs.stdenv.mkDerivation {
    name = "work";
    unpackPhase = "true";
    buildInputs = [ pkgs.makeWrapper pypkgs.python pypkgs.flask ];
    installPhase = ''
      mkdir -p $out/bin
      cp ${/root/work.py} $out/bin/work.py
      chmod a+rx $out/bin/work.py
      wrapProgram $out/bin/work.py --prefix PYTHONPATH : $PYTHONPATH
    '';
  };

  # Function which takes a network and the final octet, and returns a container
  mkContainer = net: octet: {
    privateNetwork = true;
    hostAddress = "${net}.1";
    localAddress = "${net}.${octet}";

    autoStart = true;

    config = { config, pkgs, ... }:
      {
        users.mutableUsers = false;
        # Use this only for debugging, login the machine with machinectl
        users.extraUsers.root.password = "root";
        # Let consul run check scripts
        users.extraUsers.consul.shell = "/run/current-system/sw/bin/bash";

        environment.etc."consul/prime.json".text = builtins.toJSON {
          service = {
            name = "prime";
            tags = [ "nginx" ];
            port = 8080;
            check = {
              script = "${pkgs.curl}/bin/curl localhost:8080 >/dev/null 2>&1";
              interval = "30s";
            };
          };
        };

        systemd.services.prime = {
          wantedBy = [ "multi-user.target" ];

          serviceConfig = {
            ExecStart = "${work}/bin/work.py";
          };
        };

        services.consul = {
          enable = true;
          extraConfig = { 
            server = false;
            start_join = [ "${net}.1" ];
          };
          extraConfigFiles = [ "/etc/consul/prime.json" ];
        };

        networking.firewall = {
          allowedTCPPorts = [ 8080 8400 ];
          allowPing = true;
        };
      };
  };

  nginxTmpl = pkgs.writeText "prime.conf" ''
    upstream primes {
        {{range service "prime"}}
        server {{.Address}}:8080;{{end}}
    }
  '';

in
{
  containers.prime1 = mkContainer "10.50.0" "2";
  containers.prime2 = mkContainer "10.50.0" "3";
  containers.prime3 = mkContainer "10.50.0" "4";
  containers.prime4 = mkContainer "10.50.0" "5";

  services.consul = {
    enable = true;
    extraConfig = {
      bootstrap = true;
      server = true;
    };
  };

  services.nginx = {
    enable = true;
    httpConfig = ''
      include /etc/nginx/prime.conf;

      server {
        listen 80;

        location / {
          proxy_pass http://primes;
        }
      }
    '';
  };

  systemd.services.nginx = {
    preStart = ''
      mkdir -p /etc/nginx
      touch -a /etc/nginx/prime.conf
    '';

    serviceConfig = {
      Restart = "on-failure";
      RestartSec = "1s";
    };
  };

  # Start order: consul -> consul-template -> nginx
  systemd.services.consul-template = {
    wantedBy = [ "nginx.service" ];
    before = [ "nginx.service" ];
    wants = [ "consul.service" ];
    after = [ "consul.service" ];

    serviceConfig = {
      Restart = "on-failure";
      RestartSec = "1s";
      ExecStart = "${pkgs.consul-template}/bin/consul-template -template '${nginxTmpl}:/etc/nginx/prime.conf:systemctl kill -s SIGHUP nginx'";
    };
  };

  boot.kernel.sysctl."net.ipv4.ip_forward" = true;
}

Differences with the original post:

• We only create 4 containers instead of 10. I was lazy here. If you are lazy too, you can still automatize the process with nix functions (for example map).
• We define some ordering in how services start and how they restart with systemd.
• For simplicity we include the prime.conf nginx config instead of rewriting the whole nginx config with consul-template.
• We create a self-contained package for our python service, so that anywhere it runs the dependencies will be satisfied.

Finally import this config in your /etc/nixos/configuration.nix with imports = [ ./prime.nix ];.

Step 3: apply the configuration

Type nixos-rebuild switch and then curl http://localhost. You may have to wait some seconds before consul writes the nginx config. In the while, nginx may have failed to start. If it exceeded the StartTime conditions, you can systemctl start nginx manually.
Fixing this is about tweaking the systemd service values about the StartTime.

Each container consumes practically no disk space at base. Everything else is shared through the host nix store, except logs, consul state, ecc. of course.

Have fun!

Developing in golang with Nix package manager

I've been using Go since several months. It's a pleasant language, even though it has its own drawbacks.

In our Nixpkgs repository we have support for several programming languages: perl, python, ruby, haskell, lua, ... We've merged a better support for Go.

What kind of support are we talking about? In Nix, you never install libraries. Instead, you define an environment in which to use a certain library compiled for a certain version of the language. The library will be available only within this environment.

Think of it like virtualenv for python, except for any language, and also being able to mix them.
On the other hand Nix requires the src url and the checksum of every dependency of your project. So before starting, make sure you are willing to write nix packages that are not currently present in nixpkgs.

Also you probably have to wait a couple of days before this PR will be available in the unstable channel, at the time of this writing (otherwise git clone https://github.com/NixOS/nixpkgs.git).

Using nix-shell -p

First a quick example using nix-shell -p for your own project:

$ nix-shell '<nixpkgs>' -p goPackages.go goPackages.net goPackages.osext
[nix-shell]$ echo $GOPATH
/nix/store/kw9dryid364ac038zmbzq72bnh3zsinz-go-1.4.1-go.net-3338d5f109e9/share/go:...

That's how nix mostly works, it's as simple as that. The GOPATH is set for every package that provides a directory share/go.

What is goPackages? Currently it's go14Packages, which is all the go packages we have, compiled with go 1.4. There's also go13Packages, you know some particular packages don't work with go 1.4 yet.

Writing a nix file

A more structured example by writing a default.nix file in your project:

with import <nixpkgs> {}; with goPackages;

buildGoPackage rec {
  name = "yourproject";
  buildInputs = [ net osext ];
  goPackagePath = "github.com/you/yourproject";
}

Then you can just run nix-shell in your project directory and have your dev environment ready to compile your code.
The goPackagePath is something needed by buildGoPackage, in case you are going to run nix-build. Ignore it for now.

Writing a dependency

But nixpkgs doesn't have listed all the possible go projects. What if you need to use a particular library?
Let's take for example github.com/kr/pty. Write something like this in a pty.nix file:

{ goPackages, fetchFromGitHub }:

goPackages.buildGoPackage rec {
  rev = "67e2db24c831afa6c64fc17b4a143390674365ef";
  name = "pty-${rev}";
  goPackagePath = "github.com/kr/pty";
  src = fetchFromGitHub {
    inherit rev;
    owner = "kr";
    repo = "pty";
    sha256 = "1l3z3wbb112ar9br44m8g838z0pq2gfxcp5s3ka0xvm1hjvanw2d";
  };
}

Then in your default.nix:

with import <nixpkgs> {}; with goPackages;

let
  pty = callPackage ./pty.nix {};
in
buildGoPackage rec {
  name = "yourproject";
  buildInputs = [ net osext pty ];
  goPackagePath = "github.com/you/yourproject";
}

Type nix-shell and now you will also have pty in your dev environment.
So as you can see, for each go package nix requires a name, the go path, where to fetch the sources, and the hash.

You may be wondering how do you get the sha256: a dirty trick is to write a wrong sha, then nix will tell you the correct sha.

Conclusion and references

Nix looks a little complex and boring due to writing a package for each dependency. On the other hand you get for free:

• Exact build and runtime dependencies
• Sharing build and runtime dependencies between multiple projects
• Easily test newer or older versions of libraries, without messing with system-wide installations
• Mix with other programming languages, using a similar approach
• Packages using C libraries don't need to be compiled manually by you: define the nix package once, reuse everywhere

For installing nix, follow the manual. Make sure you read the entire document to learn the nix syntax.

For more examples on how to write dependencies, you can look at nixpkgs goPackages itself.

Drop by #nixos on irc.freenode.net for any doubts.