Packaging a Haskell project as a Nix flake

2023-09-02

Introduction

I have been using nix for local development for over a year now, but want to start learning more, so three weeks ago I posted some question on Mastodon:

Okay. Time to learn some more #nix than just using it for local development!

I have a server running #nixos 23.05 and I want to set up some services that I have written myself. The first service is a simple IRC bot written in Haskell.

Where do I begin? 🤷

It should basically end up with a binary on the server and a service that is pretty much just this:
[Service]
ExecStart=/some/where/my_silly_bot
Restart=always
Do I create some config in the git repo and refer to it from the nixos server? Am I creating a “module”? (I duckduckwent it, but not even knowing what concepts I should search for made it a bit difficult. 😇)

I got some helpful pointers and realized there will be two steps to this:

Package the project for building with nix (this post)
Set up the service on a NixOS server (next post) # Creating a flake for my Haskell project

The project I’m going to package is a simple IRC bot: https://github.com/ehamberg/tribot.

So far, I’ve only written flakes to get a local dev environment when opening a project in a shell or in an editor (by using direnv and nix-direnv), but now I also want to be able to nix build the project.

In other words, there are two goals the nix flake should fulfil:

Make it possible to build the project with nix build
Make it possible to get a local development environment for the project using direnv + nix-direnv (or nix develop)

Attempt 1: A basic setup for Haskell projects

Fortunately, Gabriella Gonzalez has written a great post on incrementally packaging a Haskell program, so we’re off to a great start:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { nixpkgs, flake-utils, ... }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        config = { };

        overlay = pkgsNew: pkgsOld: {
          tribot = pkgsNew.haskell.lib.justStaticExecutables
            pkgsNew.haskellPackages.tribot;

          haskellPackages = pkgsOld.haskellPackages.override (old: {
            overrides =
              pkgsNew.haskell.lib.packageSourceOverrides { tribot = ./.; };
          });
        };

        pkgs = import nixpkgs {
          inherit config system;
          overlays = [ overlay ];
        };

      in rec {
        packages.default = pkgs.haskellPackages.tribot;

        apps.default = {
          type = "app";
          program = "${pkgs.tribot}/bin/tribot";
        };

        devShells = {
            default = pkgs.mkShell { buildInputs = with pkgs; [
                haskellPackages.haskell-language-server
                haskellPackages.hlint
                haskellPackages.cabal-fmt
                haskellPackages.ormolu
                cabal-install
                sqlite
                zlib
            ]; };
        };
      });
}

This makes sense for the most part, even though there are some pieces that are somewhat mysterious¹.

This would normally be the end of packaging a Haskell project, but while the dev shell still works, nix build does not:

❯ nix build
error: Package ‘simpleirc-0.3.1’ in /nix/store/qx67jipw01zps1rqgmmpl7as1irff275-source/pkgs/development/haskell-modules/hackage-packages.nix:268059 is marked as broken, refusing to evaluate.
a) To temporarily allow broken packages, you can use an environment variable
          for a single invocation of the nix tools.

            $ export NIXPKGS_ALLOW_BROKEN=1
		Note: For `nix shell`, `nix build`, `nix develop` or any other Nix 2.4+
        (Flake) command, `--impure` must be passed in order to read this
        environment variable.

Ooof!

Attempt 2: Allow broken‽

Allowing broken packages sounds like a really bad idea and – at best – a short-term solution, but sure, let’s try:

❯ NIXPKGS_ALLOW_BROKEN=1 nix build --impure
warning: Git tree '/Users/ehamberg/Developer/tribot' is dirty
error: builder for '/nix/store/a0cr7b79dcmak5m59aiam7ksxihgfrjc-simpleirc-0.3.1.drv' failed with exit code 1;
       last 10 log lines:
       >   |
       > 3 | import           Test.Hspec.Monadic
       >   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       >
       > tests/Spec.hs:4:1: error:
       >     Could not find module ‘CoreSpec’
       >     Use -v (or `:set -v` in ghci) to see a list of the files searched for.
       >   |
       > 4 | import qualified CoreSpec
       >   | ^^^^^^^^^^^^^^^^^^^^^^^^^
       For full logs, run 'nix log /nix/store/a0cr7b79dcmak5m59aiam7ksxihgfrjc-simpleirc-0.3.1.drv'.
error: 1 dependencies of derivation '/nix/store/48ds1qxc73masqwrp6yagpj1d5zdn8r7-tribot-0.4.0.0.drv' failed to build

Oh no! Looks like a test is broken.

Attempt 3: Disable the tests!

Hmm. Since we’re already doing naughty things, let’s see how to disable testing for a haskell package. Spoiler: We can use haskell.lib.dontCheck: Let’s fix our flake to use this to override our package with one without tests:

@@ -14,8 +14,21 @@
             pkgsNew.haskellPackages.tribot;

           haskellPackages = pkgsOld.haskellPackages.override (old: {
-            overrides =
-              pkgsNew.haskell.lib.packageSourceOverrides { tribot = ./.; };
+            overrides = let
+              oldOverrides = old.overrides or (_: _: { });
+
+              manualOverrides = haskellPackagesNew: haskellPackagesOld: {
+                simpleirc =
+                  pkgsNew.haskell.lib.dontCheck haskellPackagesOld.simpleirc;
+              };
+
+              sourceOverrides =
+                pkgsNew.haskell.lib.packageSourceOverrides { tribot = ./.; };
+
+            in pkgsNew.lib.fold pkgsNew.lib.composeExtensions oldOverrides ([
+              sourceOverrides
+              manualOverrides
+            ]);
           });
         };

Yay!

❯ NIXPKGS_ALLOW_BROKEN=1 nix build --impure
warning: Git tree '/Users/ehamberg/Developer/tribot' is dirty
error: builder for '/nix/store/k1g0n2xgpk9mf2hm0dp321paihpfmr65-tribot-0.4.0.0.drv' failed with exit code 1;
       last 10 log lines:
       >
       > app/Main.hs:12:1: error:
       >     Could not find module ‘Network.SimpleIRC.Sasl’
       >     Perhaps you meant
       >       Network.SimpleIRC.Core (from simpleirc-0.3.1)
       >       Network.SimpleIRC (from simpleirc-0.3.1)
       >     Use -v (or `:set -v` in ghci) to see a list of the files searched for.
       >    |
       > 12 | import Network.SimpleIRC.Sasl (SaslPlainArgs (..), saslPlain)
       >    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       For full logs, run 'nix log /nix/store/k1g0n2xgpk9mf2hm0dp321paihpfmr65-tribot-0.4.0.0.drv'.

No yay! Turns out all of this was a red herring. My project actually depends on a newer version of SimpleIRC than the version available on Hackage (that is marked as broken…).

Turns out I had this tucked away in a long-forgotten cabal.project file:

source-repository-package
  type: git
  location: git://github.com/dom96/SimpleIRC.git

packages: ./tribot.cabal

Attempt 4: Straight to the source!

Okay. So we need to fetch SimpleIRC straight from Github. Turns out that there’s a fetchGit function in Nix (or nixpkgs? how do I know?), so we can use this to fetch a given revision from Github (and reenable tests 😮‍💨):

@@ -18,8 +18,14 @@
               oldOverrides = old.overrides or (_: _: { });

               manualOverrides = haskellPackagesNew: haskellPackagesOld: {
-                simpleirc =
-                  pkgsNew.haskell.lib.dontCheck haskellPackagesOld.simpleirc;
+                simpleirc = let
+                  src = builtins.fetchGit {
+                    url = "https://github.com/dom96/SimpleIrc";
+                    ref = "master";
+                    rev = "8d156a89801be2c9b6923d85e6b199c8173e445a";
+                  };
+                in pkgs.haskell.lib.dontCheck
+                (haskellPackagesOld.callCabal2nix "simpleirc" src { });
               };

               sourceOverrides =

And boom! nix build works! ## The final flake.nix:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { nixpkgs, flake-utils, ... }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        config = { };

        overlay = pkgsNew: pkgsOld: {
          tribot = pkgsNew.haskell.lib.justStaticExecutables
            pkgsNew.haskellPackages.tribot;

          haskellPackages = pkgsOld.haskellPackages.override (old: {
            overrides = let
              oldOverrides = old.overrides or (_: _: { });

              manualOverrides = haskellPackagesNew: haskellPackagesOld: {
                simpleirc = let
                  src = builtins.fetchGit {
                    url = "https://github.com/dom96/SimpleIrc";
                    ref = "master";
                    rev = "8d156a89801be2c9b6923d85e6b199c8173e445a";
                  };
                in pkgs.haskell.lib.dontCheck
                (haskellPackagesOld.callCabal2nix "simpleirc" src { });
              };

              sourceOverrides =
                pkgsNew.haskell.lib.packageSourceOverrides { tribot = ./.; };

            in pkgsNew.lib.fold pkgsNew.lib.composeExtensions oldOverrides ([
              sourceOverrides
              manualOverrides
            ]);
          });
        };

        pkgs = import nixpkgs {
          inherit config system;
          overlays = [ overlay ];
        };

      in rec {
        packages.default = pkgs.haskellPackages.tribot;

        apps.default = {
          type = "app";
          program = "${pkgs.tribot}/bin/tribot";
        };

        devShells = {
          default = pkgs.mkShell {
            buildInputs = with pkgs; [
              haskellPackages.haskell-language-server
              haskellPackages.hlint
              haskellPackages.cabal-fmt
              haskellPackages.ormolu
              cabal-install
              zlib
              sqlite
            ];
          };
        };
      });
}

Conclusion

This was a somewhat confusing journey, but to be fair, if I didn’t depend on an unreleased version of a library, the very first flake.nix would probably have worked.

Now, the next step is to create a service on a NixOS server. That’s the next post.

For example the justStaticExecutables function(?), this is documented in nixpkgs and seems to be used for reducing the size of the built artifact:

justStaticExecutables drv: Only build and install the executables produced by drv, removing everything that may refer to other Haskell packages’ store paths (like libraries and documentation). This dramatically reduces the closure size of the resulting derivation. Note that the executables are only statically linked against their Haskell dependencies, but will still link dynamically against libc, GMP and other system library dependencies.

↩︎

Erlend Hamberg