Packaging a Haskell project as a Nix flake

2023-09-02

Introduction

I have been using nix for local development for over a year now, but want to start learning more, so three weeks ago I posted some question on Mastodon:

Okay. Time to learn some more #nix than just using it for local development!

I have a server running #nixos 23.05 and I want to set up some services that I have written myself. The first service is a simple IRC bot written in Haskell.

Where do I begin? 🤷

It should basically end up with a binary on the server and a service that is pretty much just this:
[Service]
ExecStart=/some/where/my_silly_bot
Restart=always
Do I create some config in the git repo and refer to it from the nixos server? Am I creating a “module”? (I duckduckwent it, but not even knowing what concepts I should search for made it a bit difficult. 😇)

I got some helpful pointers and realized there will be two steps to this:

Package the project for building with nix (this post)
Set up the service on a NixOS server (next post) # Creating a flake for my Haskell project

The project I’m going to package is a simple IRC bot: https://github.com/ehamberg/tribot.

So far, I’ve only written flakes to get a local dev environment when opening a project in a shell or in an editor (by using direnv and nix-direnv), but now I also want to be able to nix build the project.

In other words, there are two goals the nix flake should fulfil:

Make it possible to build the project with nix build
Make it possible to get a local development environment for the project using direnv + nix-direnv (or nix develop)

Attempt 1: A basic setup for Haskell projects

Fortunately, Gabriella Gonzalez has written a great post on incrementally packaging a Haskell program, so we’re off to a great start:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { nixpkgs, flake-utils, ... }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        config = { };

        overlay = pkgsNew: pkgsOld: {
          tribot = pkgsNew.haskell.lib.justStaticExecutables
            pkgsNew.haskellPackages.tribot;

          haskellPackages = pkgsOld.haskellPackages.override (old: {
            overrides =
              pkgsNew.haskell.lib.packageSourceOverrides { tribot = ./.; };
          });
        };

        pkgs = import nixpkgs {
          inherit config system;
          overlays = [ overlay ];
        };

      in rec {
        packages.default = pkgs.haskellPackages.tribot;

        apps.default = {
          type = "app";
          program = "${pkgs.tribot}/bin/tribot";
        };

        devShells = {
            default = pkgs.mkShell { buildInputs = with pkgs; [
                haskellPackages.haskell-language-server
                haskellPackages.hlint
                haskellPackages.cabal-fmt
                haskellPackages.ormolu
                cabal-install
                sqlite
                zlib
            ]; };
        };
      });
}

This makes sense for the most part, even though there are some pieces that are somewhat mysterious¹.

This would normally be the end of packaging a Haskell project, but while the dev shell still works, nix build does not:

❯ nix build
error: Package ‘simpleirc-0.3.1’ in /nix/store/qx67jipw01zps1rqgmmpl7as1irff275-source/pkgs/development/haskell-modules/hackage-packages.nix:268059 is marked as broken, refusing to evaluate.
a) To temporarily allow broken packages, you can use an environment variable
          for a single invocation of the nix tools.

            $ export NIXPKGS_ALLOW_BROKEN=1
		Note: For `nix shell`, `nix build`, `nix develop` or any other Nix 2.4+
        (Flake) command, `--impure` must be passed in order to read this
        environment variable.

Ooof!

Attempt 2: Allow broken‽

Allowing broken packages sounds like a really bad idea and – at best – a short-term solution, but sure, let’s try:

❯ NIXPKGS_ALLOW_BROKEN=1 nix build --impure
warning: Git tree '/Users/ehamberg/Developer/tribot' is dirty
error: builder for '/nix/store/a0cr7b79dcmak5m59aiam7ksxihgfrjc-simpleirc-0.3.1.drv' failed with exit code 1;
       last 10 log lines:
       >   |
       > 3 | import           Test.Hspec.Monadic
       >   | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       >
       > tests/Spec.hs:4:1: error:
       >     Could not find module ‘CoreSpec’
       >     Use -v (or `:set -v` in ghci) to see a list of the files searched for.
       >   |
       > 4 | import qualified CoreSpec
       >   | ^^^^^^^^^^^^^^^^^^^^^^^^^
       For full logs, run 'nix log /nix/store/a0cr7b79dcmak5m59aiam7ksxihgfrjc-simpleirc-0.3.1.drv'.
error: 1 dependencies of derivation '/nix/store/48ds1qxc73masqwrp6yagpj1d5zdn8r7-tribot-0.4.0.0.drv' failed to build

Oh no! Looks like a test is broken.

Attempt 3: Disable the tests!

Hmm. Since we’re already doing naughty things, let’s see how to disable testing for a haskell package. Spoiler: We can use haskell.lib.dontCheck: Let’s fix our flake to use this to override our package with one without tests:

@@ -14,8 +14,21 @@
             pkgsNew.haskellPackages.tribot;

           haskellPackages = pkgsOld.haskellPackages.override (old: {
-            overrides =
-              pkgsNew.haskell.lib.packageSourceOverrides { tribot = ./.; };
+            overrides = let
+              oldOverrides = old.overrides or (_: _: { });
+
+              manualOverrides = haskellPackagesNew: haskellPackagesOld: {
+                simpleirc =
+                  pkgsNew.haskell.lib.dontCheck haskellPackagesOld.simpleirc;
+              };
+
+              sourceOverrides =
+                pkgsNew.haskell.lib.packageSourceOverrides { tribot = ./.; };
+
+            in pkgsNew.lib.fold pkgsNew.lib.composeExtensions oldOverrides ([
+              sourceOverrides
+              manualOverrides
+            ]);
           });
         };

Yay!

❯ NIXPKGS_ALLOW_BROKEN=1 nix build --impure
warning: Git tree '/Users/ehamberg/Developer/tribot' is dirty
error: builder for '/nix/store/k1g0n2xgpk9mf2hm0dp321paihpfmr65-tribot-0.4.0.0.drv' failed with exit code 1;
       last 10 log lines:
       >
       > app/Main.hs:12:1: error:
       >     Could not find module ‘Network.SimpleIRC.Sasl’
       >     Perhaps you meant
       >       Network.SimpleIRC.Core (from simpleirc-0.3.1)
       >       Network.SimpleIRC (from simpleirc-0.3.1)
       >     Use -v (or `:set -v` in ghci) to see a list of the files searched for.
       >    |
       > 12 | import Network.SimpleIRC.Sasl (SaslPlainArgs (..), saslPlain)
       >    | ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       For full logs, run 'nix log /nix/store/k1g0n2xgpk9mf2hm0dp321paihpfmr65-tribot-0.4.0.0.drv'.

No yay! Turns out all of this was a red herring. My project actually depends on a newer version of SimpleIRC than the version available on Hackage (that is marked as broken…).

Turns out I had this tucked away in a long-forgotten cabal.project file:

source-repository-package
  type: git
  location: git://github.com/dom96/SimpleIRC.git

packages: ./tribot.cabal

Attempt 4: Straight to the source!

Okay. So we need to fetch SimpleIRC straight from Github. Turns out that there’s a fetchGit function in Nix (or nixpkgs? how do I know?), so we can use this to fetch a given revision from Github (and reenable tests 😮‍💨):

@@ -18,8 +18,14 @@
               oldOverrides = old.overrides or (_: _: { });

               manualOverrides = haskellPackagesNew: haskellPackagesOld: {
-                simpleirc =
-                  pkgsNew.haskell.lib.dontCheck haskellPackagesOld.simpleirc;
+                simpleirc = let
+                  src = builtins.fetchGit {
+                    url = "https://github.com/dom96/SimpleIrc";
+                    ref = "master";
+                    rev = "8d156a89801be2c9b6923d85e6b199c8173e445a";
+                  };
+                in pkgs.haskell.lib.dontCheck
+                (haskellPackagesOld.callCabal2nix "simpleirc" src { });
               };

               sourceOverrides =

And boom! nix build works! ## The final flake.nix:

{
  inputs = {
    nixpkgs.url = "github:NixOS/nixpkgs/nixpkgs-unstable";
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { nixpkgs, flake-utils, ... }:
    flake-utils.lib.eachDefaultSystem (system:
      let
        config = { };

        overlay = pkgsNew: pkgsOld: {
          tribot = pkgsNew.haskell.lib.justStaticExecutables
            pkgsNew.haskellPackages.tribot;

          haskellPackages = pkgsOld.haskellPackages.override (old: {
            overrides = let
              oldOverrides = old.overrides or (_: _: { });

              manualOverrides = haskellPackagesNew: haskellPackagesOld: {
                simpleirc = let
                  src = builtins.fetchGit {
                    url = "https://github.com/dom96/SimpleIrc";
                    ref = "master";
                    rev = "8d156a89801be2c9b6923d85e6b199c8173e445a";
                  };
                in pkgs.haskell.lib.dontCheck
                (haskellPackagesOld.callCabal2nix "simpleirc" src { });
              };

              sourceOverrides =
                pkgsNew.haskell.lib.packageSourceOverrides { tribot = ./.; };

            in pkgsNew.lib.fold pkgsNew.lib.composeExtensions oldOverrides ([
              sourceOverrides
              manualOverrides
            ]);
          });
        };

        pkgs = import nixpkgs {
          inherit config system;
          overlays = [ overlay ];
        };

      in rec {
        packages.default = pkgs.haskellPackages.tribot;

        apps.default = {
          type = "app";
          program = "${pkgs.tribot}/bin/tribot";
        };

        devShells = {
          default = pkgs.mkShell {
            buildInputs = with pkgs; [
              haskellPackages.haskell-language-server
              haskellPackages.hlint
              haskellPackages.cabal-fmt
              haskellPackages.ormolu
              cabal-install
              zlib
              sqlite
            ];
          };
        };
      });
}

Conclusion

This was a somewhat confusing journey, but to be fair, if I didn’t depend on an unreleased version of a library, the very first flake.nix would probably have worked.

Now, the next step is to create a service on a NixOS server. That’s the next post.

Erlend Hamberg