evenstore 3.5.0 on NixOS

KurrentDB
1

Hi,

I was trying to update the install script for NixOS 3.5.0 but hit a small pb.

I could start the install of EventStore 3.5.0 on ubuntu after installing

sudo apt-get install build-essential g++

and issuing

./scripts/build-js1/build-js1-linux.sh werror=no

by trial and error (I have no idea if I “have a newer compiler which treats warnings appearing as a result of compiling the Google V8 codebase as errors.”)

not adding the werror=no leads to

make: *** empty string invalid as file name. Stop.

But when I try to do a manual source install on NixOS 15.09, I get

…

  • CXXFLAGS=-fPIC
  • make x64.release werror=no
    PYTHONPATH=“/tmp/nix-shell-es/EventStore-oss-v3.5.0-src/scripts/build-js1/v8/tools/generate_shim_headers:”
    GYP_GENERATORS=make
    build/gyp/gyp --generator-output=“out” build/all.gyp
    -Ibuild/standalone.gypi --depth=.
    -Dv8_target_arch=x64
    -Dv8_optimized_debug=0
    -S.x64.release -Dv8_enable_backtrace=1 -Dwerror=‘’ -Darm_fpu=default -Darm_float_abi=default
    /bin/sh: build/gyp/gyp: /bin/bash: bad interpreter: No such file or directory
    Makefile:389: recipe for target ‘out/Makefile.x64.release’ failed
    make: *** [out/Makefile.x64.release] Error 126

And indeed there is no bash program in /bin/

[nix-shell:/tmp/nix-shell-es/EventStore-oss-v3.5.0-src]$ ls -la /bin/
total 12
drwxr-xr-x 2 root root 4096 Feb 24 12:25 .
drwxr-xr-x 18 root root 4096 Feb 24 12:16 …
lrwxrwxrwx 1 root root 63 Feb 24 12:25 sh → /nix/store/l80ddf18bbig2icv6cmgjfws9a2vm3jj-bash-4.3-p42/bin/sh

However when I inspect the script I dont know which program is looking for such thing.

As an aside, I imagine that’s probably why the script wisely starts with

#!/usr/bin/env bash

and not the direct location of bash.

Anyone knows a blessed way to deal with this ?

2

So for this, there was indeed a blessed way, graciously hinted to me :

you want to run the patchShebangs shell function on your script. The function is automatically run on files installed to the store, but not for source files.

That is when the files are outside the Nix store, their references to tools like bash need to be update.

The header of the script then changed after run to :

[nix-shell:/tmp/nix-shell-es/EventStore-oss-v3.5.0-src]$ cat ./scripts/build-js1/build-js1-linux.sh
#!/nix/store/12h8h2drnvla4005hvhx1dm259ck0la9-bash-4.3-p42/bin/bash
set -e
…

After doing this, the script goes on, but blocks at a new place :

  • pushd /tmp/nix-shell-es/EventStore-oss-v3.5.0-src/scripts/build-js1/v8
  • pushd /tmp/nix-shell-es/EventStore-oss-v3.5.0-src/scripts/build-js1/v8
  • CFLAGS=-fPIC
  • CXXFLAGS=-fPIC
  • make x64.release werror=no
    PYTHONPATH=“/tmp/nix-shell-es/EventStore-oss-v3.5.0-src/scripts/build-js1/v8/tools/generate_shim_headers:/nix/store/mg4kcymmz1m5s14mw2vwyp39ff8n94bd-python-2.7.11/lib/python2.7/site-packages”
    GYP_GENERATORS=make
    build/gyp/gyp --generator-output=“out” build/all.gyp
    -Ibuild/standalone.gypi --depth=.
    -Dv8_target_arch=x64
    -Dv8_optimized_debug=0
    -S.x64.release -Dv8_enable_backtrace=1 -Dwerror=‘’ -Darm_fpu=default -Darm_float_abi=default
    /bin/sh: which: command not found
    gyp: Call to ‘($(echo ${CXX_host:-$(which g++)}) -m64 -E - > /dev/null 2>&1 < /dev/null) && echo “-m64” || true’ returned exit status 0.
    Makefile:389: recipe for target ‘out/Makefile.x64.release’ failed
    make: *** [out/Makefile.x64.release] Error 1

maybe the answer is in the nixos wiki on ‘standard environment’

3

which is a standard unix command that returns where something is.

http://linux.die.net/man/1/which

4

yeah that’s clearly something NixOS specific. will dig and report.

5

so adding which as a buildInputs fixed it.

I know have the same problem on Ubuntu and NixOs, that of file expansion

  • make x64.release werror=no
    make[1]: Entering directory ‘/tmp/nix-shell-es/EventStore-oss-v3.5.0-src/scripts/build-js1/v8/out’
    make[1]: Nothing to be done for ‘all’.
    make[1]: Leaving directory ‘/tmp/nix-shell-es/EventStore-oss-v3.5.0-src/scripts/build-js1/v8/out’
  • popd
  • local outputDir=/tmp/nix-shell-es/EventStore-oss-v3.5.0-src/scripts/build-js1/…/…/src/libs/x64/nixos-
  • [[ -d /tmp/nix-shell-es/EventStore-oss-v3.5.0-src/scripts/build-js1/…/…/src/libs/x64/nixos- ]]
  • pushd /tmp/nix-shell-es/EventStore-oss-v3.5.0-src/scripts/build-js1/…/…/src/EventStore.Projections.v8Integration/
  • local outputObj=/tmp/nix-shell-es/EventStore-oss-v3.5.0-src/scripts/build-js1/…/…/src/libs/x64/nixos-/libjs1.so
  • g++ -I ‘/tmp/nix-shell-es/EventStore-oss-v3.5.0-src/scripts/build-js1/v8/include *.cpp’ -o /tmp/nix-shell-es/EventStore-oss-v3.5.0-src/scripts/build-js1/…/…/src/libs/x64/nixos-/libjs1.so -fPIC -shared -std=c++0x -lstdc++ -Wl,–start-group ‘/tmp/nix-shell-es/EventStore-oss-v3.5.0-src/scripts/build-js1/v8/out/x64.release/obj.target/tools/gyp/libv8_{base.x64,nosnapshot.x64}.a’ ‘/tmp/nix-shell-es/EventStore-oss-v3.5.0-src/scripts/build-js1/v8/out/x64.release/obj.target/third_party/icu/libicu{i18n,uc,data}.a’ -Wl,–end-group -lrt -lpthread
    g++: error: /tmp/nix-shell-es/EventStore-oss-v3.5.0-src/scripts/build-js1/v8/out/x64.release/obj.target/tools/gyp/libv8_{base.x64,nosnapshot.x64}.a: No such file or directory
    g++: error: /tmp/nix-shell-es/EventStore-oss-v3.5.0-src/scripts/build-js1/v8/out/x64.release/obj.target/third_party/icu/libicu{i18n,uc,data}.a: No such file or directory

although the files exists

6

I just removed the file expansion from the build script file and it works.

Maybe some version of bash handles that gracefully, it t’s the culprit, but it does not seem to be the case for 4.3 on NixOs / ubuntu

7

our builds are on ubuntu.

8

Nicolas, which version of Ubuntu does the build fail on? The build runs on Ubuntu 14.04

9

I saw your build was on Ubuntu, that’s why I used it to serve me as a reference for NixOS, which has its own idiosyncracies.

I use a 12.04.5 LTS fresh from vagrant, but the problem is probably elsewhere.

Configuration is always a mess, that’s why NixOs is so attractive.

10

We use 14.04 (and 15.??)

11

It’s now available on NixOS thanks to Brian McKenna aka Puffnfresh’s experience :

One thing which was not casher when I attempted to blindly mirror the install instructions and package that as immutable, was the script making network access to pull and build V8. Nix was made for those kind of dependencies issues, and asks the environment to provide for one such variable in the nix expression

The command to build js1 gets reduced to

cc -o …/libs/x64/nixos/libjs1.so -fPIC -lv8 -shared -std=c++0x *.cpp

Hopefully this is correct on a few platforms, I have not tested that yet…

Now eventstore is available on NixOS which is really a nice to run services, as it’s immutable infrastructure…