Skip to content

Passing -j32 for ninja no longer works #53176

Closed
#53187
#53181
Closed
Passing -j32 for ninja no longer works#53176
#53187
#53181
Labels
buildIssues and PRs related to Node.js builds or CI infrastructure.macosIssues and PRs related to the macOS platform.

Description

@anonrig
anonrig
opened on May 27, 2024

Running ./configure --ninja -C and later running -make -j32 no longer works on main.

Possibly related to. 19f0bca

cc @tniessen

node on  main via ⬢ v22.2.0
❯ make -j32
ninja -C out/Release  -j
ninja: option requires an argument -- j
usage: ninja [options] [targets...]

if targets are unspecified, builds the 'default' target (see manual).

options:
  --version      print ninja version ("1.12.1")
  -v, --verbose  show all command lines while building
  --quiet        don't show progress status, just command output

  -C DIR   change to DIR before doing anything else
  -f FILE  specify input build file [default=build.ninja]

  -j N     run N jobs in parallel (0 means infinity) [default=12 on this system]
  -k N     keep going until N jobs fail (0 means infinity) [default=1]
  -l N     do not start new jobs if the load average is greater than N
  -n       dry run (don't run commands but act like they succeeded)

  -d MODE  enable debugging (use '-d list' to list modes)
  -t TOOL  run a subtool (use '-t list' to list subtools)
    terminates toplevel options; further flags are passed to the tool
  -w FLAG  adjust warnings (use '-w list' to list warnings)
make: *** [node] Error 1

Activity

  1. anonrig
    changed the title [-]Passing ninja job no longer works[/-] [+]Passing `-j32` for ninja no longer works[/+] on May 27, 2024

  2. tniessen commented on May 28, 2024

    @tniessen
    tniessen
    on May 28, 2024
    Member

    @anonrig What OS distribution is this, and what flavor of make? For my GNU make, 19f0bca appears to have fixed the handling of -j. Maybe this is due to some difference between different make implementations?

  3. tniessen commented on May 28, 2024

    @tniessen
    tniessen
    on May 28, 2024
    Member

    @anonrig @jakecastelli This Dockerfile demonstrates that passing -j from make to ninja works on Ubuntu on the main branch:

    FROM ubuntu:latest

RUN apt-get update && apt-get install -y git build-essential ninja-build python3

WORKDIR /nodejs
RUN git clone --depth=1 https://github.com/nodejs/node.git
WORKDIR /nodejs/node
RUN ./configure --ninja
RUN make -j3
    $ docker build --progress=plain .
...
#10 [7/7] RUN make -j3
#10 0.519 ninja -C out/Release  -j3
#10 0.521 ninja: Entering directory `out/Release'

  4. targos commented on May 28, 2024

    @targos
    targos
    on May 28, 2024
    Member

    I'm on macOS and it doesn't work:

    $ make -v
GNU Make 3.81
Copyright (C) 2006  Free Software Foundation, Inc.
This is free software; see the source for copying conditions.
There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A
PARTICULAR PURPOSE.

This program built for i386-apple-darwin11.3.0
  5. tniessen
    added
    buildIssues and PRs related to Node.js builds or CI infrastructure.
    macosIssues and PRs related to the macOS platform.
    on May 28, 2024

  6. jakecastelli commented on May 28, 2024

    @jakecastelli
    jakecastelli
    on May 28, 2024
    Member

    I am on MacOS as well, exactly the same GNU Make as yours @targos - the current work around for me is to use make JOBS=n

  7. tniessen commented on May 28, 2024

    @tniessen
    tniessen
    on May 28, 2024
    Member

    On Linux, it seems that GNU Make 3.81 was obsolete in 2010 and last distributed in Ubuntu 14 — a decade ago... is that the most recent official release for macOS?

    I found this StackOverflow thread that seems related, and which is also about macOS. It also mentions differences in how MAKEFLAGS is passed between make 3.x and 4.x.

  8. targos commented on May 28, 2024

    @targos
    targos
    on May 28, 2024
    Member

    The default make version available on macOS probably won't change in the near future, but it's possible to install the latest using homebrew: https://formulae.brew.sh/formula/make#default

  9. jakecastelli commented on May 28, 2024

    @jakecastelli
    jakecastelli
    on May 28, 2024
    Member

    I upgraded my make to v4.4.1 and I can verify it works on main (-jn can be properly propagated).

    $ make -v
GNU Make 4.4.1
Built for aarch64-apple-darwin22.3.0
Copyright (C) 1988-2023 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

    I think your fix 19f0bca is legit! @tniessen we might need to update the README for mac users.

  10. jakecastelli commented on May 28, 2024

    @jakecastelli
    jakecastelli
    on May 28, 2024
    Member

    I think in conclusion for mac users there are two options:

    1. use make JOBS=n instead of make -jn if you are using the macOS default make.
    2. upgrade make to v4.x through homebrew and run make -jn.

    Are you guys happy for me to update the BUILDING.md in a separate PR?

  11. tniessen commented on May 28, 2024

    @tniessen
    tniessen
    on May 28, 2024
    Member

    @jakecastelli Thank you so much for confirming! Let me try to add a workaround for ancient versions of GNU make to our Makefile.

    @targos Since you have access to macOS and are still using GNU Make 3.81, could you please test if this patch fixes the propagation?

    diff --git a/Makefile b/Makefile
index 5187d6a950..8f579bada1 100644
--- a/Makefile
+++ b/Makefile
@@ -147,7 +147,11 @@ ifdef JOBS
        NINJA_ARGS := $(NINJA_ARGS) -j$(JOBS)
 else
        IMMEDIATE_NINJA_ARGS := $(NINJA_ARGS)
-       NINJA_ARGS = $(IMMEDIATE_NINJA_ARGS) $(filter -j%,$(MAKEFLAGS))
+       ifneq (3.81,$(MAKE_VERSION))
+               NINJA_ARGS = $(IMMEDIATE_NINJA_ARGS) $(filter -j%,$(MAKEFLAGS))
+       else
+               NINJA_ARGS := $(IMMEDIATE_NINJA_ARGS) $(filter -j%,$(MAKEFLAGS))
+       endif
 endif
 $(NODE_EXE): config.gypi out/Release/build.ninja
        $(NINJA) -C out/Release $(NINJA_ARGS)

    If it does, I can open a PR and add an explanation to the Makefile.

  12. targos commented on May 28, 2024

    @targos
    targos
    on May 28, 2024
    Member

    I notice one change of behavior with make v4. When configuring with --debug, it now runs the ninja commands for both Release and Debug builds in parallel (with v3, it was in sequence):

    $ make -j6
ninja -C out/Release  -j6
ninja -C out/Debug  -j6
ninja: Entering directory `out/Release'
ninja: Entering directory `out/Debug'

  13. targos commented on May 28, 2024

    @targos
    targos
    on May 28, 2024
    Member

    @tniessen I wouldn't say it fixes the propagation, but at least it goes back to the previous "working" behavior that doesn't propagate the -j flag.

  14. jakecastelli commented on May 28, 2024

    @jakecastelli
    jakecastelli
    on May 28, 2024
    Member

    If I remember correctly with make v3 it builds the release and then the debug build separately right?

    edit: sorry you have already mentioned here

    with v3, it was in sequence

  15. 11 remaining items

  16. jakecastelli commented on May 28, 2024

    @jakecastelli
    jakecastelli
    on May 28, 2024
    Member

    Would you like to add a note to building-node-with-ninja.md about this issue with ancient make versions?

    Sure, I can do that 👍

  17. nodejs-github-bot
    closed this as completedin #53187on May 30, 2024
  18. nodejs-github-bot
    added a commit that references this issue on May 30, 2024

    doc: add note to ninjia build for macOS using -jn flag

    47c5571
  19. targos
    added a commit that references this issue on Jun 1, 2024

    doc: add note to ninjia build for macOS using -jn flag

    812f0e9
  20. nodejs-github-bot
    added a commit that references this issue on Jun 8, 2024

    build: fix spacing before NINJA_ARGS

    5a446cc
  21. targos
    added a commit that references this issue on Jun 20, 2024

    build: fix spacing before NINJA_ARGS

    36aad8b
  22. eliphazbouye
    added 2 commits that reference this issue on Jun 20, 2024

    doc: add note to ninjia build for macOS using -jn flag

    09504bd

    build: fix spacing before NINJA_ARGS

    4cf6913
  23. dgrelaud
    added a commit that references this issue on Mar 30, 2026

    install GNU make v4 to fix make -j N option

    406ae66
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    buildIssues and PRs related to Node.js builds or CI infrastructure.macosIssues and PRs related to the macOS platform.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions