Skip to content

test-single-executable-application* tests flaking on macOS #59553

Closed as not planned
Closed as not planned
test-single-executable-application* tests flaking on macOS#59553
Labels
flaky-testIssues and PRs involving tests that fail intermittently in CI.macosIssues and PRs related to the macOS platform.single-executableIssues and PRs related to single-executable applications.staleIssues and PRs marked stale due to inactivity and scheduled for automatic closure.

Description

@richardlau
richardlau
opened on Aug 20, 2025

Test

test-single-executable-application

Platform

macOS

Console output

not ok 4699 sequential/test-single-executable-application
  ---
  duration_ms: 11491.49500
  severity: fail
  exitcode: 1
  stack: |-
    Copied /Users/admin/build/workspace/node-test-commit-osx/nodes/osx13-x64/out/Release/node to /Users/admin/node-tmp/.tmp.4698/sea
    Injected /Users/admin/node-tmp/.tmp.4698/sea-prep.blob into /Users/admin/node-tmp/.tmp.4698/sea
    Signed /Users/admin/node-tmp/.tmp.4698/sea
    [process 11198]: --- stderr ---
    
    [process 11198]: --- stdout ---
    
    [process 11198]: status = null, signal = SIGSEGV
    /Users/admin/build/workspace/node-test-commit-osx/nodes/osx13-x64/test/common/child_process.js:98
        throw error;
        ^
    
    Error: - process terminated with status null, expected 0
    - process terminated with signal SIGSEGV, expected null
        at Object.<anonymous> (/Users/admin/build/workspace/node-test-commit-osx/nodes/osx13-x64/test/sequential/test-single-executable-application.js:54:1)
        at Module._compile (node:internal/modules/cjs/loade...

Build links

Additional information

According to

Several test-single-executable-application* tests have been failing on one or more pull requests on macOS.

  • sequential/test-single-executable-application
  • sequential/test-single-executable-application-assets
  • sequential/test-single-executable-application-assets-raw
  • sequential/test-single-executable-application-disable-experimental-sea-warning
  • sequential/test-single-executable-application-exec-argv
  • sequential/test-single-executable-application-exec-argv-empty
  • sequential/test-single-executable-application-snapshot
  • sequential/test-single-executable-application-snapshot-and-code-cache
  • sequential/test-single-executable-application-snapshot-worker
  • sequential/test-single-executable-application-use-code-cache

It looks like the injected binary is crashing.

cc @nodejs/single-executable

Activity

  1. richardlau
    added
    flaky-testIssues and PRs involving tests that fail intermittently in CI.
    macosIssues and PRs related to the macOS platform.
    single-executableIssues and PRs related to single-executable applications.
    on Aug 20, 2025

  2. joyeecheung commented on Aug 20, 2025

    @joyeecheung
    joyeecheung
    on Aug 20, 2025
    Member

    They seem to have gone in the recent CI runs: https://ci.nodejs.org/job/node-test-commit-osx/

    I once ran into this after a rebase which was strangely persistent, and then it went away after another rebase. #59276 (comment) I wonder if it has something to do with the ephemeral macOS hosts and/or caching?

  3. panva commented on Aug 20, 2025

    @panva
    panva
    on Aug 20, 2025
    Member

    They seem to have gone in the recent CI runs: ci.nodejs.org/job/node-test-commit-osx

    I once ran into this after a rebase which was strangely persistent, and then it went away after another rebase. #59276 (comment) I wonder if it has something to do with the ephemeral macOS hosts and/or caching?

    I had the same experience in #59497

  4. joyeecheung commented on Sep 2, 2025

    @joyeecheung
    joyeecheung
    on Sep 2, 2025
    Member

    They seem to have come back 5 days ago and then gone, I noticed some patterns in https://github.com/nodejs/reliability/blob/main/reports/2025-09-02.md

    1. They happen in a short amount of time on a few jobs and then go away (in the last report they happened in 68906, 68910, 68914, 68916 and then disappeared)
    2. In the CI it only happens on the ephemeral macOS hosts
    3. They don't seem to happen on GitHub actions on macOS, and I haven't seen them happening locally on my macOS machine

    Given the pattern I still feel that it may be an issue with caching. Another possibility is code signing (I think it also crashes with segfault if you don't sign it properly on macOS) though I don't know how that's set up in the CI machines...

  5. richardlau commented on Sep 25, 2025

    @richardlau
    richardlau
    on Sep 25, 2025
    MemberAuthor

    I have no idea if it's the same cause, but the tests are consistently failing on the new macos15-x64 builds.

    e.g. https://ci.nodejs.org/job/node-test-commit-osx/66925/nodes=macos15-x64/#showFailuresLink

    07:53:44     [process 31240]: --- stderr ---
07:53:44     dyld[31240]: failed to set up thread local variables for '/Users/admin/node-tmp/.tmp.4757/sea': unsupported thread-local, larger than 4GB

    @nodejs/single-executable

  6. ryanaslett commented on Sep 25, 2025

    @ryanaslett
    ryanaslett
    on Sep 25, 2025

    "Another possibility is code signing" -> the only VM's capable of code signing are the release machines, unless these tests are relying on an existing binary that already been signed, I think that can be ruled out.

  7. joyeecheung commented on Sep 25, 2025

    @joyeecheung
    joyeecheung
    on Sep 25, 2025
    Member

    unless these tests are relying on an existing binary that already been signed, I think that can be ruled out.

    The tests relied on signing the generated SEAs or else they won't run at all. The tests would skip running these generated binaries if code signing is not available and therefore would not crash at running them (note that the tests don't need to sign them with an identity, the command is merely codesign --sign - /path/to/sea, I think it doesn't need any setup - I've never set up my local machine for signing with an identity and it can sign the SEA for the tests with this command just alright). Although it does need the codesign tool to work properly in the machine, while I am not sure if the macs in the CI has that...

  8. nodejs-github-bot
    added a commit that references this issue on Sep 30, 2025

    test: mark sea tests flaky on macOS x64

    f304bc2
  9. targos
    added a commit that references this issue on Oct 6, 2025

    test: mark sea tests flaky on macOS x64

    1d5cc5e

  10. joyeecheung commented on Oct 8, 2025

    @joyeecheung
    joyeecheung
    on Oct 8, 2025
    Member

    The unsupported thread-local, larger than 4GB seems to appear quite often for macos x64. I wonder if it's a bug in postject that corrupts the executable during injection and dyld ends up thinking something is TLS even though it's not. Maybe updating the lief dependency in postject would do something considering it's been a bit outdated and there might be some bug fixes..

  11. aduh95
    added a commit that references this issue on Oct 11, 2025

    test: mark sea tests flaky on macOS x64

    7067d79
  12. sxa
    added
    commit-queuePRs queued for automated landing through the Commit Queue.
    and removed
    commit-queuePRs queued for automated landing through the Commit Queue.
    on Oct 13, 2025

  13. 17 remaining items

  14. aduh95
    added a commit that references this issue on Feb 19, 2026

    test: skip sea tests on x64 macOS

    b8ef464
  15. zkochan
    added 2 commits that reference this issue on May 4, 2026

    fix(exe): drop darwin-x64 artifact (upstream Node SEA bug) (#11455)

    548208a

    fix(exe): drop darwin-x64 artifact (upstream Node SEA bug) (#11455)

    47b100e
  16. dadezzz
    added a commit that references this issue on May 11, 2026

    chore(deps): update pnpm to v11 (#621)

    3eab30b

  17. github-actions commented on May 12, 2026

    @github-actions
    github-actionsbot
    on May 12, 2026 – with GitHub Actions
    Contributor

    This issue has been marked as stale due to 210 days of inactivity.
    It will be automatically closed in 30 days if no further activity occurs. If this is still relevant, please leave a comment or update it to keep it open.

  18. github-actions
    added
    staleIssues and PRs marked stale due to inactivity and scheduled for automatic closure.
    on May 12, 2026

  19. github-actions commented on Jun 11, 2026

    @github-actions
    github-actionsbot
    on Jun 11, 2026 – with GitHub Actions
    Contributor

    This issue has been automatically closed after 30 days of inactivity following its stale status (no activity for a total of 240 days).
    If this is still relevant, feel free to reopen it or leave a comment with additional details so we can continue the discussion.

  20. github-actions
    closed this as not plannedon Jun 11, 2026
  21. Mearman
    added 2 commits that reference this issue on Aug 2, 2026

    docs: root-cause the macOS x64 SEA segfault to a known, unfixed upstr…

    1bd1469

    fix: install macOS x64 Homebrew via npm instead of the broken SEA binary

    6f2f406
  22. lldwb
    added 5 commits that reference this issue on Sep 22, 2026

    fix(ci): mac x64 自编译基底改用经典链接器，避开上游 SEA 注入损坏

    d5a7965

    fix(ci): 交叉基底改用经典链接器，绕 nodejs/node#59553 的 TLS 注入损坏

    2be3acc

    fix(ci): mac x64 自编译基底改用经典链接器，避开上游 SEA 注入损坏

    a49d268

    fix(ci): 交叉基底改用经典链接器，绕 nodejs/node#59553 的 TLS 注入损坏

    2dc0394

    feat(app-packaging): references 补四条 CI 与 SEA 注入实证坑

    3741026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    flaky-testIssues and PRs involving tests that fail intermittently in CI.macosIssues and PRs related to the macOS platform.single-executableIssues and PRs related to single-executable applications.staleIssues and PRs marked stale due to inactivity and scheduled for automatic closure.

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions