Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Running multiple nx targets simultaneously causes Nx daemon to shut down #29813

Open
1 of 4 tasks
jsmecham opened this issue Jan 30, 2025 · 2 comments
Open
1 of 4 tasks

Running multiple nx targets simultaneously causes Nx daemon to shut down #29813

jsmecham opened this issue Jan 30, 2025 · 2 comments
Assignees
@AgentEnder
Labels
scope: core core nx functionality type: bug

Comments

@jsmecham
Copy link

Current Behavior

In our large monorepo, we launch our tech stack using a Procfile that is responsible for starting various backend and frontend pieces. The frontend pieces are all managed with Nx, and so the Procfile looks similar to this:

frontend_process_a: nx start process_a
frontend_process_b: nx start process_b
some_other_task: nx build some_other_task

If the Nx daemon is not already running, it will attempt to start by multiple of these calls when Overmind launches each service with Nx. See the attached log output for what it looks like from the Nx daemon's perspective. It takes one or two more successive attempts at running Overmind for the Nx daemon to stabilize and allow each of these parallel tasks to succeed.

Expected Behavior

Nx should be able to handle this gracefully and wait for a singular daemon to be established, instead of killing one already in process of starting up.

GitHub Repo

No response

Steps to Reproduce

  1. Create a Procfile that launches multiple services via Nx
  2. Use a process runner like Overmind to launch the services from that Procfile
  3. Observe Nx exiting with an error such as Daemon process terminated and closed the connection.

Nx Report

Node           : 22.13.1
OS             : darwin-arm64
Native Target  : aarch64-macos
npm            : 11.0.0

nx                 : 20.3.3
@nx/js             : 20.3.3
@nx/eslint         : 20.3.3
@nx/workspace      : 20.3.3
@nx/cypress        : 20.3.3
@nx/devkit         : 20.3.3
@nx/eslint-plugin  : 20.3.3
@nx/storybook      : 20.3.3
@nx/vite           : 20.3.3
typescript         : 5.7.3
---------------------------------------
Community plugins:
nx-stylelint : 18.0.0

Failure Logs

NX Daemon Server] - 2025-01-30T17:27:32.097Z - Started listening on: /var/folders/8g/tjrxvphn17n2jc89dwt236680000gp/T/d2024cb0980ff5fca499/d.sock
[NX Daemon Server] - 2025-01-30T17:27:32.098Z - Started listening on: /var/folders/8g/tjrxvphn17n2jc89dwt236680000gp/T/d2024cb0980ff5fca499/d.sock
[NX Daemon Server] - 2025-01-30T17:27:32.099Z - [WATCHER]: Subscribed to changes within: /Users/Justin/[redacted]/App (native)
[NX Daemon Server] - 2025-01-30T17:27:32.098Z - Started listening on: /var/folders/8g/tjrxvphn17n2jc89dwt236680000gp/T/d2024cb0980ff5fca499/d.sock
[NX Daemon Server] - 2025-01-30T17:27:32.099Z - [WATCHER]: Subscribed to changes within: /Users/Justin/[redacted]/App (native)
[NX Daemon Server] - 2025-01-30T17:27:32.100Z - [WATCHER]: Stopping the watcher for /Users/Justin/[redacted]/App (sources)
[NX Daemon Server] - 2025-01-30T17:27:32.100Z - [WATCHER]: Subscribed to changes within: /Users/Justin/[redacted]/App (native)
[NX Daemon Server] - 2025-01-30T17:27:32.100Z - [WATCHER]: Stopping the watcher for /Users/Justin/[redacted]/App (sources)
[NX Daemon Server] - 2025-01-30T17:27:32.100Z - [WATCHER]: Stopping the watcher for /Users/Justin/[redacted]/App (outputs)
[NX Daemon Server] - 2025-01-30T17:27:32.100Z - Server stopped because: "this process is no longer the current daemon (native)"
[NX Daemon Server] - 2025-01-30T17:27:32.101Z - [WATCHER]: Stopping the watcher for /Users/Justin/[redacted]/App (outputs)
[NX Daemon Server] - 2025-01-30T17:27:32.101Z - Server stopped because: "this process is no longer the current daemon (native)"
[NX Daemon Server] - 2025-01-30T17:27:32.178Z - [WATCHER]: Processing file changes in outputs
[NX Daemon Server] - 2025-01-30T17:27:32.497Z - [REQUEST]: Updated workspace context based on watched changes, recomputing project graph...
[NX Daemon Server] - 2025-01-30T17:27:32.498Z - [REQUEST]: 
[NX Daemon Server] - 2025-01-30T17:27:32.498Z - [REQUEST]: 
[NX Daemon Server] - 2025-01-30T17:27:32.695Z - Started listening on: /var/folders/8g/tjrxvphn17n2jc89dwt236680000gp/T/d2024cb0980ff5fca499/d.sock
[NX Daemon Server] - 2025-01-30T17:27:32.696Z - [WATCHER]: Subscribed to changes within: /Users/Justin/[redacted]/App (native)
[NX Daemon Server] - 2025-01-30T17:27:32.699Z - Established a connection. Number of open connections: 1
[NX Daemon Server] - 2025-01-30T17:27:32.699Z - Established a connection. Number of open connections: 2
[NX Daemon Server] - 2025-01-30T17:27:32.700Z - Closed a connection. Number of open connections: 1
[NX Daemon Server] - 2025-01-30T17:27:32.700Z - [REQUEST]: Client Request for Project Graph Received
[NX Daemon Server] - 2025-01-30T17:27:32.702Z - [WATCHER]: Stopping the watcher for /Users/Justin/[redacted]/App (sources)
[NX Daemon Server] - 2025-01-30T17:27:32.702Z - [WATCHER]: Stopping the watcher for /Users/Justin/[redacted]/App (sources)
[NX Daemon Server] - 2025-01-30T17:27:32.702Z - [WATCHER]: Stopping the watcher for /Users/Justin/[redacted]/App (outputs)
[NX Daemon Server] - 2025-01-30T17:27:32.702Z - Server stopped because: "this process is no longer the current daemon (native)"
[NX Daemon Server] - 2025-01-30T17:27:32.711Z - Time taken for 'loadSpecifiedNxPlugins' 0.7660000000000053ms
[NX Daemon Server] - 2025-01-30T17:27:32.757Z - [WATCHER]: Processing file changes in outputs
[NX Daemon Server] - 2025-01-30T17:27:32.892Z - Time taken for 'Load Nx Plugin: /Users/Justin/[redacted]/App/node_modules/nx/src/plugins/package-json' 191.06116699999998ms
[NX Daemon Server] - 2025-01-30T17:27:32.905Z - Time taken for 'Load Nx Plugin: /Users/Justin/[redacted]/App/node_modules/nx/src/plugins/js' 204.250959ms
[NX Daemon Server] - 2025-01-30T17:27:32.932Z - [REQUEST]: Updated workspace context based on watched changes, recomputing project graph...
[NX Daemon Server] - 2025-01-30T17:27:32.932Z - [REQUEST]: 
[NX Daemon Server] - 2025-01-30T17:27:32.932Z - [REQUEST]: 
[NX Daemon Server] - 2025-01-30T17:27:32.953Z - Time taken for 'loadDefaultNxPlugins' 205.040209ms
[NX Daemon Server] - 2025-01-30T17:27:33.003Z - Started listening on: /var/folders/8g/tjrxvphn17n2jc89dwt236680000gp/T/d2024cb0980ff5fca499/d.sock
[NX Daemon Server] - 2025-01-30T17:27:33.005Z - [WATCHER]: Subscribed to changes within: /Users/Justin/[redacted]/App (native)
[NX Daemon Server] - 2025-01-30T17:27:33.017Z - Established a connection. Number of open connections: 1
[NX Daemon Server] - 2025-01-30T17:27:33.018Z - Established a connection. Number of open connections: 2
[NX Daemon Server] - 2025-01-30T17:27:33.019Z - Closed a connection. Number of open connections: 1
[NX Daemon Server] - 2025-01-30T17:27:33.019Z - Established a connection. Number of open connections: 2
[NX Daemon Server] - 2025-01-30T17:27:33.019Z - [REQUEST]: Client Request for Project Graph Received
[NX Daemon Server] - 2025-01-30T17:27:33.025Z - Time taken for 'loadSpecifiedNxPlugins' 0.6605419999999924ms
[NX Daemon Server] - 2025-01-30T17:27:33.025Z - Established a connection. Number of open connections: 3
[NX Daemon Server] - 2025-01-30T17:27:33.026Z - Closed a connection. Number of open connections: 2
[NX Daemon Server] - 2025-01-30T17:27:33.026Z - Established a connection. Number of open connections: 3
[NX Daemon Server] - 2025-01-30T17:27:33.026Z - Closed a connection. Number of open connections: 2
[NX Daemon Server] - 2025-01-30T17:27:33.026Z - Established a connection. Number of open connections: 3
[NX Daemon Server] - 2025-01-30T17:27:33.028Z - [REQUEST]: Client Request for Project Graph Received
[NX Daemon Server] - 2025-01-30T17:27:33.029Z - Established a connection. Number of open connections: 4
[NX Daemon Server] - 2025-01-30T17:27:33.029Z - [REQUEST]: Client Request for Project Graph Received
[NX Daemon Server] - 2025-01-30T17:27:33.029Z - Established a connection. Number of open connections: 5
[NX Daemon Server] - 2025-01-30T17:27:33.029Z - Closed a connection. Number of open connections: 4
[NX Daemon Server] - 2025-01-30T17:27:33.030Z - [REQUEST]: Client Request for Project Graph Received
[NX Daemon Server] - 2025-01-30T17:27:33.074Z - [WATCHER]: Processing file changes in outputs
[NX Daemon Server] - 2025-01-30T17:27:33.219Z - Time taken for 'build-project-configs' 26.47387500000002ms
[NX Daemon Server] - 2025-01-30T17:27:33.220Z - [WATCHER]: Stopping the watcher for /Users/Justin/[redacted]/App (sources)
[NX Daemon Server] - 2025-01-30T17:27:33.220Z - Closed a connection. Number of open connections: 0
[NX Daemon Server] - 2025-01-30T17:27:33.220Z - [WATCHER]: Stopping the watcher for /Users/Justin/[redacted]/App (sources)
[NX Daemon Server] - 2025-01-30T17:27:33.220Z - [WATCHER]: Stopping the watcher for /Users/Justin/[redacted]/App (sources)
[NX Daemon Server] - 2025-01-30T17:27:33.220Z - [WATCHER]: Stopping the watcher for /Users/Justin/[redacted]/App (outputs)
[NX Daemon Server] - 2025-01-30T17:27:33.220Z - Server stopped because: "this process is no longer the current daemon (native)"
[NX Daemon Server] - 2025-01-30T17:27:33.332Z - Time taken for 'Load Nx Plugin: /Users/Justin/[redacted]/App/node_modules/nx/src/plugins/package-json' 308.96325ms
[NX Daemon Server] - 2025-01-30T17:27:33.340Z - Time taken for 'Load Nx Plugin: /Users/Justin/[redacted]/App/node_modules/nx/src/plugins/js' 317.570125ms
[NX Daemon Server] - 2025-01-30T17:27:33.379Z - [REQUEST]: Updated workspace context based on watched changes, recomputing project graph...
[NX Daemon Server] - 2025-01-30T17:27:33.379Z - [REQUEST]: 
[NX Daemon Server] - 2025-01-30T17:27:33.379Z - [REQUEST]: 
[NX Daemon Server] - 2025-01-30T17:27:33.394Z - Time taken for 'loadDefaultNxPlugins' 326.72058400000003ms
[NX Daemon Server] - 2025-01-30T17:27:33.517Z - Time taken for 'build-project-configs' 25.235250000000008ms
[NX Daemon Server] - 2025-01-30T17:27:33.518Z - [WATCHER]: Processing file changes in outputs
[NX Daemon Server] - 2025-01-30T17:27:33.518Z - Closed a connection. Number of open connections: 3
[NX Daemon Server] - 2025-01-30T17:27:33.518Z - Closed a connection. Number of open connections: 2
[NX Daemon Server] - 2025-01-30T17:27:33.518Z - Closed a connection. Number of open connections: 1
[NX Daemon Server] - 2025-01-30T17:27:33.518Z - Closed a connection. Number of open connections: 0
[NX Daemon Server] - 2025-01-30T17:27:33.647Z - [SYNC]: collect registered sync generators
[NX Daemon Server] - 2025-01-30T17:27:33.648Z - [REQUEST]: Responding to the client. project-graph
[NX Daemon Server] - 2025-01-30T17:27:33.648Z - [REQUEST]: Responding to the client. project-graph
[NX Daemon Server] - 2025-01-30T17:27:33.648Z - [REQUEST]: Responding to the client. project-graph
[NX Daemon Server] - 2025-01-30T17:27:33.648Z - [REQUEST]: Responding to the client. project-graph
Error: This socket has been ended by the other party
    at genericNodeError (node:internal/errors:983:15)
    at wrappedFn (node:internal/errors:537:14)
    at Socket.writeAfterFIN [as write] (node:net:580:14)
    at /Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/shutdown-utils.js:73:16
    at new Promise (<anonymous>)
    at respondToClient (/Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/shutdown-utils.js:69:12)
    at handleResult (/Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/server.js:178:52)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async handleMessage (/Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/server.js:107:9)
    at async /Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/server.js:72:9 {
  code: 'EPIPE'
}
[NX Daemon Server] - 2025-01-30T17:27:33.648Z - Done responding to the client project-graph
Error: This socket has been ended by the other party
    at genericNodeError (node:internal/errors:983:15)
    at wrappedFn (node:internal/errors:537:14)
    at Socket.writeAfterFIN [as write] (node:net:580:14)
    at /Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/shutdown-utils.js:73:16
    at new Promise (<anonymous>)
    at respondToClient (/Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/shutdown-utils.js:69:12)
    at handleResult (/Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/server.js:178:52)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async handleMessage (/Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/server.js:107:9)
    at async /Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/server.js:72:9 {
  code: 'EPIPE'
}
[NX Daemon Server] - 2025-01-30T17:27:33.649Z - Done responding to the client project-graph
Error: This socket has been ended by the other party
    at genericNodeError (node:internal/errors:983:15)
    at wrappedFn (node:internal/errors:537:14)
    at Socket.writeAfterFIN [as write] (node:net:580:14)
    at /Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/shutdown-utils.js:73:16
    at new Promise (<anonymous>)
    at respondToClient (/Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/shutdown-utils.js:69:12)
    at handleResult (/Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/server.js:178:52)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async handleMessage (/Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/server.js:107:9)
    at async /Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/server.js:72:9 {
  code: 'EPIPE'
}
[NX Daemon Server] - 2025-01-30T17:27:33.649Z - Done responding to the client project-graph
Error: This socket has been ended by the other party
    at genericNodeError (node:internal/errors:983:15)
    at wrappedFn (node:internal/errors:537:14)
    at Socket.writeAfterFIN [as write] (node:net:580:14)
    at /Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/shutdown-utils.js:73:16
    at new Promise (<anonymous>)
    at respondToClient (/Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/shutdown-utils.js:69:12)
    at handleResult (/Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/server.js:178:52)
    at process.processTicksAndRejections (node:internal/process/task_queues:105:5)
    at async handleMessage (/Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/server.js:107:9)
    at async /Users/Justin/[redacted]/App/node_modules/nx/src/daemon/server/server.js:72:9 {
  code: 'EPIPE'
}
[NX Daemon Server] - 2025-01-30T17:27:33.649Z - Done responding to the client project-graph
[NX Daemon Server] - 2025-01-30T17:27:33.649Z - Handled REQUEST_PROJECT_GRAPH. Handling time: 629. Response time: 1.
[NX Daemon Server] - 2025-01-30T17:27:33.649Z - Handled REQUEST_PROJECT_GRAPH. Handling time: 620. Response time: 1.
[NX Daemon Server] - 2025-01-30T17:27:33.649Z - Handled REQUEST_PROJECT_GRAPH. Handling time: 619. Response time: 1.
[NX Daemon Server] - 2025-01-30T17:27:33.649Z - Handled REQUEST_PROJECT_GRAPH. Handling time: 618. Response time: 1.
[NX Daemon Server] - 2025-01-30T17:27:33.649Z - Time taken for 'total for creating and serializing project graph' 618.3367079999999ms
[NX Daemon Server] - 2025-01-30T17:27:34.480Z - [WATCHER]: Processing file changes in outputs
[NX Daemon Server] - 2025-01-30T17:27:34.741Z - [WATCHER]: Processing file changes in outputs
[NX Daemon Server] - 2025-01-30T17:27:34.998Z - [WATCHER]: Processing file changes in outputs
[NX Daemon Server] - 2025-01-30T17:27:36.319Z - [WATCHER]: Processing file changes in outputs
[NX Daemon Server] - 2025-01-30T17:27:38.517Z - [WATCHER]: Processing file changes in outputs
[NX Daemon Server] - 2025-01-30T17:41:10.364Z - [WATCHER]: Stopping the watcher for /Users/Justin/[redacted]/App (sources)
[NX Daemon Server] - 2025-01-30T17:41:10.364Z - [WATCHER]: Stopping the watcher for /Users/Justin/[redacted]/App (outputs)
[NX Daemon Server] - 2025-01-30T17:41:10.365Z - Server stopped because: "this process is no longer the current daemon (native)"
[NX Daemon Server] - 2025-01-30T17:41:10.365Z - Started listening on: /var/folders/8g/tjrxvphn17n2jc89dwt236680000gp/T/d2024cb0980ff5fca499/d.sock
[NX Daemon Server] - 2025-01-30T17:41:10.366Z - [WATCHER]: Subscribed to changes within: /Users/Justin/[redacted]/App (native)
[NX Daemon Server] - 2025-01-30T17:41:10.375Z - Established a connection. Number of open connections: 1
[NX Daemon Server] - 2025-01-30T17:41:10.375Z - Established a connection. Number of open connections: 2
[NX Daemon Server] - 2025-01-30T17:41:10.376Z - Closed a connection. Number of open connections: 1
[NX Daemon Server] - 2025-01-30T17:41:10.377Z - [REQUEST]: Client Request for Project Graph Received
[NX Daemon Server] - 2025-01-30T17:41:10.382Z - Time taken for 'loadSpecifiedNxPlugins' 0.569625000000002ms
[NX Daemon Server] - 2025-01-30T17:41:10.428Z - [WATCHER]: Processing file changes in outputs
[NX Daemon Server] - 2025-01-30T17:41:10.540Z - Time taken for 'Load Nx Plugin: /Users/Justin/[redacted]/App/node_modules/nx/src/plugins/project-json/build-nodes/project-json' 162.16500000000002ms
[NX Daemon Server] - 2025-01-30T17:41:10.545Z - Time taken for 'Load Nx Plugin: /Users/Justin/[redacted]/App/node_modules/nx/src/plugins/js' 168.02075ms
[NX Daemon Server] - 2025-01-30T17:41:10.557Z - [REQUEST]: Updated workspace context based on watched changes, recomputing project graph...
[NX Daemon Server] - 2025-01-30T17:41:10.557Z - [REQUEST]: 
[NX Daemon Server] - 2025-01-30T17:41:10.557Z - [REQUEST]: 
[NX Daemon Server] - 2025-01-30T17:41:10.572Z - Time taken for 'loadDefaultNxPlugins' 168.41812500000003ms
[NX Daemon Server] - 2025-01-30T17:41:10.679Z - Time taken for 'build-project-configs' 14.844458999999972ms
[NX Daemon Server] - 2025-01-30T17:41:10.724Z - [SYNC]: collect registered sync generators
[NX Daemon Server] - 2025-01-30T17:41:10.724Z - [REQUEST]: Responding to the client. project-graph
[NX Daemon Server] - 2025-01-30T17:41:10.725Z - Time taken for 'total for creating and serializing project graph' 347.698666ms
[NX Daemon Server] - 2025-01-30T17:41:10.727Z - Done responding to the client project-graph
[NX Daemon Server] - 2025-01-30T17:41:10.727Z - Handled REQUEST_PROJECT_GRAPH. Handling time: 347. Response time: 3.
[NX Daemon Server] - 2025-01-30T17:41:10.751Z - Closed a connection. Number of open connections: 0

Package Manager Version

No response

Operating System

  • macOS
  • Linux
  • Windows
  • Other (Please specify)

Additional Information

No response
@FrozenPandaz FrozenPandaz added the scope: core core nx functionality label Jan 31, 2025
@FrozenPandaz
Copy link
Collaborator

@AgentEnder actually implemented something just the other day to address this. Could you please see if 20.5.0-beta.0 fixes this issue for you?

@AgentEnder
Copy link
Member

The file lock actually is really only for the no-daemon case, a similar lock for the daemon starting up may also make sense though 🤔. We could reuse the file lock mechanism in the daemon client

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@AgentEnder AgentEnder

Labels
scope: core core nx functionality type: bug
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@jsmecham @AgentEnder @FrozenPandaz