psa-systems/vervain-server
Watch
9
0
Fork
You've already forked vervain-server
 0
Code Issues Pull requests 1 Projects Releases 2 Packages Wiki Activity Actions

fix(agent): detect dead agent WS via inbound-idle deadline #100

Merged
Claude-Run merged 1 commit from feat/VS-80-agent-ws-idle-deadline into main 2026-07-02 04:47:52 +02:00
Conversation 0 Commits 1 Files changed 1 +61 -8
Claude-Run commented 2026-07-02 04:46:50 +02:00
Member
Copy link

The agent control-channel session only cleared BIT_AGENT when the select loop broke on a clean Close, a failed send, or recv yielding None/Err. A host that drops abruptly (power off, cable pull, crash) leaves a half-open socket: axum/tungstenite never times it out, and a server ping buffers and returns Ok until the kernel abandons the socket (tcp_retries2, ~15 min or longer under a black-hole). The device stayed green "Online" with a frozen "Last seen" the whole time.

Add an application-level read-idle deadline. Track last_seen, reset it on every Some(Ok(_)) from socket.recv() (including Pong, so a healthy agent answering the 30s server ping stays alive), and add a tokio::time::sleep_until(last_seen + IDLE_DEADLINE) branch to the select that breaks on expiry. The existing post-loop teardown then clears BIT_AGENT, drops the sender via clear_sender_if_current, and records the agentdisconnect event, so the device flips offline in the UI and via the nodeconnect push.

IDLE_DEADLINE is a named constant set to 75s (2.5x PING_INTERVAL): two consecutive missed Pongs trip it. A guard test enforces IDLE_DEADLINE >= 2x PING_INTERVAL so nobody lowers it into false-positive territory. Also corrected the stale PING_INTERVAL comment that claimed the WebSocket layer times out stale connections; it does not.

#VS-80

The agent control-channel session only cleared BIT_AGENT when the select loop broke on a clean Close, a failed send, or recv yielding None/Err. A host that drops abruptly (power off, cable pull, crash) leaves a half-open socket: axum/tungstenite never times it out, and a server ping buffers and returns Ok until the kernel abandons the socket (tcp_retries2, ~15 min or longer under a black-hole). The device stayed green "Online" with a frozen "Last seen" the whole time. Add an application-level read-idle deadline. Track last_seen, reset it on every Some(Ok(_)) from socket.recv() (including Pong, so a healthy agent answering the 30s server ping stays alive), and add a tokio::time::sleep_until(last_seen + IDLE_DEADLINE) branch to the select that breaks on expiry. The existing post-loop teardown then clears BIT_AGENT, drops the sender via clear_sender_if_current, and records the agentdisconnect event, so the device flips offline in the UI and via the nodeconnect push. IDLE_DEADLINE is a named constant set to 75s (2.5x PING_INTERVAL): two consecutive missed Pongs trip it. A guard test enforces IDLE_DEADLINE >= 2x PING_INTERVAL so nobody lowers it into false-positive territory. Also corrected the stale PING_INTERVAL comment that claimed the WebSocket layer times out stale connections; it does not. #VS-80
fix(agent): detect dead agent WS via inbound-idle deadline
All checks were successful
Check / fmt + clippy + build + tests (pull_request) Successful in 57s
Details
Create release / Create release from merged PR (pull_request) Has been skipped
Details
f7eeadff59 
The agent control-channel session only cleared BIT_AGENT when the select loop broke on a clean Close, a failed send, or recv yielding None/Err. A host that drops abruptly (power off, cable pull, crash) leaves a half-open socket: axum/tungstenite never times it out, and a server ping buffers and returns Ok until the kernel abandons the socket (tcp_retries2, ~15 min or longer under a black-hole). The device stayed green "Online" with a frozen "Last seen" the whole time.

Add an application-level read-idle deadline. Track last_seen, reset it on every Some(Ok(_)) from socket.recv() (including Pong, so a healthy agent answering the 30s server ping stays alive), and add a tokio::time::sleep_until(last_seen + IDLE_DEADLINE) branch to the select that breaks on expiry. The existing post-loop teardown then clears BIT_AGENT, drops the sender via clear_sender_if_current, and records the agentdisconnect event, so the device flips offline in the UI and via the nodeconnect push.

IDLE_DEADLINE is a named constant set to 75s (2.5x PING_INTERVAL): two consecutive missed Pongs trip it. A guard test enforces IDLE_DEADLINE >= 2x PING_INTERVAL so nobody lowers it into false-positive territory. Also corrected the stale PING_INTERVAL comment that claimed the WebSocket layer times out stale connections; it does not.

#VS-80
Claude-Run deleted branch feat/VS-80-agent-ws-idle-deadline 2026-07-02 04:47:52 +02:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Projects
Clear projects
No items
No project
Assignees
Clear assignees
No assignees
1 participant
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
psa-systems/vervain-server!100
No description provided.