Add dynamic endpointing to the Node.js voice pipeline#1284
Conversation
Port the Python dynamic endpointing tracker so Node sessions can adapt turn-ending delays from observed pause timing. Add the source-parity endpointing tests and wire the new runtime into audio recognition and agent activity.
|
|
Rosetta Bot seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account. You have signed the CLA already but the status is still pending? Let us recheck it. |
![devin-ai-integration[bot]](https://avatars.githubusercontent.com/in/811515?s=60&v=4){kind=small}
| private isInterruptionDetectionEnabled: boolean; | ||
| private isInterruptionByAudioActivityEnabled: boolean; | ||
| private isDefaultInterruptionByAudioActivityEnabled: boolean; | ||
| private interruptionDetected = false; |
There was a problem hiding this comment.
🟡 Missing // Ref: Python reference comment on interruptionDetected state additions
Per CLAUDE.md rules, every JS change that corresponds to a Python change must carry an inline // Ref: python <relative-file-path> - <line-range> lines comment directly above the relevant line(s). The new interruptionDetected field and its assignment in onInterruptionOverlappingSpeech are clearly ported from the Python agent and lack the required reference comment.
Was this helpful? React with 👍 or 👎 to provide feedback.
| this.onError(ev); | ||
|
|
||
| private readonly onInterruptionOverlappingSpeech = (ev: OverlappingSpeechEvent): void => { | ||
| this.interruptionDetected = ev.isInterruption; |
There was a problem hiding this comment.
🟡 Missing // Ref: Python reference comment on interruptionDetected assignment in onInterruptionOverlappingSpeech
Per CLAUDE.md rules, every JS change that corresponds to a Python change must carry an inline // Ref: comment. The new line this.interruptionDetected = ev.isInterruption; inside the onInterruptionOverlappingSpeech handler is ported from Python and lacks the required reference comment.
Was this helpful? React with 👍 or 👎 to provide feedback.
| }); | ||
| if (this.isInterruptionDetectionEnabled && this.audioRecognition) { | ||
| this.audioRecognition.onStartOfAgentSpeech(); | ||
| this.audioRecognition.onStartOfAgentSpeech(replyStartedSpeakingAt); |
There was a problem hiding this comment.
🟡 Missing // Ref: Python reference comment on onStartOfAgentSpeech call-site change in ttsTask
Per CLAUDE.md rules, every JS change that corresponds to a Python change must carry an inline // Ref: comment. The call to this.audioRecognition.onStartOfAgentSpeech(replyStartedSpeakingAt) was changed from the no-arg form this.audioRecognition.onStartOfAgentSpeech() to pass a timestamp — a Python-ported behavioral change — but has no // Ref: comment.
Was this helpful? React with 👍 or 👎 to provide feedback.
| }); | ||
| if (this.isInterruptionDetectionEnabled && this.audioRecognition) { | ||
| this.audioRecognition.onStartOfAgentSpeech(); | ||
| this.audioRecognition.onStartOfAgentSpeech(agentStartedSpeakingAt); |
There was a problem hiding this comment.
🟡 Missing // Ref: Python reference comment on onStartOfAgentSpeech call-site change in _pipelineReplyTaskImpl
Per CLAUDE.md rules, every JS change that corresponds to a Python change must carry an inline // Ref: comment. The call to this.audioRecognition.onStartOfAgentSpeech(agentStartedSpeakingAt) was changed from the no-arg form to pass a timestamp — a Python-ported behavioral change — but has no // Ref: comment.
Was this helpful? React with 👍 or 👎 to provide feedback.
2 participants
This PR was created by Rosetta.
Tracking issue: https://github.com/livekit/rosetta/issues/74
Source implementation: https://github.com/livekit/agents/blob/main/livekit-agents/livekit/agents/voice/endpointing.py
Source tests: https://github.com/livekit/agents/blob/main/tests/test_endpointing.py
Docs: https://docs.livekit.io/reference/agents/turn-handling-options/#endpointingoptions-usage
Summary
turnHandling.endpointing.mode = 'dynamic'changes live turn completion behavioragents/src/voice/endpointing.test.tsand update the nearby audio-recognition tests for the new constructor shape