Refactor audio crate

[EzraEllette]

description

• Moved pcm_to_mel into screenpipe and switched to tokio runtime.
• Reapplied LRU cache for speaker embeddings
• lazy load onnx sessions
• stt is now fully async

related issue: #1318

how to test

Test using instruments.

Update your profile (~/.zshrc)

alias instruments="open /Applications/Xcode.app/Contents/Applications/Instruments.app"
get-task-allow () {
	codesign -s - -v -f --entitlements ~/.get-task-allow.ent $1
}

Create .get-task-allow.ent in ~

.get-task-allow.ent
<?xml version="1.0" encoding="UTF-8"?>

<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
    <dict>
        <key>com.apple.security.get-task-allow</key>
        <true/>
    </dict>
</plist>

Update screenpipe/Cargo.toml [release] section (optional)

[release]
debug=true

Testing (TEST THOROUGHLY)

1. Build screenpipe with --release (optional)
2. run get-task-allow <path_to_screenpipe_build>
3. Open instruments with instruments
4. Select the Leaks & your screenpipe build as the target.
5. Edit the target to add arguments.
6. Press the record button. Fix permissions. It helps to remove screenpipe completely from permissions and add it back when prompted.
7. Use screenpipe normally

Screenshots

Before

After

Ran for 13:33 and had under 400MB usage

/claim #1318

[vercel]

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name	Status	Preview	Comments	Updated (UTC)
screenpipe	✅ Ready (Inspect)	Visit Preview	💬 Add feedback	Feb 11, 2025 5:47am