Problem: The dispatcher was placed AFTER the prebuffer await, causing a deadlock:
- Pump waits for data on track_rx
- Cache waits for pump to produce PCM data
- Code awaits cache completion before reaching dispatcher
- Dispatcher never runs → pump never receives data → deadlock
Solution: Use tokio::select! to dispatch segments in parallel with prebuffer wait
Architecture now has 3 phases:
1. Phase 1: Dispatch chunks + await prebuffer (in parallel via select!)
2. Phase 2: Copy metadata + push to playlist (after prebuffer complete)
3. Phase 3: Continue dispatching until TrackBoundary
This fixes the "sans musique" blocking issue where the system would freeze
waiting for prebuffer that could never complete.
Problem: When TrackBoundary arrived, the pump was awaited before continuing,
causing file truncation when pcm_tx was dropped while data was still buffering.
Solution: Allow multiple pump tasks to run in parallel:
- Create dedicated channel (track_tx/track_rx) for each track's pump
- Main loop reads from rx and dispatches segments to current pump via track_tx
- When TrackBoundary arrives: drop track_tx (signals pump to finish) and immediately start new pump
- Old pump continues writing in background until all data is flushed
This prevents truncation in progressive cache scenario (radio streaming).
Changes in flac_cache_sink.rs:
- Replace pump_track_segments_owned() with pump_track_segments_from_channel()
- Remove rx ownership passing - each pump gets its own channel
- Dispatcher loop reads rx and forwards to active pump
- No await on pump completion - let it finish in background
Problem:
- tokio::join!() waited for both cache_future AND pump_future to complete
- cache_future returned after prebuffer (~530ms)
- pump_future read entire first track (~19s)
- Track only pushed to playlist after both finished → 19s delay
Solution (Solution A from OPTIMIZATION_PREBUFFER_TO_PLAYLIST.md):
- Created pump_track_segments_owned() that takes ownership of rx and returns it
- Spawned pump in tokio::spawn to run independently
- Wait for cache_future alone → push to playlist immediately
- Wait for pump_handle later to recover rx for next track
Results (tested with play_and_cache --null-audio):
Before:
- Prebuffer → playlist: ~19s
- Prebuffer → playback: ~19.5s
After:
- Prebuffer → playlist: ~24ms
- Prebuffer → playback: ~76ms
- Improvement: 99.6% (250x faster!)
Target was <1s, achieved 76ms (13x better than target!)
Changes:
- Added pump_track_segments_owned() in flac_cache_sink.rs:516
- Modified FlacCacheSinkLogic::process() to use tokio::spawn pattern
- Added timing logs (INFO level) for prebuffer and playlist push
- rx ownership properly managed: moved to pump, returned, recovered
Tests passed:
✅ Prebuffer completes in ~530ms (512KB downloaded)
✅ Track pushed to playlist in ~24ms after prebuffer
✅ Playback starts in ~76ms after prebuffer
✅ rx properly recovered for next tracks
✅ No panics or deadlocks
Added comprehensive logging to FlacCacheSink::process():
- Process start
- Waiting for/receiving first audio chunk
- FLAC encoder creation
- Cache ingestion and pump parallel execution
- tokio::join! completion
- Track added to cache confirmation
Testing results show PROGRESSIVE CACHING WORKS:
✅ Prebuffer reached in 0.6 seconds
✅ Track added to cache with pk
✅ Download pipeline completes successfully
✅ Playlist receives track
✅ Playback starts
Current timing:
- t=0.6s: Prebuffer complete (512KB)
- t=3.6s: Track added to playlist (after pump completes)
- t=4.5s: Playback starts
The 3s delay is because tokio::join! waits for BOTH futures:
- cache_future (returns after prebuffer ~0.6s)
- pump_future (pumps entire first track ~3s)
For true 1-2s startup, would need to refactor to push to playlist
immediately after prebuffer, without waiting for pump to complete.
Problem: All FLAC files with the same format (44.1kHz, stereo, 16-bit)
had identical headers and thus the same pk (071c5713d5cf485ca688832207bef0f9).
This caused the cache to think all tracks were the same file, regardless
of channel selection or actual content.
Solution: Skip the FLAC header (first 512 bytes) and calculate the pk
from bytes 512-1024 (actual audio content) instead. This ensures each
track gets a unique pk based on its actual audio data, not just its
format header.
Changes:
- Modified add_from_reader_with_pk() to read 1024 bytes instead of 512
- Use bytes 512-1024 for pk calculation when explicit_pk is None
- This works even with poor metadata (empty artist/title)
- Maintains backward compatibility with explicit_pk parameter
Fixes the issue where changing radio channel played the same song.
When a file was already in cache, FlacCacheSink would drain all
remaining segments (which can take 13+ seconds - the full track
duration) BEFORE adding the track to the playlist. This caused
a long delay before playback could start.
The fix reorders operations to:
1. Copy metadata to cache (fast)
2. Add pk to playlist IMMEDIATELY (fast)
3. Drain remaining segments (slow, but playback already started)
This ensures the playlist receives tracks immediately, allowing
playback to start without waiting for segment drainage to complete.
Fixes the 13-second delay when playing already-cached files.
- Add .complete marker files to track completed downloads
- Check marker instead of file size for completion detection
- Drain segments when file already in cache to avoid pipeline errors
- Consolidate() now removes incomplete files without markers
- Add new_cache_with_consolidation() for automatic cleanup on startup
Quand un fichier est déjà en cache, add_from_reader() retourne immédiatement
sans lire le stream FLAC, ce qui ferme le channel PCM. Avant cette correction,
pump_track_segments() retournait une erreur SendError, causant l'échec du
pipeline download.
Changements :
- Dans pump_track_segments(), détecter quand le channel est fermé
- Retourner Ok avec StopReason::ChannelClosed au lieu d'une erreur
- Ceci permet au pipeline de se terminer gracieusement
Cette situation est normale et attendue quand le fichier est déjà en cache.
Corrections :
- Removed unused Cursor import
- Fixed borrow checker issues by using tokio::join! instead of tokio::spawn
- Kept progressive streaming approach with add_from_reader
La solution finale utilise tokio::join! pour exécuter pump_track_segments
et add_from_reader en parallèle, évitant ainsi les problèmes de lifetime
avec tokio::spawn tout en conservant le streaming progressif.
Cette correction implémente le cache progressif et le streaming pour permettre
un démarrage quasi immédiat de la lecture pendant le téléchargement.
## Changements dans FlacCacheSink (pmoaudio-ext)
Avant :
- Accumulait tout le FLAC en mémoire dans un buffer
- Attendait la fin complète de l'encodage avant d'ajouter au cache
- Ajoutait à la playlist seulement après ingestion complète
Après :
- Passe le flux FLAC directement à add_from_reader
- add_from_reader retourne dès que le prebuffer (512 KB) est atteint
- Le PK est ajouté à la playlist immédiatement après le prebuffer
- L'encodage et l'écriture continuent en arrière-plan
## Changements dans play_and_cache.rs
- Suppression du sleep de 2 secondes avant le démarrage de la lecture
- Ajout de commentaire expliquant le mécanisme de prebuffer
- La lecture démarre dès que le prebuffer est atteint (~1-2 secondes)
## Résultat
La musique démarre maintenant presque immédiatement après le début du
téléchargement (temps du prebuffer) au lieu d'attendre la fin du
téléchargement complet du premier morceau.
Improve the architecture by configuring the playlist handle directly
in register_playlist() instead of deferring it to run().
Changes:
- Remove playlist_handle_pending field (no longer needed)
- register_playlist() now calls logic_mut() to configure immediately
- run() becomes a simple delegation with no configuration logic
- Follows proper pattern: configuration before run(), not during run()
This is cleaner than the previous approach which used a pending field
and transferred it during run(). The new approach:
1. User calls register_playlist() → directly configures logic
2. User calls run() → simple delegation to inner.run()
Architecture now properly separates configuration from execution.
- Add Node::logic_mut() method to allow post-construction configuration
of node logic before run() is called
- Fix FlacCacheSink to properly transfer playlist_handle_pending to
the inner logic using logic_mut()
- Resolves FIXME at flac_cache_sink.rs:656 about missing logic_mut()
This enables the playlist registration mechanism to work correctly:
1. User calls register_playlist() on FlacCacheSink
2. Handle is stored in playlist_handle_pending
3. During run(), handle is transferred to FlacCacheSinkLogic
4. Tracks are automatically added to playlist after caching
