Commit Graph

275 Commits

Author SHA1 Message Date
henryruhs 19f8f5f206 bring back the face store but for source and reference only
(cherry picked from commit 9d8a5be714)
2026-06-05 21:47:57 +02:00
Henry Ruhs 775985645e Push based receive with queue (#1146)
* move to push based receive

* move to push based receive, fix mocks

* fix tests

* add todos

* remove asyncio

* remove asyncio

* resolve todos

* move to queue without events

* prevent debug spam

* concurrent stream inference

stream_video.py: pipeline face-swap inference across execution_thread_count workers (ThreadPoolExecutor + bounded in-flight deque, ordered encode) to keep the GPU busy during encode

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>

* add todos

* add todos

* add missing state

---------

Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 00:18:57 +02:00
henryruhs 1f494f54db fix hashes 2026-06-03 10:43:16 +02:00
henryruhs 476a21cc7a fix hashes 2026-06-03 10:43:09 +02:00
henryruhs 121c3a5af6 update hashes 2026-06-03 10:36:00 +02:00
Henry Ruhs 2ac9b70550 Feat/remb both direction (#1145)
* revamp remb to take both directions

* extract more methods
2026-06-03 10:31:33 +02:00
Henry Ruhs 7181b41f2d Feat/finalize stream (#1144)
* break stream helper into pieces

* remove todos
2026-06-02 16:05:21 +02:00
Harisreedhar beeb1d99e9 Refactor(stream-helper): split encode/receive loops and unify audio/video structure (#1142)
* split encode loop

* unify audio and video encode loop methods

* improve variable names

* split receive methods

* test improve

* try to avoid != with even more weird approach

* remove source_path check

* remove empty variables

* fix lint

* avoid not in condition

* rename

* fix lint
2026-06-02 01:50:40 +05:30
Henry Ruhs e2e4e6a95b Refactor/stream helper testing (#1141)
* fix tests part1

* improve testing

* use real audio

* more renaming

* switch to hash assertions

* switch to hash assertions

* switch to hash assertions

* switch to hash assertions

* skip on macos
2026-06-01 18:42:37 +02:00
Harisreedhar 3eeb505c86 Refactor(stream-helper): pass peer context objects to receivers, fix codec guard, move YUV conversion into decoders (#1140)
* extract numpy.empty(0) into an empty_vision_frame variable so the sentinel intent is clear

* bring back todos

* bring back todos

* rename opus_buffer to audio

* move reshape and cvtColor into decoder modules

* add audio_codec check

* add audio_codec check

* rename audio video buffer

* simplify receive methods and cleanup test

* todo

* revert decode methods to return pointer

* remove duplicate test_stream_helper — tests live in test_api_stream_helper

Co-Authored-By: Claude Sonnet 4.6 <noreply@anthropic.com>

---------

Co-authored-by: Claude Sonnet 4.6 <noreply@anthropic.com>
2026-06-01 17:21:06 +05:30
harisreedhar 02f684e220 bring back todos 2026-06-01 15:48:10 +05:30
harisreedhar a88d8ead49 bring back todos 2026-06-01 15:45:44 +05:30
harisreedhar 7735c3740f bring back todos 2026-06-01 15:19:28 +05:30
henryruhs f2de9f1843 rename to video pack and audio pack 2026-06-01 11:41:25 +02:00
Harisreedhar aea36a9e55 Replace queues with timestamped deques and fix Audio Video sync (#1139)
* change to deque

* remove single line condition

* change threading to asyncio

* bringback todos
2026-06-01 15:03:33 +05:30
Henry Ruhs 0f5f75ba51 Cleanup/testing suite (#1136)
* clean testing suite

* clean testing suite part2

* clean testing suite part3

* add todos

* extend testing suite and kill some mutants

* fix hashes

* fix lint

* fix test

* fix test
2026-06-01 08:54:37 +02:00
Henry Ruhs 460c65004b Add available event (#1134)
* remove sleep with available event

* reorder methods, caller has to follow variable names of consumer, reorder tests methods

* more todos for naming
2026-05-30 16:39:48 +05:30
Henry Ruhs 1ac0e3e9a4 remove face store (#1132) (#1133) 2026-05-30 11:55:58 +02:00
Harisreedhar 9f6a19c1d2 feat(rtc): 2-way REMB with in-place encoder bitrate update (#1131)
* implement remb other direction

* update test
2026-05-30 04:41:10 +05:30
Harisreedhar 6b9ddd9a4f feat(rtc): REMB bitrate adaptation with in-place encoder update (#1130)
* improve test

* fix lint

* cleanup

* cleanup
2026-05-29 21:15:03 +05:30
henryruhs 871559cb6a clean and simplify tests 2026-05-29 15:44:07 +02:00
Harisreedhar 2a8672b54d Implement tier 1 REMB (#1129)
* implement tier 1

* fix lint

* cleanup

* cleanup

* fix lint

* use clear_remb method

* use single callback

* improve test

* improve test

* improve test

* improve test

* improve test
2026-05-29 19:00:27 +05:30
Harisreedhar 4fe79483ea Fix stream lifecycle bugs: threading, RTP sync, and resource cleanup (#1125)
* fix executor thread not terminating after stream deletion

* fix stream shutdown and thread lifecycle

* add todo

* cleanup

* cleanup

* cleanup

* cleanup

* audio_queue.put() → get_nowait() + put_nowait()

* rename test

* fix test

* merge tests

* cleanup tests

* cleanup tests

* cleanup tests

* simplify test logic with mock

* cleanup

* cleanup hard to read stream_helper.py

* introduce rtc_peer.has_peers

* fix lint

* add todos

* fix test hash
2026-05-22 16:29:45 +05:30
Harisreedhar 520dcbfd6b Refine stream helper: queue-based loops, AV1/WHIP support, endpoint separation (#1124)
* rearrange methods

* add test_stream_helper.py

* improve tests

* use deque

* move decoder to recieve methods

* remove cleanup_peer

* add destroy_stream

* make run_peer_loop more readable

* make video and audio method simlar

* change deque to queue to avoid extra thread event

* remove negative condition

* cleanup

* remove wait_for_frame

* cleanup

* cleanup

* fix process_image

* fix lint

* cleanup

* remove last_time

* add todos
2026-05-20 23:43:14 +05:30
Henry Ruhs 48869bedf0 Follow WHIP specs (#1123)
* follow more specs of whip

* pass the location header value via API router

* fix CI

* remove more queries
2026-05-19 17:05:53 +02:00
Henry Ruhs 927857d70d cleanup decoders tests (#1122)
* cleanup mostly decoders tests, also a bit encoders

* cleanup mostly decoders tests, also a bit encoders
2026-05-19 12:57:57 +02:00
Henry Ruhs fbacb24fcc Tiny refactor of codecs (#1121)
* improve performance using pointers

* simplify decoder's collect

* simplify decoder's collect

* add threading to decoders

* fix test

* switch back to return bytes

* fix macos
2026-05-19 10:31:53 +02:00
Henry Ruhs c00ea92f35 Migrate to WHIP (#1120)
* migrate to whip part1

* migrate to whip part2

* migrate to whip part3

* migrate to whip part4

* migrate to whip/whep with bidirectional

* migrate to whip/whep with bidirectional

* use next library

* add _next to lid datachannel files

* cleanup and add todos

* use internal helper rtcGetPayloadTypesForCodec

* fix lint

* refactor decode()

* move logic to codecs

* move logic to codecs

* break encoders and decoders into multiple files

* break encoders and decoders into multiple files

* cleanup more

* drop action for stream endpoints, keep type for self documentation

* restore the v4 store

* fix: align frame_width and frame_height to even in both collect() and read_resolution() in both decoders.

---------

Co-authored-by: harisreedhar <h4harisreedhar.s.s@gmail.com>
2026-05-18 16:16:06 +02:00
Harisreedhar c48c238f88 Combine encode loop methods (#1119)
* combine run_aom_encode_loop and run_vpx_encode_loop to encode_video_loop

* run_opus_encode_loop -> encode_audio_loop

* use else instead of continue

* rename to video_codec
2026-05-16 23:29:36 +05:30
Henry Ruhs dd1ded1408 Refactor/rtc cleanup 3 (#1118)
* tweak rtc store and make the decision to ban trivial testing

* clear todos for rtc_test, remove redundant tests

* clear todos for rtc_test, remove redundant tests

* break negotiation out of rtc flow, introduce create_sdp_answer and set_remote_description

* add todo

* move timeline control to the stream helper, clean send_audio|video_to_peers

* rename some methods

* fix test

* introduce detect_sdp_media

* introduce detect_sdp_media
2026-05-16 09:06:04 +02:00
Henry Ruhs 95435f842c use datachannel to create proper rtc_track_init (#1117)
* use datachannel to create proper rtc_track_init

* fix lint

* fix lint
2026-05-15 19:14:36 +02:00
Harisreedhar 0019d3ad0f Refactor stream_helper: queue-based audio/video loops with unified threading (#1116)
* rearrange methods following the flow

* add test_stream_helper.py

* fix lint

* fix lint

* refactor audio flow to match video by replacing dequeue with queue

* remove unused keyframe interval

* remove try block

* remove while True

* simplify run_aom_encode_loop and run_vp8_encode_loop

* cleanup names

* simplify run_opus_encode_loop

* move opus_encoder creation to run_opus_encode_loop

* add todos

* fix lint

* update todos and tests
2026-05-15 21:47:30 +05:30
Henry Ruhs 532464032b More RTC cleanup (#1115)
* reduce create_peer_connection like crazy

* turns out that we dont event and callback/while+sleep magic for create_sdp_offer and negotiate_sdp_answer

* test for audio/video to peers

* flag rtc methods to be revised

* fix lint
2026-05-15 15:10:51 +02:00
Henry Ruhs 98adce8a2b Refactor RTC structure (#1113)
* refactor rtc part1

* skip for macos

* merge create spd and create sdp offer

* fix lint

* add test for create_sdp_offer

* better naming for negotiate method as we get an answer

* extend tests based on mutations

* remove dead code

* rename rtc store and related methods

* clean store, move sender logic to stream helper under apis

* generate tests for rtc store
2026-05-15 11:46:51 +02:00
henryruhs 3def6c8fcd skip for macos 2026-05-15 08:47:28 +02:00
henryruhs 504f2240f7 make test more robust 2026-05-14 23:19:49 +02:00
henryruhs 1562fe2fee kill the stream helper in tests 2026-05-14 22:41:54 +02:00
henryruhs dc74e1c783 event driven test_stream_video 2026-05-14 22:21:36 +02:00
henryruhs a097034889 skip test for macos 2026-05-14 19:32:14 +02:00
henryruhs d8d9d5a280 fix test for macos 2026-05-14 18:44:02 +02:00
henryruhs eeb342ce36 fix macos for aom encode 2026-05-14 17:20:18 +02:00
henryruhs b8d00d6389 get rid of aom obus stuff again 2026-05-14 16:44:09 +02:00
Henry Ruhs 18a487347a av1 support integrated (#1112) 2026-05-14 16:11:23 +02:00
Henry Ruhs b607e4a99e AV1 codec support (#1111)
* restructure xxx_encoders, introduce av1 codec

* get rid of strip_temporal_delimiters

* improve testing

* fix test for macos

* improve testing
2026-05-14 13:30:18 +02:00
Henry Ruhs b1bc0ea43c aom library for av1 support (#1110) 2026-05-14 11:41:28 +02:00
Henry Ruhs 832d954df6 use bytes over pointer for opus encoder (#1109)
* use bytes over pointer for opus encoder

* use bytes over pointer for opus encoder
2026-05-13 14:41:26 +02:00
Henry Ruhs 9e1c068938 improve naming, remove flags as not needed (#1108)
* improve naming, remove flags as not needed

* fix lint
2026-05-13 13:34:37 +02:00
Harisreedhar bff222a12f try to unify structure of encode_opus_buffer and encode_vpx_buffer (#1107) 2026-05-13 16:26:30 +05:30
Henry Ruhs 9453a042a1 hash based test for test_stream_image (#1104)
* hash based test for test_stream_image

* new todo for test_video_stream

* new todo for test_video_stream
2026-05-12 19:51:19 +05:30
Harisreedhar 5e39c60b5c Improve encoder tests with hash assertion (#1103)
* improve test_encode_opus_buffer

* try different hash per os

* fix lint

* add windows check

* update windows hash

* fix test and lint

* update windows hash

* update CI for test_video_encoder.py

* update hash for macos

* update method to use single cpu

* update mac hash

* update windows hash

* cleanup

* restore ci.yml

* remove argument defaults

* selected CI tests

* selected CI tests

* restore ci.yml

---------

Co-authored-by: henryruhs <info@henryruhs.com>
2026-05-12 16:36:06 +05:30