[Ekiga-devel-list] Some points on the EKIGA/OPAL video plugin

From: Matthias Schneider <ma30002000 yahoo de>
To: ekiga-devel-list gnome org, openh323 openh323 org
Subject: [Ekiga-devel-list] Some points on the EKIGA/OPAL video plugin
Date: Wed, 25 Oct 2006 21:58:32 +0200 (CEST)

Hi all,

in the process of developing a video plugin for ekiga/opal I noticed some things that I would like
to clarify. Everything mentioned here refers to the latest HEAD version of OPAL, EKIGA nd PWLIB.I
am posting this to both openh323/opal and ekiga lists since some topics do concern both software
packages..

OPAL issue:

- in patch.cxx OpalMediaPatch::Main() we do a sleep (5) after each packet received
5ms sleeping is much too long and has lead to buffer overruns in the UDP input que in my case
(e.g. if a frame is transmitted as 4 packets at 25fps, and we need 20 ms waiting after the
received packet, leaving only another 20 ms per frame to decode and display it - not enough in my
case) Right now my workaround is to wait 5 ms only after every frame since I have not yet
understood the motivation for this sleep - the select that is called in order to collect the
packets should e blocking anyway, so we dont need any sleep? Or does this depend on the platform?
Possibly this problem doesnt appear on windows - there is an #if !defined (WIN32) in the code.
Also I do not understand which other thread should be allowed to take the mutex...

EKIGA bug:

- EKIGA does not complete a remotely initiated termination of a call (see attached file
terminate.txt) - happens with different endpoints. If video is activated, outbound video
transmission does not even stop. If the connection is terminated by ekiga everything works like
expected.

Plugin uncertainties:

- dynamic RTP
If I understand correctly, EKIGA/OPAL should support dynamic RTP payload types (like in the
speex plugin) for video as well.
- Is it correct interpretation of RFC 4566 (please advise if there is an RFC more specific on
dynamic RTP PTs) that the rtpmap attribute specifies the PT on which it will be received and that
it can be asymmetric (e.g. A sending with PT 96 while receiving the same codec on 99 because A
sent an SDP line "a=rtpmap:99 X/90000" and B "a=rtpmap:96 X/90000"?
- How do I handle dynamic RTP types in a video plugin? As I understand there are two options -
shared RTP code (e.g. speex) and separately coded RTP (e.g. h263ffmpeg.cxx). If I understand
correctly I have to register a dynamic PT codec plugin with the RTPTypeDynamic flag. - With which
PT do I register my plugin? How do I know with which RTP PT to tag the outgoing frames? Or is it
mandatory to use the shared RTP code in this case? How do I put in the correct timecode and maker
in that case? Also, if I add the RTPTypeDynamic flag instead of RTPTypeExplicit, the codec does no
longer appear in the EKIGA codec list...

- fmtp handling
How is the a=fmtp line in sip/sdp handled? I noticed that it is possible to hand an fmtp line
from the encoder side to the SDP handler like in speex. However this happens BEFORE options like
width and height of frame are set up, making it impossible to include anything based on that in
the fmtp line, which is sometimes necessary. Also I am wondering how the received a=fmtp line is
passed to the decoder.

General Questions:
- cerr and PTRACEs commented out in h323ffmpeg.cxx
All log output in h323ffmpeg.cxx is commented out. What would be the correct way for outputting
log information in a video plugin without introducing unneccessary dependencies?

EKIGA enhancement proposals for discussion:

- Bitrate widget in ekiga
The bitrate slide is in kbyte/s, however I am used rather to kbit/s and always have to
calculate... I wonder how you feel about that, actually the codec plugins also handle the setting
kbit/s. Also I would like to propose to higher the bitrate range to at least 2mbit = 250 kbyte/s.

- make the max. RTP payload size configurable (also OPAL)
Perhaps even more options should be configurable in the plugins? (right now I know only of
width, height, bitrate and a quality variable)

- FPS counter (also OPAL)
It would be nice to have an frames/s counter in ekiga similar to the audio and video bitrate
counter

- Jitter buffer
In ekiga CVS where has the jitter buffer display gone to? I also would like to ask if the
current layout will be the final one - I like the "old" with the embedded sliders and the simpler
embedded video window from ekiga 2.0.3 better...

It would be nice to get a some advice and possibly some changes based on the above.. In case of
questions dont hesitate to ask for me to clarify

Thanks in advance,
Matthias

___________________________________________________________
Der frühe Vogel fängt den Wurm. Hier gelangen Sie zum neuen Yahoo! Mail: http://mail.yahoo.de

2006/10/25 20:58:41.727   0:13.458       Housekeeper    RTP     Found existing session 1
2006/10/25 20:58:42.356   0:14.087      RTP Jitter...er:8472da8 RTP     Receive statistics:  packets=101 octets=16160 lost=0 tooLate=0 order=0 avgTime=16 maxTime=31 minTime=0 jitter=7 maxJitter=7
2006/10/25 20:58:42.727   0:14.458       Housekeeper    RTP     Found existing session 1
2006/10/25 20:58:42.760   0:14.491      Media Patc...ch:846afd8 RTP     Transmit statistics:  packets=101 octets=16160 avgTime=20 maxTime=41 minTime=0
2006/10/25 20:58:43.727   0:15.458       Housekeeper    RTP     Found existing session 1
2006/10/25 20:58:44.357   0:16.088      RTP Jitter...er:8472da8 RTP     Receive statistics:  packets=201 octets=32160 lost=0 tooLate=0 order=0 avgTime=20 maxTime=30 minTime=10 jitter=0 maxJitter=8
2006/10/25 20:58:44.727   0:16.458       Housekeeper    RTP     Found existing session 1
2006/10/25 20:58:44.760   0:16.491      Media Patc...ch:846afd8 RTP     Transmit statistics:  packets=201 octets=32160 avgTime=20 maxTime=40 minTime=0
2006/10/25 20:58:45.162   0:16.893      SIP Transp...rt:843f910 SIP     PDU Received on udp$192.168.134.1:5060<if=udp$192.168.134.100:5063>
BYE sip:matthias-softphone 192 168 134 100:5063 SIP/2.0
CSeq: 102 BYE
Via: SIP/2.0/UDP 192.168.134.1:5060;branch=z9hG4bK0b444cd5;rport
User-Agent: X-Lite release 1103m
From: <sip:4711 192 168 134 1>;tag=as73e7415e
Call-ID: 16241d7d-c862-db11-8100-0012f018b0b9 matthias-laptop
To: "Matthias Schneider" <sip:matthias-softphone 192 168 134 1>;tag=322b1d7d-c862-db11-8100-0012f018b0b9
Contact: <sip:4711 192 168 134 1>
Content-Length: 0
Max-Forwards: 70


2006/10/25 20:58:45.163   0:16.895      SIP Transp...rt:843f910 SIP     Queueing PDU: 102 BYE sip:matthias-softphone 192 168 134 100:5063
2006/10/25 20:58:45.164   0:16.895      SIP Transp...rt:843f910 SIP     Waiting for PDU on udp$192.168.134.1:5060<if=udp$192.168.134.100:5063>
2006/10/25 20:58:45.165   0:16.897      SIP Handle...er:8465880 SIP     Handling PDU 102 BYE sip:matthias-softphone 192 168 134 100:5063
2006/10/25 20:58:45.167   0:16.898      SIP Handle...er:8465880 SIP     BYE received for call 16241d7d-c862-db11-8100-0012f018b0b9 matthias-laptop
2006/10/25 20:58:45.169   0:16.900      SIP Handle...er:8465880 SIP     Sending PDU on udp$192.168.134.1:5060<if=udp$192.168.134.100:5063>
SIP/2.0 200 OK
CSeq: 102 BYE
Via: SIP/2.0/UDP 192.168.134.1:5060;branch=z9hG4bK0b444cd5;rport=5060;received=192.168.134.1
From: <sip:4711 192 168 134 1>;tag=as73e7415e
Call-ID: 16241d7d-c862-db11-8100-0012f018b0b9 matthias-laptop
To: "Matthias Schneider" <sip:matthias-softphone 192 168 134 1>;tag=322b1d7d-c862-db11-8100-0012f018b0b9
Allow: INVITE,ACK,OPTIONS,BYE,CANCEL,NOTIFY,REFER,MESSAGE,INFO,PING
Content-Length: 0


2006/10/25 20:58:45.172   0:16.903      SIP Handle...er:8465880 OpalCon SetPhase from EstablishedPhase to ReleasingPhase
2006/10/25 20:58:45.173   0:16.904      SIP Handle...er:8465880 OpalCon Releasing Call[1]-EP<sip>[16241d7d-c862-db11-8100-0012f018b0b9 matthias-laptop]
2006/10/25 20:58:45.175   0:16.906      SIP Handle...er:8465880 OpalCon Call end reason for 16241d7d-c862-db11-8100-0012f018b0b9 matthias-laptop set to EndedByRemoteUser
2006/10/25 20:58:45.176   0:16.907      SIP Handle...er:8465880 SIP     Awaiting next PDU.
2006/10/25 20:58:45.176   0:16.907      OnRelease:...se:8465528 SIP     OnReleased: Call[1]-EP<sip>[16241d7d-c862-db11-8100-0012f018b0b9 matthias-laptop], phase = ReleasingPhase
2006/10/25 20:58:45.178   0:16.910      OnRelease:...se:8465528 OpalCon SetPhase from ReleasingPhase to ReleasingPhase
2006/10/25 20:58:45.182   0:16.914      OnRelease:...se:8465528 Media   Closing RTP for OpalRTPMediaStream-Sink-G.711-ALaw-64k
2006/10/25 20:58:45.184   0:16.915      OnRelease:...se:8465528 RTP_UDP Session 1, Shutting down write.
2006/10/25 20:58:45.185   0:16.916      OnRelease:...se:8465528 Media   Closing stream OpalRTPMediaStream-Sink-G.711-ALaw-64k
2006/10/25 20:58:45.187   0:16.918      OnRelease:...se:8465528 Media   Disconnecting OpalRTPMediaStream-Sink-G.711-ALaw-64k from patch thread Patch OpalAudioMediaStream-Source-PCM-16
2006/10/25 20:58:45.211   0:16.942      Media Patc...ch:846afd8 RTP_UDP Session 1, Write shutdown.
2006/10/25 20:58:45.212   0:16.943      OnRelease:...se:8465528 Patch   Removing media stream sink OpalRTPMediaStream-Sink-G.711-ALaw-64k
2006/10/25 20:58:45.727   0:17.458       Housekeeper    RTP     Found existing session 1
2006/10/25 20:58:46.727   0:18.458       Housekeeper    RTP     Found existing session 1
2006/10/25 20:58:47.727   0:19.458       Housekeeper    RTP     Found existing session 1
2006/10/25 20:58:48.727   0:20.459       Housekeeper    RTP     Found existing session 1
2006/10/25 20:58:49.240   0:20.971      RTP Jitter...er:8472da8 RTP     SentSenderReport:  ssrc=141150149 ntp=3370791529.1031581972 rtp=35360 psent=222 osent=35520
2006/10/25 20:58:49.242   0:20.973      RTP Jitter...er:8472da8 RTP     SentReceiverReport: ssrc=661912853 fraction=0 lost=0 last_seq=0 jitter=2 lsr=0 dlsr=0
2006/10/25 20:58:49.244   0:20.975      RTP Jitter...er:8472da8 RTP     Sending SDES: matthias matthias-laptop
2006/10/25 20:58:49.728   0:21.459       Housekeeper    RTP     Found existing session 1
2006/10/25 20:58:50.728   0:22.459       Housekeeper    RTP     Found existing session 1
2006/10/25 20:58:51.728   0:23.459       Housekeeper    RTP     Found existing session 1
2006/10/25 20:58:52.728   0:24.459       Housekeeper    RTP     Found existing session 1
2006/10/25 20:58:53.728   0:25.459       Housekeeper    RTP     Found existing session 1
2006/10/25 20:58:54.728   0:26.459       Housekeeper    RTP     Found existing session 1
2006/10/25 20:58:55.728   0:27.459       Housekeeper    RTP     Found existing session 1
2006/10/25 20:58:56.728   0:28.459       Housekeeper    RTP     Found existing session 1
2006/10/25 20:58:57.728   0:29.459       Housekeeper    RTP     Found existing session 1
2006/10/25 20:58:58.728   0:30.459       Housekeeper    RTP     Found existing session 1
2006/10/25 20:58:59.728   0:31.459       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:00.728   0:32.459       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:01.664   0:33.396      RTP Jitter...er:8472da8 RTP     SentSenderReport:  ssrc=141150149 ntp=3370791541.2854543850 rtp=35360 psent=222 osent=35520
2006/10/25 20:59:01.666   0:33.398      RTP Jitter...er:8472da8 RTP     SentReceiverReport: ssrc=661912853 fraction=0 lost=0 last_seq=19535 jitter=2 lsr=0 dlsr=0
2006/10/25 20:59:01.669   0:33.400      RTP Jitter...er:8472da8 RTP     Sending SDES: matthias matthias-laptop
2006/10/25 20:59:01.728   0:33.459       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:02.728   0:34.459       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:03.728   0:35.459       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:04.732   0:36.464       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:05.733   0:37.464       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:06.733   0:38.464       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:07.733   0:39.464       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:08.733   0:40.464       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:08.833   0:40.564       Housekeeper    SIP     Set state Terminated_Success for transaction 1 INVITE
2006/10/25 20:59:09.733   0:41.464       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:10.733   0:42.464       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:11.733   0:43.464       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:12.729   0:44.460       Housekeeper    SIP     Set state Terminated_Success for transaction 2 INVITE
2006/10/25 20:59:12.733   0:44.464       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:13.733   0:45.464       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:14.733   0:46.464       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:15.733   0:47.464       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:15.773   0:47.504      RTP Jitter...er:8472da8 RTP     SentSenderReport:  ssrc=141150149 ntp=3370791555.3321821224 rtp=35360 psent=222 osent=35520
2006/10/25 20:59:15.773   0:47.504      RTP Jitter...er:8472da8 RTP     SentReceiverReport: ssrc=661912853 fraction=0 lost=0 last_seq=19535 jitter=2 lsr=0 dlsr=0
2006/10/25 20:59:15.773   0:47.504      RTP Jitter...er:8472da8 RTP     Sending SDES: matthias matthias-laptop
2006/10/25 20:59:16.733   0:48.464       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:17.733   0:49.464       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:18.733   0:50.464       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:19.733   0:51.465       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:20.733   0:52.465       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:21.734   0:53.465       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:22.734   0:54.465       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:23.734   0:55.465       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:24.734   0:56.465       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:25.742   0:57.473       Housekeeper    RTP     Found existing session 1
2006/10/25 20:59:26.742   0:58.473       Housekeeper    RTP     Found existing session 1

Follow-Ups:
- Re: [Ekiga-devel-list] [OpenH323]Some points on the EKIGA/OPAL video plugin
  - From: Craig Southeren
- Re: [Ekiga-devel-list] Some points on the EKIGA/OPAL video plugin
  - From: Damien Sandras

[Date Prev][Date Next] [Thread Prev][Thread Next] [Thread Index] [Date Index] [Author Index]