Re: Discussing defaults (Was: [gentoo-dev] Re: [gentoo-commits] gentoo-x86 commit in virtual/ffmpeg: ffmpeg-9.ebuild ChangeLog ffmpeg-0.10.2-r1.ebuild)

[Alexis Ballier]

On Mon, 11 Feb 2013 12:25:36 +0100
Luca Barbato <lu_zero@g.o> wrote:

> On 11/02/13 03:01, Alexis Ballier wrote:
> > Sorry, I was away this week end...
>
> Not a problem, I should be reachable anytime today.
>

Will ping you.

> > This is only because libav people do not care at all about what
> > FFmpeg defines, while FFmpeg seems to care more about its consumers
> > and users by trying to provide a compatible ABI and API.
>
> The reasons about that could be multiple, the facts are those. If you
> want you application to run in both you pick the Libav API.

Well, then this raises the question: If libav does not care about what
is done around it, why should we care about libav?

> > Moreover, this is not
> > true at all when it comes to the parts where supporting both forks
> > is painful: libavfilter and audio resampling. It is pretty clear
> > that the audio resampling wheel was reinvented on the libav part,
> > with a different library name and in 95% of its use cases going
> > from ffmpeg's audio resampling to libav's is mainly a matter of
> > 'sed s/swr/avr/'.
>
> Not really, the resample library idea was brewing for long even
> pre-fork, it got released first on FFmpeg because there is this
> bulimic tendency to add more features in order to be compelling in
> FFmpeg.

There was a need for it, FFmpeg made it available with 0.10, one year
before libav-9. Why not re-using libswr into libav, possibly reworking
it entirely but offering a compatible API? This all sounded to me like
'we do not care, users and applications can take the mess, it is not
our problem'.
There is a problem we are facing today: Some audio decoders started to
output planar audio when historically ffmpeg/libav audio decoders
produced only iterleaved audio, how do you suggest to fix this in
applications? Manually converting planar audio to interleaved (ala
mplayer)? This does not seem very future-proof, this will have to be
adapted everytime a new format is added (which will not change lib
major and thus break application at runtime for poor usage of the
provided API). Or maybe use a library as a fallback that ensures you it
will be able to convert any audio format produced by ffmpeg/libav to
what you need (ala xbmc)? Which one to chose? If I want to support
ffmpeg then I have to use libswr, because that is the one that has this
property; If I want to support libav, then I have to use libavr... All
of this because ~10 people cannot work together, well, really, thank
you :)


> I said already what happened when I started to develop the pulse
> capture and the generic segmenter in my github as another example of
> stuff developed by unrelated people and curiously landing before
> deemed ready by the author in ffmpeg git. (prores could be considered
> yet another funny example).

Can you point out any important problem? Otherwise, this is simply
how open source work: this has been published with a free enough
license, if I were working on these parts I'd not start from scratch
but from the code available to me... If you want to avoid this, don't
publish it or publish it with a restrictive license and relicense your
code once pushing to libav.
Another funny example are the multithreaded decoders (ffmpeg-mt),
which was one of the main publicity material at the beginning of libav,
and suffered from numerous problems in the early days... nevertheless it
was merged by libav. You are criticizing something that has been done on
both sides :)

> > Some parts of libavfilter also appeared later in libav, sometimes
> > with a slightly different API, making it really awful to support
> > both forks in applications.
>
> libavfilter is deemed still not ready for general consumption in Libav
> and just the set of calls that won't change (hopefully) in the future
> are provided. Meanwhile we are trying to make the whole buffer
> management less copy-happy and a little more rational.
>
> (not sure if you tried to use libavfilter but it is a _bit_ hard to
> use if compared to vlc filter layer and such)

Yes, well, if forces were not scattered due to the split then I'm
pretty sure libavfilter would be in a much better state. IIRC, ffmpeg's
libavfilter is much more 'copy-less' than current libav's version.

[...]
> > It is funny to see how libav ignores completely ffmpeg even for
> > bugfixes, I stumbled over this recently and it sounded familiar:
> > http://git.libav.org/?p=libav.git;a=commitdiff;h=89f11f498b9c15bc71494a11a7ec560f4adf630d
> > vs
> > http://git.videolan.org/?p=ffmpeg.git;a=commitdiff;h=1af91978dbab35ba9fdede187577c00d643ae33b
> > now, you can look at the dates, there's a one year lag in libav for
> > reinventing the same bugfix. This is for a format almost nobody
> > uses, but in any case I do not consider this attitude sane.
>
> Nobody is perfect and not everybody has the time to track another
> project tree every day.

As a Gentoo developer, if you notice a bug in a package, usually the
first thing you do is pull upstream's trunk repo and see if there is a
fix available. If you don't do that, you may just be wasting your time.
Maybe if the time spent reinventing the wheel was spent more
efficiently by looking at what is done in FFmpeg, then more important
improvements could be achieved... 

> >> and a bit more tested over a wider range of architectures.
> >
> > If you are talking about fate, then I cannot comment. fate started
> > before the split, so I assume the owners of said test machines took
> > them within their respective fork.
>
> That speaks a *bit* about the general support maybe?

fate is quite complete on the ffmpeg side too. For me, this speaks more
about a useless scattering of forces than anything else...

> > I fully agree there, but you should be careful with who you include
> > in 'our'. In the case of a default then it is clearly 'the users'.
> > Now, considering there are a couple (very few) of packages that do
> > not compile with libav, do you consider having it as default a good
> > idea?
>
> I have no opinion, I stayed out of the discussion and decision about
> that before because I know I have a bias. I let other people decide.

Note: If pro-libav people would be doing the work of fixing the tree
and ensuring *everything* works with libav I would not mind at all what
the default is. So far, I have been doing most of the porting work
since years, there are packages still not working with libav-0.8, the
introduction of some libav stuff in the tree caused problems
(virtual/ffmpeg, libpostproc; nothing inherent to libav, its just it
was poorly done), there are still numerous patches to support libav
(mainly because of some libavutil headers missing). I consider FFmpeg
superior, but can understand there are different opinions, however, if
this is to lower the tree quality then it is obvious libav shouldn't be
the default.

> > Those packages can very well be fixed, but if patches do not go
> > upstream, this is not going to work in the long term. Also, the only
> > reason why I have not pinned those packages to depend only on
> > media-video/ffmpeg is because libav is not the default (not entirely
> > true btw, see later), meaning those that use libav are those that
> > are willing to help and know what they are doing. If people are to
> > get libav by default and then hit compile failures to be told to use
> > ffmpeg, then it is QA 101 that these packages should be depending on
> > media-video/ffmpeg.
>
> You can, if you really want, offset ffmpeg so it doesn't clash with
> Libav.

Ahahah :) Or maybe, despite the claims of some libav devs, libav should
realize they are the actual fork (this is now pretty clear since the
takeover failed and ffmpeg didn't collapse...) and also rename their
libraries so that the internal libav/ffmpeg fights would not affect
their users anymore and projects could use what they think best...


[...]
>  > Probably true, but I still consider a quick fix disabling the
> > vulnerability to be released quickly is a good thing.
>
> No, it is in general a really stupid idea. You are basically telling
> the interested attackers where the problem is. Then the attacker does
> +1 and you have your new, freshly installed application compromised.
>
> It is the _worst_ possible service you could do.
>
> Think about a random rpc system with 4 calls that are similar and go
> down the same codepaths, some fuzzer discovers that sending a message
> with a certain shape let you crash the application.
>
> The fix pushed and published with much fanfare is "if $shape then
> error" for that specific rpc call.
> Pity that that similar shapes would crash the other rpc calls, thus
> anybody with interest in crashing the application have a good example
> on how to do that.
>
> Unrelated to the discussion please do not do that in any project you
> contribute. It is that _bad_

Fully agree, but a quick fix can very well be 'if overflow then error'
on the common code path, disabling completely the vulnerability while
likely rejecting valid requests too (that never worked) and then a
complete fix would be to rework the code so that there cannot be an
overflow...

Alexis.