[gentoo-dev] Revisiting version-related tree policies

[Michał Górny]

Hi, everyone.

As part of our work on version operators, we've noticed some issues
with our version policies. ulm has done some additional research on
the topic and now I'd like to open a discussion on our rules.


== PMS rules ==
PMS specifies only minimal syntax for versions, that is allowed types
and order of components. It does not define any range, length or
count limits. In other words, your versions can be infinitely long,
with infinitely many components and thanks to 'negative' suffixes such
as _alpha.._rc, also with infinite precision. Revisions can grow up to
infinity as well.

Fun fact: for every existing versions (without considering revisions!)
A and B, you can always create a new version X, so that A < X < B. For
example, if A = 1.4, B = 1.4_p1, X can be 1.4_p1_pre. For A = 1.4,
B = 1.4_p1_pre, X = 1.4_p1_pre_pre and so on.


== Current Gentoo policy ==
ulm has found a tiny note in the devmanual [1] stating:

| No integer part of the version may be longer than 18 digits.

The rationale is supposedly to be able to practically hold each
component in 64-bit integer type.

[1]:https://devmanual.gentoo.org/ebuild-writing/file-format/index.html#file-naming-rules


== Practical implications ==
Aside to purely technical matters, I think the free-form versioning has
two major drawbacks:

1. Some of the more creative versions are confusing to everyone (e.g.
when you are trying to figure out what particular components mean)
and really hard to type correctly,

2. Getting safe lower or upper bound for <, <=, >=, > deps is sometimes
hard to impossible. For example, >=foo-1.4_alpha wouldn't catch
1.4_alpha_rc which is valid. And <=foo-1.4-r9999 wouldn't catch
foo-1.4.r-10000.


== Currently used versions in ::gentoo ==
[Note: after noting this all down I've noticed my results don't include
masked packages]

=== Version lengths (not counting revisions) ===
The longest version used is 23 characters long. The longest are:

23 app-emacs/limit 1.14.10_pre200811252332
23 app-i18n/man-pages-ru 3.71.2209.1992.20140911
23 app-i18n/man-pages-ru 3.81.2230.2080.20160117
23 net-ftp/pybootd 1.5.0_pre20110524131526
22 sys-auth/google-authenticator 1.01_pre20160307231538

Key|Ct    (Pct)    Histogram
 5|14597 (39.72%) ******************************************************
 6| 6496 (17.68%) ************************
 3| 3954 (10.76%) ***************
 7| 3396  (9.24%) *************
 4| 2914  (7.93%) ***********
10| 2288  (6.23%) *********
 8| 1155  (3.14%) *****
 9|  478  (1.30%) **
 2|  271  (0.74%) *
 1|  239  (0.65%) *

I don't see any problem here.


=== Revision values ===
The highest revisions are:

2014120900 www-servers/xsp 2014.12
9999 app-crypt/keylookup 2.2
500 kde-misc/openofficeorg-thumbnail 1.0.0
301 dev-libs/libappindicator 12.10.0
301 dev-libs/libindicator 12.10.1

Key|Ct    (Pct)    Histogram
  0|26311 (71.60%) *****************************************************
  1| 6133 (16.69%) *************
  2| 1792  (4.88%) ****
  3|  841  (2.29%) **
  4|  526  (1.43%) **
  5|  380  (1.03%) *
  6|  319  (0.87%) *
 10|  275  (0.75%) *
  7|   40  (0.11%) *
  8|   26  (0.07%) *
  9|   23  (0.06%) *
100|   19  (0.05%) *
 11|   10  (0.03%) *
200|    7  (0.02%) *
300|    6  (0.02%) *

As expected, the most common are revisions increasing monotonically.
However, multiples of 100 are also popular.

The revision number of 9999 is suspicious, and 2014120900 is clearly
pathological -- and probably should be replaced by _pre or _p.

It should be noted that e.g. ::progress overlay is known to use
revisions 10000+ to override Gentoo ebuilds.


=== Numeric version component lengths ===
The longest numeric version component is 14 characters long.
The longest are:

14 20141110122616 dev-vcs/pwclient 20141110122616
14 20140414130214 dev-ruby/arel 5.0.1.20140414130214
14 20121105131501 dev-vcs/pwclient 20121105131501
12 201607172312 sys-apps/gradm 3.1.201607172312
12 201607021514 app-crypt/gentoo-keys 201607021514
12 201606062304 mail-filter/opensmtpd-extras 5.9.2.201606062304
12 201603152148 sys-apps/gradm 3.1.201603152148
12 201507191652 sys-apps/gradm 3.1.201507191652
12 201506180105 app-misc/xmind 3.5.3.201506180105
12 201505061057 net-libs/libasr 1.0.1.201505061057
12 201411201906 app-misc/xmind 3.5.1.201411201906
12 201401221918 app-misc/xmind 3.4.1.201401221918
10 2016020301 dev-perl/Regexp-Common 2016020301.0.0
10 2013031301 dev-perl/Regexp-Common 2013031301.0.0
10 2009041301 dev-perl/Geography-Countries 2009041301.0.0

Key|Ct    (Pct)    Histogram
 1|81628 (82.51%) ******************************************************
 2|11700 (11.83%) ********
 8| 2772  (2.80%) **
 3| 2063  (2.09%) **
 4|  665  (0.67%) *
 5|   38  (0.04%) *
 6|   28  (0.03%) *
 7|   20  (0.02%) *
12|    9  (0.01%) *
14|    3  (0.00%) *
10|    3  (0.00%) *

All longer values seem to be reserved for timestamps with various
precisions.


=== Version suffix lengths ===
The longest version suffix is 17 characters long (14 digits). The
longest are:

17 pre20161004153257 net-irc/kvirc 5.0_pre20161004153257
17 pre20160801092805 net-irc/kvirc 5.0_pre20160801092805
17 pre20160307231538 sys-auth/google-authenticator 1.01_pre20160307231538
17 pre20110524131526 net-ftp/pybootd 1.5.0_pre20110524131526
15 pre200811252332 app-emacs/limit 1.14.10_pre200811252332
15 p20160215155418 sys-apps/net-tools 1.60_p20160215155418
...
13 p200709030413 app-emacs/mu-cite 8.1_p200709030413
13 alpha20110303 dev-python/pivy 0.5_alpha20110303
12 beta20150411 app-vim/rust-vim 1_beta20150411
12 beta20150411 app-shells/rust-zshcomp 1_beta20150411
12 beta20150411 app-emacs/rust-mode 1_beta20150411
11 pre20161029 media-sound/tomahawk 0.9.0_pre20161029

Key|Ct  (Pct)    Histogram
 9|299 (24.67%) ********************************************************
11|208 (17.16%) ***************************************
 2|197 (16.25%) *************************************
 3|168 (13.86%) *******************************
 5|144 (11.88%) ***************************
 4| 96  (7.92%) ******************
 6| 38  (3.14%) *******
 7| 33  (2.72%) *******
 8|  9  (0.74%) **
13|  7  (0.58%) **
17|  4  (0.33%) *
15|  4  (0.33%) *
12|  3  (0.25%) *
10|  2  (0.17%) *

The situation is similar to numeric components. The longest components
are various kinds of timestamps, increased by appropriate keyword
lengths. (note: actually, all those should be +1 since I didn't count
the '_').


=== Version suffix counts ===
There are no more than 2 suffixes used in versions simultaneously.
The packages using two suffixes are:

_beta_p dev-java/protobuf-java 3.0.0_beta3_p1
_beta_p dev-libs/protobuf 3.0.0_beta3_p1
_beta_p dev-python/protobuf-python 3.0.0_beta3_p1
_beta_p dev-util/xxdiff 4.0_beta1_p20110426
_beta_p net-analyzer/fping 2.4_beta2_p161
_beta_p net-misc/fatrat 1.2.0_beta2_p20150803
_beta_p net-misc/freerdp 1.1.0_beta1_p20130710
_beta_p x11-plugins/wmtime 1.0_beta2_p9
_p_p net-analyzer/tcptrace 6.6.7_p4_p1
_p_p x11-libs/xosd 2.2.14_p2_p1
_p_p x11-misc/xkbset 0.5_p5_p1
_pre_p app-arch/unp 2.0_pre7_p1
_pre_p x11-misc/imwheel 1.0.0_pre13_p20100827
_rc_p dev-libs/hidapi 0.8.0_rc1_p20140719
_rc_p dev-lua/busted 2.0_rc11_p0
_rc_p dev-lua/busted 2.0_rc12_p1
_rc_p media-libs/openglide 0.09_rc9_p20160913

    Key|Ct (Pct)    Histogram
_beta_p|8 (44.44%) *****************************************************
  _rc_p|4 (22.22%) ***************************
 _pre_p|3 (16.67%) ********************
   _p_p|3 (16.67%) ********************

It seems that double suffixes are used to either indicate snapshots of
corresponding upstream versions, or to cover non-Gentoo-friendly
versions such as '6.6.7-4.1'.

Note: there is also a pathological case of bash-4.3_p39_pre0 that is
non-keyworded. In this case, it seems that -pre0 doesn't correspond to
any upstream version.


== Policy changes? ==
I think that the following new policies could make sense:

1. Revision number must be no longer than 9999:
1a. to make <=X-r9999 reliable,
1b. to prevent pathological uses of revision as date.

2. I think we could use a policy to make >=X_alpha reliable. However, I
have no clue how to word it without making it weird and artificially
restricting valid version numbers.

What do you think?

-- 
Best regards,
Michał Górny
<http://dev.gentoo.org/~mgorny/>