Stableperl, Coro, and in general the support status of my modules with perl 5.22.0

[Marc Lehmann]

Hi!

I sent this mail to you because you are likely interested because you use
one of my modules, reported that Coro doesn't work, or were otherwise
involved in this whole mess. You will not automatically receive further
mails from me on this topic and it is safe to stop here if you are not
currently doing perl stuff.

As some of you already realised, perl 5.22.0 is out, and of course it
breaks Coro. It also breaks Coro pretty badly, by revoking access to
public data structures that Coro needs to manipulate.

While this could probably be worked around somehow, by trying to access those
structures in rather dirty and version-specific ways, or copying those
structures and patching in-memory objects, what is too much is too much.

This is certainly not the first time that interesting features had to be
shelved (for example, call/cc support in Coro had to be dropped because
perl5porters, the group maintaining the standard version of perl, no
longer allowed certain data structures to be accessed).

Some of you are doubtlessly aware that the quality of my interaction with
perl5porters has been constantly degrading over the years: every new perl
release breaks dozens, often hundreds of CPAN modules, and the new regime
of perl5porters cares increasingly less. I can tell the difference because
I am contributing to Perl in some way for two decades now, and follow
perl5 releases since the first release: perl 5 releases now regularly
break many more modules, in much smaller intervals, than in the first 15
years of perl 5's existance.

For example, the so-called security patch that broke hundreds of modules
when it changed hash ordering turned out to be complete snake oil,
invented by a member of perl5porters who thinks he is capable of making a
secure hashing algorithm by gobbling together enough bitshifts, no matter
how much the slowdown or collateral damage. But back then perl5porters at
least showed some interest in helping to clean up the mess.

But while perl5porters have an official policy (see man perlpolicy)
which states that backwards compatibility is paramount, including
bug-compatibility, those parts of the policy have lost their meaning,
with many core developers stating that it's not their responsiblity to
fix broken CPAN modules (modules broken by them for example by removing
documented syntax features), or stating that they will remove or break any
feature at a whim, just because they can do so.

Instead, perl5porters give us exciting-sounding experimental features that
never work reliably, break existing code, and almost always get removed
after a few releases when they turn out to be misdesigned. perl5 has
simply become a toy for them to experiment with, at the expense of those
actually working with perl.

And while they fail to update the policy to reflect this lack of
compatibility (even after asking for and receiving a rewrite), they
did update the policy to be able to block dissenting users from their
mailinglist, rt.cpan.org, and even blogs.perl.org. Yes: you can now
officially be blocked on all these services for "merely stating facts",
and in fact, this has already happened multiple times for people
criticising recent changes in perl.

Now, I would like to expand on this in more detail at some point (I'm just
scratching the surface), but it really isn't relevant, and is overall
way too depressing to everybody involved, and boring to everybody not
involved, so now, what is the situation, how will I deal with it, and what
does that mean for you?

What is the situation?

I'm not willing to drop modules or features just because perl5porters
give a fuck about them: Not many years ago, perl was a place where
even outrageous ideas (such as Coro, in 2001) could be conceived and
implemented, far from bondage-and-discipline hard-liners who think modules
should be limited to the most trivial of things.

The repeated breakage and non-caring attitude of perl5porters sends a
clear message: They don't want interesting perl modules, they don't care
about CPAN, and ultimately, they do not care about the people who use
perl.

How will I deal with it?

Two things: I will not support perl-5.22.0 in any of my modules. Modules
that get updated will use Canary::Stability to check for this, and alert
you to this fact (without keeping you from installing, of course).

Second, I plan to provide "stableperl" variants of those perl releases (if
possible), which have the same features, but are also compatible with CPAN
(foremost my own modules, of course, but it's really open for all).

If you then want to install one of those modules on perl 5.22.0 (or any
other unsupported version), you then have three options:

- Ignore the alert and continue, hoping for the best. This will obviously
  not work for Coro, but most of my other modules seem to work with perl
  5.22.0 (for example, AnyEvent and JSON::XS). Still, if anything breaks,
  you get to keep the pieces, that's what "no support" means.

- Downgrade to perl 5.20.x or earlier - most of my modules support perl
  5.8 to 5.20.

- Sidegrade to stableperl-5.22.0-1.001 or another version of stableperl.

Stableperl is basically a branch (or fork) of standard perl, designed
to be fully drop-in compatible with the version of perl they are
based on. The only current release (5.22.0-1.001) is already quite
promising: Coro works with minimal adjustments (both in perl and Coro),
hashing is about 40% faster (more to come, but it basically reinstates 5.8
speeds), and it is already more portable than 5.22.0.

Next planned step is to fix the longstanding data corruption bugs during
global destruction - you are not the only one suffering from these.

You can learn more about stableperl and its modest goals here:

   http://schplog.schmorp.de/2015-06-06-a-stable-perl.html
   http://schplog.schmorp.de/2015-06-06-stableperl-faq.html

I am aware that this might mean the end of Coro (and possibly other
modules), because it likely won't work with future standard perl versions,
but the main user of my modules is me, and I was forced to implement these
changes simply because not only do I love to hack perl, I also need to use
perl and my modules for a living, and I need to ensure that perl actually
works.

The steps I implemented (providing a separate version of 5.22.0 that works
with more modules, and the stability canary) hopefully provide all of my
users with the necessary machinery to either continue using my modules,
or to slowly switch away if they so desire.

These steps are also very painful for me, as I publish my modules in the
sincere hope that they come in useful for others, and this likely reduces
their usefulness for others, or creates problems for them similar to the
problems I have to endure on my own.

I'm just sorry that it took me so long - I first had to analyze the
situation, get my blog up, something I had planned for years, get some
kind of canary system working, fix perl and publish my own version, and of
course, I always hoped that perl5porters would wake up one day and realise
how important CPAN is.

Hope dies last, as they say.

-- 
                The choice of a       Deliantra, the free code+content MORPG
      -----==-     _GNU_              http://www.deliantra.net
      ----==-- _       generation
      ---==---(_)__  __ ____  __      Marc Lehmann
      --==---/ / _ \/ // /\ \/ /      schmorp at schmorp.de
      -=====/_/_//_/\_,_/ /_/\_\