Repository navigation
a more deliberate release process #1854
Description
Activity
This is the price for being the front-runner, you manage to break things on the way.
NodeJS, in contrast, didn't rush things like this, faced no such predicament, and lagged behind happily for years to come.
I believe the truth is in the middle. Therefore, looking forward to the merge of the two platforms.
I think quite a lot of issues brought up in io.js focus far too inside the box, the whole ongoing thing with V8 4.3 onward has focused heavily on how will io.js / nan solve this, I think there needs to be a bigger effort to external thought, and the ecosystem as a whole.
Quite often a lot of these small changes to clean the code base are based on quick and dirty searches on google, github and npm, and the assumption this is good enough to put in a deprecation notice (which, are mostly useless in my experience, as the people who care enough to do something are generally those who are eager to maintain their software anyway).
It would be great if some external tooling were to start to happen, for example; the chrome guys have metrics that they use to inform changes, it would be great if npmjs could do some static analysis on packages to establish some real connectivity of code use and its flow between packages. This would be really interesting outside of this issue as well, and could be used to establish quality of packages over time, and perhaps even some day some automation around automatically updating dependencies.
I'm not saying the above is the answer, or even a good one- but its trying to use aspects of the ecosystem that aren't just code. Aside to this, there's so much worry about upgrading V8 too often and how code can solve that, but what about outside of that? can npm help? can community outreach help?
More on topic, and I'm really out of my depth here not having any experience bar following along reading issues most days, the node/iojs build + test does seem woefully inadequate for how important this platform is.
First I'd like to say thank you to everyone and anyone who took the time to spend the Sunday to try and resolve this particular issue.
I think #2, as @othiym23 has suggested, may be a good compromise. If at all possible, I'll be willing to add something like
iojs-latest-rcto.travis.yml(or some CI platform), and maybe a lot of projects would as well; and this would the cheapest way to test the waters that a new release is stable enough for the rest of the ecosystem.@meandmycode Static analysis is already in the works. See @chrisdickinson's estoc module.
@meandmycode +1, you hit the nail on the head.
The focus of io.js (and for that matter, Node as well, more often than not) has been on polishing and improving io.js.
There are still features of the platform that are not documented, not tested, and perhaps maintainers of io.js may assume are "internal".
However, we need to change our thinking on that.
If it's exposed, it's public API. Period.
Imo, changes to the public API require, at minimum:
- A very damn good reason. ("It's ugly" or "it should be internal" is not satisfactory. Too late, sorry.)
- Explicit deprecation in the docs for a major release.
- Deprecation via
util.deprecatefor another major release. - Assuming that no riots form when we do (3), remove it in the next major release.
We can add new stuff all day long, or provide better APIs in the meantime. But we can't break stuff without breaking users.
Related: #1807 (comment)
Sorry, still haven't gotten around to writing up the issue for a stable release checklist. I think we're seeing from this fiasco that it's not just a matter of keeping up with breaking V8 changes, but also pure-js changes as well.
Btw. When I asked in IRC why was
.clientdeprecated in a minor version (after the 2.2.0 got released), I was told that deprecating in a minor is ok, along with actual removing in a major. If that is not so, you should probably form a policy for such things and put it somewhere.@ChALkeR I was refering to this part of semver:
Minor version Y (x.Y.z | x > 0) MUST be incremented if new, backwards compatible functionality is introduced to the public API. It MUST be incremented if any public API functionality is marked as deprecated.
The
.clientdeprecation itself was improperly handled which lead to #1850.I don't expect people to run
test-npmfor every commit, but it should be a requirement for every release at least, as it's been proven again that npm itself is a good test for the ecosystem.@silverwind Running
make testin npm will be less of a good test of the ecosystem once we finish making all of the npm tests run network-free. A much better test of the ecosystem is selecting the top 100 modules, and making sure that you can donpm install && npm teston each of those. (npm itself would presumably be in that set.) In the case of 2.2.0, this would have been 0/100 passing, sincenpm installis broken.@ChALkeR Everything that is exposed is exposed. We've had a firm "this is internal" policy regarding
process.binding(...)stuff, but even then, I think we should verify that breaking changes won't break user code.In the case of 1eec5f0 specifically, even though the
clientproperty was not used internally, because it was exposed, it was a part of the public API, documented or not. Ifclientis to be deprecated in favor ofsocket, then that ought to be changed in 3.0.0 by adding a note to the docs, then in 4.0.0 by usingutil.deprecate, and then in 5.0.0 by removing it entirely. Moving it to an underscore-prefixed property is inappropriate in this case.Of course, mistakes happen, and slip past review. That's why we need an explicit objective release gating process, so that we can get away from the (currently valid!) stigma that io.js is unstable, prone to break in minor releases, and not trustworthy.
I thought that when the topic of nightlies/RCs was originally discussed months ago, the conclusion was that nobody would likely test/use them (especially with a
-nightly-xxxxxor similar version number it would not work with npm)? Also it looks like the last nightly was built on 5/14/15.Nightly usage has been pretty minimal. That might change once we converge, but for now, they're not getting the exposure they need.
Everything that is exposed is exposed. We've had a firm "this is internal" policy regarding process.binding(...) stuff, but even then, I think we should verify that breaking changes won't break user code.
I always thought that
_-prefixed means that there are no guarantees and that whoever uses such properties does it on their own risk.Technically,
Symbol-based properties are also accessible. Will you treat those as «exposed»? Technically, in c++ private members of a class are also accessible. Are those «exposed»?The question is how far does the user code go in breaking the conventions and to what extent do want to support that.
To get more nightly usage, we probably need some kind of auto-update mechanism. That would of course require installing into a user-writeable directory so we can update without needing root.
Another unresolved issue with nightly builds is that their version strings don't satisfy the
enginerequirement of packages, which results in npm printing a warning for every single dependency installed, but I guess there could be made an exception in npm.If it would help, I would be pleased to extend
nvmto support installingio.jsnightlies, provided we can agree on formats/structure/URLs/etc, and provided it will remain the same withnodepost-merge - that would allow easy testing ontravis-ci, as well as easy testing on devs' local machines.21 remaining items
integrated into tools like nvm would go a long way to getting them used more.
That would give great exposure, and be fairly safe as long as they -rc's didn't automatically get installed as the latest.
I'd still prefer some sort of auto-update mechanism for maximum exposure.
A debian ppm for -rc packages might work for deb-based systems. All the OS X auto-updates I've seen seem to require a UI to be running, and a user to interact with.
How would you see this working for node?
How would you see this working for node?
I could see some sort of startup hook build into nightly binaries that does a single HTTP request per day to check for an update. If there's one available, download it and put it into
~/.node/binwith PATH pointing there, pretty similar whatnvmdoes.This would primarily target people that use node as a tool and not servers. Also, there must be an opt-out and the timestamp of the last update check must be saved to disk. Thinking about this, maybe it's better to keep such things to
nvmthough, these things can get messy.Last time we discussed putting out RCs we looked at the nightly builds for next and said "hey, we're already doing everything we would do to put out an RC."
What do you people advocating for RCs over nightlies see happening here? A marketing change perhaps---better links from the homepage?
Nobody really wants to do the extra infrastructure work to change the name from nightly.yyyymmdd to rc.increasingnumericsequence, so hopefully that's not what you're looking for.
Mostly branding: That the exact build that will become 'latest' bakes for a week as 'next' first. Then promoted, rather than a rebuild or similar -- it makes the difference between 'testing' and 'safe to use' very very small, just earlier -- rather than nightlies that are thrown away, and most never become a 'release'.
@domenic https://iojs.org/download/next-nightly/ - they only build when there are changes which is why the list is short (and because of the accidental downtime last month)
I have some unrelated things to do today but if I get time I'm going to set up yet another release process in Jenkins for "rc" builds with special "rc" labels so they don't look so scary - perhaps we can get nvm to pick those up too?
3.0.0-rc1@ https://iojs.org/download/rc/3.0.0-rc1@rvagg if the listing in
https://iojs.org/download/rcis an identical format to that inhttps://iojs.org/dist, then I should be able to get something out this or next weekend, and merged totravis-ci. When they deploy that, I can't tell you, but it'd be included in their next VM image update.neato, all 3 of the current directories are using the same format, https://iojs.org/download/next https://iojs.org/download/next-nightly and https://iojs.org/download/release/ which is aliased at https://iojs.org/dist/, they all use index.tab and index.json in the same way
Great! I'll build it optimistically hoping the future merged
nodereleases follow the same patterns :-)I think you'll find that optimisim well founded
Please let me know when a release candidate exists in
https://iojs.org/download/rcso I can test with it :-)Further discussion at #1997
<whining>I'm going to be straight-up emotional for just a second here: this is the second Sunday in the last month where I've woken up to a brewing crisis involving io.js and npm (#1850, #1591). It is incredibly annoying, as well as demotivating, to have to clean up these situations on what should be one of my days off. I try very hard to be a conscientious project lead / OSS maintainer / io.js collaborator, but for human psychological and physiological reasons I need some downtime, and the weekends are that time for me. The suggestions below are offered in the spirit of making it so that I don't have to choose between being responsible to the io.js and npm user communities, and being responsible to myself. Thank you for reading this with that in mind.</whining>Also, for all of the following, I frame things in terms of npm, but this is just as true of V8 and changes made to Node's own APIs.
About a year ago this time, npm introduced a new release process that included as one of its key features the notion of publishing every new release with a
dist-tagofnext, and only promoting it tolatestafter at least a few days (generally a week) so that interested members of the community could use / test it and identify any latent issues with the release. This was done as a reaction to what used to be an all-too-common pattern with Node.js releases, where a new release of Node.js would be followed almost immediately by another due to something messed up inside the version of npm bundled with Node, which was embarrassing and wasteful. This basically hasn't happened since npm's new process was put into place, which I think everybody agrees is a big improvement.#1850 shows that this only deals with one side of the problem, though. If there's something in a new Node release that makes it incompatible with npm, we can end up putting out versions of Node that render npm unusable, which cripples the release. There's a lot of room for improvement in how npm is tested and integrated with io.js's CI, but even with that situation substantially improved (which a number of us are working on), there's no substitute for real use of the product to give it some real-world smoke-testing.
io.js has a much quicker release cadence than Node.js has had for a while, and this is one of the primary drivers of io.js's quick progress (and is great). New versions come out frequently, and when things break, a new release that fixes them is generally pretty quick to appear. However, because things like Travis pin to the latest version, and many many projects take advantage of this, official io.js releases can magnify issues into an avalanche of issue-tracker and support traffic.
Therefore, I have a few suggestions I'd like to make about how io.js is released:
Note that these suggestions are orthogonal to adding more / better tests of dependencies, better integration of the tests we do have into CI, acceptance tests, and other prophylaxes intended to improve release quality. All of those things would be great, as would some kind of defined QA process for the project. I still think, however, that Node and npm are both complex and widely-used enough that there's no real substitute for actual, hands-on use to help ensure quality.