[Csnd] Csound release management, esp re Csound 6

[Csnd] Csound release management, esp re Csound 6

Date	2012-02-13 00:00
From	Iain Duncan
Subject	[Csnd] Csound release management, esp re Csound 6
	Ok, I will start by apologizing for my less than diplomatic tone regarding build issues on the development list so far. Honest apologies. I will try to keep this email to facts and questions, and humbly offered opinions. Posting to both lists as I believe there is a good chance that there are users who have also been frustrated but are not on the dev list and may wish to weigh in. From what I can tell from Steven's thread on Csound 6, a major thrust of Csound 6 is to improve Csound as a library that can be used as an audio engine by other apps. This is great news to me, that's what I'm doing. My project is a C++ app using Csound as one possible output layer. This does mean that Csound is seriously putting itself forth as something that can be reliably used as a dependency for other projects, and I believe that has implications. I am proposing that if Csound is going to be targeting this segment, as part of this plan, an improved policy for Csound source code release management is required. I would hazard that developers using csound as an engine will likely be providing instructions for their developers to build from source, and likely to users as well in the initial alpha or beta stages of a project. In the last two months, I've had two production snapshot releases not build, and another had debug printf output that wasn't supposed to be there and rendered it unusable for the project. These were Csound 5.14.2 for linux, Csound 5.15.0 on OS X, Csound 5.16.1 on OS X. To clarify, all were production tarballs, not pulling from git. I was advised to fix from git, however I have been told that the git trunk is not stable and not guaranteed to work 100% of the time. ( reasonable for trunk ). This has been very frustrating to me, and I apologize for not handling it as gracefully as I could have. That said, were my project at the point of alpha releases to others, it would have been frustrating for my users and developers too. It is my personal belief from my observation of other projects that I work with on a daily basis with my python business, that projects being used as dependencies by other projects ( as Csound is intending ) generally make sure that the production release always builds and always passes all the regression tests. There are indeed a number of great tools out there that are free and help make this possible. The process I see the most is as follows: - a "we're getting close to a release" announcement goes out on the mailing list ( this happens with Csound ) - trunk is branched for a release candidate - the rc branch is tested by a build system or by users/developers who do this - any issues are fixed in the rc branch only, nothing else goes on in this branch - when the branch is known as passing all tests and building on all targeted platforms, the release manager then creates a frozen production snapshot and releases - the bug fixes from the rc branch are then merged back into trunk - development continues on trunk and the process begins again It does not seem like this is happening with Csound. So, my questions are: - do others think this is important? am I in a minority, or are there others who are concerned and are just not saying anything? - do others share my opinion that this will become more important if Csound 6 is meant to target developer users of the CsoundAPI? - is there a current release policy and if so, what is the release policy? - why is the release procedure the way it is right now? not a rhetorical question, I mean what are the real reasons? IE lack of resources, time, experience with continuous integration tools, etc? What is the bottleneck? - what do we need to do to make an improved release policy that guarantees production releases build 100% of the time and pass all tests? IE, what can provide to help this happen? Some things I can think of that could be done: - a page documenting the release policy and who is in charge of what (is there a csound release manager? could there be one?) - adoption of at least a system of branching and merging for each release. This would help even in the absence of a continuous integration system, as people could build on virtual machines to test. - Maybe flagging fewer releases as production release if the above is too onerous - setting up a buildbot system that builds nightly from trunk and is used to check release candidates - we could approach various hosting companies who like to demonstrate that they support open source and asking for donation of VPS's if resources are an issue - a continuous integration system where trunk is built nightly so that people checking in code that breaks the build are alerted right away, before we even branch for releases Feeback please. iain

Date	2012-02-13 02:36
From	Iain Duncan
Subject	[Csnd] Re: Csound release management, esp re Csound 6
	Answering myself, as I just thought of another way to look at this. Csound has an API to allow other apps to embed it and have it as a dependency. When I use Csound in this way, my relationship to Csound is the same as Csound's relationship to its dependencies, libsndfile, fltk, portaudio, etc. Personally, I think the future of csound is in this direction, and that this is very important, but YMMV there. I personally believe the Csound development community should hold itself to the same standard of reliability as we get from those packages that Csound depends on. We don't ever have to worry about whether libdsnfile, jack, scons, fltk, qt, portaudio, etc will build. Their maintainers make sure that the canonical production release is always dependable, and that is taken into account when choosing dependencies. I evaluated many audio output layers for my app, and chose Jack because I know the truck number is good, and it always works. I don't have to get on the jack mailing list or keep track of what's going in there unless I'm interested, and I wouldn't want to have to get on the list for libsndfile to have to figure out how to build Csound. So perhaps this is a growing pain of csound becoming an audio engine library as well as a product. I believe Csound release management should be taken just as seriously as it is by our upstream components. thanks for listening. iain

Date	2012-02-13 12:06
From	luis jure
Subject	Re: [Csnd] Csound release management, esp re Csound 6
	i'm the last one who should chime in (i'm not one of the developers, i'm not even a programmer), but since no-one responded yet to your mail (at least on this list) i'll give my opinion as a simple user. >- do others think this is important? am I in a minority, or are there >others who are concerned and are just not saying anything? yes, i think it's very important and i share your concern about all the issues you mention (of course this doesn't mean anything about who is the "majority"). >- do others share my opinion that this will become more important if >Csound 6 is meant to target developer users of the CsoundAPI? well, i think it's as important as it could possibly be just now. but perhaps you're right and it would be even "more" important with Csound6. in fact, i've been pondering about some of these issues for quite some time. i may be completely off the mark here, but my conclusion is that csound has insufficient resources for such a complex project. i'm not following the dev-list, but my impression is that there are relatively few developers actively working on csound, and many (or most, or all) with only limited time to dedicate to the project. also, there's no institutional support that i know of (again, i might be wrong here). my uneducated opinion is that more human resources would be needed in project development AND management. but then there's the fact that the user base of Csound is much smaller than the other projects you mention (libsndfile, portaudio, jack, etc.). it might not be very easy to get those resources... just my two devalued pesos...

Date	2012-02-15 18:10
From	Steven Yi
Subject	Re: [Csnd] Csound release management, esp re Csound 6
	I have to say, I do find the tone of some of the earlier emails a bit overly harsh and to some degree offensive, though at the same time I do appreciate where it is coming from, what it is trying to address, and do not doubt the sincerity. On the whole, I would say, there are a lot of things going on with Csound that people don't see or notice, and that there aren't a lot of resources. The fact that this is cross-platform software further complicates the matter greatly. There have been a number of occasions where someone has checked in code that works on one platform but not another, or on one variant of an operating system but not another. I think a defined, reproducible release strategy is good. Building releases from tagged sources is a good practice, and automating everything should be the goal. Having a testing period is ideal, though our past experiences for the developers is that we did at one point have beta periods and did not get much of a response, such that it added more work than what we got out of it. We can certainly try it again though. Having nightly builds and continuous integration is something we've discussed already on the developers list. John and I discussed pretty much the same thing Iain raised regarding machines, and went as far as to discuss with Dr. B to use storage on Csounds.com to post the nightly builds. However, other things for Csound development took attention away from that initiative, and we never managed to get everything put in place. I know John has had a system building nightly builds for a while now though, but it's not well known. If someone has the time to set up the CI system and has resources (servers) to dedicate to it, then I think that would certainly be welcome. As for branching strategies, at the last company I worked for, we had quite some issues with different strategies. We tried version branches, feature branches, etc. Some of these things are suited much more for centralized versioning systems like Subversion than they are to distributed systems, IMO. With the number of people we currently have and that we're mostly working on related areas of code, I'd rather use as simple a scheme as possible. One thing that might be easiest is to tag with a release candidate, i.e. "5.16.1_RC", do builds, see if there are issues, ask at that point for volunteers, then if everyone makes it through the build, re-tag with "5.16.1" and rebuild once more. If everything is automated and using the same tag, then it shouldn't be too painful. The one tricky bit is to test the source release, as that can easily build on one computer but not the other. Also, I would suggest that mistakes can and will happen, and that if we're learning from them and making improvements, it's fine. Thanks, steven On Mon, Feb 13, 2012 at 12:06 PM, luis jure wrote: > > i'm the last one who should chime in (i'm not one of the developers, i'm > not even a programmer), but since no-one responded yet to your mail (at > least on this list) i'll give my opinion as a simple user. > > >>- do others think this is important? am I in a minority, or are there >>others who are concerned and are just not saying anything? > > yes, i think it's very important and i share your concern about all the > issues you mention (of course this doesn't mean anything about who is the > "majority"). > > >>- do others share my opinion that this will become more important if >>Csound 6 is meant to target developer users of the CsoundAPI? > > well, i think it's as important as it could possibly be just now. but > perhaps you're right and it would be even "more" important with Csound6. > > > in fact, i've been pondering about some of these issues for quite some > time. i may be completely off the mark here, but my conclusion is that > csound has insufficient resources for such a complex project. i'm not > following the dev-list, but my impression is that there are relatively few > developers actively working on csound, and many (or most, or all) with > only limited time to dedicate to the project. also, there's no > institutional support that i know of (again, i might be wrong here). my > uneducated opinion is that more human resources would be needed in project > development AND management. but then there's the fact that the user base of > Csound is much smaller than the other projects you mention (libsndfile, > portaudio, jack, etc.). it might not be very easy to get those resources... > > > just my two devalued pesos... > > > Send bugs reports to the Sourceforge bug tracker > https://sourceforge.net/tracker/?group_id=81968&atid=564599 > Discussions of bugs and features can be posted here > To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound" >

Date	2012-02-15 18:47
From	Iain Duncan
Subject	Re: [Csnd] Csound release management, esp re Csound 6
	Thanks for your thoughtful response, some further thoughts below. On Wed, Feb 15, 2012 at 10:10 AM, Steven Yi <stevenyi@gmail.com> wrote: I have to say, I do find the tone of some of the earlier emails a bit overly harsh and to some degree offensive, though at the same time I do appreciate where it is coming from, what it is trying to address, Thanks and again, apologies ( as I think you noted). To some degree, I think a lot of my frustration stemmed from the fact that the dev responses to my very first emails came across to me as "there's no problem" and it made me feel like I was talking to a wall. This has since changed, though it seems like a lot of muck had to be stirred up before anyone acknowledged that there was a point under my frustrations. and do not doubt the sincerity. On the whole, I would say, there are a lot of things going on with Csound that people don't see or notice, That is an issue that should be addressed. The EXCELLENT book, "Producing Open Source Software" by Karl Fogel of SVN addresses this very issue. His point is that when decisions and discussions go on offlist, they absolutely need to be mirrored to the list or it leads to misinterpretations and misunderstandings. So I would humbly put forth that it's worth having someone type up a synopsis of these developments and discussions and make sure they always go on the dev list. ( Also, the book is absolutely worth reading ) and that there aren't a lot of resources. The fact that this is cross-platform software further complicates the matter greatly. There have been a number of occasions where someone has checked in code that works on one platform but not another, or on one variant of an operating system but not another. That is a real problem, and should probably be addressed by some kind of policy and testing system. If I decided to write some new Csound code, I would have no expectations that it would get included in trunk unless I could demonstrate that it doesn't break the build on any systems. That's a reasonable policy, but probably requires some kind of buildbot system that all developers have access to to check their work across the whole field before pushing their changes or merging their branches. I think a defined, reproducible release strategy is good. Building releases from tagged sources is a good practice, and automating everything should be the goal. Having a testing period is ideal, though our past experiences for the developers is that we did at one point have beta periods and did not get much of a response, such that it added more work than what we got out of it. We can certainly try it again though. I think in this case one has to accept that there won't be a response. IE no response is a good response. But not doing it can generate a negative response. One of which unfortunately is silent, the developers who decide not to get further involved as a result of these issues, as we've now heard on the dev list. Fogel talks a lot about that phenomena in the book too. Having nightly builds and continuous integration is something we've discussed already on the developers list. John and I discussed pretty much the same thing Iain raised regarding machines, and went as far as to discuss with Dr. B to use storage on Csounds.com to post the nightly builds. However, other things for Csound development took attention away from that initiative, and we never managed to get everything put in place. I know John has had a system building nightly builds for a while now though, but it's not well known. If someone has the time to set up the CI system and has resources (servers) to dedicate to it, then I think that would certainly be welcome. I am interested in contributing in this way, but only if I know it will actually get used religiously. Setting up a buildbot system and build farm isn't much good unless it's backed by a release policy that forces everyone to use it. I know in some companies for instance, you don't get to check code in that doesn't pass the buildbot, period. And if you break the build you stay at the office until it's fixed. IMHO only be stressing the absolute importance of the build running at all times is it going to get used. My pref would be buildbot over Hudson/Jenkins. It's python, and lots of use python. I can test it on my machines, but don't want it to depend on me in the long term as my two businesses are quite volatile and some takes take over life badly for a bit. Thanks for your considered input. I would love an opportunity to demonstrate that I'm not just ranting for ranting's sake, and I'd love to be able know Csound is usable as a dependency for external projects, both in the technical and managerial sense. iain

Date	2012-02-15 19:05
From	luis jure
Subject	Re: [Csnd] Csound release management, esp re Csound 6
	on 2012-02-15 at 18:10 Steven Yi wrote: >I have to say, I do find the tone of some of the earlier emails a bit >overly harsh and to some degree offensive, i'm very sorry for this, my intention was quite the opposite! > On the whole, I would say, there are a lot of things going on with > Csound that people don't see or notice, and that there aren't a lot of > resources. precisely! that's exactly what i expressed in my mail. i thought that had implicit my recognition and gratitude to everyone involved in developing csound, making it the great software it is. again, i'm sorry if the way i expressed myself didn't make that clear. best, lj

Date	2012-02-15 19:24
From	Iain Duncan
Subject	Re: [Csnd] Csound release management, esp re Csound 6
	On Wed, Feb 15, 2012 at 11:05 AM, luis jure <ljc@internet.com.uy> wrote: on 2012-02-15 at 18:10 Steven Yi wrote: >I have to say, I do find the tone of some of the earlier emails a bit >overly harsh and to some degree offensive, i'm very sorry for this, my intention was quite the opposite! Pretty sure that was aimed at my comments on the dev list. ;-) iain

Date	2012-02-15 19:48
From	luis jure
Subject	Re: [Csnd] Csound release management, esp re Csound 6
	on 2012-02-15 at 11:24 Iain Duncan wrote: >Pretty sure that was aimed at my comments on the dev list. ;-) oh... well, here at least on the user list it appeared as a reply to my message, and it really got me worried... anyway, every opportunity is good to reiterate my gratitude to the csound development team. best, lj BTW, i think i should follow more closely the dev-list. i see there's a discussion about migrating from scons to cmake. which for me, as a user who builds csound from sources, is sweet news... :-)

Date	2012-02-15 20:55
From	Michael Gogins
Subject	Re: [Csnd] Csound release management, esp re Csound 6
	I think there are not enough people working on this project to get too fancy. If I can build the Windows installer binaries with CMake, which I will try soon to do, I'm happy to switch. For the rest, the beta release idea was a bust. I think we could accomplish much the same objective by incrementing the release with bug fixes and removing unfixed versions, i.e. if 5.16 has a bug fixed in 5.16.1, we should remove 5.16 and show only 5.16.1. But we would leave the last semi-major release, i.e. if we were at 5.16.5 and then jumpted to 5.17, we would leave 5.16.5 and introduce 5.17.0. I think it just needs to be clearer what is stable and what is not... I'm open to better ideas. Regards, Mike On Wed, Feb 15, 2012 at 2:48 PM, luis jure wrote: > > on 2012-02-15 at 11:24 Iain Duncan wrote: > >>Pretty sure that was aimed at my comments on the dev list. ;-) > > > oh... > > > well, here at least on the user list it appeared as a reply to my message, > and it really got me worried... anyway, every opportunity is good to > reiterate my gratitude to the csound development team. > > > best, > > > lj > > > BTW, i think i should follow more closely the dev-list. i see there's a > discussion about migrating from scons to cmake. which for me, as a user > who builds csound from sources, is sweet news... :-) > > > Send bugs reports to the Sourceforge bug tracker > https://sourceforge.net/tracker/?group_id=81968&atid=564599 > Discussions of bugs and features can be posted here > To unsubscribe, send email sympa@lists.bath.ac.uk with body "unsubscribe csound" > -- Michael Gogins Irreducible Productions http://www.michael-gogins.com Michael dot Gogins at gmail dot com

Date	2012-02-15 22:20
From	Iain Duncan
Subject	Re: [Csnd] Csound release management, esp re Csound 6
	On Wed, Feb 15, 2012 at 12:55 PM, Michael Gogins <michael.gogins@gmail.com> wrote: I think there are not enough people working on this project to get too fancy. If I can build the Windows installer binaries with CMake, which I will try soon to do, I'm happy to switch. For the rest, the beta release idea was a bust. I think we could accomplish much the same objective by incrementing the release with bug fixes and removing unfixed versions, i.e. if 5.16 has a bug fixed in 5.16.1, we should remove 5.16 and show only 5.16.1. But we would leave the last semi-major release, i.e. if we were at 5.16.5 and then jumpted to 5.17, we would leave 5.16.5 and introduce 5.17.0. I think it just needs to be clearer what is stable and what is not... I agree with that. But maybe a better option would be to label the moving target releases as betas for the next point? To me, that's really what they are as far as the normal use of the term. IE, it should be labelled beta or rc if it is not a known stable tested build, and anything labelled normally should be stable. Then sourceforge could have have something like this: 5.16 - > production build 5.17b1 - snapshot for developers, not guaranteed 5.17rc1 - we're thinking this one will become 5.17 real soon now if it passes muster nightly snapshot - for developers, not guaranteed I'm open to how it's done too, but we really need to have something on sourceforge that is clearly what you download if you want the latest stable known good release, and that ought to build reliably on some specified set of platforms across some specified set of options. IHMO, the current situation is just too confusing and frustrating to someone new who just goes there to build csound from source but isn't necessarily planning on becoming heavily involved in csound ( ie, someone using csound as a dependency for another app ) thanks iain

Date	2012-02-16 00:26
From	Michael Gogins
Subject	Re: [Csnd] Csound release management, esp re Csound 6
	For some time, Linus Torvalds managed the Linux release process by having even numbered releases be "developer" and odd numbered releases be "release" versions, or maybe it was the other way around. That would be fine with me. This could work by always working on "developer" releases, and when most of the outstanding bugs are finished up, or enough time has gone by, or something, John can announce a push for the next "release" version, and then we focus on that and when it's done, go back to working on the "release" version. Regards, Mike On Wed, Feb 15, 2012 at 5:20 PM, Iain Duncan wrote: > On Wed, Feb 15, 2012 at 12:55 PM, Michael Gogins > wrote: >> >> I think there are not enough people working on this project to get too >> fancy. >> >> If I can build the Windows installer binaries with CMake, which I will >> try soon to do, I'm happy to switch. >> >> For the rest, the beta release idea was a bust. I think we could >> accomplish much the same objective by incrementing the release with >> bug fixes and removing unfixed versions, i.e. if 5.16 has a bug fixed >> in 5.16.1, we should remove 5.16 and show only 5.16.1. But we would >> leave the last semi-major release, i.e. if we were at 5.16.5 and then >> jumpted to 5.17, we would leave 5.16.5 and introduce 5.17.0. >> >> I think it just needs to be clearer what is stable and what is not... > > > I agree with that. But maybe a better option would be to label the moving > target releases as betas for the next point? To me, that's really what > they are as far as the normal use of the term. IE, it should be labelled > beta or rc if it is not a known stable tested build, and anything labelled > normally should be stable. > > Then sourceforge could have have something like this: > > 5.16 - > production build > 5.17b1 - snapshot for developers, not guaranteed > 5.17rc1 - we're thinking this one will become 5.17 real soon now if it > passes muster > nightly snapshot - for developers, not guaranteed > > I'm open to how it's done too, but we really need to have something on > sourceforge that is clearly what you download if you want the latest stable > known good release, and that ought to build reliably on some specified set > of platforms across some specified set of options. IHMO, the current > situation is just too confusing and frustrating to someone new who just goes > there to build csound from source but isn't necessarily planning on becoming > heavily involved in csound ( ie, someone using csound as a dependency for > another app ) > > thanks > iain -- Michael Gogins Irreducible Productions http://www.michael-gogins.com Michael dot Gogins at gmail dot com

Date	2012-02-16 02:06
From	Iain Duncan
Subject	Re: [Csnd] Csound release management, esp re Csound 6
	On Wed, Feb 15, 2012 at 4:26 PM, Michael Gogins <michael.gogins@gmail.com> wrote: For some time, Linus Torvalds managed the Linux release process by having even numbered releases be "developer" and odd numbered releases be "release" versions, or maybe it was the other way around. That would be fine with me. This could work by always working on "developer" releases, and when most of the outstanding bugs are finished up, or enough time has gone by, or something, John can announce a push for the next "release" version, and then we focus on that and when it's done, go back to working on the "release" version. Absolutely! +1000 That worked great for linux because it was well published, and the release management policy is strictly enforced. I think that would be great system for csound, and would screw up people's workflows the least. We could make buildbots for the release-releases and keep it as others see fit for the developer releases. iain