Hello Wolfgang, Tom,
Am 06.11.2013 08:50, schrieb Wolfgang Denk:
Dear Tom,
In message20131105203736.GM5925@bill-the-cat you wrote:
We have the real problem, that we have a lot of old boards, which are unmaintained in U-Boot, and we have no chance to find out, if this boards are longer used/tested ...
We also have a feature, lots of hardware support because lots of things don't change drastically, frequently. That's not to say that I wouldn't mind dropping old platforms (even that ones I have sentimental feelings towards), and would certainly like to see more and frequent sanity testing.
Yep, thats the reason I had in mind. We must have more real tests on real hardware (or at least, get back the info, that this is done somewhere ...) ... thats the idea behind to force the board maintainers to give back us such info.
I think Heiko's idea of documenting test reports is pretty cool - but of course we need to discuss in detail how to implement this, and also decide wether we use this for (semi-automatic) code cleanup (by removing boards that have not been tested for a long time).
I vote for removing boards from which we get no such test info back. The board support code is just availiable, just not for current releases.
If code does not change drastically, such a "compile and try it on the hardware, give feedback to mainline" should not cost much time the board maintainer, and maybe we add a script, which the board maintainer just have to call, to send an "board tested EMail" to the mailinglist ...
Maybe a lot of boardmaintainers do this tests, and this info is important for mainline, but we have no mechanism to collect this info!
And if code changes drastically (which it currently does and will do in future I think) a board maintainer must decide, if it makes sense to sync the board with current mainline or the board get dropped from mainline ...
And I have this problem currently with i2c subsystem ... a lot of boards to convert to new framework, and I cannot decide, which are really maintained... or if currently, the converted boards, really work. (And I think, not only the i2c subsystem has this problem)
This problem comes up when we talk about doing big changes, and I think that's the right time to talk about things. And I think the answer should be, we try and convert things forward and when it's not obvious if things will still work correctly, or how to do it, that's when we need to make a hard push on the board maintainers to find some time to work on things.
Agreed. And here information how recently (or maybe even how frequently) a board has been tested (build tested, run on actual hardware) would come in really handy. we can probbaly automate build testing one way or another, but for actual runtime tests we will lways depend on the board maintainers, or board users.
Hmm.. Is it always obvious, if changes are big? Do we really get feedback when doing such big changes? I just reworking the i2c framework, and I have not really much feedback from board maintainers for their boards I changed ... Does this change really work on all boards I converted? I have no chance to get this info. But if we have such a livetime feature, I can be sure, that after n releases all boards in mainline are tested ... automatically ... I want such a feature ...
So, the question raises, should we introduce a column in boards.cfg, which shows the "livetime" of a board support in U-Boot?
I sense a lot of conflicting patches.
Again I agree. Also, I fear that boards.cfg is becoming more and more unreadable by adding even more stuff. If I see this correctly, the maximum line length in boards.cfg already exceeds 360 characters :-(
Right, boards.cfg gets unhandy ... Hmm .. what with the column "Staus" ... instead of "Active" it would be more informative to have there the livetime counter, and a single digit saves some characters ;-)
But you are right, that approach leads in a lot of conflicting patches ... but I think, we just pooled board information in boards.cfg, so this would be the right place in my eyes ...
Maybe we get such Information "a Boards is tested with current mainline" inform of an EMail with an Text "Board xy tested with commit mm. Please update livetime" ... and we can add a script, which updates the livetime for this board, so we can prevent conflicting patches ... ?
So nstead of adding this information to boards.cfg we could probably use separate files for such information. We could provide tools to make test reports really easy, say something like
scripts/build_test scripts/run_test
which the user would just call with a "passed" or "failed" argument; the scripts could then auto-detect which configuration and which exact U-Boot version were in use, and send an email. Whether that would be a patch against the source code or something that get's auto-added to a wiki page is just an implementation detail. But if we had something like this, we could get a muchbetter understanding how actively boards are being tested.
Yes, that sound also good. I want to see the test information in the sourcecode, not somewhere on a wiki...
So when you're once again doing some change that requires touching files for some othe rboards, you could simply check that database. If you see that 3 out of the last 5 releases have reported succesful run-time tests you will probably decide to accept the needed efforts,
Hmm.. that works, if you have to touch some (some < 5) boards. But If you have to touch > 5 boards, this gets unhandy...
but when you see the last test report is more than 5 years old, you will probably rather decide to initiate a code removal process.
If we decide to delete older boards after n release cycles without testreports, we must not decide nor look in a database. We are sure, we have only "good and working" boards ... and we just do the necessary work for new features ... and we are sure, that we get back testreports within n release cycles ...
So let us decide first, if we want to go this way ...
bye, Heiko