U-Boot btrfs implementation - will there by syncing?
Hello Qu and others,
one of the points of Qu's reimplementation of U-Boot's btrfs code that made it somehow synced with btrfs-progs, was that the old implementation was, quote
pretty different from btrfs-progs nor kernel, making it pretty hard to sync code between different projects
Another point was that afterwards Qu wanted to bring some of the missing btrfs features into U-Boot, since they should be much easier to implement after the reimplementation.
I was looking at current btrfs-progs, and noticed that there were many changes since then, but only one or two were ever synced to U-Boot.
I would like to know if Qu, or anyone else, is planning to do this code-syncing, and maybe implement some of the missing features?
The reason I am asking this is that in the meantime I've reached the opinion that the idea of code-syncing in this sense almost never really works. It is usually proposed by one person who ports the code from another project, and then keeps it synced for a time, but then stops doing it because they have too much other work, or change employer, or another reason.
It happened multiple times. One example is U-Boot's Hush shell, which was imported from busybox, with the intention to keep it in sync, but the reality is that U-Boot's implementation is now years (decade?) old, and there were so many ad-hoc fixes done in U-Boot that currently the only way to sync is basically to change the whole code (which will also cause issues with existing user scripts, but maybe it should be done anyway).
Another reason why I am writing this is that I think that with the amount of work Qu put into that reimplementation last year, he could have instead implemented some of the new features in the old implementation and spend a similar amount of time doing that, and the result would be those new features in U-Boot for over a year by now.
Also, I've come to the opinion that maybe two different and maintained implementations are a good thing for such a project as a filesystem, similar to how multiple implementations of C/C++ compiler are a good thing.
For the last few months I was on-and-off thinking about whether it would make sense to revert to the original implementation and then implement some of the missing features. But it would not make sense if Qu, or anyone else, is planning to do that code syncing and bringing of new features from btrfs-progs, so that's why I am asking.
Please give your opinions, people.
Thanks.
Marek
On 2021/12/17 07:05, Marek Behún wrote:
Hello Qu and others,
one of the points of Qu's reimplementation of U-Boot's btrfs code that made it somehow synced with btrfs-progs, was that the old implementation was, quote
pretty different from btrfs-progs nor kernel, making it pretty hard to sync code between different projects
Another point was that afterwards Qu wanted to bring some of the missing btrfs features into U-Boot, since they should be much easier to implement after the reimplementation.
I was looking at current btrfs-progs, and noticed that there were many changes since then, but only one or two were ever synced to U-Boot.
Yep, those changes are mostly preparation for the incoming extent-tree v2 feature, they bring no real behavior change for now.
And I'm planning to cross-port them just to make later cross-port easier.
Thanks for the reminder.
I would like to know if Qu, or anyone else, is planning to do this code-syncing, and maybe implement some of the missing features?
The reason I am asking this is that in the meantime I've reached the opinion that the idea of code-syncing in this sense almost never really works. It is usually proposed by one person who ports the code from another project, and then keeps it synced for a time, but then stops doing it because they have too much other work, or change employer, or another reason.
It happened multiple times. One example is U-Boot's Hush shell, which was imported from busybox, with the intention to keep it in sync, but the reality is that U-Boot's implementation is now years (decade?) old, and there were so many ad-hoc fixes done in U-Boot that currently the only way to sync is basically to change the whole code (which will also cause issues with existing user scripts, but maybe it should be done anyway).
Another reason why I am writing this is that I think that with the amount of work Qu put into that reimplementation last year, he could have instead implemented some of the new features in the old implementation and spend a similar amount of time doing that, and the result would be those new features in U-Boot for over a year by now.
I doubt that, the original interface is completely different than the interface we have in kernel/progs.
That means, if one wants to cross port the recent extent-tree v2 changes, it will be a hell.
So in the long run, it's still beneficial, even they get de-synced somehow.
Also, I've come to the opinion that maybe two different and maintained implementations are a good thing for such a project as a filesystem, similar to how multiple implementations of C/C++ compiler are a good thing.
For the last few months I was on-and-off thinking about whether it would make sense to revert to the original implementation and then implement some of the missing features. But it would not make sense if Qu, or anyone else, is planning to do that code syncing and bringing of new features from btrfs-progs, so that's why I am asking.
The worst example for different implementations are another bootloader, GRUB.
Its btrfs implementation has way less features, no sanity checks, and is super easy to craft an image to cause infinite loop for GRUB. And it even can not handle the new default mkfs features (NO_HOLES, which is there for long long time).
Thus I even re-implemented a new license compatible project from scratch, btrfs-fuse, to provide a basis for later cross-port. (That only has read-only support, and is mostly MIT licesnsed, the interface is almost the same as progs/kernel).
It may be possible for simpler filesystems, but for complex filesysmtes like btrfs, a completely different implementation with completely different interfaces is really not an ideal situation.
I don't really want to do it again and again, using different (and sometimes very crappy) interfaces to do the same work.
Thanks, Qu
Please give your opinions, people.
Thanks.
Marek
On 2021/12/17 07:45, Qu Wenruo wrote:
On 2021/12/17 07:05, Marek Behún wrote:
Hello Qu and others,
one of the points of Qu's reimplementation of U-Boot's btrfs code that made it somehow synced with btrfs-progs, was that the old implementation was, quote
pretty different from btrfs-progs nor kernel, making it pretty hard to sync code between different projects
Another point was that afterwards Qu wanted to bring some of the missing btrfs features into U-Boot, since they should be much easier to implement after the reimplementation.
I was looking at current btrfs-progs, and noticed that there were many changes since then, but only one or two were ever synced to U-Boot.
Yep, those changes are mostly preparation for the incoming extent-tree v2 feature, they bring no real behavior change for now.
And I'm planning to cross-port them just to make later cross-port easier.
Forgot to mention that, a lot of btrfs-progs changes are not to the core implementation, but ioctls/user-interafaces/check/image tools, thus you may need to do more filtering to figure out what's really needed to be backported.
But I may not cover all changes, so if you find some aspects missing, feel free to point it out, or even better, try to crossport if possible.
Thanks for the reminder.
I would like to know if Qu, or anyone else, is planning to do this code-syncing, and maybe implement some of the missing features?
Some missing features, are blocked by lacking of interfaces and knoledge.
Currently the missing hash (BLAKE2) needs BLAKE2 support in U-boot, which is still not yet in U-boot.
If really needed, I may crossport BLAKE2B to Uboot then implement the support in btrfs.
The similar case is for multi-device support, I'm not familiar enough on how to iterate through all block devices in Uboot.
If there is already such interfaces or someone could provide some help, I'm pretty happy to implement it in Uboot.
The reason I am asking this is that in the meantime I've reached the opinion that the idea of code-syncing in this sense almost never really works. It is usually proposed by one person who ports the code from another project, and then keeps it synced for a time, but then stops doing it because they have too much other work, or change employer, or another reason.
Personally speaking, my current employer (SUSE) is super awesome, and there is no plan to move to another employer.
Furthermore, SUSE is a super strong supporter for btrfs, thus maintaining a common base to provide better btrfs support is definitely benefiting SUSE.
Thus you could count on such long term support, and personally speaking I'm pretty happy to work on btrfs, even not paid for.
So the long term support, at least from myself, should be something you can rely on.
Although there are indeed cases where I'm buried by other btrfs work, thus such reminder is really helpful.
Thanks, Qu
It happened multiple times. One example is U-Boot's Hush shell, which was imported from busybox, with the intention to keep it in sync, but the reality is that U-Boot's implementation is now years (decade?) old, and there were so many ad-hoc fixes done in U-Boot that currently the only way to sync is basically to change the whole code (which will also cause issues with existing user scripts, but maybe it should be done anyway).
Another reason why I am writing this is that I think that with the amount of work Qu put into that reimplementation last year, he could have instead implemented some of the new features in the old implementation and spend a similar amount of time doing that, and the result would be those new features in U-Boot for over a year by now.
I doubt that, the original interface is completely different than the interface we have in kernel/progs.
That means, if one wants to cross port the recent extent-tree v2 changes, it will be a hell.
So in the long run, it's still beneficial, even they get de-synced somehow.
Also, I've come to the opinion that maybe two different and maintained implementations are a good thing for such a project as a filesystem, similar to how multiple implementations of C/C++ compiler are a good thing.
For the last few months I was on-and-off thinking about whether it would make sense to revert to the original implementation and then implement some of the missing features. But it would not make sense if Qu, or anyone else, is planning to do that code syncing and bringing of new features from btrfs-progs, so that's why I am asking.
The worst example for different implementations are another bootloader, GRUB.
Its btrfs implementation has way less features, no sanity checks, and is super easy to craft an image to cause infinite loop for GRUB. And it even can not handle the new default mkfs features (NO_HOLES, which is there for long long time).
Thus I even re-implemented a new license compatible project from scratch, btrfs-fuse, to provide a basis for later cross-port. (That only has read-only support, and is mostly MIT licesnsed, the interface is almost the same as progs/kernel).
It may be possible for simpler filesystems, but for complex filesysmtes like btrfs, a completely different implementation with completely different interfaces is really not an ideal situation.
I don't really want to do it again and again, using different (and sometimes very crappy) interfaces to do the same work.
Thanks, Qu
Please give your opinions, people.
Thanks.
Marek
participants (2)
-
Marek Behún -
Qu Wenruo