[U-Boot] [ANN] git server, FTP server
Hello all,
we're planning some reorganization / updates for our servers:
1. We will move all git repositories to a new, faster machine. In this process, the web interface will change (moving from gitweb to cgit). In this process, for a (hopefully) short time, access to the git repositories will be impossible. I will send more details when we have decided about the schedule.
2. For some time now, we provide not only the classic FTP server for download of the U-Boot release tarballs, but also a public directory in the Amazon Cloud Drive [1]. The ACD is supposed to provide much better connectivity (especially for non-european users) and much higher download rates.
So far, I have received zero feedback about this. I would like to hear about your experiences (good and bad), and what you think about dropping the FTP server completely.
Thanks in advance.
[1] https://www.amazon.com/clouddrive/share/IvfObauv9EsZau4GLr7i6BsLX5Zh8THmHXDi...
Best regards,
Wolfgang Denk
Hi Wolfgang,
On 07/15/15 10:24, Wolfgang Denk wrote:
Hello all,
we're planning some reorganization / updates for our servers:
We will move all git repositories to a new, faster machine. In this process, the web interface will change (moving from gitweb to cgit). In this process, for a (hopefully) short time, access to the git repositories will be impossible. I will send more details when we have decided about the schedule.
For some time now, we provide not only the classic FTP server for download of the U-Boot release tarballs, but also a public directory in the Amazon Cloud Drive [1]. The ACD is supposed to provide much better connectivity (especially for non-european users) and much higher download rates.
So far, I have received zero feedback about this. I would like to hear about your experiences (good and bad), and what you think about dropping the FTP server completely.
Seems very reasonable. We came to the same conclusion recently for some of our needs.
Thanks in advance.
[1] https://www.amazon.com/clouddrive/share/IvfObauv9EsZau4GLr7i6BsLX5Zh8THmHXDi...
Works great, although all the files are relatively small (<10MB). Anyway, it took about ~5-8 seconds to download the u-boot-2015.07.tar.bz2.
Hi Wolfgang,
On Wed, Jul 15, 2015 at 3:24 PM, Wolfgang Denk wd@denx.de wrote:
Hello all,
we're planning some reorganization / updates for our servers:
- We will move all git repositories to a new, faster machine. In this process, the web interface will change (moving from gitweb to cgit). In this process, for a (hopefully) short time, access to the git repositories will be impossible. I will send more details when we have decided about the schedule.
Is the U-Boot mailing list hosted on the same machine as git? Will this move also fix the mailing list occasional hiccup issue?
For some time now, we provide not only the classic FTP server for download of the U-Boot release tarballs, but also a public directory in the Amazon Cloud Drive [1]. The ACD is supposed to provide much better connectivity (especially for non-european users) and much higher download rates.
So far, I have received zero feedback about this. I would like to hear about your experiences (good and bad), and what you think about dropping the FTP server completely.
Thanks in advance.
[1] https://www.amazon.com/clouddrive/share/IvfObauv9EsZau4GLr7i6BsLX5Zh8THmHXDi...
I just downloaded several tarballs from ACD. Looks the download speed is OK for me (average 200KB/s).
Regards, Bin
Dear Bin,
In message CAEUhbmUXS5DE_un0fvJo+cAwALFP-1Gu1J+Ra_wpLypJfO0e_Q@mail.gmail.com you wrote:
Is the U-Boot mailing list hosted on the same machine as git? Will this move also fix the mailing list occasional hiccup issue?
Yes, at the moment this is on the same, old and slow machine. We will move services one by one, the mailing lists are next after git. And yes, this is also supposed to help with the mailing list issues.
[1] https://www.amazon.com/clouddrive/share/IvfObauv9EsZau4GLr7i6BsLX5Zh8THmHXDi...
I just downloaded several tarballs from ACD. Looks the download speed is OK for me (average 200KB/s).
Ouch, that's awfully slow. Are you sure the bottleneck is on the ACD side? How fast is your link usually?
Best regards,
Wolfgang Denk
Hi Wolfgang,
On Wed, Jul 15, 2015 at 8:35 PM, Wolfgang Denk wd@denx.de wrote:
Dear Bin,
In message CAEUhbmUXS5DE_un0fvJo+cAwALFP-1Gu1J+Ra_wpLypJfO0e_Q@mail.gmail.com you wrote:
Is the U-Boot mailing list hosted on the same machine as git? Will this move also fix the mailing list occasional hiccup issue?
Yes, at the moment this is on the same, old and slow machine. We will move services one by one, the mailing lists are next after git. And yes, this is also supposed to help with the mailing list issues.
[1] https://www.amazon.com/clouddrive/share/IvfObauv9EsZau4GLr7i6BsLX5Zh8THmHXDi...
I just downloaded several tarballs from ACD. Looks the download speed is OK for me (average 200KB/s).
Ouch, that's awfully slow. Are you sure the bottleneck is on the ACD side? How fast is your link usually?
I have a 15Mbps bandwidth link to the internet. Is the ACD supposed to choose wherever the nearest download site to the downloader for better speed? The reason that I felt it is OK for me, is that for the original ftp.denx.de I could only get average 30KB/s when loading a tarball. Thus I am thankful for the new ACD :)
Regards, Bin
Dear Bin,
In message CAEUhbmUYatxaYtbBAaKWrxNopaNfxJiXchw-3Y_NP1FxZ9RWjw@mail.gmail.com you wrote:
I just downloaded several tarballs from ACD. Looks the download speed is OK for me (average 200KB/s).
Ouch, that's awfully slow. Are you sure the bottleneck is on the ACD side? How fast is your link usually?
I have a 15Mbps bandwidth link to the internet. Is the ACD supposed to choose wherever the nearest download site to the downloader for better speed?
Yes.
Hm, other prople have reported (ok - for bigger files, say 4+ GiB) download rates of 11 MiB/s (MegaBYTES per second) on a 100 Mbps fibre link, which is close to the theoretical limit there. So you should get better rates, I think.
The reason that I felt it is OK for me, is that for the original ftp.denx.de I could only get average 30KB/s when loading a tarball. Thus I am thankful for the new ACD :)
I'm glad if it helps, even though I can't explain why you don't get the full bandwidth.
Best regards,
Wolfgang Denk
Wolfgang wrote...
{snip}
For some time now, we provide not only the classic FTP server for download of the U-Boot release tarballs, but also a public directory in the Amazon Cloud Drive [1]. The ACD is supposed to provide much better connectivity (especially for non-european users) and much higher download rates.
So far, I have received zero feedback about this. I would like to hear about your experiences (good and bad), and what you think about dropping the FTP server completely.
Today it took about 5 seconds of manual effort to copy the classic FTP URL from the announce email and use it with wget which then took just under 6 seconds to download it.
So far, two attempts at using the Amazon Cloud web interface have failed. Clicking on the "Download" button result in the message "Download starting" but no file appears. It is probably just Chrome being a PITA but I know which one currently gets my vote!
Andy.
Hello Wolfgang,
Wolfgang Denk wrote on 2015-07-15:
For some time now, we provide not only the classic FTP server for download of the U-Boot release tarballs, but also a public directory in the Amazon Cloud Drive [1]. The ACD is supposed to provide much better connectivity (especially for non-european users) and much higher download rates.
So far, I have received zero feedback about this. I would like to hear about your experiences (good and bad), and what you think about dropping the FTP server completely.
In addition to the mail from Andy:
I tried to find a way to download a file with wget or a similar tool, which would be used by a distribution builder (like Yocto, Buildroot, OpenWrt, ...). Up to now I failed, as the download button depends on javascript and the generated download link for the file [1] has no indication of the file name (and because of "templink" inside I am also not sure, how constant this would be).
If somebody can provide a way to download the tarballs by name from a command line I would accept the ACD as a replacement. Otherwise I would prefer a server (like ftp) with some kind of stable URLs.
Best Regards, Thomas
[1] https://content-na.drive.amazonaws.com/cdproxy/templink/No5u9y7HLlaVIEwnz3VG...
Dear Thomas,
In message 593AEF6C47F46446852B067021A273D6FBC788CC@MUCSE037.lantiq.com you wrote:
I tried to find a way to download a file with wget or a similar tool, which would be used by a distribution builder (like Yocto, Buildroot, OpenWrt, ...).
Why would any build environment use tarballs? can you not just reference the git repository? This is much more efficient, IMHO.
Up to now I failed, as the download button depends on javascript and the generated download link for the file [1] has no indication of the file name (and because of "templink" inside I am also not sure, how constant this would be).
Sorry, I cannot say much about this.
If somebody can provide a way to download the tarballs by name from a command line I would accept the ACD as a replacement.
There is tools like acd_cli.py (on github) which also allow a FUSE mount on ACD data, so there are other options for fancy uses. But I don't know if (or how) these would work with shared data.
Otherwise I would prefer a server (like ftp) with some kind of stable URLs.
Well, the ACD URLs _are_ stable, just not really conveniend to use from the command line.
Thanks for your input.
Best regards,
Wolfgang Denk
Dear Wolfgang,
Am 15.07.2015 um 14:51 schrieb Wolfgang Denk:
In message 593AEF6C47F46446852B067021A273D6FBC788CC@MUCSE037.lantiq.com you wrote:
I tried to find a way to download a file with wget or a similar tool, which would be used by a distribution builder (like Yocto, Buildroot, OpenWrt, ...).
Why would any build environment use tarballs? can you not just reference the git repository? This is much more efficient, IMHO.
Open Build Service uses tarballs, too. We usually specify full URLs in the RPM .spec file [1] both for documentation of origin and so that the mirrored tarball can be verified via gpg or checksum where available. As the actual build is intentionally done offline, at some point a tarball is needed. The URL could happily be a redirect to some mirror.
When developing, Git is great of course.
Regards, Andreas
[1] https://build.opensuse.org/package/view_file/Base:System/u-boot/u-boot.spec....
I tried to find a way to download a file with wget or a similar tool, which would be used by a distribution builder (like Yocto, Buildroot, OpenWrt, ...).
Why would any build environment use tarballs? can you not just reference the git repository? This is much more efficient, IMHO.
Open Build Service uses tarballs, too. We usually specify full URLs in the RPM .spec file [1] both for documentation of origin and so that the mirrored tarball can be verified via gpg or checksum where available. As the actual build is intentionally done offline, at some point a tarball is needed. The URL could happily be a redirect to some mirror.
This is the same in Fedora, the builds are done in a constrained environment without connectivity to the general internet.
Peter
Dear Peter,
In message CALeDE9NJiqCaE3zhdrS80Fe_P4bfxCG2naW8YhzrHM+WqSHWfw@mail.gmail.com you wrote:
I tried to find a way to download a file with wget or a similar tool, which would be used by a distribution builder (like Yocto, Buildroot, OpenWrt, ...).
Why would any build environment use tarballs? can you not just reference the git repository? This is much more efficient, IMHO.
Open Build Service uses tarballs, too. We usually specify full URLs in the RPM .spec file [1] both for documentation of origin and so that the mirrored tarball can be verified via gpg or checksum where available. As the actual build is intentionally done offline, at some point a tarball is needed. The URL could happily be a redirect to some mirror.
This is the same in Fedora, the builds are done in a constrained environment without connectivity to the general internet.
This is normal. It's the same in any Yocto based environment. And probably in any other buil environment that alloes to reliably reproduce a specific build. But you can still use git to fetch your sources - instead of storing a tarball locally, you store a clone of the git repo.
Best regards,
Wolfgang Denk
Dear Andreas,
In message 55ABD78C.2060108@suse.de you wrote:
Why would any build environment use tarballs? can you not just reference the git repository? This is much more efficient, IMHO.
Open Build Service uses tarballs, too. We usually specify full URLs in the RPM .spec file [1] both for documentation of origin and so that the mirrored tarball can be verified via gpg or checksum where available.
OK, I get the argument.
Well, technically a git repository is better even in this aspect, as it is inherently self-verifying :-)
But no worries, we will keep FTP if there are people who find it useful.
Best regards,
Wolfgang Denk
participants (7)
-
Andreas Färber -
Andy Pont -
Bin Meng -
Igor Grinberg -
Peter Robinson -
thomas.langer＠lantiq.com -
Wolfgang Denk