[U-Boot] [RFC PATCH v2 0/2] Make Python scripts compatible with older versions
It was reported by Miao Yan that the kconfig-related Python scripts are not working on Python 2.4.
Do we have any consensus in terms of Python version requirement?
This series avoids using "with ... as ..." and "except ... as ..." statements. But "with ... as ..." is used everywhere in buildman, which means buildman requires at least Python 2.6.
It is true we hope tools can work on broad range of versions, but it also means we have more restrictions when writing Python scripts.
I have no idea which version we should expect at least. Your comments are welcome.
Changes in v2: - Fix git-description. s/exception/except/ - Fix git-description. s/exception/except/
Masahiro Yamada (2): kconfig: make multiconfig.py compatible with Python 2.4 tools: make genboardscfg.py compatible with Python 2.5
scripts/multiconfig.py | 73 +++++++++++++++++++++++--------------------------- tools/genboardscfg.py | 4 +-- 2 files changed, 36 insertions(+), 41 deletions(-)
The statements "with ... as ..." and "except ... as ..." are available in Python 2.6 or lator. Do not use them to support older version of Python. Tested on Python 2.4.6.
Signed-off-by: Masahiro Yamada yamada.m@jp.panasonic.com ---
Changes in v2: - Fix git-description. s/exception/except/
scripts/multiconfig.py | 73 +++++++++++++++++++++++--------------------------- 1 file changed, 34 insertions(+), 39 deletions(-)
diff --git a/scripts/multiconfig.py b/scripts/multiconfig.py index 749abcb..de5ec51 100755 --- a/scripts/multiconfig.py +++ b/scripts/multiconfig.py @@ -165,7 +165,7 @@ def mkdirs(*dirs): for d in dirs: try: os.makedirs(d) - except OSError as exception: + except OSError, exception: # Ignore 'File exists' error if exception.errno != errno.EEXIST: raise @@ -175,7 +175,7 @@ def rmfiles(*files): for f in files: try: os.remove(f) - except OSError as exception: + except OSError, exception: # Ignore 'No such file or directory' error if exception.errno != errno.ENOENT: raise @@ -187,7 +187,7 @@ def rmdirs(*dirs): for d in dirs: try: os.rmdir(d) - except OSError as exception: + except OSError, exception: # Ignore 'No such file or directory' # and 'Directory not empty' error if exception.errno != errno.ENOENT and \ @@ -253,15 +253,14 @@ def get_enabled_subimages(ignore_error=False): for img in SUB_IMAGES ] try: f = open(KCONFIG_CONFIG) - except IOError as exception: + except IOError, exception: if not ignore_error or exception.errno != errno.ENOENT: raise return enabled - with f: - for line in f: - for img, pattern in match_patterns: - if line == pattern: - enabled += (img,) + for line in f: + for img, pattern in match_patterns: + if line == pattern: + enabled += (img,) return enabled
def do_silentoldconfig(cmd): @@ -307,8 +306,7 @@ def do_tmp_defconfig(output_lines, img): TMP_DIRS = ('arch', 'configs') defconfig_path = os.path.join('configs', TMP_DEFCONFIG) mkdirs(*TMP_DIRS) - with open(defconfig_path, 'w') as f: - f.write(''.join(output_lines[img])) + open(defconfig_path, 'w').write(''.join(output_lines[img])) cleanup = lambda: (rmfiles(defconfig_path), rmdirs(*TMP_DIRS)) run_make_config(TMP_DEFCONFIG, img, cleanup) cleanup() @@ -321,15 +319,14 @@ def do_board_defconfig(cmd): """ defconfig_path = os.path.join(srctree, 'configs', cmd) output_lines = dict([ (img, []) for img in IMAGES ]) - with open(defconfig_path) as f: - for line in f: - m = PATTERN_SYMBOL.match(line) - if m: - for idx, img in enumerate(IMAGES): - if m.group(idx + 1): - output_lines[img].append(m.group(4) + '\n') - continue - output_lines[''].append(line) + for line in open(defconfig_path): + m = PATTERN_SYMBOL.match(line) + if m: + for idx, img in enumerate(IMAGES): + if m.group(idx + 1): + output_lines[img].append(m.group(4) + '\n') + continue + output_lines[''].append(line) do_tmp_defconfig(output_lines, '') for img in get_enabled_subimages(): do_tmp_defconfig(output_lines, img) @@ -356,29 +353,27 @@ def do_savedefconfig(cmd): run_make_config(cmd, '') output_lines = [] prefix = {} - with open(DEFCONFIG) as f: - for line in f: - output_lines.append(line) - prefix[line] = '+' + for line in open(DEFCONFIG): + output_lines.append(line) + prefix[line] = '+' for img in subimages: run_make_config(cmd, img) unmatched_lines = [] - with open(DEFCONFIG) as f: - for line in f: - if line in output_lines: - index = output_lines.index(line) - output_lines[index:index] = unmatched_lines - unmatched_lines = [] - prefix[line] += SYMBOL_MAP[img] - else: - ummatched_lines.append(line) - prefix[line] = SYMBOL_MAP[img] - with open(DEFCONFIG, 'w') as f: - for line in output_lines: - if prefix[line] == '+': - f.write(line) + for line in open(DEFCONFIG): + if line in output_lines: + index = output_lines.index(line) + output_lines[index:index] = unmatched_lines + unmatched_lines = [] + prefix[line] += SYMBOL_MAP[img] else: - f.write(prefix[line] + ':' + line) + ummatched_lines.append(line) + prefix[line] = SYMBOL_MAP[img] + f = open(DEFCONFIG, 'w') + for line in output_lines: + if prefix[line] == '+': + f.write(line) + else: + f.write(prefix[line] + ':' + line)
def do_others(cmd): """Run the make command other than 'silentoldconfig', 'defconfig',
The statement "exception ... as ..." can be used in Python 2.6 or lator. Avoid using it. Tested on Python 2.5.6.
Signed-off-by: Masahiro Yamada yamada.m@jp.panasonic.com ---
Changes in v2: - Fix git-description. s/exception/except/
tools/genboardscfg.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/genboardscfg.py b/tools/genboardscfg.py index 734d90b..9e4892f 100755 --- a/tools/genboardscfg.py +++ b/tools/genboardscfg.py @@ -57,7 +57,7 @@ def get_terminal_columns(): arg = struct.pack('hhhh', 0, 0, 0, 0) try: ret = fcntl.ioctl(sys.stdout.fileno(), termios.TIOCGWINSZ, arg) - except IOError as exception: + except IOError, exception: if exception.errno != errno.ENOTTY: raise # If 'Inappropriate ioctl for device' error occurs, @@ -473,7 +473,7 @@ def gen_boards_cfg(jobs): # We should remove incomplete boards.cfg try: os.remove(BOARD_FILE) - except OSError as exception: + except OSError, exception: # Ignore 'No such file or directory' error if exception.errno != errno.ENOENT: raise
Tom,
I noticed this patch's gone to RFC.
Did you decide to drop python2.5 from support?
If so, I will use "except ... as ..." and "with ... as ..." statements in my other patches.
Best Regards Masahiro Yamada
On Mon, 4 Aug 2014 19:23:14 +0900 Masahiro Yamada yamada.m@jp.panasonic.com wrote:
The statement "exception ... as ..." can be used in Python 2.6 or lator. Avoid using it. Tested on Python 2.5.6.
Signed-off-by: Masahiro Yamada yamada.m@jp.panasonic.com
Changes in v2:
- Fix git-description. s/exception/except/
tools/genboardscfg.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/genboardscfg.py b/tools/genboardscfg.py index 734d90b..9e4892f 100755 --- a/tools/genboardscfg.py +++ b/tools/genboardscfg.py @@ -57,7 +57,7 @@ def get_terminal_columns(): arg = struct.pack('hhhh', 0, 0, 0, 0) try: ret = fcntl.ioctl(sys.stdout.fileno(), termios.TIOCGWINSZ, arg)
except IOError as exception:
except IOError, exception: if exception.errno != errno.ENOTTY: raise # If 'Inappropriate ioctl for device' error occurs,@@ -473,7 +473,7 @@ def gen_boards_cfg(jobs): # We should remove incomplete boards.cfg try: os.remove(BOARD_FILE)
except OSError as exception:
except OSError, exception: # Ignore 'No such file or directory' error if exception.errno != errno.ENOENT: raise-- 1.9.1
Tom,
A litte more info:
Buildman does not work on python 2.5. (only 2.6 and 2.7) No complaint about it so far.
This tools is intended to be a work-around for using MAKEALL and Buildman, but we are removing the former.
In that case, having only this tool support python 2.5 seems meaningless.
Best Regards Masahiro Yamada
On Fri, 22 Aug 2014 15:01:30 +0900 Masahiro Yamada yamada.m@jp.panasonic.com wrote:
Tom,
I noticed this patch's gone to RFC.
Did you decide to drop python2.5 from support?
If so, I will use "except ... as ..." and "with ... as ..." statements in my other patches.
Best Regards Masahiro Yamada
On Mon, 4 Aug 2014 19:23:14 +0900 Masahiro Yamada yamada.m@jp.panasonic.com wrote:
The statement "exception ... as ..." can be used in Python 2.6 or lator. Avoid using it. Tested on Python 2.5.6.
Signed-off-by: Masahiro Yamada yamada.m@jp.panasonic.com
Changes in v2:
- Fix git-description. s/exception/except/
tools/genboardscfg.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/genboardscfg.py b/tools/genboardscfg.py index 734d90b..9e4892f 100755 --- a/tools/genboardscfg.py +++ b/tools/genboardscfg.py @@ -57,7 +57,7 @@ def get_terminal_columns(): arg = struct.pack('hhhh', 0, 0, 0, 0) try: ret = fcntl.ioctl(sys.stdout.fileno(), termios.TIOCGWINSZ, arg)
except IOError as exception:
except IOError, exception: if exception.errno != errno.ENOTTY: raise # If 'Inappropriate ioctl for device' error occurs,@@ -473,7 +473,7 @@ def gen_boards_cfg(jobs): # We should remove incomplete boards.cfg try: os.remove(BOARD_FILE)
except OSError as exception:
except OSError, exception: # Ignore 'No such file or directory' error if exception.errno != errno.ENOENT: raise-- 1.9.1
U-Boot mailing list U-Boot@lists.denx.de http://lists.denx.de/mailman/listinfo/u-boot
On Fri, Aug 22, 2014 at 03:01:30PM +0900, Masahiro Yamada wrote:
Tom,
I noticed this patch's gone to RFC.
Did you decide to drop python2.5 from support?
If so, I will use "except ... as ..." and "with ... as ..." statements in my other patches.
So, I'm not 100% sure just yet how to handle this. I'm biased about supporting python2.6 since that's what my big compute resources are pinned to (until Ubuntu 10.04 LTS is no longer supported). And I'm making them switch to buildman too. It's also possible (and encouraged, relatively speaking, by the admins) to add required new software /over/there. So how hard is it to keep what MAKEALL/buildman needs (so genboardscfg.py) and maybe even convert buildman to python2.5-or-later ?
Hi Tom,
On Sat, 23 Aug 2014 08:44:49 -0400 Tom Rini trini@ti.com wrote:
On Fri, Aug 22, 2014 at 03:01:30PM +0900, Masahiro Yamada wrote:
Tom,
I noticed this patch's gone to RFC.
Did you decide to drop python2.5 from support?
If so, I will use "except ... as ..." and "with ... as ..." statements in my other patches.
So, I'm not 100% sure just yet how to handle this. I'm biased about supporting python2.6 since that's what my big compute resources are pinned to (until Ubuntu 10.04 LTS is no longer supported). And I'm making them switch to buildman too. It's also possible (and encouraged, relatively speaking, by the admins) to add required new software /over/there. So how hard is it to keep what MAKEALL/buildman needs (so genboardscfg.py) and maybe even convert buildman to python2.5-or-later ?
Unfortunately, python 2.5 is missing some important statements which are often used these days.
We can change "except ... as ..." to "except ... , ..." automatically, but we have to convert "with ... as ..." by hand.
I am not happy (and I guess Simon neither) about converting already working scripts.
OK. Let's support only python 2.6 or lator (but not 3.x) and see what will happen.
Perhaps someone may complain about it, but I think we can excuse here because they are no-core utilities.
Anyway, I think python 2.5 is too old, so this is an acceptable limition.
Best Regards Masahiro Yamada
Hi Masahiro,
On 4 August 2014 04:23, Masahiro Yamada yamada.m@jp.panasonic.com wrote:
It was reported by Miao Yan that the kconfig-related Python scripts are not working on Python 2.4.
Do we have any consensus in terms of Python version requirement?
This series avoids using "with ... as ..." and "except ... as ..." statements. But "with ... as ..." is used everywhere in buildman, which means buildman requires at least Python 2.6.
It is true we hope tools can work on broad range of versions, but it also means we have more restrictions when writing Python scripts.
I have no idea which version we should expect at least. Your comments are welcome.
Since it's reasonably easy to do for Kconfig it seems worth doing. For buildman I'm less sure, since it has been around for a year with no complaints. I supposse people can install a back-ported python 2.6?
Let's see what Tom thinks.
Regards, Simon
Hi Masahiro,
On 08/04/14 13:23, Masahiro Yamada wrote:
It was reported by Miao Yan that the kconfig-related Python scripts are not working on Python 2.4.
Do we have any consensus in terms of Python version requirement?
This series avoids using "with ... as ..." and "except ... as ..." statements. But "with ... as ..." is used everywhere in buildman, which means buildman requires at least Python 2.6.
Hmmmm.... Funny... a bit more fire:
-------------------cut---------------------------- $ make cm_t335_defconfig HOSTCC scripts/basic/fixdep GEN /home/grinberg/bin-temp/u-boot/Makefile File "/home/grinberg/git-repo/u-boot/scripts/multiconfig.py", line 344 print "*** Default configuration is based on '%s'" % KBUILD_DEFCONFIG ^ SyntaxError: invalid syntax make[1]: *** [cm_t335_defconfig] Error 1 make: *** [sub-make] Error 2
$ python --version Python 3.3.5 -------------------cut----------------------------
Apparently, print should be used as function - with parenthesis... It seems that those scripts only work on Python versions 2.6 - 2.7?
It is true we hope tools can work on broad range of versions, but it also means we have more restrictions when writing Python scripts.
I have no idea which version we should expect at least. Your comments are welcome.
I'd propose to not bring any new dependency on python at all... At least not for the simple task of building U-Boot... Can't we use a bit more stable API then this of python?
Changes in v2:
- Fix git-description. s/exception/except/
- Fix git-description. s/exception/except/
Masahiro Yamada (2): kconfig: make multiconfig.py compatible with Python 2.4 tools: make genboardscfg.py compatible with Python 2.5
scripts/multiconfig.py | 73 +++++++++++++++++++++++--------------------------- tools/genboardscfg.py | 4 +-- 2 files changed, 36 insertions(+), 41 deletions(-)
Hi Igor,
On Mon, 04 Aug 2014 14:21:32 +0300 Igor Grinberg grinberg@compulab.co.il wrote:
Hi Masahiro,
On 08/04/14 13:23, Masahiro Yamada wrote:
It was reported by Miao Yan that the kconfig-related Python scripts are not working on Python 2.4.
Do we have any consensus in terms of Python version requirement?
This series avoids using "with ... as ..." and "except ... as ..." statements. But "with ... as ..." is used everywhere in buildman, which means buildman requires at least Python 2.6.
Hmmmm.... Funny... a bit more fire:
-------------------cut---------------------------- $ make cm_t335_defconfig HOSTCC scripts/basic/fixdep GEN /home/grinberg/bin-temp/u-boot/Makefile File "/home/grinberg/git-repo/u-boot/scripts/multiconfig.py", line 344 print "*** Default configuration is based on '%s'" % KBUILD_DEFCONFIG ^ SyntaxError: invalid syntax make[1]: *** [cm_t335_defconfig] Error 1 make: *** [sub-make] Error 2
$ python --version Python 3.3.5 -------------------cut----------------------------
Sorry, the script does not work on 3.x
I think make PYTHON=python2 cm_t335_defconfig will work if Python 2.x exists as "python2" in your command path.
Apparently, print should be used as function - with parenthesis...
Yes, a famous difference between 2.x and 3.x "print" is a statement in 2.x but a function in 3.x.
At first, when I was writing tools/genboardscfg.py, I tried to make it compatible with both 2.x and 3.x.
But I found it was more difficult than I had expected because of the varous differences. Finially I gave up.
As for scripts/multiconfig.py, perhaps I can modify it for 3.x compatible.
It seems that those scripts only work on Python versions 2.6 - 2.7?
Yes.
It is true we hope tools can work on broad range of versions, but it also means we have more restrictions when writing Python scripts.
I have no idea which version we should expect at least. Your comments are welcome.
I'd propose to not bring any new dependency on python at all... At least not for the simple task of building U-Boot... Can't we use a bit more stable API then this of python?
Yeah, some people want to run it on older version such as 2.4.x and some people want to use Python 3.x.
Now I found it so difficult to satisfy everyone. Maybe I made a mistake in the first place.
Best Regards Masahiro Yamada
Hi.
It seems that those scripts only work on Python versions 2.6 - 2.7?
I took a quick look and I found Python 3.x is not comatible 2.x at all.
3.x requires the "print" is called like print(msg, file=sys.stderr) but it failes in Python 2.7.x.
Moreover, "except ... , ..." fails in 3.x while "except ... as ..." fails in 2.5 or earlier.
If the compatibility with python 3 is the requirement, I can't do this. I must throw Python scripts away.
Best Regards Masahiro Yamada
Hi Masahiro,
On 08/04/14 15:59, Masahiro Yamada wrote:
Hi.
It seems that those scripts only work on Python versions 2.6 - 2.7?
I took a quick look and I found Python 3.x is not comatible 2.x at all.
3.x requires the "print" is called like print(msg, file=sys.stderr) but it failes in Python 2.7.x.
Last time I checked the print function, it worked with parenthesis on both versions 2.7 and 3.x.
Moreover, "except ... , ..." fails in 3.x while "except ... as ..." fails in 2.5 or earlier.
If the compatibility with python 3 is the requirement, I can't do this. I must throw Python scripts away.
I think we'd better replace these with something more stable in terms of API... bash? perl?
Hi Igor,
On Mon, 04 Aug 2014 16:17:00 +0300 Igor Grinberg grinberg@compulab.co.il wrote:
Hi Masahiro,
On 08/04/14 15:59, Masahiro Yamada wrote:
Hi.
It seems that those scripts only work on Python versions 2.6 - 2.7?
I took a quick look and I found Python 3.x is not comatible 2.x at all.
3.x requires the "print" is called like print(msg, file=sys.stderr) but it failes in Python 2.7.x.
Last time I checked the print function, it worked with parenthesis on both versions 2.7 and 3.x.
I think
print("helloworld")
works on both 2.x and 3.x.
On 2.x, ( ) is just meaningless parantheses and simply omitted. On 3.x, ( ) is mandatory for function call.
print("helloworld", file=sys.stderr)
never works on 2.x because the print statement does not take named arguments
Best Masahiro Yamada
On Mon, Aug 04, 2014 at 04:17:00PM +0300, Igor Grinberg wrote:
Hi Masahiro,
On 08/04/14 15:59, Masahiro Yamada wrote:
Hi.
It seems that those scripts only work on Python versions 2.6 - 2.7?
I took a quick look and I found Python 3.x is not comatible 2.x at all.
3.x requires the "print" is called like print(msg, file=sys.stderr) but it failes in Python 2.7.x.
Last time I checked the print function, it worked with parenthesis on both versions 2.7 and 3.x.
Moreover, "except ... , ..." fails in 3.x while "except ... as ..." fails in 2.5 or earlier.
If the compatibility with python 3 is the requirement, I can't do this. I must throw Python scripts away.
I think we'd better replace these with something more stable in terms of API... bash? perl?
I don't think API is an argument against python, we just need /usr/bin/env python2 as how we invoke our scripts.
The question is, what helper scripts do we really need to have around and expect many people to use.
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 08/07/14 13:57, Tom Rini wrote:
On Mon, Aug 04, 2014 at 04:17:00PM +0300, Igor Grinberg wrote:
Hi Masahiro,
On 08/04/14 15:59, Masahiro Yamada wrote:
Hi.
It seems that those scripts only work on Python versions 2.6 - 2.7?
I took a quick look and I found Python 3.x is not comatible 2.x at all.
3.x requires the "print" is called like print(msg, file=sys.stderr) but it failes in Python 2.7.x.
Last time I checked the print function, it worked with parenthesis on both versions 2.7 and 3.x.
Moreover, "except ... , ..." fails in 3.x while "except ... as ..." fails in 2.5 or earlier.
If the compatibility with python 3 is the requirement, I can't do this. I must throw Python scripts away.
I think we'd better replace these with something more stable in terms of API... bash? perl?
I don't think API is an argument against python,
Not the API as stand alone, of course, but the burden to support its changes and breakages. Why should U-Boot even have python as a build prerequisite? I really think that having shell, make, and $(cross)gcc should be enough for the basic source build.
we just need /usr/bin/env python2 as how we invoke our scripts.
This means impose python version dependency for U-Boot source build? Correct me if you think I'm wrong, but I don't think this is a good practice... I think that for tools like buildman, patman, etc. - this is perfectly fine to impose an interpreter/compiler version, but not for the basic source builds.
The question is, what helper scripts do we really need to have around and expect many people to use.
IMO, helper scripts, that are not involved in U-Boot source build can be written in any language/script.
- -- Regards, Igor.
On Thu, Aug 07, 2014 at 04:17:21PM +0300, Igor Grinberg wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 08/07/14 13:57, Tom Rini wrote:
On Mon, Aug 04, 2014 at 04:17:00PM +0300, Igor Grinberg wrote:
Hi Masahiro,
On 08/04/14 15:59, Masahiro Yamada wrote:
Hi.
It seems that those scripts only work on Python versions 2.6 - 2.7?
I took a quick look and I found Python 3.x is not comatible 2.x at all.
3.x requires the "print" is called like print(msg, file=sys.stderr) but it failes in Python 2.7.x.
Last time I checked the print function, it worked with parenthesis on both versions 2.7 and 3.x.
Moreover, "except ... , ..." fails in 3.x while "except ... as ..." fails in 2.5 or earlier.
If the compatibility with python 3 is the requirement, I can't do this. I must throw Python scripts away.
I think we'd better replace these with something more stable in terms of API... bash? perl?
I don't think API is an argument against python,
Not the API as stand alone, of course, but the burden to support its changes and breakages. Why should U-Boot even have python as a build prerequisite? I really think that having shell, make, and $(cross)gcc should be enough for the basic source build.
we just need /usr/bin/env python2 as how we invoke our scripts.
This means impose python version dependency for U-Boot source build? Correct me if you think I'm wrong, but I don't think this is a good practice... I think that for tools like buildman, patman, etc. - this is perfectly fine to impose an interpreter/compiler version, but not for the basic source builds.
I agree. You don't need MAKEALL or buildman to do basic source builds. Doing 'make foo_defconfig' doesn't require re-creating boards.cfg.
To me, the gray area is people doing SoC level (or higher) changes that want to be good and test more areas. That's when MAKEALL or buildman become handy and some sort of win over a shell forloop.
On 08/07/2014 10:57 AM, Tom Rini wrote:
On Thu, Aug 07, 2014 at 04:17:21PM +0300, Igor Grinberg wrote:
On 08/07/14 13:57, Tom Rini wrote:
..
we just need /usr/bin/env python2 as how we invoke our scripts.
This means impose python version dependency for U-Boot source build? Correct me if you think I'm wrong, but I don't think this is a good practice... I think that for tools like buildman, patman, etc. - this is perfectly fine to impose an interpreter/compiler version, but not for the basic source builds.
I agree. You don't need MAKEALL or buildman to do basic source builds. Doing 'make foo_defconfig' doesn't require re-creating boards.cfg.
To me, the gray area is people doing SoC level (or higher) changes that want to be good and test more areas. That's when MAKEALL or buildman become handy and some sort of win over a shell forloop.
Why on earth isn't relying specifically on either Python2 (with the current script code) or Python3 (after porting the code) a good practice?
There was a major release of the language, and the language changed. Such a change is just like any other scripting language. Scripts written for one particular version of the language should specify python2 or python3 not just python in their #! line. Problem solved.
For now at least, all distros I know of provide both python2 and python3, so there's no issue doing this. In N years when there's only python3 available, I assumed the scripts would be updated, in the same way we use new C features (e.g. bool) and drop support for older tools of other kinds.
Banning or replacing the use of Python just because they cleaned up their language seems like poking your eye out to spite your nose (or whatever the expression is). The same thing will happen with Perl, and happened with dtc, etc.
On Thu, Aug 07, 2014 at 11:33:35AM -0600, Stephen Warren wrote:
On 08/07/2014 10:57 AM, Tom Rini wrote:
On Thu, Aug 07, 2014 at 04:17:21PM +0300, Igor Grinberg wrote:
On 08/07/14 13:57, Tom Rini wrote:
..
we just need /usr/bin/env python2 as how we invoke our scripts.
This means impose python version dependency for U-Boot source build? Correct me if you think I'm wrong, but I don't think this is a good practice... I think that for tools like buildman, patman, etc. - this is perfectly fine to impose an interpreter/compiler version, but not for the basic source builds.
I agree. You don't need MAKEALL or buildman to do basic source builds. Doing 'make foo_defconfig' doesn't require re-creating boards.cfg.
To me, the gray area is people doing SoC level (or higher) changes that want to be good and test more areas. That's when MAKEALL or buildman become handy and some sort of win over a shell forloop.
Why on earth isn't relying specifically on either Python2 (with the current script code) or Python3 (after porting the code) a good practice?
We can and should (and will) rely on python2 (or python3, but probably 2 due to RHEL/CentOS5/Ubuntu 10.04) to fix the first problem here that cropped up, of /usr/bin/env python being not the best idea.
On 08/07/14 20:33, Stephen Warren wrote:
On 08/07/2014 10:57 AM, Tom Rini wrote:
On Thu, Aug 07, 2014 at 04:17:21PM +0300, Igor Grinberg wrote:
On 08/07/14 13:57, Tom Rini wrote:
..
we just need /usr/bin/env python2 as how we invoke our scripts.
This means impose python version dependency for U-Boot source build? Correct me if you think I'm wrong, but I don't think this is a good practice... I think that for tools like buildman, patman, etc. - this is perfectly fine to impose an interpreter/compiler version, but not for the basic source builds.
I agree. You don't need MAKEALL or buildman to do basic source builds. Doing 'make foo_defconfig' doesn't require re-creating boards.cfg.
To me, the gray area is people doing SoC level (or higher) changes that want to be good and test more areas. That's when MAKEALL or buildman become handy and some sort of win over a shell forloop.
Why on earth isn't relying specifically on either Python2 (with the current script code) or Python3 (after porting the code) a good practice?
Because I think (I can think this way, right?) it is not a good practice to bring another host machine dependency (moreover, version dependency) for the simple source code build (now it also backfired in OE).
Banning or replacing the use of Python just because they cleaned up their language seems like poking your eye out to spite your nose (or whatever the expression is). The same thing will happen with Perl, and happened with dtc, etc.
Did I say ban python or something? No, I did not say that. What I'm saying is: Right now, we have compiler dependency (a must as you can't practically produce any code without it), and we have dtc (a must if you want to compile dts), and we have make, and we have shell (this one is found on every host, although windows users have to use cygwin or such, but who cares, so no problem), and now we also add python to the soup?
Don't get me wrong, I don't have any problem with python... I can have any number of python versions I want (I think gentoo is the best at supporting this kind of stuff...). I just don't think that basic code compile should depend on even more stuff being added. I've compiled various bootloaders and have seen huge dependencies on tools and after all the build time and complexity got worth and worth. I don't really want U-Boot to go that direction, but more to keep it simple and stupid.
On Sun, Aug 10, 2014 at 11:49:12AM +0300, Igor Grinberg wrote:
On 08/07/14 20:33, Stephen Warren wrote:
On 08/07/2014 10:57 AM, Tom Rini wrote:
On Thu, Aug 07, 2014 at 04:17:21PM +0300, Igor Grinberg wrote:
On 08/07/14 13:57, Tom Rini wrote:
..
we just need /usr/bin/env python2 as how we invoke our scripts.
This means impose python version dependency for U-Boot source build? Correct me if you think I'm wrong, but I don't think this is a good practice... I think that for tools like buildman, patman, etc. - this is perfectly fine to impose an interpreter/compiler version, but not for the basic source builds.
I agree. You don't need MAKEALL or buildman to do basic source builds. Doing 'make foo_defconfig' doesn't require re-creating boards.cfg.
To me, the gray area is people doing SoC level (or higher) changes that want to be good and test more areas. That's when MAKEALL or buildman become handy and some sort of win over a shell forloop.
Why on earth isn't relying specifically on either Python2 (with the current script code) or Python3 (after porting the code) a good practice?
Because I think (I can think this way, right?) it is not a good practice to bring another host machine dependency (moreover, version dependency) for the simple source code build (now it also backfired in OE).
Banning or replacing the use of Python just because they cleaned up their language seems like poking your eye out to spite your nose (or whatever the expression is). The same thing will happen with Perl, and happened with dtc, etc.
Did I say ban python or something? No, I did not say that. What I'm saying is: Right now, we have compiler dependency (a must as you can't practically produce any code without it), and we have dtc (a must if you want to compile dts), and we have make, and we have shell (this one is found on every host, although windows users have to use cygwin or such, but who cares, so no problem), and now we also add python to the soup?
Maybe I'm being thick then. What's the use case you need MAKEALL or buildman for?
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 08/10/14 14:14, Tom Rini wrote:
On Sun, Aug 10, 2014 at 11:49:12AM +0300, Igor Grinberg wrote:
On 08/07/14 20:33, Stephen Warren wrote:
On 08/07/2014 10:57 AM, Tom Rini wrote:
On Thu, Aug 07, 2014 at 04:17:21PM +0300, Igor Grinberg wrote:
On 08/07/14 13:57, Tom Rini wrote:
..
we just need /usr/bin/env python2 as how we invoke our scripts.
This means impose python version dependency for U-Boot source build? Correct me if you think I'm wrong, but I don't think this is a good practice... I think that for tools like buildman, patman, etc. - this is perfectly fine to impose an interpreter/compiler version, but not for the basic source builds.
I agree. You don't need MAKEALL or buildman to do basic source builds. Doing 'make foo_defconfig' doesn't require re-creating boards.cfg.
To me, the gray area is people doing SoC level (or higher) changes that want to be good and test more areas. That's when MAKEALL or buildman become handy and some sort of win over a shell forloop.
Why on earth isn't relying specifically on either Python2 (with the current script code) or Python3 (after porting the code) a good practice?
Because I think (I can think this way, right?) it is not a good practice to bring another host machine dependency (moreover, version dependency) for the simple source code build (now it also backfired in OE).
Banning or replacing the use of Python just because they cleaned up their language seems like poking your eye out to spite your nose (or whatever the expression is). The same thing will happen with Perl, and happened with dtc, etc.
Did I say ban python or something? No, I did not say that. What I'm saying is: Right now, we have compiler dependency (a must as you can't practically produce any code without it), and we have dtc (a must if you want to compile dts), and we have make, and we have shell (this one is found on every host, although windows users have to use cygwin or such, but who cares, so no problem), and now we also add python to the soup?
Maybe I'm being thick then. What's the use case you need MAKEALL or buildman for?
Now, I feel like we are talking about different things... I'll try to be a bit more clear on this:
Simple use case: build U-Boot for a board: ================================ $ make mrproper && make cm_t335_config && make -j12 > /dev/null CLEAN examples/standalone CLEAN tools CLEAN u-boot.lds CLEAN spl/arch spl/board spl/common spl/disk spl/drivers spl/fs spl/lib spl/spl spl/u-boot-spl spl/u-boot-spl.bin spl/u-boot-spl.lds spl/u-boot-spl.map CLEAN u-boot.map u-boot.bin u-boot.srec u-boot u-boot.img MLO System.map CLEAN include/generated spl CLEAN include/autoconf.mk include/autoconf.mk.dep include/config.h etags HOSTCC scripts/basic/fixdep GEN /home/grinberg/bin-temp/u-boot/Makefile File "/home/grinberg/git-repo/u-boot/scripts/multiconfig.py", line 344 print "*** Default configuration is based on '%s'" % KBUILD_DEFCONFIG ^ SyntaxError: invalid syntax make[1]: *** [cm_t335_config] Error 1 make: *** [sub-make] Error 2 =======================================
The build failed and I need to either use python 2.7 or patch the multiconfig.py file. The error message is really not important as for different system settings (python version) it will differ. So the above means that for a simple source code build for single board, one needs to cope with a new dependency... and that is something I personally dislike.
Another use case: I'm changing common code and I want to send patches, so I need to make sure I did not break anything. I run MAKEALL or use buildman to cover relevant/all architectures and boards. That is perfectly fine with me to use any python dependency as in this case I use developer tools for achieving the goal.
- -- Regards, Igor.
On Mon, Aug 11, 2014 at 01:18:40PM +0300, Igor Grinberg wrote:
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1
On 08/10/14 14:14, Tom Rini wrote:
On Sun, Aug 10, 2014 at 11:49:12AM +0300, Igor Grinberg wrote:
On 08/07/14 20:33, Stephen Warren wrote:
On 08/07/2014 10:57 AM, Tom Rini wrote:
On Thu, Aug 07, 2014 at 04:17:21PM +0300, Igor Grinberg wrote:
On 08/07/14 13:57, Tom Rini wrote:
..
> we just need > /usr/bin/env python2 as how we invoke our scripts.
This means impose python version dependency for U-Boot source build? Correct me if you think I'm wrong, but I don't think this is a good practice... I think that for tools like buildman, patman, etc. - this is perfectly fine to impose an interpreter/compiler version, but not for the basic source builds.
I agree. You don't need MAKEALL or buildman to do basic source builds. Doing 'make foo_defconfig' doesn't require re-creating boards.cfg.
To me, the gray area is people doing SoC level (or higher) changes that want to be good and test more areas. That's when MAKEALL or buildman become handy and some sort of win over a shell forloop.
Why on earth isn't relying specifically on either Python2 (with the current script code) or Python3 (after porting the code) a good practice?
Because I think (I can think this way, right?) it is not a good practice to bring another host machine dependency (moreover, version dependency) for the simple source code build (now it also backfired in OE).
Banning or replacing the use of Python just because they cleaned up their language seems like poking your eye out to spite your nose (or whatever the expression is). The same thing will happen with Perl, and happened with dtc, etc.
Did I say ban python or something? No, I did not say that. What I'm saying is: Right now, we have compiler dependency (a must as you can't practically produce any code without it), and we have dtc (a must if you want to compile dts), and we have make, and we have shell (this one is found on every host, although windows users have to use cygwin or such, but who cares, so no problem), and now we also add python to the soup?
Maybe I'm being thick then. What's the use case you need MAKEALL or buildman for?
Now, I feel like we are talking about different things... I'll try to be a bit more clear on this:
Simple use case: build U-Boot for a board:
$ make mrproper && make cm_t335_config && make -j12 > /dev/null CLEAN examples/standalone CLEAN tools CLEAN u-boot.lds CLEAN spl/arch spl/board spl/common spl/disk spl/drivers spl/fs spl/lib spl/spl spl/u-boot-spl spl/u-boot-spl.bin spl/u-boot-spl.lds spl/u-boot-spl.map CLEAN u-boot.map u-boot.bin u-boot.srec u-boot u-boot.img MLO System.map CLEAN include/generated spl CLEAN include/autoconf.mk include/autoconf.mk.dep include/config.h etags HOSTCC scripts/basic/fixdep GEN /home/grinberg/bin-temp/u-boot/Makefile File "/home/grinberg/git-repo/u-boot/scripts/multiconfig.py", line 344 print "*** Default configuration is based on '%s'" % KBUILD_DEFCONFIG
Ug, crap, OK, I had missed that one. Masahiro, how hard would it be to turn multiconfig.py into a bash script? Or even handle it within make?
Hi Tom.
2014-08-11 22:12 GMT+09:00 Tom Rini trini@ti.com:
Ug, crap, OK, I had missed that one. Masahiro, how hard would it be to turn multiconfig.py into a bash script? Or even handle it within make?
I will try a shell script. (Perhaps some parts are too complicated for a shell script, but I will take a look anyway.)
How urgent is this issue? Must it be fixed until -rc2 ?
On Tue, Aug 12, 2014 at 08:03:28AM +0900, Masahiro YAMADA wrote:
Hi Tom.
2014-08-11 22:12 GMT+09:00 Tom Rini trini@ti.com:
Ug, crap, OK, I had missed that one. Masahiro, how hard would it be to turn multiconfig.py into a bash script? Or even handle it within make?
I will try a shell script. (Perhaps some parts are too complicated for a shell script, but I will take a look anyway.)
How urgent is this issue? Must it be fixed until -rc2 ?
I'd really like to know where we stand on this at least by -rc2.
Hi.
On Mon, 11 Aug 2014 21:53:47 -0400 Tom Rini trini@ti.com wrote:
On Tue, Aug 12, 2014 at 08:03:28AM +0900, Masahiro YAMADA wrote:
Hi Tom.
2014-08-11 22:12 GMT+09:00 Tom Rini trini@ti.com:
Ug, crap, OK, I had missed that one. Masahiro, how hard would it be to turn multiconfig.py into a bash script? Or even handle it within make?
I will try a shell script. (Perhaps some parts are too complicated for a shell script, but I will take a look anyway.)
How urgent is this issue? Must it be fixed until -rc2 ?
I'd really like to know where we stand on this at least by -rc2.
-- Tom
The patch is on Patchwork now.
http://patchwork.ozlabs.org/patch/380553/
I did not use bash-extension because I wanted to keep it sh-compatible.
I tested it with dash on Ubuntu.
Beforehand, let's clear up a trivial fix and a document: http://patchwork.ozlabs.org/patch/380316/ http://patchwork.ozlabs.org/patch/380310/
Best Regards Masahiro Yamada
On 08/10/2014 02:49 AM, Igor Grinberg wrote:
On 08/07/14 20:33, Stephen Warren wrote:
On 08/07/2014 10:57 AM, Tom Rini wrote:
On Thu, Aug 07, 2014 at 04:17:21PM +0300, Igor Grinberg wrote:
On 08/07/14 13:57, Tom Rini wrote:
..
we just need /usr/bin/env python2 as how we invoke our scripts.
This means impose python version dependency for U-Boot source build? Correct me if you think I'm wrong, but I don't think this is a good practice... I think that for tools like buildman, patman, etc. - this is perfectly fine to impose an interpreter/compiler version, but not for the basic source builds.
I agree. You don't need MAKEALL or buildman to do basic source builds. Doing 'make foo_defconfig' doesn't require re-creating boards.cfg.
To me, the gray area is people doing SoC level (or higher) changes that want to be good and test more areas. That's when MAKEALL or buildman become handy and some sort of win over a shell forloop.
Why on earth isn't relying specifically on either Python2 (with the current script code) or Python3 (after porting the code) a good practice?
Because I think (I can think this way, right?) it is not a good practice to bring another host machine dependency (moreover, version dependency) for the simple source code build (now it also backfired in OE).
Banning or replacing the use of Python just because they cleaned up their language seems like poking your eye out to spite your nose (or whatever the expression is). The same thing will happen with Perl, and happened with dtc, etc.
Did I say ban python or something? No, I did not say that.
I only see two alternatives:
a) We allow Python to be used to implement parts of the U-Boot build process and/or associated utility scripts. This introduces a dependency on Python, and such a dependency has to be on a specific (major) version of Python because the language in Python2 and Python3 is a bit different.
b) We avoid introducing a dependency on Python by disallowing parts of the build process/scripts to be implemented in Python.
What I'm saying is: Right now, we have compiler dependency (a must as you can't practically produce any code without it), and we have dtc (a must if you want to compile dts), and we have make, and we have shell (this one is found on every host, although windows users have to use cygwin or such, but who cares, so no problem), and now we also add python to the soup?
Don't get me wrong, I don't have any problem with python... I can have any number of python versions I want (I think gentoo is the best at supporting this kind of stuff...). I just don't think that basic code compile should depend on even more stuff being added. I've compiled various bootloaders and have seen huge dependencies on tools and after all the build time and complexity got worth and worth. I don't really want U-Boot to go that direction, but more to keep it simple and stupid.
To me, that sounds *exactly* like you're saying ban Python (at least as part of the core build process).
On 08/11/14 19:58, Stephen Warren wrote:
On 08/10/2014 02:49 AM, Igor Grinberg wrote:
On 08/07/14 20:33, Stephen Warren wrote:
On 08/07/2014 10:57 AM, Tom Rini wrote:
On Thu, Aug 07, 2014 at 04:17:21PM +0300, Igor Grinberg wrote:
On 08/07/14 13:57, Tom Rini wrote:
..
we just need /usr/bin/env python2 as how we invoke our scripts.
This means impose python version dependency for U-Boot source build? Correct me if you think I'm wrong, but I don't think this is a good practice... I think that for tools like buildman, patman, etc. - this is perfectly fine to impose an interpreter/compiler version, but not for the basic source builds.
I agree. You don't need MAKEALL or buildman to do basic source builds. Doing 'make foo_defconfig' doesn't require re-creating boards.cfg.
To me, the gray area is people doing SoC level (or higher) changes that want to be good and test more areas. That's when MAKEALL or buildman become handy and some sort of win over a shell forloop.
Why on earth isn't relying specifically on either Python2 (with the current script code) or Python3 (after porting the code) a good practice?
Because I think (I can think this way, right?) it is not a good practice to bring another host machine dependency (moreover, version dependency) for the simple source code build (now it also backfired in OE).
Banning or replacing the use of Python just because they cleaned up their language seems like poking your eye out to spite your nose (or whatever the expression is). The same thing will happen with Perl, and happened with dtc, etc.
Did I say ban python or something? No, I did not say that.
I only see two alternatives:
a) We allow Python to be used to implement parts of the U-Boot build process and/or associated utility scripts. This introduces a dependency on Python, and such a dependency has to be on a specific (major) version of Python because the language in Python2 and Python3 is a bit different.
I think we can also handle multiple versions support, it will require more thinking and work though...
b) We avoid introducing a dependency on Python by disallowing parts of the build process/scripts to be implemented in Python.
Why be so strict in choosing the way? It sounds like we have to choose yes or no. I would separate the cases and thus let developers use any dependency they like in helper tools/scripts, yet keeping the core build sustained.
What I'm saying is: Right now, we have compiler dependency (a must as you can't practically produce any code without it), and we have dtc (a must if you want to compile dts), and we have make, and we have shell (this one is found on every host, although windows users have to use cygwin or such, but who cares, so no problem), and now we also add python to the soup?
Don't get me wrong, I don't have any problem with python... I can have any number of python versions I want (I think gentoo is the best at supporting this kind of stuff...). I just don't think that basic code compile should depend on even more stuff being added. I've compiled various bootloaders and have seen huge dependencies on tools and after all the build time and complexity got worth and worth. I don't really want U-Boot to go that direction, but more to keep it simple and stupid.
To me, that sounds *exactly* like you're saying ban Python (at least as part of the core build process).
Not exactly, but I do want to separate the cases here (at least for now), the core build process and the utility scripting.
For the core build process: currently, I don't think python dependency is needed... It is only the multiconfig.py which was not here until recently. Linux, does not have the python dependency for the core builds... Does Linux core build simpler than U-Boot core build? Well, of course this is unrelated, but just to get a bit of comparison.
Nevertheless, if we are about to use python in the core build process, the decision must be done in a educated manner and documented appropriately as a dependency (and may be including some documentation or links to description on how to setup the correct version) and not the way it was done, by just sneaking it in (Masahiro, I'm not accusing you, please don't get offended, you've done a great job!).
For the utility scripting: it might be more convenient, and add a lot of power to the scripts, and developers will be very happy to write python stuff, so no problem with that at all. And I think, if that is the case also for java, then I have no problem with that also and will use java... What, I'm saying is that I don't have any particular/personal problem with python.
On Mon, Aug 04, 2014 at 07:23:12PM +0900, Masahiro Yamada wrote:
It was reported by Miao Yan that the kconfig-related Python scripts are not working on Python 2.4.
OK, what host distribution are you using? I'm going to guess RHEL5 (or compatible)... If so, yes, I think we need to support that as a host platform so long as it's still supported by RedHat.
OK, what host distribution are you using? I'm going to guess RHEL5 (or compatible)... If so, yes, I think we need to support that as a host platform so long as it's still supported by RedHat.
It's CentOS 5, which still has several years of life time :-(
Thanks, Miao
On Tue, Aug 05, 2014 at 02:05:20AM +0000, Yan, Miao wrote:
OK, what host distribution are you using? I'm going to guess RHEL5 (or compatible)... If so, yes, I think we need to support that as a host platform so long as it's still supported by RedHat.
It's CentOS 5, which still has several years of life time :-(
That's what I figured, so yes, we need to work there.
participants (7)
-
Igor Grinberg -
Masahiro YAMADA -
Masahiro Yamada
-
Simon Glass -
Stephen Warren -
Tom Rini -
Yan, Miao