[U-Boot] [PATCH v2 00/10] Python 3.x support for patman & dtoc

This series makes patman & dtoc run on python 3.x, which allows them to work without hacks on distributions where the python binary is python 3.x. All changes are made in a way that continue to work on python 2.x, so both tools should work on either major version of python.
Paul Burton (10): patman: Replace tabs with spaces patman: Make print statements python 3.x safe patman: Make exception handling python 3.x safe patman: Import 'configparser' lower case to be python 3.x safe patman: Decode stdout/stderr as utf8, be python 3.x safe patman: Use items() to iterate over dictionaries patman: Fix doctest StringIO import for python 3.x dtoc: Use items() to iterate over dictionaries in python 3.x dtoc: Decode strings for struct.unpack on python 3.x dtoc: Make integer division python 3.x safe
tools/dtoc/dtoc.py | 10 +++++----- tools/dtoc/fdt_fallback.py | 2 +- tools/dtoc/fdt_util.py | 3 +++ tools/patman/checkpatch.py | 16 ++++++++-------- tools/patman/command.py | 2 +- tools/patman/cros_subprocess.py | 6 +++--- tools/patman/get_maintainer.py | 2 +- tools/patman/gitutil.py | 18 +++++++++--------- tools/patman/patchstream.py | 6 +++--- tools/patman/patman.py | 12 ++++++------ tools/patman/series.py | 42 +++++++++++++++++++++-------------------- tools/patman/settings.py | 37 ++++++++++++++++++++++-------------- tools/patman/terminal.py | 12 +++++++----- tools/patman/test.py | 2 +- 14 files changed, 93 insertions(+), 77 deletions(-)

In preparation for running on python 3.x, which will refuse to run scripts which mix tabs & spaces for indentation, replace 2 tab characters present in series.py with spaces.
Signed-off-by: Paul Burton paul.burton@imgtec.com Acked-by: Simon Glass sjg@chromium.org ---
Changes in v2: None
tools/patman/series.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/patman/series.py b/tools/patman/series.py index cc6f80b..1ce30c6 100644 --- a/tools/patman/series.py +++ b/tools/patman/series.py @@ -225,7 +225,7 @@ class Series(dict): raise_on_error=raise_on_error) list += gitutil.BuildEmailList(commit.cc_list, raise_on_error=raise_on_error) - if add_maintainers: + if add_maintainers: list += get_maintainer.GetMaintainer(commit.patch) all_ccs += list print >>fd, commit.patch, ', '.join(set(list)) @@ -259,7 +259,7 @@ class Series(dict): """ git_prefix = gitutil.GetDefaultSubjectPrefix() if git_prefix: - git_prefix = '%s][' % git_prefix + git_prefix = '%s][' % git_prefix else: git_prefix = ''

On 27 September 2016 at 09:03, Paul Burton paul.burton@imgtec.com wrote:
In preparation for running on python 3.x, which will refuse to run scripts which mix tabs & spaces for indentation, replace 2 tab characters present in series.py with spaces.
Signed-off-by: Paul Burton paul.burton@imgtec.com Acked-by: Simon Glass sjg@chromium.org
Changes in v2: None
tools/patman/series.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
Applied to u-boot-dm, thanks!

In python 3.x, print must be used as a function call. Convert all print statements to the function call style, importing from __future__ where we print with no trailing newline or print to a file object.
Signed-off-by: Paul Burton paul.burton@imgtec.com Acked-by: Simon Glass sjg@chromium.org ---
Changes in v2: None
tools/patman/checkpatch.py | 16 ++++++++-------- tools/patman/get_maintainer.py | 2 +- tools/patman/gitutil.py | 6 +++--- tools/patman/patchstream.py | 6 +++--- tools/patman/patman.py | 12 ++++++------ tools/patman/series.py | 38 ++++++++++++++++++++------------------ tools/patman/settings.py | 16 +++++++++------- tools/patman/terminal.py | 12 +++++++----- tools/patman/test.py | 2 +- 9 files changed, 58 insertions(+), 52 deletions(-)
diff --git a/tools/patman/checkpatch.py b/tools/patman/checkpatch.py index 3eef6de..be78fc5 100644 --- a/tools/patman/checkpatch.py +++ b/tools/patman/checkpatch.py @@ -83,7 +83,7 @@ def CheckPatch(fname, verbose=False):
for line in result.stdout.splitlines(): if verbose: - print line + print(line)
# A blank line indicates the end of a message if not line and item: @@ -151,17 +151,17 @@ def CheckPatches(verbose, args): error_count += result.errors warning_count += result.warnings check_count += result.checks - print '%d errors, %d warnings, %d checks for %s:' % (result.errors, - result.warnings, result.checks, col.Color(col.BLUE, fname)) + print('%d errors, %d warnings, %d checks for %s:' % (result.errors, + result.warnings, result.checks, col.Color(col.BLUE, fname))) if (len(result.problems) != result.errors + result.warnings + result.checks): - print "Internal error: some problems lost" + print("Internal error: some problems lost") for item in result.problems: - print GetWarningMsg(col, item.get('type', '<unknown>'), + print(GetWarningMsg(col, item.get('type', '<unknown>'), item.get('file', '<unknown>'), - item.get('line', 0), item.get('msg', 'message')) + item.get('line', 0), item.get('msg', 'message'))) print - #print stdout + #print(stdout) if error_count or warning_count or check_count: str = 'checkpatch.pl found %d error(s), %d warning(s), %d checks(s)' color = col.GREEN @@ -169,6 +169,6 @@ def CheckPatches(verbose, args): color = col.YELLOW if error_count: color = col.RED - print col.Color(color, str % (error_count, warning_count, check_count)) + print(col.Color(color, str % (error_count, warning_count, check_count))) return False return True diff --git a/tools/patman/get_maintainer.py b/tools/patman/get_maintainer.py index 00b4939..2deb5db 100644 --- a/tools/patman/get_maintainer.py +++ b/tools/patman/get_maintainer.py @@ -40,7 +40,7 @@ def GetMaintainer(fname, verbose=False): get_maintainer = FindGetMaintainer() if not get_maintainer: if verbose: - print "WARNING: Couldn't find get_maintainer.pl" + print("WARNING: Couldn't find get_maintainer.pl") return []
stdout = command.Output(get_maintainer, '--norolestats', fname) diff --git a/tools/patman/gitutil.py b/tools/patman/gitutil.py index bb7c9e0..c0fe093 100644 --- a/tools/patman/gitutil.py +++ b/tools/patman/gitutil.py @@ -491,7 +491,7 @@ def LookupEmail(lookup_name, alias=None, raise_on_error=True, level=0): if raise_on_error: raise OSError, msg else: - print col.Color(col.RED, msg) + print(col.Color(col.RED, msg)) return out_list
if lookup_name: @@ -500,7 +500,7 @@ def LookupEmail(lookup_name, alias=None, raise_on_error=True, level=0): if raise_on_error: raise ValueError, msg else: - print col.Color(col.RED, msg) + print(col.Color(col.RED, msg)) return out_list for item in alias[lookup_name]: todo = LookupEmail(item, alias, raise_on_error, level + 1) @@ -508,7 +508,7 @@ def LookupEmail(lookup_name, alias=None, raise_on_error=True, level=0): if not new_item in out_list: out_list.append(new_item)
- #print "No match for alias '%s'" % lookup_name + #print("No match for alias '%s'" % lookup_name) return out_list
def GetTopLevel(): diff --git a/tools/patman/patchstream.py b/tools/patman/patchstream.py index 69d5cfb..cd4667f 100644 --- a/tools/patman/patchstream.py +++ b/tools/patman/patchstream.py @@ -480,12 +480,12 @@ def FixPatches(series, fnames): commit.patch = fname result = FixPatch(backup_dir, fname, series, commit) if result: - print '%d warnings for %s:' % (len(result), fname) + print('%d warnings for %s:' % (len(result), fname)) for warn in result: - print '\t', warn + print('\t', warn) print count += 1 - print 'Cleaned %d patches' % count + print('Cleaned %d patches' % count) return series
def InsertCoverLetter(fname, series, count): diff --git a/tools/patman/patman.py b/tools/patman/patman.py index fe50eb4..fdbee67 100755 --- a/tools/patman/patman.py +++ b/tools/patman/patman.py @@ -93,11 +93,11 @@ elif options.test: suite.run(result)
# TODO: Surely we can just 'print' result? - print result + print(result) for test, err in result.errors: - print err + print(err) for test, err in result.failures: - print err + print(err)
# Called from git with a patch filename as argument # Printout a list of additional CC recipients for this patch @@ -110,7 +110,7 @@ elif options.cc_cmd: for cc in match.group(2).split(', '): cc = cc.strip() if cc: - print cc + print(cc) fd.close()
elif options.full_help: @@ -166,12 +166,12 @@ else: options.dry_run, not options.ignore_bad_tags, cc_file, in_reply_to=options.in_reply_to, thread=options.thread) else: - print col.Color(col.RED, "Not sending emails due to errors/warnings") + print(col.Color(col.RED, "Not sending emails due to errors/warnings"))
# For a dry run, just show our actions as a sanity check if options.dry_run: series.ShowActions(args, cmd, options.process_tags) if not its_a_go: - print col.Color(col.RED, "Email would not be sent") + print(col.Color(col.RED, "Email would not be sent"))
os.remove(cc_file) diff --git a/tools/patman/series.py b/tools/patman/series.py index 1ce30c6..38a452e 100644 --- a/tools/patman/series.py +++ b/tools/patman/series.py @@ -3,6 +3,8 @@ # SPDX-License-Identifier: GPL-2.0+ #
+from __future__ import print_function + import itertools import os
@@ -101,38 +103,38 @@ class Series(dict): cc_set = set(gitutil.BuildEmailList(self.cc));
col = terminal.Color() - print 'Dry run, so not doing much. But I would do this:' - print - print 'Send a total of %d patch%s with %scover letter.' % ( + print('Dry run, so not doing much. But I would do this:') + print() + print('Send a total of %d patch%s with %scover letter.' % ( len(args), '' if len(args) == 1 else 'es', - self.get('cover') and 'a ' or 'no ') + self.get('cover') and 'a ' or 'no '))
# TODO: Colour the patches according to whether they passed checks for upto in range(len(args)): commit = self.commits[upto] - print col.Color(col.GREEN, ' %s' % args[upto]) + print(col.Color(col.GREEN, ' %s' % args[upto])) cc_list = list(self._generated_cc[commit.patch]) for email in set(cc_list) - to_set - cc_set: if email == None: email = col.Color(col.YELLOW, "<alias '%s' not found>" % tag) if email: - print ' Cc: ',email + print(' Cc: ', email) print for item in to_set: - print 'To:\t ', item + print('To:\t ', item) for item in cc_set - to_set: - print 'Cc:\t ', item - print 'Version: ', self.get('version') - print 'Prefix:\t ', self.get('prefix') + print('Cc:\t ', item) + print('Version: ', self.get('version')) + print('Prefix:\t ', self.get('prefix')) if self.cover: - print 'Cover: %d lines' % len(self.cover) + print('Cover: %d lines' % len(self.cover)) cover_cc = gitutil.BuildEmailList(self.get('cover_cc', '')) all_ccs = itertools.chain(cover_cc, *self._generated_cc.values()) for email in set(all_ccs) - to_set - cc_set: - print ' Cc: ',email + print(' Cc: ', email) if cmd: - print 'Git command: %s' % cmd + print('Git command: %s' % cmd)
def MakeChangeLog(self, commit): """Create a list of changes for each version. @@ -191,13 +193,13 @@ class Series(dict): else: if version > 1: str = 'Change log missing for v%d' % version - print col.Color(col.RED, str) + print(col.Color(col.RED, str)) for version in changes_copy: str = 'Change log for unknown version v%d' % version - print col.Color(col.RED, str) + print(col.Color(col.RED, str)) elif self.changes: str = 'Change log exists, but no version is set' - print col.Color(col.RED, str) + print(col.Color(col.RED, str))
def MakeCcFile(self, process_tags, cover_fname, raise_on_error, add_maintainers): @@ -228,12 +230,12 @@ class Series(dict): if add_maintainers: list += get_maintainer.GetMaintainer(commit.patch) all_ccs += list - print >>fd, commit.patch, ', '.join(set(list)) + print(commit.patch, ', '.join(set(list)), file=fd) self._generated_cc[commit.patch] = list
if cover_fname: cover_cc = gitutil.BuildEmailList(self.get('cover_cc', '')) - print >>fd, cover_fname, ', '.join(set(cover_cc + all_ccs)) + print(cover_fname, ', '.join(set(cover_cc + all_ccs)), file=fd)
fd.close() return fname diff --git a/tools/patman/settings.py b/tools/patman/settings.py index ba2a68f..01f6c38 100644 --- a/tools/patman/settings.py +++ b/tools/patman/settings.py @@ -3,6 +3,8 @@ # SPDX-License-Identifier: GPL-2.0+ #
+from __future__ import print_function + import ConfigParser import os import re @@ -156,7 +158,7 @@ def ReadGitAliases(fname): try: fd = open(fname, 'r') except IOError: - print "Warning: Cannot find alias file '%s'" % fname + print("Warning: Cannot find alias file '%s'" % fname) return
re_line = re.compile('alias\s+(\S+)\s+(.*)') @@ -167,7 +169,7 @@ def ReadGitAliases(fname):
m = re_line.match(line) if not m: - print "Warning: Alias file line '%s' not understood" % line + print("Warning: Alias file line '%s' not understood" % line) continue
list = alias.get(m.group(1), []) @@ -200,10 +202,10 @@ def CreatePatmanConfigFile(config_fname): try: f = open(config_fname, 'w') except IOError: - print "Couldn't create patman config file\n" + print("Couldn't create patman config file\n") raise
- print >>f, "[alias]\nme: %s <%s>" % (name, email) + print("[alias]\nme: %s <%s>" % (name, email), file=f) f.close();
def _UpdateDefaults(parser, config): @@ -233,7 +235,7 @@ def _UpdateDefaults(parser, config): val = config.getint('settings', name) parser.set_default(name, val) else: - print "WARNING: Unknown setting %s" % name + print("WARNING: Unknown setting %s" % name)
def _ReadAliasFile(fname): """Read in the U-Boot git alias file if it exists. @@ -258,7 +260,7 @@ def _ReadAliasFile(fname): continue alias[words[1]] = [s.strip() for s in words[2].split(',')] if bad_line: - print bad_line + print(bad_line)
def Setup(parser, project_name, config_fname=''): """Set up the settings module by reading config files. @@ -276,7 +278,7 @@ def Setup(parser, project_name, config_fname=''): config_fname = '%s/.patman' % os.getenv('HOME')
if not os.path.exists(config_fname): - print "No config file found ~/.patman\nCreating one...\n" + print("No config file found ~/.patman\nCreating one...\n") CreatePatmanConfigFile(config_fname)
config.read(config_fname) diff --git a/tools/patman/terminal.py b/tools/patman/terminal.py index e78a7c1..d8a9e4c 100644 --- a/tools/patman/terminal.py +++ b/tools/patman/terminal.py @@ -8,6 +8,8 @@ This module handles terminal interaction including ANSI color codes. """
+from __future__ import print_function + import os import sys
@@ -52,9 +54,9 @@ def Print(text='', newline=True, colour=None): if colour: col = Color() text = col.Color(colour, text) - print text, + print(text, end='') if newline: - print + print()
def SetPrintTestMode(): """Go into test mode, where all printing is recorded""" @@ -79,11 +81,11 @@ def EchoPrintTestLines(): for line in print_test_list: if line.colour: col = Color() - print col.Color(line.colour, line.text), + print(col.Color(line.colour, line.text), end='') else: - print line.text, + print(line.text, end='') if line.newline: - print + print()
class Color(object): diff --git a/tools/patman/test.py b/tools/patman/test.py index e8f7472..8c39f66 100644 --- a/tools/patman/test.py +++ b/tools/patman/test.py @@ -181,7 +181,7 @@ index 0000000..2234c87 elif data_type == 'indent': indent = tab else: - print 'not implemented' + print('not implemented') return data % (signoff, tab, indent, tab)
def SetupData(self, data_type):

On 27 September 2016 at 09:03, Paul Burton paul.burton@imgtec.com wrote:
In python 3.x, print must be used as a function call. Convert all print statements to the function call style, importing from __future__ where we print with no trailing newline or print to a file object.
Signed-off-by: Paul Burton paul.burton@imgtec.com Acked-by: Simon Glass sjg@chromium.org
Changes in v2: None
tools/patman/checkpatch.py | 16 ++++++++-------- tools/patman/get_maintainer.py | 2 +- tools/patman/gitutil.py | 6 +++--- tools/patman/patchstream.py | 6 +++--- tools/patman/patman.py | 12 ++++++------ tools/patman/series.py | 38 ++++++++++++++++++++------------------ tools/patman/settings.py | 16 +++++++++------- tools/patman/terminal.py | 12 +++++++----- tools/patman/test.py | 2 +- 9 files changed, 58 insertions(+), 52 deletions(-)
Applied to u-boot-dm, thanks!

Syntax for exception handling is a little more strict in python 3.x. Convert all uses to a form accepted by both python 2.x & python 3.x.
Signed-off-by: Paul Burton paul.burton@imgtec.com Acked-by: Simon Glass sjg@chromium.org ---
Changes in v2: None
tools/patman/command.py | 2 +- tools/patman/cros_subprocess.py | 2 +- tools/patman/gitutil.py | 12 ++++++------ 3 files changed, 8 insertions(+), 8 deletions(-)
diff --git a/tools/patman/command.py b/tools/patman/command.py index d1f0ca5..bebc495 100644 --- a/tools/patman/command.py +++ b/tools/patman/command.py @@ -85,7 +85,7 @@ def RunPipe(pipe_list, infile=None, outfile=None,
try: last_pipe = cros_subprocess.Popen(cmd, cwd=cwd, **kwargs) - except Exception, err: + except Exception as err: result.exception = err if raise_on_error: raise Exception("Error running '%s': %s" % (user_pipestr, str)) diff --git a/tools/patman/cros_subprocess.py b/tools/patman/cros_subprocess.py index 0fc4a06..ebd4300 100644 --- a/tools/patman/cros_subprocess.py +++ b/tools/patman/cros_subprocess.py @@ -166,7 +166,7 @@ class Popen(subprocess.Popen): while read_set or write_set: try: rlist, wlist, _ = select.select(read_set, write_set, [], 0.2) - except select.error, e: + except select.error as e: if e.args[0] == errno.EINTR: continue raise diff --git a/tools/patman/gitutil.py b/tools/patman/gitutil.py index c0fe093..0d23079 100644 --- a/tools/patman/gitutil.py +++ b/tools/patman/gitutil.py @@ -139,7 +139,7 @@ def GetUpstream(git_dir, branch): leaf = merge.split('/')[-1] return '%s/%s' % (remote, leaf), None else: - raise ValueError, ("Cannot determine upstream branch for branch " + raise ValueError("Cannot determine upstream branch for branch " "'%s' remote='%s', merge='%s'" % (branch, remote, merge))
@@ -224,7 +224,7 @@ def Checkout(commit_hash, git_dir=None, work_tree=None, force=False): result = command.RunPipe([pipe], capture=True, raise_on_error=False, capture_stderr=True) if result.return_code != 0: - raise OSError, 'git checkout (%s): %s' % (pipe, result.stderr) + raise OSError('git checkout (%s): %s' % (pipe, result.stderr))
def Clone(git_dir, output_dir): """Checkout the selected commit for this build @@ -236,7 +236,7 @@ def Clone(git_dir, output_dir): result = command.RunPipe([pipe], capture=True, cwd=output_dir, capture_stderr=True) if result.return_code != 0: - raise OSError, 'git clone: %s' % result.stderr + raise OSError('git clone: %s' % result.stderr)
def Fetch(git_dir=None, work_tree=None): """Fetch from the origin repo @@ -252,7 +252,7 @@ def Fetch(git_dir=None, work_tree=None): pipe.append('fetch') result = command.RunPipe([pipe], capture=True, capture_stderr=True) if result.return_code != 0: - raise OSError, 'git fetch: %s' % result.stderr + raise OSError('git fetch: %s' % result.stderr)
def CreatePatches(start, count, series): """Create a series of patches from the top of the current branch. @@ -489,7 +489,7 @@ def LookupEmail(lookup_name, alias=None, raise_on_error=True, level=0): if level > 10: msg = "Recursive email alias at '%s'" % lookup_name if raise_on_error: - raise OSError, msg + raise OSError(msg) else: print(col.Color(col.RED, msg)) return out_list @@ -498,7 +498,7 @@ def LookupEmail(lookup_name, alias=None, raise_on_error=True, level=0): if not lookup_name in alias: msg = "Alias '%s' not found" % lookup_name if raise_on_error: - raise ValueError, msg + raise ValueError(msg) else: print(col.Color(col.RED, msg)) return out_list

On 27 September 2016 at 09:03, Paul Burton paul.burton@imgtec.com wrote:
Syntax for exception handling is a little more strict in python 3.x. Convert all uses to a form accepted by both python 2.x & python 3.x.
Signed-off-by: Paul Burton paul.burton@imgtec.com Acked-by: Simon Glass sjg@chromium.org
Changes in v2: None
tools/patman/command.py | 2 +- tools/patman/cros_subprocess.py | 2 +- tools/patman/gitutil.py | 12 ++++++------ 3 files changed, 8 insertions(+), 8 deletions(-)
Applied to u-boot-dm, thanks!

In python 3.x module names used in import statements are case sensitive, and the configparser module is named in all lower-case. Import it as such in order to avoid errors when running with python 3.x.
Signed-off-by: Paul Burton paul.burton@imgtec.com Acked-by: Simon Glass sjg@chromium.org ---
Changes in v2: None
tools/patman/settings.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/tools/patman/settings.py b/tools/patman/settings.py index 01f6c38..3caf379 100644 --- a/tools/patman/settings.py +++ b/tools/patman/settings.py @@ -5,7 +5,11 @@
from __future__ import print_function
-import ConfigParser +try: + import configparser as ConfigParser +except: + import ConfigParser + import os import re

On 27 September 2016 at 09:03, Paul Burton paul.burton@imgtec.com wrote:
In python 3.x module names used in import statements are case sensitive, and the configparser module is named in all lower-case. Import it as such in order to avoid errors when running with python 3.x.
Signed-off-by: Paul Burton paul.burton@imgtec.com Acked-by: Simon Glass sjg@chromium.org
Changes in v2: None
tools/patman/settings.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
Applied to u-boot-dm, thanks!

In python 3.x reading stdout or stdin will produce a bytestring rather than a string. Decode it in CommunicateFilter such that the rest of the code can continue to deal with strings. This works fine with python 2.x too.
Signed-off-by: Paul Burton paul.burton@imgtec.com Acked-by: Simon Glass sjg@chromium.org ---
Changes in v2: None
tools/patman/cros_subprocess.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/patman/cros_subprocess.py b/tools/patman/cros_subprocess.py index ebd4300..922e560 100644 --- a/tools/patman/cros_subprocess.py +++ b/tools/patman/cros_subprocess.py @@ -189,7 +189,7 @@ class Popen(subprocess.Popen): data = "" # We will get an error on read if the pty is closed try: - data = os.read(self.stdout.fileno(), 1024) + data = os.read(self.stdout.fileno(), 1024).decode('utf8') except OSError: pass if data == "": @@ -204,7 +204,7 @@ class Popen(subprocess.Popen): data = "" # We will get an error on read if the pty is closed try: - data = os.read(self.stderr.fileno(), 1024) + data = os.read(self.stderr.fileno(), 1024).decode('utf8') except OSError: pass if data == "":

Hi Paul,
On 27 September 2016 at 09:03, Paul Burton paul.burton@imgtec.com wrote:
In python 3.x reading stdout or stdin will produce a bytestring rather than a string. Decode it in CommunicateFilter such that the rest of the code can continue to deal with strings. This works fine with python 2.x too.
Signed-off-by: Paul Burton paul.burton@imgtec.com Acked-by: Simon Glass sjg@chromium.org
Changes in v2: None
tools/patman/cros_subprocess.py | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
diff --git a/tools/patman/cros_subprocess.py b/tools/patman/cros_subprocess.py index ebd4300..922e560 100644 --- a/tools/patman/cros_subprocess.py +++ b/tools/patman/cros_subprocess.py @@ -189,7 +189,7 @@ class Popen(subprocess.Popen): data = "" # We will get an error on read if the pty is closed try:
data = os.read(self.stdout.fileno(), 1024)
data = os.read(self.stdout.fileno(), 1024).decode('utf8') except OSError: pass if data == "":
@@ -204,7 +204,7 @@ class Popen(subprocess.Popen): data = "" # We will get an error on read if the pty is closed try:
data = os.read(self.stderr.fileno(), 1024)
data = os.read(self.stderr.fileno(), 1024).decode('utf8') except OSError: pass if data == "":
-- 2.10.0
Unfortunately this causes an error:
$ buildman -b dm-push sandbox boards.cfg is up to date. Nothing to do. Building 16 commits for 3 boards (3 threads, 3 jobs per thread) Traceback (most recent call last): File "/home/sglass/bin/buildman", line 64, in <module> ret_code = control.DoBuildman(options, args) File "/scratch/sglass/cosarm/src/third_party/u-boot/files/tools/buildman/control.py", line 300, in DoBuildman options.keep_outputs, options.verbose) File "/scratch/sglass/cosarm/src/third_party/u-boot/files/tools/buildman/builder.py", line 1433, in BuildBoards self._PrepareOutputSpace() File "/scratch/sglass/cosarm/src/third_party/u-boot/files/tools/buildman/builder.py", line 1399, in _PrepareOutputSpace dir_list.append(self._GetOutputDir(commit_upto)) File "/scratch/sglass/cosarm/src/third_party/u-boot/files/tools/buildman/builder.py", line 449, in _GetOutputDir subject = commit.subject.translate(trans_valid_chars) TypeError: character mapping must return integer, None or unicode
Regards, Simon

In python 3.x the iteritems() method has been removed from dictionaries, and the items() method does effectively the same thing. On python 2.x using items() is a little less efficient since it involves copying data, but as speed isn't a concern in this code switch to using items() anyway for simplicity.
Signed-off-by: Paul Burton paul.burton@imgtec.com
---
Changes in v2: - Just use items() for all python versions
tools/patman/settings.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/patman/settings.py b/tools/patman/settings.py index 3caf379..7ef0ab0 100644 --- a/tools/patman/settings.py +++ b/tools/patman/settings.py @@ -94,7 +94,7 @@ class _ProjectConfigParser(ConfigParser.SafeConfigParser): if not self.has_section(project_settings): self.add_section(project_settings) project_defaults = _default_settings.get(project_name, {}) - for setting_name, setting_value in project_defaults.iteritems(): + for setting_name, setting_value in project_defaults.items(): self.set(project_settings, setting_name, setting_value)
def get(self, section, option, *args, **kwargs):

On 27 September 2016 at 09:03, Paul Burton paul.burton@imgtec.com wrote:
In python 3.x the iteritems() method has been removed from dictionaries, and the items() method does effectively the same thing. On python 2.x using items() is a little less efficient since it involves copying data, but as speed isn't a concern in this code switch to using items() anyway for simplicity.
Signed-off-by: Paul Burton paul.burton@imgtec.com
Changes in v2:
- Just use items() for all python versions
tools/patman/settings.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Acked-by: Simon Glass sjg@chromium.org

On 27 September 2016 at 11:55, Simon Glass sjg@chromium.org wrote:
On 27 September 2016 at 09:03, Paul Burton paul.burton@imgtec.com wrote:
In python 3.x the iteritems() method has been removed from dictionaries, and the items() method does effectively the same thing. On python 2.x using items() is a little less efficient since it involves copying data, but as speed isn't a concern in this code switch to using items() anyway for simplicity.
Signed-off-by: Paul Burton paul.burton@imgtec.com
Changes in v2:
- Just use items() for all python versions
tools/patman/settings.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Acked-by: Simon Glass sjg@chromium.org
Applied to u-boot-dm, thanks!

In python 3.x StringIO is no longer a module, and the class can instead be found in the io module. Adjust the code in the doctest input to account for both.
Signed-off-by: Paul Burton paul.burton@imgtec.com
---
Changes in v2: - New patch, need found by running --test.
tools/patman/settings.py | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
diff --git a/tools/patman/settings.py b/tools/patman/settings.py index 7ef0ab0..5f207f5 100644 --- a/tools/patman/settings.py +++ b/tools/patman/settings.py @@ -36,7 +36,10 @@ class _ProjectConfigParser(ConfigParser.SafeConfigParser): - Merge general default settings/aliases with project-specific ones.
# Sample config used for tests below... - >>> import StringIO + >>> try: + ... from StringIO import StringIO + ... except ImportError: + ... from io import StringIO >>> sample_config = ''' ... [alias] ... me: Peter P. likesspiders@example.com @@ -54,25 +57,25 @@ class _ProjectConfigParser(ConfigParser.SafeConfigParser):
# Check to make sure that bogus project gets general alias. >>> config = _ProjectConfigParser("zzz") - >>> config.readfp(StringIO.StringIO(sample_config)) + >>> config.readfp(StringIO(sample_config)) >>> config.get("alias", "enemies") 'Evil evil@example.com'
# Check to make sure that alias gets overridden by project. >>> config = _ProjectConfigParser("sm") - >>> config.readfp(StringIO.StringIO(sample_config)) + >>> config.readfp(StringIO(sample_config)) >>> config.get("alias", "enemies") 'Green G. ugly@example.com'
# Check to make sure that settings get merged with project. >>> config = _ProjectConfigParser("linux") - >>> config.readfp(StringIO.StringIO(sample_config)) + >>> config.readfp(StringIO(sample_config)) >>> sorted(config.items("settings")) [('am_hero', 'True'), ('process_tags', 'False')]
# Check to make sure that settings works with unknown project. >>> config = _ProjectConfigParser("unknown") - >>> config.readfp(StringIO.StringIO(sample_config)) + >>> config.readfp(StringIO(sample_config)) >>> sorted(config.items("settings")) [('am_hero', 'True')] """

On 27 September 2016 at 09:03, Paul Burton paul.burton@imgtec.com wrote:
In python 3.x StringIO is no longer a module, and the class can instead be found in the io module. Adjust the code in the doctest input to account for both.
Signed-off-by: Paul Burton paul.burton@imgtec.com
Changes in v2:
- New patch, need found by running --test.
tools/patman/settings.py | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
Acked-by: Simon Glass sjg@chromium.org

On 27 September 2016 at 11:55, Simon Glass sjg@chromium.org wrote:
On 27 September 2016 at 09:03, Paul Burton paul.burton@imgtec.com wrote:
In python 3.x StringIO is no longer a module, and the class can instead be found in the io module. Adjust the code in the doctest input to account for both.
Signed-off-by: Paul Burton paul.burton@imgtec.com
Changes in v2:
- New patch, need found by running --test.
tools/patman/settings.py | 13 ++++++++----- 1 file changed, 8 insertions(+), 5 deletions(-)
Acked-by: Simon Glass sjg@chromium.org
Applied to u-boot-dm, thanks!

In python 3.x the iteritems() method has been removed from dictionaries, and the items() method does effectively the same thing. On python 2.x using items() is a little less efficient since it involves copying data, but as speed isn't a concern in the affected code switch to using items() anyway for simplicity.
Signed-off-by: Paul Burton paul.burton@imgtec.com
---
Changes in v2: - Just use items() for all python versions
tools/dtoc/dtoc.py | 8 ++++---- tools/dtoc/fdt_fallback.py | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-)
diff --git a/tools/dtoc/dtoc.py b/tools/dtoc/dtoc.py index 518aa51..12aa990 100755 --- a/tools/dtoc/dtoc.py +++ b/tools/dtoc/dtoc.py @@ -224,14 +224,14 @@ class DtbPlatdata: fields = {}
# Get a list of all the valid properties in this node. - for name, prop in node.props.iteritems(): + for name, prop in node.props.items(): if name not in PROP_IGNORE_LIST and name[0] != '#': fields[name] = copy.deepcopy(prop)
# If we've seen this node_name before, update the existing struct. if node_name in structs: struct = structs[node_name] - for name, prop in fields.iteritems(): + for name, prop in fields.items(): oldprop = struct.get(name) if oldprop: oldprop.Widen(prop) @@ -246,7 +246,7 @@ class DtbPlatdata: for node in self._valid_nodes: node_name = self.GetCompatName(node) struct = structs[node_name] - for name, prop in node.props.iteritems(): + for name, prop in node.props.items(): if name not in PROP_IGNORE_LIST and name[0] != '#': prop.Widen(struct[name]) upto += 1 @@ -298,7 +298,7 @@ class DtbPlatdata: var_name = Conv_name_to_c(node.name) self.Buf('static struct %s%s %s%s = {\n' % (STRUCT_PREFIX, struct_name, VAL_PREFIX, var_name)) - for pname, prop in node.props.iteritems(): + for pname, prop in node.props.items(): if pname in PROP_IGNORE_LIST or pname[0] == '#': continue ptype = TYPE_NAMES[prop.type] diff --git a/tools/dtoc/fdt_fallback.py b/tools/dtoc/fdt_fallback.py index 0c0ebbc..8a531ff 100644 --- a/tools/dtoc/fdt_fallback.py +++ b/tools/dtoc/fdt_fallback.py @@ -58,7 +58,7 @@ class Node(NodeBase): This fills in the props and subnodes properties, recursively searching into subnodes so that the entire tree is built. """ - for name, byte_list_str in self._fdt.GetProps(self.path).iteritems(): + for name, byte_list_str in self._fdt.GetProps(self.path).items(): prop = Prop(self, name, byte_list_str) self.props[name] = prop

On 27 September 2016 at 09:03, Paul Burton paul.burton@imgtec.com wrote:
In python 3.x the iteritems() method has been removed from dictionaries, and the items() method does effectively the same thing. On python 2.x using items() is a little less efficient since it involves copying data, but as speed isn't a concern in the affected code switch to using items() anyway for simplicity.
Signed-off-by: Paul Burton paul.burton@imgtec.com
Changes in v2:
- Just use items() for all python versions
tools/dtoc/dtoc.py | 8 ++++---- tools/dtoc/fdt_fallback.py | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-)
Acked-by: Simon Glass sjg@chromium.org

On 27 September 2016 at 11:55, Simon Glass sjg@chromium.org wrote:
On 27 September 2016 at 09:03, Paul Burton paul.burton@imgtec.com wrote:
In python 3.x the iteritems() method has been removed from dictionaries, and the items() method does effectively the same thing. On python 2.x using items() is a little less efficient since it involves copying data, but as speed isn't a concern in the affected code switch to using items() anyway for simplicity.
Signed-off-by: Paul Burton paul.burton@imgtec.com
Changes in v2:
- Just use items() for all python versions
tools/dtoc/dtoc.py | 8 ++++---- tools/dtoc/fdt_fallback.py | 2 +- 2 files changed, 5 insertions(+), 5 deletions(-)
Acked-by: Simon Glass sjg@chromium.org
Applied to u-boot-dm, thanks!

On python 3.x struct.unpack will complain if we provide it with a string since it expects to operate on a bytes object. In order to satisfy this requirement, encode the string to a bytes object when running on python 3.x.
Signed-off-by: Paul Burton paul.burton@imgtec.com Acked-by: Simon Glass sjg@chromium.org ---
Changes in v2: None
tools/dtoc/fdt_util.py | 3 +++ 1 file changed, 3 insertions(+)
diff --git a/tools/dtoc/fdt_util.py b/tools/dtoc/fdt_util.py index 3a10838..e6d523b 100644 --- a/tools/dtoc/fdt_util.py +++ b/tools/dtoc/fdt_util.py @@ -8,6 +8,7 @@
import os import struct +import sys import tempfile
import command @@ -22,6 +23,8 @@ def fdt32_to_cpu(val): Return: A native-endian integer value """ + if sys.version_info > (3, 0): + val = val.encode('raw_unicode_escape') return struct.unpack('>I', val)[0]
def EnsureCompiled(fname):

On 27 September 2016 at 09:03, Paul Burton paul.burton@imgtec.com wrote:
On python 3.x struct.unpack will complain if we provide it with a string since it expects to operate on a bytes object. In order to satisfy this requirement, encode the string to a bytes object when running on python 3.x.
Signed-off-by: Paul Burton paul.burton@imgtec.com Acked-by: Simon Glass sjg@chromium.org
Changes in v2: None
tools/dtoc/fdt_util.py | 3 +++ 1 file changed, 3 insertions(+)
Applied to u-boot-dm, thanks!

If we use the '/' operator then python 3.x will produce a float, and refuse to multiply the string sequence in Conv_name_to_c by it with:
TypeError: can't multiply sequence by non-int of type 'float'
Use the '//' operator instead to enforce that we want integer rather than floating point division.
Signed-off-by: Paul Burton paul.burton@imgtec.com Acked-by: Simon Glass sjg@chromium.org
---
Changes in v2: None
tools/dtoc/dtoc.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/dtoc/dtoc.py b/tools/dtoc/dtoc.py index 12aa990..11050b6 100755 --- a/tools/dtoc/dtoc.py +++ b/tools/dtoc/dtoc.py @@ -60,7 +60,7 @@ def Conv_name_to_c(name): def TabTo(num_tabs, str): if len(str) >= num_tabs * 8: return str + ' ' - return str + '\t' * (num_tabs - len(str) / 8) + return str + '\t' * (num_tabs - len(str) // 8)
class DtbPlatdata: """Provide a means to convert device tree binary data to platform data

On 27 September 2016 at 09:03, Paul Burton paul.burton@imgtec.com wrote:
If we use the '/' operator then python 3.x will produce a float, and refuse to multiply the string sequence in Conv_name_to_c by it with:
TypeError: can't multiply sequence by non-int of type 'float'
Use the '//' operator instead to enforce that we want integer rather than floating point division.
Signed-off-by: Paul Burton paul.burton@imgtec.com Acked-by: Simon Glass sjg@chromium.org
Changes in v2: None
tools/dtoc/dtoc.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Applied to u-boot-dm, thanks!
participants (2)
-
Paul Burton
-
Simon Glass