[U-Boot] [PATCH] buildman: Translate more strings to latin-1
This is a follow-up patch for commit fbeb33752999e7317113199ef89873d6b6916814.
This fixes following exception:
Exception in thread Thread-7: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/u-boot/tools/buildman/builderthread.py", line 475, in run self.RunJob(job) File "/u-boot/tools/buildman/builderthread.py", line 456, in RunJob self._WriteResult(result, job.keep_outputs) File "/u-boot/tools/buildman/builderthread.py", line 333, in _WriteResult print >>fd, dump_result.stdout, UnicodeEncodeError: 'ascii' codec can't encode characters in position 75-76: ordinal not in range(128)
Signed-off-by: Daniel Schwierzeck daniel.schwierzeck@gmail.com
---
tools/buildman/builderthread.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index acaf5007f5..0a460878f3 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -330,7 +330,7 @@ class BuilderThread(threading.Thread): objdump = self.builder.GetObjdumpFile(result.commit_upto, result.brd.target, fname) with open(objdump, 'w') as fd: - print >>fd, dump_result.stdout, + print >>fd, dump_result.stdout.encode('latin-1', 'ignore'), for line in dump_result.stdout.splitlines(): fields = line.split() if len(fields) > 5 and fields[1] == '.rodata':
Hi Daniel,
On 31 May 2017 at 07:40, Daniel Schwierzeck daniel.schwierzeck@gmail.com wrote:
This is a follow-up patch for commit fbeb33752999e7317113199ef89873d6b6916814.
This fixes following exception:
Exception in thread Thread-7: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/u-boot/tools/buildman/builderthread.py", line 475, in run self.RunJob(job) File "/u-boot/tools/buildman/builderthread.py", line 456, in RunJob self._WriteResult(result, job.keep_outputs) File "/u-boot/tools/buildman/builderthread.py", line 333, in _WriteResult print >>fd, dump_result.stdout, UnicodeEncodeError: 'ascii' codec can't encode characters in position 75-76: ordinal not in range(128)
Signed-off-by: Daniel Schwierzeck daniel.schwierzeck@gmail.com
tools/buildman/builderthread.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index acaf5007f5..0a460878f3 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -330,7 +330,7 @@ class BuilderThread(threading.Thread): objdump = self.builder.GetObjdumpFile(result.commit_upto, result.brd.target, fname) with open(objdump, 'w') as fd:
print >>fd, dump_result.stdout,
print >>fd, dump_result.stdout.encode('latin-1', 'ignore'),
I am worried that this will produce gibberish in the output. Do you know what specific characters are causing problems? Can we encode to utf-8? BTW I recently sent a patman series to try to resolve unicode issues in patman.
for line in dump_result.stdout.splitlines(): fields = line.split() if len(fields) > 5 and fields[1] == '.rodata':-- 2.13.0
Regards, Simon
Am 01.06.2017 um 15:21 schrieb Simon Glass:
Hi Daniel,
On 31 May 2017 at 07:40, Daniel Schwierzeck daniel.schwierzeck@gmail.com wrote:
This is a follow-up patch for commit fbeb33752999e7317113199ef89873d6b6916814.
This fixes following exception:
Exception in thread Thread-7: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/u-boot/tools/buildman/builderthread.py", line 475, in run self.RunJob(job) File "/u-boot/tools/buildman/builderthread.py", line 456, in RunJob self._WriteResult(result, job.keep_outputs) File "/u-boot/tools/buildman/builderthread.py", line 333, in _WriteResult print >>fd, dump_result.stdout, UnicodeEncodeError: 'ascii' codec can't encode characters in position 75-76: ordinal not in range(128)
Signed-off-by: Daniel Schwierzeck daniel.schwierzeck@gmail.com
tools/buildman/builderthread.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index acaf5007f5..0a460878f3 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -330,7 +330,7 @@ class BuilderThread(threading.Thread): objdump = self.builder.GetObjdumpFile(result.commit_upto, result.brd.target, fname) with open(objdump, 'w') as fd:
print >>fd, dump_result.stdout,
print >>fd, dump_result.stdout.encode('latin-1', 'ignore'),I am worried that this will produce gibberish in the output. Do you know what specific characters are causing problems? Can we encode to utf-8? BTW I recently sent a patman series to try to resolve unicode issues in patman.
objdump uses localized output. If I use LC_ALL=C, the exception doesn't occur. My system's default language is German. Thus the offending character is the German 'ö':
... Sektionen: Idx Name Größe VMA LMA Datei-Off Ausr. 0 .text 00032a94 9f000000 9f000000 000000c0 2**4 CONTENTS, ALLOC, LOAD, CODE 1 .rodata 00008e50 9f032aa0 9f032aa0 00032b60 2**4 CONTENTS, ALLOC, LOAD, READONLY, DATA ...
Actually I'd chosen UTF-8 too, but I followed Tom's patch which used latin-1 ;)
So we should always encode the output of tools such as binutils to UTF-8, shouldn't we? I can send an updated patch.
On Fri, Jun 02, 2017 at 11:18:27AM +0200, Daniel Schwierzeck wrote:
Am 01.06.2017 um 15:21 schrieb Simon Glass:
Hi Daniel,
On 31 May 2017 at 07:40, Daniel Schwierzeck daniel.schwierzeck@gmail.com wrote:
This is a follow-up patch for commit fbeb33752999e7317113199ef89873d6b6916814.
This fixes following exception:
Exception in thread Thread-7: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/u-boot/tools/buildman/builderthread.py", line 475, in run self.RunJob(job) File "/u-boot/tools/buildman/builderthread.py", line 456, in RunJob self._WriteResult(result, job.keep_outputs) File "/u-boot/tools/buildman/builderthread.py", line 333, in _WriteResult print >>fd, dump_result.stdout, UnicodeEncodeError: 'ascii' codec can't encode characters in position 75-76: ordinal not in range(128)
Signed-off-by: Daniel Schwierzeck daniel.schwierzeck@gmail.com
tools/buildman/builderthread.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index acaf5007f5..0a460878f3 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -330,7 +330,7 @@ class BuilderThread(threading.Thread): objdump = self.builder.GetObjdumpFile(result.commit_upto, result.brd.target, fname) with open(objdump, 'w') as fd:
print >>fd, dump_result.stdout,
print >>fd, dump_result.stdout.encode('latin-1', 'ignore'),I am worried that this will produce gibberish in the output. Do you know what specific characters are causing problems? Can we encode to utf-8? BTW I recently sent a patman series to try to resolve unicode issues in patman.
objdump uses localized output. If I use LC_ALL=C, the exception doesn't occur. My system's default language is German. Thus the offending character is the German 'ö':
... Sektionen: Idx Name Größe VMA LMA Datei-Off Ausr. 0 .text 00032a94 9f000000 9f000000 000000c0 2**4 CONTENTS, ALLOC, LOAD, CODE 1 .rodata 00008e50 9f032aa0 9f032aa0 00032b60 2**4 CONTENTS, ALLOC, LOAD, READONLY, DATA ...
Actually I'd chosen UTF-8 too, but I followed Tom's patch which used latin-1 ;)
So we should always encode the output of tools such as binutils to UTF-8, shouldn't we? I can send an updated patch.
I'd be fine with UTF-8 everywhere too, I just borrowed an existing example that was latin-1 :)
Hi Daniel,
On 2 June 2017 at 07:21, Tom Rini trini@konsulko.com wrote:
On Fri, Jun 02, 2017 at 11:18:27AM +0200, Daniel Schwierzeck wrote:
Am 01.06.2017 um 15:21 schrieb Simon Glass:
Hi Daniel,
On 31 May 2017 at 07:40, Daniel Schwierzeck daniel.schwierzeck@gmail.com wrote:
This is a follow-up patch for commit fbeb33752999e7317113199ef89873d6b6916814.
This fixes following exception:
Exception in thread Thread-7: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/u-boot/tools/buildman/builderthread.py", line 475, in run self.RunJob(job) File "/u-boot/tools/buildman/builderthread.py", line 456, in RunJob self._WriteResult(result, job.keep_outputs) File "/u-boot/tools/buildman/builderthread.py", line 333, in _WriteResult print >>fd, dump_result.stdout, UnicodeEncodeError: 'ascii' codec can't encode characters in position 75-76: ordinal not in range(128)
Signed-off-by: Daniel Schwierzeck daniel.schwierzeck@gmail.com
tools/buildman/builderthread.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index acaf5007f5..0a460878f3 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -330,7 +330,7 @@ class BuilderThread(threading.Thread): objdump = self.builder.GetObjdumpFile(result.commit_upto, result.brd.target, fname) with open(objdump, 'w') as fd:
print >>fd, dump_result.stdout,
print >>fd, dump_result.stdout.encode('latin-1', 'ignore'),I am worried that this will produce gibberish in the output. Do you know what specific characters are causing problems? Can we encode to utf-8? BTW I recently sent a patman series to try to resolve unicode issues in patman.
objdump uses localized output. If I use LC_ALL=C, the exception doesn't occur. My system's default language is German. Thus the offending character is the German 'ö':
... Sektionen: Idx Name Größe VMA LMA Datei-Off Ausr. 0 .text 00032a94 9f000000 9f000000 000000c0 2**4 CONTENTS, ALLOC, LOAD, CODE 1 .rodata 00008e50 9f032aa0 9f032aa0 00032b60 2**4 CONTENTS, ALLOC, LOAD, READONLY, DATA ...
Actually I'd chosen UTF-8 too, but I followed Tom's patch which used latin-1 ;)
So we should always encode the output of tools such as binutils to UTF-8, shouldn't we? I can send an updated patch.
I'd be fine with UTF-8 everywhere too, I just borrowed an existing example that was latin-1 :)
Are you planning to send an updated patch?
Regards, Simon
Am 07.06.2017 um 13:55 schrieb Simon Glass: ...
Actually I'd chosen UTF-8 too, but I followed Tom's patch which used latin-1 ;)
So we should always encode the output of tools such as binutils to UTF-8, shouldn't we? I can send an updated patch.
I'd be fine with UTF-8 everywhere too, I just borrowed an existing example that was latin-1 :)
Are you planning to send an updated patch?
sure, I forgot to send it ;)
binutils are using localized output. On systems with locales and special characters this leads to exceptions like this:
Exception in thread Thread-7: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/u-boot/tools/buildman/builderthread.py", line 475, in run self.RunJob(job) File "/u-boot/tools/buildman/builderthread.py", line 456, in RunJob self._WriteResult(result, job.keep_outputs) File "/u-boot/tools/buildman/builderthread.py", line 333, in _WriteResult print >>fd, dump_result.stdout, UnicodeEncodeError: 'ascii' codec can't encode characters in position 75-76: ordinal not in range(128)
Always encode the value of CommandResult::stdout to UTF-8 before writing it to stdout or a log file.
Signed-off-by: Daniel Schwierzeck daniel.schwierzeck@gmail.com
---
Changes in v2: - encode all occurences of CommandResult::stdout after spawning binutils to UTF-8 - reworded commit message
tools/buildman/builderthread.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index acaf5007f5..4ca2f2eb90 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -280,13 +280,13 @@ class BuilderThread(threading.Thread): outfile = os.path.join(build_dir, 'log') with open(outfile, 'w') as fd: if result.stdout: - fd.write(result.stdout.encode('latin-1', 'ignore')) + fd.write(result.stdout.encode('UTF-8', 'ignore'))
errfile = self.builder.GetErrFile(result.commit_upto, result.brd.target) if result.stderr: with open(errfile, 'w') as fd: - fd.write(result.stderr.encode('latin-1', 'ignore')) + fd.write(result.stderr.encode('UTF-8', 'ignore')) elif os.path.exists(errfile): os.remove(errfile)
@@ -319,7 +319,7 @@ class BuilderThread(threading.Thread): nm = self.builder.GetFuncSizesFile(result.commit_upto, result.brd.target, fname) with open(nm, 'w') as fd: - print >>fd, nm_result.stdout, + print >>fd, nm_result.stdout.encode('UTF-8'),
cmd = ['%sobjdump' % self.toolchain.cross, '-h', fname] dump_result = command.RunPipe([cmd], capture=True, @@ -330,7 +330,7 @@ class BuilderThread(threading.Thread): objdump = self.builder.GetObjdumpFile(result.commit_upto, result.brd.target, fname) with open(objdump, 'w') as fd: - print >>fd, dump_result.stdout, + print >>fd, dump_result.stdout.encode('UTF-8'), for line in dump_result.stdout.splitlines(): fields = line.split() if len(fields) > 5 and fields[1] == '.rodata': @@ -352,7 +352,7 @@ class BuilderThread(threading.Thread): sizes = self.builder.GetSizesFile(result.commit_upto, result.brd.target) with open(sizes, 'w') as fd: - print >>fd, '\n'.join(lines) + print >>fd, '\n'.join(lines).encode('UTF-8')
# Write out the configuration files, with a special case for SPL for dirname in ['', 'spl', 'tpl']:
Hi Daniel,
On 7 June 2017 at 06:19, Daniel Schwierzeck daniel.schwierzeck@gmail.com wrote:
binutils are using localized output. On systems with locales and special characters this leads to exceptions like this:
Exception in thread Thread-7: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/u-boot/tools/buildman/builderthread.py", line 475, in run self.RunJob(job) File "/u-boot/tools/buildman/builderthread.py", line 456, in RunJob self._WriteResult(result, job.keep_outputs) File "/u-boot/tools/buildman/builderthread.py", line 333, in _WriteResult print >>fd, dump_result.stdout, UnicodeEncodeError: 'ascii' codec can't encode characters in position 75-76: ordinal not in range(128)
Always encode the value of CommandResult::stdout to UTF-8 before writing it to stdout or a log file.
Signed-off-by: Daniel Schwierzeck daniel.schwierzeck@gmail.com
Changes in v2:
- encode all occurences of CommandResult::stdout after spawning binutils to UTF-8
- reworded commit message
tools/buildman/builderthread.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
I'm still seeing a rare failure, e.g.:
https://travis-ci.org/sglass68/u-boot/jobs/240350621
Any thoughts on that?
Regards, Simon
Am 07.06.2017 um 17:52 schrieb Simon Glass:
Hi Daniel,
On 7 June 2017 at 06:19, Daniel Schwierzeck daniel.schwierzeck@gmail.com wrote:
binutils are using localized output. On systems with locales and special characters this leads to exceptions like this:
Exception in thread Thread-7: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/u-boot/tools/buildman/builderthread.py", line 475, in run self.RunJob(job) File "/u-boot/tools/buildman/builderthread.py", line 456, in RunJob self._WriteResult(result, job.keep_outputs) File "/u-boot/tools/buildman/builderthread.py", line 333, in _WriteResult print >>fd, dump_result.stdout, UnicodeEncodeError: 'ascii' codec can't encode characters in position 75-76: ordinal not in range(128)
Always encode the value of CommandResult::stdout to UTF-8 before writing it to stdout or a log file.
Signed-off-by: Daniel Schwierzeck daniel.schwierzeck@gmail.com
Changes in v2:
- encode all occurences of CommandResult::stdout after spawning binutils to UTF-8
- reworded commit message
tools/buildman/builderthread.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
I'm still seeing a rare failure, e.g.:
https://travis-ci.org/sglass68/u-boot/jobs/240350621
Any thoughts on that?
I'll look into it
Hi Daniel,
On 7 June 2017 at 10:03, Daniel Schwierzeck daniel.schwierzeck@gmail.com wrote:
Am 07.06.2017 um 17:52 schrieb Simon Glass:
Hi Daniel,
On 7 June 2017 at 06:19, Daniel Schwierzeck daniel.schwierzeck@gmail.com wrote:
binutils are using localized output. On systems with locales and special characters this leads to exceptions like this:
Exception in thread Thread-7: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/u-boot/tools/buildman/builderthread.py", line 475, in run self.RunJob(job) File "/u-boot/tools/buildman/builderthread.py", line 456, in RunJob self._WriteResult(result, job.keep_outputs) File "/u-boot/tools/buildman/builderthread.py", line 333, in _WriteResult print >>fd, dump_result.stdout, UnicodeEncodeError: 'ascii' codec can't encode characters in position 75-76: ordinal not in range(128)
Always encode the value of CommandResult::stdout to UTF-8 before writing it to stdout or a log file.
Signed-off-by: Daniel Schwierzeck daniel.schwierzeck@gmail.com
Changes in v2:
- encode all occurences of CommandResult::stdout after spawning binutils to UTF-8
- reworded commit message
tools/buildman/builderthread.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
I'm still seeing a rare failure, e.g.:
https://travis-ci.org/sglass68/u-boot/jobs/240350621
Any thoughts on that?
I'll look into it
Thanks. I believe the two new patches you sent supersede this one.
Regards, Simon
Am 09.06.2017 um 14:27 schrieb Simon Glass:
Hi Daniel,
On 7 June 2017 at 10:03, Daniel Schwierzeck daniel.schwierzeck@gmail.com wrote:
Am 07.06.2017 um 17:52 schrieb Simon Glass:
Hi Daniel,
On 7 June 2017 at 06:19, Daniel Schwierzeck daniel.schwierzeck@gmail.com wrote:
binutils are using localized output. On systems with locales and special characters this leads to exceptions like this:
Exception in thread Thread-7: Traceback (most recent call last): File "/usr/lib/python2.7/threading.py", line 801, in __bootstrap_inner self.run() File "/u-boot/tools/buildman/builderthread.py", line 475, in run self.RunJob(job) File "/u-boot/tools/buildman/builderthread.py", line 456, in RunJob self._WriteResult(result, job.keep_outputs) File "/u-boot/tools/buildman/builderthread.py", line 333, in _WriteResult print >>fd, dump_result.stdout, UnicodeEncodeError: 'ascii' codec can't encode characters in position 75-76: ordinal not in range(128)
Always encode the value of CommandResult::stdout to UTF-8 before writing it to stdout or a log file.
Signed-off-by: Daniel Schwierzeck daniel.schwierzeck@gmail.com
Changes in v2:
- encode all occurences of CommandResult::stdout after spawning binutils to UTF-8
- reworded commit message
tools/buildman/builderthread.py | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
I'm still seeing a rare failure, e.g.:
https://travis-ci.org/sglass68/u-boot/jobs/240350621
Any thoughts on that?
I'll look into it
Thanks. I believe the two new patches you sent supersede this one.
yes, this one is superseded.
This series fixes and optimizes the unicode handling in buildman. It's based on the similar series for patman and applies on top of u-boot-dm/master.
This has been tested by forcibly adding unicode characters in the stdout and stderr stream.
Examples:
The patch
--- a/arch/arm/config.mk +++ b/arch/arm/config.mk @@ -69,7 +69,7 @@ checkgcc6: @if test "$(call cc-name)" = "gcc" -a \ "$(call cc-version)" -lt "0600"; then \ echo -n '*** Your GCC is older than 6.0 and will not be '; \ - echo 'supported starting in v2018.01.'; \ + echo 'supported starting <C3><B6> in v2018.01.'; \ fi
called with `tools/buildman/buildman -P xilinx_zynqmp_ep` results in stdout:
*** Your GCC is older than 6.0 and will not be supported starting ? in v2018.01.
The patch
--- a/arch/sandbox/cpu/os.c +++ b/arch/sandbox/cpu/os.c @@ -40,6 +40,7 @@ ssize_t os_read(int fd, void *buf, size_t count) ssize_t os_read_no_block(int fd, void *buf, size_t count) { const int flags = fcntl(fd, F_GETFL, 0); + const char dummy[] = "aaa<C3><A4><C3><B6>cc";
fcntl(fd, F_SETFL, flags | O_NONBLOCK); return os_read(fd, buf, count);
called with `tools/buildman/buildman -P sandbox` results in stderr:
arch/sandbox/cpu/os.c: In function ?os_read_no_block?: arch/sandbox/cpu/os.c:43:13: warning: unused variable ?dummy? [-Wunused-variable] const char dummy[] = "aaa??cc"; ^
Daniel Schwierzeck (2): buildman: disable localized and unicode output of all build tools buildman: properly translate strings for log and err files to ASCII
tools/buildman/builderthread.py | 6 ++++-- tools/buildman/toolchain.py | 6 +++++- 2 files changed, 9 insertions(+), 3 deletions(-)
Build tools like Make, gcc or binutils support localized output or unicode encoded output dependent on the default system locale. This is not useful for buildman, where we want reproducible warning or error messages or where the output of binutils is further processed.
Signed-off-by: Daniel Schwierzeck daniel.schwierzeck@gmail.com ---
tools/buildman/toolchain.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
diff --git a/tools/buildman/toolchain.py b/tools/buildman/toolchain.py index 5cf97ac814..2076323d5d 100644 --- a/tools/buildman/toolchain.py +++ b/tools/buildman/toolchain.py @@ -144,7 +144,9 @@ class Toolchain: """Returns an environment for using the toolchain.
Thie takes the current environment and adds CROSS_COMPILE so that - the tool chain will operate correctly. + the tool chain will operate correctly. This also disables localized + output and possibly unicode encoded output of all build tools by + adding LC_ALL=C.
Args: full_path: Return the full path in CROSS_COMPILE and don't set @@ -159,6 +161,8 @@ class Toolchain: env['CROSS_COMPILE'] = wrapper + self.cross env['PATH'] = self.path + ':' + env['PATH']
+ env['LC_ALL'] = 'C' + return env
Build tools like Make, gcc or binutils support localized output or unicode encoded output dependent on the default system locale. This is not useful for buildman, where we want reproducible warning or error messages or where the output of binutils is further processed.
Signed-off-by: Daniel Schwierzeck daniel.schwierzeck@gmail.com ---
tools/buildman/toolchain.py | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
Applied to u-boot-dm, thanks!
The build output can still produce unicode encoded output. But in the buildman's log and err files we only want plain ASCII characters.
To handle all situations with unicode and non-unicode output, encode the stdout and stderr strings to UTF-8 and afterwards to ASCII with replacing all special characters.
Signed-off-by: Daniel Schwierzeck daniel.schwierzeck@gmail.com
---
tools/buildman/builderthread.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
diff --git a/tools/buildman/builderthread.py b/tools/buildman/builderthread.py index acaf5007f5..9e8ca80c5b 100644 --- a/tools/buildman/builderthread.py +++ b/tools/buildman/builderthread.py @@ -280,13 +280,15 @@ class BuilderThread(threading.Thread): outfile = os.path.join(build_dir, 'log') with open(outfile, 'w') as fd: if result.stdout: - fd.write(result.stdout.encode('latin-1', 'ignore')) + # We don't want unicode characters in log files + fd.write(result.stdout.decode('UTF-8').encode('ASCII', 'replace'))
errfile = self.builder.GetErrFile(result.commit_upto, result.brd.target) if result.stderr: with open(errfile, 'w') as fd: - fd.write(result.stderr.encode('latin-1', 'ignore')) + # We don't want unicode characters in log files + fd.write(result.stderr.decode('UTF-8').encode('ASCII', 'replace')) elif os.path.exists(errfile): os.remove(errfile)
The build output can still produce unicode encoded output. But in the buildman's log and err files we only want plain ASCII characters.
To handle all situations with unicode and non-unicode output, encode the stdout and stderr strings to UTF-8 and afterwards to ASCII with replacing all special characters.
Signed-off-by: Daniel Schwierzeck daniel.schwierzeck@gmail.com
---
tools/buildman/builderthread.py | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
Applied to u-boot-dm, thanks!
participants (4)
-
Daniel Schwierzeck -
Simon Glass -
sjg＠google.com -
Tom Rini