[U-Boot] [PATCH 1/2] tools: moveconfig: support wildcards in --defconfigs file
Supporting shell-style wildcards for the --defconfigs option will be useful to run the moveconfig tool against a specific platform. For example, "uniphier*" in the file passed by --defconfigs option will be expanded to defconfig files that start with "uniphier". This is easier than listing out all the UniPhier defconfig files.
Signed-off-by: Masahiro Yamada yamada.masahiro@socionext.com ---
tools/moveconfig.py | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py index 87e2bb2..67cf5f0 100755 --- a/tools/moveconfig.py +++ b/tools/moveconfig.py @@ -127,7 +127,8 @@ Available options standard commit message is used which may need to be edited.
-d, --defconfigs - Specify a file containing a list of defconfigs to move + Specify a file containing a list of defconfigs to move. The defconfig + files can be given with shell-style wildcards.
-n, --dry-run Perform a trial run that does not make any changes. It is useful to @@ -180,6 +181,7 @@ import copy import difflib import filecmp import fnmatch +import glob import multiprocessing import optparse import os @@ -284,6 +286,17 @@ def get_make_cmd(): sys.exit('GNU Make not found') return ret[0].rstrip()
+def get_matched_defconfigs(defconfigs_file): + """Get all the defconfig files that match the patterns in a file.""" + # use a set rather than a list in order to avoid double-counting + defconfigs = set() + for line in open(defconfigs_file): + line = os.path.join('configs', line.strip()) + defconfigs |= set(glob.glob(line)) + defconfigs |= set(glob.glob(line + '_defconfig')) + + return [ defconfig[len('configs') + 1:] for defconfig in defconfigs ] + def get_all_defconfigs(): """Get all the defconfig files under the configs/ directory.""" defconfigs = [] @@ -1204,13 +1217,7 @@ def move_config(configs, options): reference_src_dir = None
if options.defconfigs: - defconfigs = [line.strip() for line in open(options.defconfigs)] - for i, defconfig in enumerate(defconfigs): - if not defconfig.endswith('_defconfig'): - defconfigs[i] = defconfig + '_defconfig' - if not os.path.exists(os.path.join('configs', defconfigs[i])): - sys.exit('%s - defconfig does not exist. Stopping.' % - defconfigs[i]) + defconfigs = get_matched_defconfigs(options.defconfigs) else: defconfigs = get_all_defconfigs()
This will make the log more readable. The log is not deterministic in case of parallel processing, though.
Signed-off-by: Masahiro Yamada yamada.masahiro@socionext.com ---
tools/moveconfig.py | 2 ++ 1 file changed, 2 insertions(+)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py index 67cf5f0..49cb1e2 100755 --- a/tools/moveconfig.py +++ b/tools/moveconfig.py @@ -1221,6 +1221,8 @@ def move_config(configs, options): else: defconfigs = get_all_defconfigs()
+ defconfigs.sort() + progress = Progress(len(defconfigs)) slots = Slots(configs, options, progress, reference_src_dir)
On Mon, Oct 17, 2016 at 02:17:23AM +0900, Masahiro Yamada wrote:
This will make the log more readable. The log is not deterministic in case of parallel processing, though.
Signed-off-by: Masahiro Yamada yamada.masahiro@socionext.com
I'm honestly ambivalent about this since single threaded is basically never the case, right?
Hi Tom,
2016-10-17 5:12 GMT+09:00 Tom Rini trini@konsulko.com:
On Mon, Oct 17, 2016 at 02:17:23AM +0900, Masahiro Yamada wrote:
This will make the log more readable. The log is not deterministic in case of parallel processing, though.
Signed-off-by: Masahiro Yamada yamada.masahiro@socionext.com
I'm honestly ambivalent about this since single threaded is basically never the case, right?
Right. If you want to diff two logs, you need to pass -j1 option.
But, my main motivation is not that. You will be able to find the log of boards you are interested in.
For example, when some boards are listed in "suspicious results" at the end of conversion, you may want to check the log to see what happened. If the log is sorted *almost* alphabetically, it will be easier to jump there.
On Wed, Oct 19, 2016 at 01:23:24AM +0900, Masahiro Yamada wrote:
Hi Tom,
2016-10-17 5:12 GMT+09:00 Tom Rini trini@konsulko.com:
On Mon, Oct 17, 2016 at 02:17:23AM +0900, Masahiro Yamada wrote:
This will make the log more readable. The log is not deterministic in case of parallel processing, though.
Signed-off-by: Masahiro Yamada yamada.masahiro@socionext.com
I'm honestly ambivalent about this since single threaded is basically never the case, right?
Right. If you want to diff two logs, you need to pass -j1 option.
But, my main motivation is not that. You will be able to find the log of boards you are interested in.
For example, when some boards are listed in "suspicious results" at the end of conversion, you may want to check the log to see what happened. If the log is sorted *almost* alphabetically, it will be easier to jump there.
I'm still ambivalent. I 'less' the log and search doesn't care about order. And I try and fix one broken board at a time since it's often the case that N boards will have the same form of thinko.
2016-10-19 4:25 GMT+09:00 Tom Rini trini@konsulko.com:
On Wed, Oct 19, 2016 at 01:23:24AM +0900, Masahiro Yamada wrote:
Hi Tom,
2016-10-17 5:12 GMT+09:00 Tom Rini trini@konsulko.com:
On Mon, Oct 17, 2016 at 02:17:23AM +0900, Masahiro Yamada wrote:
This will make the log more readable. The log is not deterministic in case of parallel processing, though.
Signed-off-by: Masahiro Yamada yamada.masahiro@socionext.com
I'm honestly ambivalent about this since single threaded is basically never the case, right?
Right. If you want to diff two logs, you need to pass -j1 option.
But, my main motivation is not that. You will be able to find the log of boards you are interested in.
For example, when some boards are listed in "suspicious results" at the end of conversion, you may want to check the log to see what happened. If the log is sorted *almost* alphabetically, it will be easier to jump there.
I'm still ambivalent. I 'less' the log and search doesn't care about order. And I try and fix one broken board at a time since it's often the case that N boards will have the same form of thinko.
OK, fine.
Please drop this patch.
I sent v2 only for 1/2.
On Mon, Oct 17, 2016 at 02:17:22AM +0900, Masahiro Yamada wrote:
Supporting shell-style wildcards for the --defconfigs option will be useful to run the moveconfig tool against a specific platform. For example, "uniphier*" in the file passed by --defconfigs option will be expanded to defconfig files that start with "uniphier". This is easier than listing out all the UniPhier defconfig files.
Signed-off-by: Masahiro Yamada yamada.masahiro@socionext.com
tools/moveconfig.py | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py index 87e2bb2..67cf5f0 100755 --- a/tools/moveconfig.py +++ b/tools/moveconfig.py @@ -127,7 +127,8 @@ Available options standard commit message is used which may need to be edited.
-d, --defconfigs
- Specify a file containing a list of defconfigs to move
Specify a file containing a list of defconfigs to move. The defconfig
files can be given with shell-style wildcards.
-n, --dry-run Perform a trial run that does not make any changes. It is useful to
@@ -180,6 +181,7 @@ import copy import difflib import filecmp import fnmatch +import glob import multiprocessing import optparse import os @@ -284,6 +286,17 @@ def get_make_cmd(): sys.exit('GNU Make not found') return ret[0].rstrip()
+def get_matched_defconfigs(defconfigs_file):
- """Get all the defconfig files that match the patterns in a file."""
- # use a set rather than a list in order to avoid double-counting
- defconfigs = set()
- for line in open(defconfigs_file):
line = os.path.join('configs', line.strip())defconfigs |= set(glob.glob(line))defconfigs |= set(glob.glob(line + '_defconfig'))- return [ defconfig[len('configs') + 1:] for defconfig in defconfigs ]
def get_all_defconfigs(): """Get all the defconfig files under the configs/ directory.""" defconfigs = [] @@ -1204,13 +1217,7 @@ def move_config(configs, options): reference_src_dir = None
if options.defconfigs:
defconfigs = [line.strip() for line in open(options.defconfigs)]for i, defconfig in enumerate(defconfigs):if not defconfig.endswith('_defconfig'):defconfigs[i] = defconfig + '_defconfig'if not os.path.exists(os.path.join('configs', defconfigs[i])):sys.exit('%s - defconfig does not exist. Stopping.' %defconfigs[i])
defconfigs = get_matched_defconfigs(options.defconfigs)
I don't see error handling in the case of files (and now globs) not existing now, is this just handled by other parts of the code? Thanks!
2016-10-17 5:15 GMT+09:00 Tom Rini trini@konsulko.com:
On Mon, Oct 17, 2016 at 02:17:22AM +0900, Masahiro Yamada wrote:
Supporting shell-style wildcards for the --defconfigs option will be useful to run the moveconfig tool against a specific platform. For example, "uniphier*" in the file passed by --defconfigs option will be expanded to defconfig files that start with "uniphier". This is easier than listing out all the UniPhier defconfig files.
Signed-off-by: Masahiro Yamada yamada.masahiro@socionext.com
tools/moveconfig.py | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py index 87e2bb2..67cf5f0 100755 --- a/tools/moveconfig.py +++ b/tools/moveconfig.py @@ -127,7 +127,8 @@ Available options standard commit message is used which may need to be edited.
-d, --defconfigs
- Specify a file containing a list of defconfigs to move
Specify a file containing a list of defconfigs to move. The defconfig
files can be given with shell-style wildcards.
-n, --dry-run Perform a trial run that does not make any changes. It is useful to
@@ -180,6 +181,7 @@ import copy import difflib import filecmp import fnmatch +import glob import multiprocessing import optparse import os @@ -284,6 +286,17 @@ def get_make_cmd(): sys.exit('GNU Make not found') return ret[0].rstrip()
+def get_matched_defconfigs(defconfigs_file):
- """Get all the defconfig files that match the patterns in a file."""
- # use a set rather than a list in order to avoid double-counting
- defconfigs = set()
- for line in open(defconfigs_file):
line = os.path.join('configs', line.strip())defconfigs |= set(glob.glob(line))defconfigs |= set(glob.glob(line + '_defconfig'))- return [ defconfig[len('configs') + 1:] for defconfig in defconfigs ]
def get_all_defconfigs(): """Get all the defconfig files under the configs/ directory.""" defconfigs = [] @@ -1204,13 +1217,7 @@ def move_config(configs, options): reference_src_dir = None
if options.defconfigs:
defconfigs = [line.strip() for line in open(options.defconfigs)]for i, defconfig in enumerate(defconfigs):if not defconfig.endswith('_defconfig'):defconfigs[i] = defconfig + '_defconfig'if not os.path.exists(os.path.join('configs', defconfigs[i])):sys.exit('%s - defconfig does not exist. Stopping.' %defconfigs[i])
defconfigs = get_matched_defconfigs(options.defconfigs)I don't see error handling in the case of files (and now globs) not existing now, is this just handled by other parts of the code? Thanks!
No, it will be removed.
With this patch, each line will be a pattern that may match multiple defconfigs, or that may match nothing.
Is it important to check users' possible typos?
On Wed, Oct 19, 2016 at 01:14:58AM +0900, Masahiro Yamada wrote:
2016-10-17 5:15 GMT+09:00 Tom Rini trini@konsulko.com:
On Mon, Oct 17, 2016 at 02:17:22AM +0900, Masahiro Yamada wrote:
Supporting shell-style wildcards for the --defconfigs option will be useful to run the moveconfig tool against a specific platform. For example, "uniphier*" in the file passed by --defconfigs option will be expanded to defconfig files that start with "uniphier". This is easier than listing out all the UniPhier defconfig files.
Signed-off-by: Masahiro Yamada yamada.masahiro@socionext.com
tools/moveconfig.py | 23 +++++++++++++++-------- 1 file changed, 15 insertions(+), 8 deletions(-)
diff --git a/tools/moveconfig.py b/tools/moveconfig.py index 87e2bb2..67cf5f0 100755 --- a/tools/moveconfig.py +++ b/tools/moveconfig.py @@ -127,7 +127,8 @@ Available options standard commit message is used which may need to be edited.
-d, --defconfigs
- Specify a file containing a list of defconfigs to move
Specify a file containing a list of defconfigs to move. The defconfig
files can be given with shell-style wildcards.
-n, --dry-run Perform a trial run that does not make any changes. It is useful to
@@ -180,6 +181,7 @@ import copy import difflib import filecmp import fnmatch +import glob import multiprocessing import optparse import os @@ -284,6 +286,17 @@ def get_make_cmd(): sys.exit('GNU Make not found') return ret[0].rstrip()
+def get_matched_defconfigs(defconfigs_file):
- """Get all the defconfig files that match the patterns in a file."""
- # use a set rather than a list in order to avoid double-counting
- defconfigs = set()
- for line in open(defconfigs_file):
line = os.path.join('configs', line.strip())defconfigs |= set(glob.glob(line))defconfigs |= set(glob.glob(line + '_defconfig'))- return [ defconfig[len('configs') + 1:] for defconfig in defconfigs ]
def get_all_defconfigs(): """Get all the defconfig files under the configs/ directory.""" defconfigs = [] @@ -1204,13 +1217,7 @@ def move_config(configs, options): reference_src_dir = None
if options.defconfigs:
defconfigs = [line.strip() for line in open(options.defconfigs)]for i, defconfig in enumerate(defconfigs):if not defconfig.endswith('_defconfig'):defconfigs[i] = defconfig + '_defconfig'if not os.path.exists(os.path.join('configs', defconfigs[i])):sys.exit('%s - defconfig does not exist. Stopping.' %defconfigs[i])
defconfigs = get_matched_defconfigs(options.defconfigs)I don't see error handling in the case of files (and now globs) not existing now, is this just handled by other parts of the code? Thanks!
No, it will be removed.
With this patch, each line will be a pattern that may match multiple defconfigs, or that may match nothing.
Ah, OK.
Is it important to check users' possible typos?
Well, will a list of invalid config names say something useful, or fail silently or fail in unexpected ways?
On Tue, Oct 18, 2016 at 2:23 PM, Tom Rini trini@konsulko.com wrote:
On Wed, Oct 19, 2016 at 01:14:58AM +0900, Masahiro Yamada wrote:
2016-10-17 5:15 GMT+09:00 Tom Rini trini@konsulko.com:
On Mon, Oct 17, 2016 at 02:17:22AM +0900, Masahiro Yamada wrote: I don't see error handling in the case of files (and now globs) not existing now, is this just handled by other parts of the code? Thanks!
No, it will be removed.
With this patch, each line will be a pattern that may match multiple defconfigs, or that may match nothing.
Ah, OK.
Is it important to check users' possible typos?
Well, will a list of invalid config names say something useful, or fail silently or fail in unexpected ways?
It seems like it would be good to warn if a glob (or otherwise) results in an empty set, because that it probably not what was intended.
-Joe
participants (3)
-
Joe Hershberger -
Masahiro Yamada -
Tom Rini