[U-Boot] [PATCH] libfdt: fix build with Python 3
From: Stefan Agner stefan.agner@toradex.com
For some reason Python 3 seems to think it does not need to build the library. Using the --force parameter makes sure that the library gets built always. This is especially important since we move the library in the next step of the Makefile, hence forcing a rebuild every time the higher level Makefile triggers a rebuild is required to make sure the library is always there.
Signed-off-by: Stefan Agner stefan.agner@toradex.com ---
lib/libfdt/setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/libfdt/setup.py b/lib/libfdt/setup.py index 62e7bcc1ac..845a0c2b10 100644 --- a/lib/libfdt/setup.py +++ b/lib/libfdt/setup.py @@ -27,7 +27,7 @@ libfdt_module = Extension( extra_compile_args = cflags )
-sys.argv = [progname, '--quiet', 'build_ext', '--inplace'] +sys.argv = [progname, '--quiet', 'build_ext', '--inplace', '--force']
setup (name = 'libfdt', version = '0.1',
On 29 March 2017 at 13:41, Stefan Agner stefan@agner.ch wrote:
From: Stefan Agner stefan.agner@toradex.com
For some reason Python 3 seems to think it does not need to build the library. Using the --force parameter makes sure that the library gets built always. This is especially important since we move the library in the next step of the Makefile, hence forcing a rebuild every time the higher level Makefile triggers a rebuild is required to make sure the library is always there.
Signed-off-by: Stefan Agner stefan.agner@toradex.com
lib/libfdt/setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Acked-by: Simon Glass sjg@chromium.org
FYI the Python libfdt bindings were accepted upstream. I just need to get the proper setup.py script up there now. Then we can replace all of this. Hopefully after this release.
Regards, Simon
On 2017-03-31 21:23, Simon Glass wrote:
On 29 March 2017 at 13:41, Stefan Agner stefan@agner.ch wrote:
From: Stefan Agner stefan.agner@toradex.com
For some reason Python 3 seems to think it does not need to build the library. Using the --force parameter makes sure that the library gets built always. This is especially important since we move the library in the next step of the Makefile, hence forcing a rebuild every time the higher level Makefile triggers a rebuild is required to make sure the library is always there.
Signed-off-by: Stefan Agner stefan.agner@toradex.com
lib/libfdt/setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Acked-by: Simon Glass sjg@chromium.org
Simon, Tom, not sure, through what tree will this get merged? Just wanted to note that I think this is ready to be merged, the discussion with Stefano which branches off this thread is somewhat unrelated and should not really hold us back from merging this.
-- Stefan
FYI the Python libfdt bindings were accepted upstream. I just need to get the proper setup.py script up there now. Then we can replace all of this. Hopefully after this release.
Regards, Simon
Hi Stefan,
On 5 April 2017 at 12:08, Stefan Agner stefan@agner.ch wrote:
On 2017-03-31 21:23, Simon Glass wrote:
On 29 March 2017 at 13:41, Stefan Agner stefan@agner.ch wrote:
From: Stefan Agner stefan.agner@toradex.com
For some reason Python 3 seems to think it does not need to build the library. Using the --force parameter makes sure that the library gets built always. This is especially important since we move the library in the next step of the Makefile, hence forcing a rebuild every time the higher level Makefile triggers a rebuild is required to make sure the library is always there.
Signed-off-by: Stefan Agner stefan.agner@toradex.com
lib/libfdt/setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Acked-by: Simon Glass sjg@chromium.org
Simon, Tom, not sure, through what tree will this get merged? Just wanted to note that I think this is ready to be merged, the discussion with Stefano which branches off this thread is somewhat unrelated and should not really hold us back from merging this.
OK I've just added it to my queue. Tom, feel free to pick if up if you like.
-- Stefan
FYI the Python libfdt bindings were accepted upstream. I just need to get the proper setup.py script up there now. Then we can replace all of this. Hopefully after this release.
Regards, Simon
Regards, Simon
On 5 April 2017 at 12:11, Simon Glass sjg@chromium.org wrote:
Hi Stefan,
On 5 April 2017 at 12:08, Stefan Agner stefan@agner.ch wrote:
On 2017-03-31 21:23, Simon Glass wrote:
On 29 March 2017 at 13:41, Stefan Agner stefan@agner.ch wrote:
From: Stefan Agner stefan.agner@toradex.com
For some reason Python 3 seems to think it does not need to build the library. Using the --force parameter makes sure that the library gets built always. This is especially important since we move the library in the next step of the Makefile, hence forcing a rebuild every time the higher level Makefile triggers a rebuild is required to make sure the library is always there.
Signed-off-by: Stefan Agner stefan.agner@toradex.com
lib/libfdt/setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Acked-by: Simon Glass sjg@chromium.org
Simon, Tom, not sure, through what tree will this get merged? Just wanted to note that I think this is ready to be merged, the discussion with Stefano which branches off this thread is somewhat unrelated and should not really hold us back from merging this.
OK I've just added it to my queue. Tom, feel free to pick if up if you like.
Applied to u-boot-fdt, thanks!
Sorry, I got stuck in Android-land for a while, but this does not fix the build on a fresh build box with python 3.4 as default system python. Switching to python 2.7 does build u-boot successfully. Note this is with u-boot v2017.05 (which does have --force in setup.py) and DE1 SoC config, which does not have any libfdt stuff enabled in defconfig.
Also note that "fresh" means "freshly built itself" including all the bits 'n pieces so the build environment is definitely not fubar'd.
Portage 2.3.6 (python 2.7.13-final-0, hardened/linux/amd64, gcc-5.4.0, glibc-2.24-r3, 4.9.16-gentoo x86_64) ================================================================= System uname: Linux-4.9.16-gentoo-x86_64-AMD_FX-tm-6300_Six-Core_Processor-with-gentoo-2.4.1 KiB Mem: 8170760 total, 2746504 free KiB Swap: 20971512 total, 20971384 free Timestamp of repository gentoo: Fri, 23 Jun 2017 05:30:01 +0000 sh bash 4.3_p48-r1 ld GNU ld (Gentoo 2.28 p1.2) 2.28 app-shells/bash: 4.3_p48-r1::gentoo dev-lang/perl: 5.24.1-r2::gentoo dev-lang/python: 2.7.13::gentoo, 3.4.6::gentoo dev-util/cmake: 3.8.2::gentoo dev-util/pkgconfig: 0.29.2::gentoo sys-apps/baselayout: 2.4.1::gentoo sys-apps/openrc: 0.27.2::gentoo sys-apps/sandbox: 2.10-r4::gentoo sys-devel/autoconf: 2.13::gentoo, 2.69-r3::gentoo sys-devel/automake: 1.11.6-r2::gentoo, 1.13.4-r1::gentoo, 1.15.1::gentoo sys-devel/binutils: 2.28-r2::gentoo sys-devel/gcc: 5.4.0-r3::gentoo, 6.3.0::gentoo sys-devel/gcc-config: 1.8-r1::gentoo sys-devel/libtool: 2.4.6-r4::gentoo sys-devel/make: 4.2.1-r1::gentoo sys-kernel/linux-headers: 4.10::gentoo (virtual/os-headers) sys-libs/glibc: 2.24-r3::gentoo Repositories:
gentoo location: /usr/portage sync-type: rsync sync-uri: rsync://rsync.gentoo.org/gentoo-portage priority: -1000
crossdev location: /usr/local/armv7-a masters: gentoo priority: 0
ACCEPT_KEYWORDS="amd64 ~amd64" ACCEPT_LICENSE="* -@EULA @GPL-COMPATIBLE @OSI-APPROVED @EULA dlj-1.1 skype-eula googleearth AdobeFlash-10.1" CBUILD="x86_64-pc-linux-gnu" CFLAGS="-march=athlon64 -mtune=bdver1 -O2 -pipe -flto=6 -ftree-vectorize -floop-interchange -ftree-loop-distribution -floop-strip-mine -floop-block" CHOST="x86_64-pc-linux-gnu" CONFIG_PROTECT="/etc /usr/share/gnupg/qualified.txt" CONFIG_PROTECT_MASK="/etc/ca-certificates.conf /etc/dconf /etc/env.d /etc/fonts/fonts.conf /etc/gconf /etc/gentoo-release /etc/revdep-rebuild /etc/sandbox.d /etc/terminfo /etc/texmf/language.dat.d /etc/texmf/language.def.d /etc/texmf/updmap.d /etc/texmf/web2c" CXXFLAGS="-march=athlon64 -mtune=bdver1 -O2 -pipe -flto=6 -ftree-vectorize -floop-interchange -ftree-loop-distribution -floop-strip-mine -floop-block" DISTDIR="/usr/portage/distfiles" FCFLAGS="-O2 -pipe" FEATURES="assume-digests binpkg-logs config-protect-if-modified distlocks ebuild-locks fixlafiles merge-sync multilib-strict news parallel-fetch preserve-libs protect-owned sandbox sfperms strict unknown-features-warn unmerge-logs unmerge-orphans userfetch userpriv usersandbox usersync xattr" FFLAGS="-O2 -pipe" GENTOO_MIRRORS="http://distfiles.gentoo.org" LANG="en_US.utf8" LDFLAGS="-march=athlon64 -mtune=bdver1 -O2 -pipe -flto=6 -ftree-vectorize -floop-interchange -ftree-loop-distribution -floop-strip-mine -floop-block -fuse-linker-plugin" MAKEOPTS="-j7" PKGDIR="/usr/portage/packages" PORTAGE_BZIP2_COMMAND="/bin/bzip2" PORTAGE_CONFIGROOT="/" PORTAGE_RSYNC_OPTS="--recursive --links --safe-links --perms --times --omit-dir-times --compress --force --whole-file --delete --stats --human-readable --timeout=180 --exclude=/distfiles --exclude=/local --exclude=/packages --exclude=/.git" PORTAGE_TMPDIR="/var/tmp" USE="X a52 aac acl acpi alsa amd64 amr apache2 apng aspell avahi berkdb bluetooth bzip2 bzlib cairo caps cdda cdr cli consolekit cracklib crypt cups curl cxx dbus diff djvu dnotify dri drm dts dvd eds enchant exif expat fam firefox3 flac fortran gbm gd gdbm gif git glade glamor gmp gnome-keyring gnome-online-accounts gnutls graphite gstreamer gtk gtk3 hardened iconv icu imagemagick inotify introspection ipv6 jpeg json justify lcms ldap libav libnotify libsecret lua lz4 mng modules mp3 multilib nautilus ncurses networking networkmanager nfs nls nptl nptlonly nsplugin ogg opengl openmp orc pam pango pax_kernel pcre pda pdf pic pie pkcs11 png policykit postgres pulseaudio python qt5 readline rtc samba sasl sdl seamonkey seccomp session sip slp snmp spell sqlite ssh ssl ssp startup-notification svg system-cairo system-harfbuzz system-icu system-jpeg system-libvpx system-sqlite tcl tcpd text theora threads thunar tiff tk truetype udev udisks unicode upower urandom usb v4l vaapi vala vdpau vorbis wayland webp wheel wifi x264 xa xattr xcb xkb xml xtpax xvmc zeroconf zlib" ABI_X86="64 32" ALSA_CARDS="emu10k1 hda-intel usb-audio" APACHE2_MODULES="actions alias auth_digest auth_basic authn_alias access_compat authn_core authn_dbd authz_core cache_disk cern_meta cgi cgid charset_lite authn_anon authn_dbm authn_default authn_file authz_dbm authz_default authz_groupfile authz_host authz_owner authz_user autoindex cache dav dav_fs dav_lock deflate dir disk_cache env expires ext_filter file_cache filter headers include info log_config logio mem_cache mime mime_magic negotiation proxy proxy_connect proxy_http rewrite setenvif speling status unique_id userdir usertrack vhost_alias unixd socache_shmcb" CALLIGRA_FEATURES="kexi words flow plan sheets stage tables krita karbon braindump author" CAMERAS="adc65 agfa_cl20 aox ax203 barbie canon casio_qv clicksmart310 digigr8 digita dimagev dimera3500 directory enigma13 fuji gsmart300 hp215 iclick jamcam jd11 jl2005a jl2005c kodak_dc120 kodak_dc210 kodak_dc240 kodak_dc3200 kodak_ez200 konica konica_qm150 largan lg_gsm mars mustek panasonic_coolshot panasonic_l859 panasonic_dc1000 panasonic_dc1580 pccam300 pccam600 pentax polaroid_pdc320 polaroid_pdc640 polaroid_pdc700 ptp2 ricoh ricoh_g3 samsung sierra sipix_blink2 sipix_web2 smal sonix sony_dscf1 sony_dscf55 soundvision spca50x sq905 st2205 stv0674 stv0680 sx330z toshiba_pdrm11 topfield tp6801" COLLECTD_PLUGINS="df interface irq load memory rrdtool swap syslog" CPU_FLAGS_X86="aes avx f16c fma3 fma4 mmx mmxext popcnt sse sse2 sse3 sse4_1 sse4_2 sse4a ssse3 xop" ELIBC="glibc" GPSD_PROTOCOLS="ashtech aivdm earthmate evermore fv18 garmin garmintxt gpsclock isync itrax mtk3301 nmea ntrip navcom oceanserver oldstyle oncore rtcm104v2 rtcm104v3 sirf skytraq superstar2 timing tsip tripmate tnt ublox ubx" GRUB_PLATFORMS="efi-32 efi-64 pc" INPUT_DEVICES="keyboard mouse joystick evdev" KERNEL="linux" LCD_DEVICES="bayrad cfontz cfontz633 glk hd44780 lb216 lcdm001 mtxorb ncurses text" LIBREOFFICE_EXTENSIONS="presenter-minimizer wiki-publisher" LINGUAS="en_US en" NETBEANS_MODULES="apisupport java javafx profiler websvccommon cnd dlight ergonomics groovy mobility webcommon" OFFICE_IMPLEMENTATION="libreoffice" PHP_TARGETS="php5-6" PYTHON_SINGLE_TARGET="python3_4" PYTHON_TARGETS="python2_7 python3_4" QEMU_SOFTMMU_TARGETS="arm i386 mipsel ppc ppc64 x86_64" QEMU_USER_TARGETS="arm armeb i386 mipsel ppc ppc64 x86_64" RUBY_TARGETS="ruby21 ruby22" SANE_BACKENDS="net" USERLAND="GNU" VIDEO_CARDS="fbdev radeon r600" XTABLES_ADDONS="quota2 psd pknock lscan length2 ipv4options ipset ipp2p iface geoip fuzzy condition tee tarpit sysrq steal rawnat logmark ipmark dhcpmac delude chaos account" Unset: CC, CPPFLAGS, CTARGET, CXX, EMERGE_DEFAULT_OPTS, INSTALL_MASK, LC_ALL, PORTAGE_BUNZIP2_COMMAND, PORTAGE_COMPRESS, PORTAGE_COMPRESS_FLAGS, PORTAGE_RSYNC_EXTRA_OPTS, USE_PYTHON
On Mon, Apr 10, 2017 at 1:52 PM, Simon Glass sjg@chromium.org wrote:
On 5 April 2017 at 12:11, Simon Glass sjg@chromium.org wrote:
Hi Stefan,
On 5 April 2017 at 12:08, Stefan Agner stefan@agner.ch wrote:
On 2017-03-31 21:23, Simon Glass wrote:
On 29 March 2017 at 13:41, Stefan Agner stefan@agner.ch wrote:
From: Stefan Agner stefan.agner@toradex.com
For some reason Python 3 seems to think it does not need to build the library. Using the --force parameter makes sure that the library gets built always. This is especially important since we move the library in the next step of the Makefile, hence forcing a rebuild every time the higher level Makefile triggers a rebuild is required to make sure the library is always there.
Signed-off-by: Stefan Agner stefan.agner@toradex.com
lib/libfdt/setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
Acked-by: Simon Glass sjg@chromium.org
Simon, Tom, not sure, through what tree will this get merged? Just wanted to note that I think this is ready to be merged, the discussion with Stefano which branches off this thread is somewhat unrelated and should not really hold us back from merging this.
OK I've just added it to my queue. Tom, feel free to pick if up if you
like.
Applied to u-boot-fdt, thanks! _______________________________________________ U-Boot mailing list U-Boot@lists.denx.de https://lists.denx.de/listinfo/u-boot
Hi Stefan,
On 29/03/2017 21:41, Stefan Agner wrote:
From: Stefan Agner stefan.agner@toradex.com
For some reason Python 3 seems to think it does not need to build the library. Using the --force parameter makes sure that the library gets built always. This is especially important since we move the library in the next step of the Makefile, hence forcing a rebuild every time the higher level Makefile triggers a rebuild is required to make sure the library is always there.
Signed-off-by: Stefan Agner stefan.agner@toradex.com
lib/libfdt/setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/libfdt/setup.py b/lib/libfdt/setup.py index 62e7bcc1ac..845a0c2b10 100644 --- a/lib/libfdt/setup.py +++ b/lib/libfdt/setup.py @@ -27,7 +27,7 @@ libfdt_module = Extension( extra_compile_args = cflags )
-sys.argv = [progname, '--quiet', 'build_ext', '--inplace'] +sys.argv = [progname, '--quiet', 'build_ext', '--inplace', '--force']
setup (name = 'libfdt', version = '0.1',
It looks to me we have a genaral problem about which python is called. In fact, I am getting another issue when I build 2017.03 in Yocto. The headers like "Python.h" are taken from the distro, and not from the Yocto's buildsystem (nativepython), that makes them available.
It can be easy tested removing the package "libpython" from the host. A "bitbake u-boot", when u-boot 2017.03 is available, fails because no headers are found - but they are all available in Yocto's toolchain.
We set the "PYTHON" variable in the makefile, but this is not used consistent, as far as I see, because in some files as setup.py, "/usr/bin/env python" points to the python installed on distro - adding "PYTHON=nativepython" in u-boot recipe does not work.
Best regards, Stefano
Hi Stefano,
On 2017-04-03 03:30, Stefano Babic wrote:
Hi Stefan,
On 29/03/2017 21:41, Stefan Agner wrote:
From: Stefan Agner stefan.agner@toradex.com
For some reason Python 3 seems to think it does not need to build the library. Using the --force parameter makes sure that the library gets built always. This is especially important since we move the library in the next step of the Makefile, hence forcing a rebuild every time the higher level Makefile triggers a rebuild is required to make sure the library is always there.
Signed-off-by: Stefan Agner stefan.agner@toradex.com
lib/libfdt/setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/libfdt/setup.py b/lib/libfdt/setup.py index 62e7bcc1ac..845a0c2b10 100644 --- a/lib/libfdt/setup.py +++ b/lib/libfdt/setup.py @@ -27,7 +27,7 @@ libfdt_module = Extension( extra_compile_args = cflags )
-sys.argv = [progname, '--quiet', 'build_ext', '--inplace'] +sys.argv = [progname, '--quiet', 'build_ext', '--inplace', '--force']
setup (name = 'libfdt', version = '0.1',
It looks to me we have a genaral problem about which python is called. In fact, I am getting another issue when I build 2017.03 in Yocto. The headers like "Python.h" are taken from the distro, and not from the Yocto's buildsystem (nativepython), that makes them available.
It can be easy tested removing the package "libpython" from the host. A "bitbake u-boot", when u-boot 2017.03 is available, fails because no headers are found - but they are all available in Yocto's toolchain.
We set the "PYTHON" variable in the makefile, but this is not used consistent, as far as I see, because in some files as setup.py, "/usr/bin/env python" points to the python installed on distro - adding
Not sure whether that conclusion is correct. As far as I can see we call setup.py in tools/Makefile using python explicitly:
tools/_libfdt.so: $(patsubst %.o,%.c,$(LIBFDT_OBJS)) tools/libfdt_wrap.c LDFLAGS="$(HOSTLDFLAGS)" CFLAGS= python $(srctree)/lib/libfdt/setup.py \ "$(_hostc_flags)" $^ mv _libfdt.so $@
So we are not using /usr/bin/env in this case.
Not sure though whether a call to "python" is different in that case from using "/usr/bin/env python"
-- Stefan
Hi Stefan,
On 03/04/2017 18:41, Stefan Agner wrote:
Hi Stefano,
On 2017-04-03 03:30, Stefano Babic wrote:
Hi Stefan,
On 29/03/2017 21:41, Stefan Agner wrote:
From: Stefan Agner stefan.agner@toradex.com
For some reason Python 3 seems to think it does not need to build the library. Using the --force parameter makes sure that the library gets built always. This is especially important since we move the library in the next step of the Makefile, hence forcing a rebuild every time the higher level Makefile triggers a rebuild is required to make sure the library is always there.
Signed-off-by: Stefan Agner stefan.agner@toradex.com
lib/libfdt/setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/libfdt/setup.py b/lib/libfdt/setup.py index 62e7bcc1ac..845a0c2b10 100644 --- a/lib/libfdt/setup.py +++ b/lib/libfdt/setup.py @@ -27,7 +27,7 @@ libfdt_module = Extension( extra_compile_args = cflags )
-sys.argv = [progname, '--quiet', 'build_ext', '--inplace'] +sys.argv = [progname, '--quiet', 'build_ext', '--inplace', '--force']
setup (name = 'libfdt', version = '0.1',
It looks to me we have a genaral problem about which python is called. In fact, I am getting another issue when I build 2017.03 in Yocto. The headers like "Python.h" are taken from the distro, and not from the Yocto's buildsystem (nativepython), that makes them available.
It can be easy tested removing the package "libpython" from the host. A "bitbake u-boot", when u-boot 2017.03 is available, fails because no headers are found - but they are all available in Yocto's toolchain.
We set the "PYTHON" variable in the makefile, but this is not used consistent, as far as I see, because in some files as setup.py, "/usr/bin/env python" points to the python installed on distro - adding
Not sure whether that conclusion is correct.
Quite. I forget to mention...
As far as I can see we call setup.py in tools/Makefile using python explicitly:
Exactly. This makes PYTHON workless.
tools/_libfdt.so: $(patsubst %.o,%.c,$(LIBFDT_OBJS)) tools/libfdt_wrap.c LDFLAGS="$(HOSTLDFLAGS)" CFLAGS= python $(srctree)/lib/libfdt/setup.py \ "$(_hostc_flags)" $^ mv _libfdt.so $@
So we are not using /usr/bin/env in this case.
No, but we are using "python" that can be taken just from PATH and not from the PYTHON variable.
Not sure though whether a call to "python" is different in that case from using "/usr/bin/env python"
I am not sure, too, but I am expecting that shebangs is not evaluated because python is already running. But this breaks when I build in Yocto, because I cannot override with "PYTHON=nativepython" the default interpreter.
Regards, Stefano
On 2017-04-03 10:35, Stefano Babic wrote:
Hi Stefan,
On 03/04/2017 18:41, Stefan Agner wrote:
Hi Stefano,
On 2017-04-03 03:30, Stefano Babic wrote:
Hi Stefan,
On 29/03/2017 21:41, Stefan Agner wrote:
From: Stefan Agner stefan.agner@toradex.com
For some reason Python 3 seems to think it does not need to build the library. Using the --force parameter makes sure that the library gets built always. This is especially important since we move the library in the next step of the Makefile, hence forcing a rebuild every time the higher level Makefile triggers a rebuild is required to make sure the library is always there.
Signed-off-by: Stefan Agner stefan.agner@toradex.com
lib/libfdt/setup.py | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/libfdt/setup.py b/lib/libfdt/setup.py index 62e7bcc1ac..845a0c2b10 100644 --- a/lib/libfdt/setup.py +++ b/lib/libfdt/setup.py @@ -27,7 +27,7 @@ libfdt_module = Extension( extra_compile_args = cflags )
-sys.argv = [progname, '--quiet', 'build_ext', '--inplace'] +sys.argv = [progname, '--quiet', 'build_ext', '--inplace', '--force']
setup (name = 'libfdt', version = '0.1',
It looks to me we have a genaral problem about which python is called. In fact, I am getting another issue when I build 2017.03 in Yocto. The headers like "Python.h" are taken from the distro, and not from the Yocto's buildsystem (nativepython), that makes them available.
It can be easy tested removing the package "libpython" from the host. A "bitbake u-boot", when u-boot 2017.03 is available, fails because no headers are found - but they are all available in Yocto's toolchain.
We set the "PYTHON" variable in the makefile, but this is not used consistent, as far as I see, because in some files as setup.py, "/usr/bin/env python" points to the python installed on distro - adding
Not sure whether that conclusion is correct.
Quite. I forget to mention...
As far as I can see we call setup.py in tools/Makefile using python explicitly:
Exactly. This makes PYTHON workless.
tools/_libfdt.so: $(patsubst %.o,%.c,$(LIBFDT_OBJS)) tools/libfdt_wrap.c LDFLAGS="$(HOSTLDFLAGS)" CFLAGS= python $(srctree)/lib/libfdt/setup.py \ "$(_hostc_flags)" $^ mv _libfdt.so $@
So we are not using /usr/bin/env in this case.
No, but we are using "python" that can be taken just from PATH and not from the PYTHON variable.
Not sure though whether a call to "python" is different in that case from using "/usr/bin/env python"
I am not sure, too, but I am expecting that shebangs is not evaluated because python is already running. But this breaks when I build in
I am expecting that too.
Yocto, because I cannot override with "PYTHON=nativepython" the default interpreter.
But then, I don't expect that "/usr/bin/env python" is looking at PYTHON.
As far as I understand env, it just looks up the current environment to run its COMMAND argument. It is helpful as a shebang, but it does not seem to make any difference to calling "python" directly (since that also just looks up the current environment PATH and executes the first python it finds...
-- Stefan
Hi Stefan,
On 03/04/2017 23:02, Stefan Agner wrote:
But then, I don't expect that "/usr/bin/env python" is looking at PYTHON.
As far as I understand env, it just looks up the current environment to run its COMMAND argument.
Agree, this is also mz understanding.
It is helpful as a shebang, but it does not seem to make any difference to calling "python" directly (since that also just looks up the current environment PATH and executes the first python it finds...
Agree - so that is disturbing is the fix calls to "python" inside Makefile. That means it remains to change:
tools/_libfdt.so: $(patsubst %.o,%.c,$(LIBFDT_OBJS)) tools/libfdt_wrap.c LDFLAGS="$(HOSTLDFLAGS)" CFLAGS= python $(srctree)/lib/libfdt/setup.py
to:
tools/_libfdt.so: $(patsubst %.o,%.c,$(LIBFDT_OBJS)) tools/libfdt_wrap.c LDFLAGS="$(HOSTLDFLAGS)" CFLAGS= ${PYTHON} $(srctree)/lib/libfdt/setup.py
Regards, Stefano
On 2017-04-04 01:53, Stefano Babic wrote:
Hi Stefan,
On 03/04/2017 23:02, Stefan Agner wrote:
But then, I don't expect that "/usr/bin/env python" is looking at PYTHON.
As far as I understand env, it just looks up the current environment to run its COMMAND argument.
Agree, this is also mz understanding.
It is helpful as a shebang, but it does not seem to make any difference to calling "python" directly (since that also just looks up the current environment PATH and executes the first python it finds...
Agree - so that is disturbing is the fix calls to "python" inside Makefile. That means it remains to change:
tools/_libfdt.so: $(patsubst %.o,%.c,$(LIBFDT_OBJS)) tools/libfdt_wrap.c LDFLAGS="$(HOSTLDFLAGS)" CFLAGS= python $(srctree)/lib/libfdt/setup.py
to:
tools/_libfdt.so: $(patsubst %.o,%.c,$(LIBFDT_OBJS)) tools/libfdt_wrap.c LDFLAGS="$(HOSTLDFLAGS)" CFLAGS= ${PYTHON} $(srctree)/lib/libfdt/setup.py
On my host
$ echo Python is "$PYTHON" Python is ""
Wouldn't that break regular builds? Probably wouldn't since shebang kicks in, but is that really what we want?
What I thought OE was doing is adding the native/x86_64 sysroots in your path at the beginning, which should already make sure that a regular "python" call executes OE python... But maybe there is more to it, especially since setup.py even compiles things...
Maybe we need to peek into distutils/setuptools class to understand how that works for regular Python modules: https://github.com/openembedded/openembedded/blob/master/classes/setuptools.... https://github.com/openembedded/openembedded/blob/master/classes/distutils.b...
-- Stefan
Hi Stefan,
On 04/04/2017 20:44, Stefan Agner wrote:
On 2017-04-04 01:53, Stefano Babic wrote:
Hi Stefan,
On 03/04/2017 23:02, Stefan Agner wrote:
But then, I don't expect that "/usr/bin/env python" is looking at PYTHON.
As far as I understand env, it just looks up the current environment to run its COMMAND argument.
Agree, this is also mz understanding.
It is helpful as a shebang, but it does not seem to make any difference to calling "python" directly (since that also just looks up the current environment PATH and executes the first python it finds...
Agree - so that is disturbing is the fix calls to "python" inside Makefile. That means it remains to change:
tools/_libfdt.so: $(patsubst %.o,%.c,$(LIBFDT_OBJS)) tools/libfdt_wrap.c LDFLAGS="$(HOSTLDFLAGS)" CFLAGS= python $(srctree)/lib/libfdt/setup.py
to:
tools/_libfdt.so: $(patsubst %.o,%.c,$(LIBFDT_OBJS)) tools/libfdt_wrap.c LDFLAGS="$(HOSTLDFLAGS)" CFLAGS= ${PYTHON} $(srctree)/lib/libfdt/setup.py
On my host
$ echo Python is "$PYTHON" Python is ""
Sent a patch today, easier to test.
The thing is that PYTHON is set into the main Makefile as "python", and this should not break your build. At least, I hope so...
Wouldn't that break regular builds? Probably wouldn't since shebang kicks in, but is that really what we want?
The main Makefile already sets most tools to default if they are not current set. This should ensure that the normal build works, but should even allow to override it from command line - or adding EXTRA_OEMAKE in recipe.
What I thought OE was doing is adding the native/x86_64 sysroots in your path at the beginning,
Right.
which should already make sure that a regular "python" call executes OE python...
"nativepython" should have a symbolic link to "python", but what I noted is some mix between distro and SDK.
But maybe there is more to it, especially since setup.py even compiles things...
Right. I have seen in some recipes that /usr/bin/env is replaced, for example:
meta/recipes-support/createrepo/createrepo_0.4.11.bb: sed -i -e 's|^#!.*/usr/bin/env python|#! /usr/bin/env nativepython|' $i
There are a few of recipes doing this.
Maybe we need to peek into distutils/setuptools class to understand how that works for regular Python modules: https://github.com/openembedded/openembedded/blob/master/classes/setuptools.... https://github.com/openembedded/openembedded/blob/master/classes/distutils.b...
But there are for building python-modules, and we are just simply want to build u-boot. Then, if I understand your concept, we need to build a python module outside u-boot, and built before u-boot (ok, I am just thinking about Yocto build..), as nativesdk-pythonsetuppy or whatever, and this is just called when u-boot is built. But this makes things much more complicated...
Stefano
On 2017-04-05 10:26, Stefano Babic wrote:
Hi Stefan,
On 04/04/2017 20:44, Stefan Agner wrote:
On 2017-04-04 01:53, Stefano Babic wrote:
Hi Stefan,
On 03/04/2017 23:02, Stefan Agner wrote:
But then, I don't expect that "/usr/bin/env python" is looking at PYTHON.
As far as I understand env, it just looks up the current environment to run its COMMAND argument.
Agree, this is also mz understanding.
It is helpful as a shebang, but it does not seem to make any difference to calling "python" directly (since that also just looks up the current environment PATH and executes the first python it finds...
Agree - so that is disturbing is the fix calls to "python" inside Makefile. That means it remains to change:
tools/_libfdt.so: $(patsubst %.o,%.c,$(LIBFDT_OBJS)) tools/libfdt_wrap.c LDFLAGS="$(HOSTLDFLAGS)" CFLAGS= python $(srctree)/lib/libfdt/setup.py
to:
tools/_libfdt.so: $(patsubst %.o,%.c,$(LIBFDT_OBJS)) tools/libfdt_wrap.c LDFLAGS="$(HOSTLDFLAGS)" CFLAGS= ${PYTHON} $(srctree)/lib/libfdt/setup.py
On my host
$ echo Python is "$PYTHON" Python is ""
Sent a patch today, easier to test.
The thing is that PYTHON is set into the main Makefile as "python", and this should not break your build. At least, I hope so...
Ok, yeah I guess than its fine to do such a change.
Wouldn't that break regular builds? Probably wouldn't since shebang kicks in, but is that really what we want?
The main Makefile already sets most tools to default if they are not current set. This should ensure that the normal build works, but should even allow to override it from command line - or adding EXTRA_OEMAKE in recipe.
What I thought OE was doing is adding the native/x86_64 sysroots in your path at the beginning,
Right.
which should already make sure that a regular "python" call executes OE python...
"nativepython" should have a symbolic link to "python", but what I noted is some mix between distro and SDK.
And that symlink is part of the path, so in a OE environment, "python" really should call the nativepython implementation...?
What do you mean by mix between distro and SDK?
But maybe there is more to it, especially since setup.py even compiles things...
Right. I have seen in some recipes that /usr/bin/env is replaced, for example:
meta/recipes-support/createrepo/createrepo_0.4.11.bb: sed -i -e 's|^#!.*/usr/bin/env python|#! /usr/bin/env nativepython|' $i
There are a few of recipes doing this.
Maybe we need to peek into distutils/setuptools class to understand how that works for regular Python modules: https://github.com/openembedded/openembedded/blob/master/classes/setuptools.... https://github.com/openembedded/openembedded/blob/master/classes/distutils.b...
But there are for building python-modules, and we are just simply want to build u-boot. Then, if I understand your concept, we need to build a python module outside u-boot, and built before u-boot (ok, I am just thinking about Yocto build..), as nativesdk-pythonsetuppy or whatever, and this is just called when u-boot is built. But this makes things much more complicated...
I guess there is a reason why we build that module as part of the U-Boot build, probably because later steps use the module to script some automation...? So yeah, agreed we should not split that out.
My point was more that those classes could be a source of inspiration how OE handles building Python modules in a sysroot...
-- Stefan
participants (4)
-
Simon Glass -
Stefan Agner -
Stefano Babic -
Stephen Arnold