I've just been digging into a problem where I've got both CONFIG_ENV_IS_NOWHERE set and CONFIG_BOOTDELAY set to -2 and it turns out in env_default.h we have:
#if defined(CONFIG_BOOTDELAY) && (CONFIG_BOOTDELAY >= 0) "bootdelay=" __stringify(CONFIG_BOOTDELAY) "\0" #endif
So the -ve values never make it into the default environment, which means I don't have it at all when CONFIG_ENV_IS_NOWHERE.
The (CONFIG_BOOTDELAY >= 0) seems to have been there forever (c609719b8d1b2dca590e0ed499016d041203e403, Sun Nov 3 00:24:07 2002 +0000 is as far back as I've gone), but we've then changed the behaviours of the bootdelay values in (the commit I was looking at was 2fbb8462b0e18893b4b739705db047ffda82d4fc from Mon Jun 27 16:23:01 2016 +0900, but I'm not sure that's really the right one)
I think we should change the code to a simple #if defined(CONFIG_BOOTDELAY)
?