I'm a Yocto newbie porting an SBC project from Buildroot, setting up a custom layer called meta-x. I initially added meta-x to the bblayers.conf in /build/conf/ but wanted to run the yocto-check-layer script -- it returned that I should remove meta-x from the /build/conf/bblayers.conf and run it again:
INFO: meta-remora already in /home/hal/build/conf/bblayers.conf. To capture initial
signatures, layer under test should not present in BBLAYERS. Please remove meta-remora
from BBLAYERS.`
The yocto-check-layers script then runs normally, but hangs on the "Getting initial signatures …" stage. Sending a keyboard interrupt prints the following error output:
Traceback (most recent call last):
File "/home/hal/poky/scripts/yocto-check-layer", line 251, in <module>
ret = main()
File "/home/hal/poky/scripts/yocto-check-layer", line 214, in main
td['sigs'], td['tunetasks'] = get_signatures(td['builddir'])
File "/home/hal/poky/scripts/lib/checklayer/__init__.py", line 316, in get_signatures
cmd, builddir)
File "/home/hal/poky/scripts/lib/checklayer/__init__.py", line 284, in check_command
output, _ = p.communicate()
File "/usr/lib/python3.6/subprocess.py", line 850, in communicate
stdout = self.stdout.read()
File "/home/hal/poky/scripts/yocto-check-layer", line 160, in cleanup_bblayers
shutil.copyfile(bblayersconf + '.backup', bblayersconf)
File "/usr/lib/python3.6/shutil.py", line 120, in copyfile
with open(src, 'rb') as fsrc:
FileNotFoundError: [Errno 2] No such file or directory:
'/home/hal/build/conf/bblayers.conf.backup'
My build setup has Ubuntu 18.04 LTS in WSL2.
Neither restarting the shell nor restarting WSL2 have any effect. Without diving into the source (a bit too new for that) I have not found any references to a bblayers.conf.backup file online. Is this common knowledge? If I have to create the file what should it include - it seems like something that should be auto-generated.
I followed the Yocto Quick-Build guide to get where I am now, and have been using the basic core-image-base recipe to build images.
Thanks,
hal.
Edit 1 - top output during Getting initial signatures stage which I believe hangs:
PID USER PR NI VIRT RES SHR S %CPU %MEM TIME+ COMMAND
104 hal 20 0 688068 621324 10496 S 0.3 4.8 0:07.55 Cooker
113 hal 20 0 204756 58184 7288 S 0.3 0.5 0:00.13 Cooker
1 root 20 0 908 540 472 S 0.0 0.0 0:00.01 init
7 root 20 0 908 88 20 S 0.0 0.0 0:00.00 init
8 root 20 0 908 88 20 S 0.0 0.0 0:00.04 init
9 hal 20 0 4636 800 736 S 0.0 0.0 0:00.00 sh
12 hal 20 0 20320 4024 3416 S 0.0 0.0 0:00.02 bash
55 hal 20 0 167824 32984 9432 S 0.0 0.3 0:00.17 python3
101 hal 20 0 4640 812 744 S 0.0 0.0 0:00.00 sh
102 hal 20 0 161780 30052 9364 S 0.0 0.2 0:00.56 KnottyUI
152 root 20 0 908 88 20 S 0.0 0.0 0:00.00 init
153 root 20 0 908 88 20 S 0.0 0.0 0:00.00 init
154 hal 20 0 4636 860 796 S 0.0 0.0 0:00.00 sh
157 hal 20 0 20320 3868 3420 S 0.0 0.0 0:00.00 bash
165 hal 20 0 38272 3436 2992 R 0.0 0.0 0:00.00 top
Edit 2 - interesting bitbake output after interrupting the previously frozen yocto-check-layers stage (error output earlier in post).
hal@disguise-000557:~/build$ bitbake
^CTraceback (most recent call last):
File "/home/hal/poky/bitbake/bin/bitbake", line 37, in <module>
cookerdata.CookerConfiguration()))
File "/home/hal/poky/bitbake/lib/bb/main.py", line 360, in bitbake_main
server_connection, ui_module = setup_bitbake(configParams)
File "/home/hal/poky/bitbake/lib/bb/main.py", line 449, in setup_bitbake
server_connection = bb.server.process.connectProcessServer(sockname, featureset)
File "/home/hal/poky/bitbake/lib/bb/server/process.py", line 608, in connectProcessServer
server_connection.connection.updateFeatureSet(featureset)
File "/home/hal/poky/bitbake/lib/bb/server/process.py", line 415, in updateFeatureSet
_, error = self.runCommand(["setFeatures", featureset])
File "/home/hal/poky/bitbake/lib/bb/server/process.py", line 403, in runCommand
if not self.recv.poll(30):
File "/home/hal/poky/bitbake/lib/bb/server/process.py", line 719, in poll
return self.reader.poll(timeout)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 257, in poll
return self._poll(timeout)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 414, in _poll
r = wait([self], timeout)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 911, in wait
ready = selector.select(timeout)
File "/usr/lib/python3.6/selectors.py", line 376, in select
fd_event_list = self._poll.poll(timeout)
KeyboardInterrupt
NOTE: Reconnecting to bitbake server...
Edit 3 - the contents of the poky/scripts/lib/checklayer/__init.py__ file that is generating the initial error
290 def get_signatures(builddir, failsafe=False, machine=None, extravars=None):
291 import re
292
293 # some recipes needs to be excluded like meta-world-pkgdata
294 # because a layer can add recipes to a world build so signature
295 # will be change
296 exclude_recipes = ('meta-world-pkgdata',)
297
298 sigs = {}
299 tune2tasks = {}
300
301 cmd = 'BB_ENV_PASSTHROUGH_ADDITIONS="$BB_ENV_PASSTHROUGH_ADDITIONS BB_SIGNATURE_HANDLER" BB_SIGNATURE_HANDLER="OEBasicHash" '
302 if extravars:
303 cmd += extravars
304 cmd += ' '
305 if machine:
306 cmd += 'MACHINE=%s ' % machine
307 cmd += 'bitbake '
308 if failsafe:
309 cmd += '-k '
310 cmd += '-S none world'
311 sigs_file = os.path.join(builddir, 'locked-sigs.inc')
312 if os.path.exists(sigs_file):
313 os.unlink(sigs_file)
314 try:
315 check_command('Generating signatures failed. This might be due to some parse error and/or general layer incompatibilities.',
316 cmd, builddir)
317 except RuntimeError as ex:
318 if failsafe and os.path.exists(sigs_file):
319 # Ignore the error here. Most likely some recipes active
320 # in a world build lack some dependencies. There is a
321 # separate test_machine_world_build which exposes the
322 # failure.
323 pass
324 else:
325 raise
326
327 sig_regex = re.compile("^(?P<task>.*:.*):(?P<hash>.*) .$")
328 tune_regex = re.compile("(^|\s)SIGGEN_LOCKEDSIGS_t-(?P<tune>\S*)\s*=\s*")
329 current_tune = None
330 with open(sigs_file, 'r') as f:
331 for line in f.readlines():
332 line = line.strip()
333 t = tune_regex.search(line)
334 if t:
335 current_tune = t.group('tune')
336 s = sig_regex.match(line)
337 if s:
338 exclude = False
339 for er in exclude_recipes:
340 (recipe, task) = s.group('task').split(':')
341 if er == recipe:
342 exclude = True
343 break
344 if exclude:
345 continue
346
347 sigs[s.group('task')] = s.group('hash')
348 tune2tasks.setdefault(current_tune, []).append(s.group('task'))
349
350 if not sigs:
351 raise RuntimeError('Can\'t load signatures from %s' % sigs_file)
352
353 return (sigs, tune2tasks)
Edit 4 - output of the fail command line 301 in the yocto-check-layer
BB_ENV_PASSTHROUGH_ADDITIONS="$BB_ENV_PASSTHROUGH_ADDITIONS BB_SIGNATURE_HANDLER" BB_SIGNATURE_HANDLER="OEBasicHash" bitbake -S none world
^CTraceback (most recent call last):
File "/home/hal/poky/bitbake/bin/bitbake", line 37, in <module>
cookerdata.CookerConfiguration()))
File "/home/hal/poky/bitbake/lib/bb/main.py", line 360, in bitbake_main
server_connection, ui_module = setup_bitbake(configParams)
File "/home/hal/poky/bitbake/lib/bb/main.py", line 449, in setup_bitbake
server_connection = bb.server.process.connectProcessServer(sockname, featureset)
File "/home/hal/poky/bitbake/lib/bb/server/process.py", line 608, in connectProcessServer
server_connection.connection.updateFeatureSet(featureset)
File "/home/hal/poky/bitbake/lib/bb/server/process.py", line 415, in updateFeatureSet
_, error = self.runCommand(["setFeatures", featureset])
File "/home/hal/poky/bitbake/lib/bb/server/process.py", line 405, in runCommand
if not self.recv.poll(30):
File "/home/hal/poky/bitbake/lib/bb/server/process.py", line 719, in poll
return self.reader.poll(timeout)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 257, in poll
return self._poll(timeout)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 414, in _poll
r = wait([self], timeout)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 911, in wait
ready = selector.select(timeout)
File "/usr/lib/python3.6/selectors.py", line 376, in select
fd_event_list = self._poll.poll(timeout)
KeyboardInterrupt
NOTE: Reconnecting to bitbake server...
NOTE: No reply from server in 30s
Edit 5 - ps aux | grep bitbake output while the previous cmd is run
ps aux | grep bitbake
hal 63 0.4 0.2 168348 28872 pts/0 Sl+ 17:51 0:00 python3 /home/hal/poky/bitbake/bin/bitbake -v -S none world
hal 65 4.5 4.8 690048 622924 ? S 17:51 0:08 bitbake-server /home/hal/poky/bitbake/bin/bitbake-server decafbad 3 5 /home/hal/build/bitbake-cookerdaemon.log /home/hal/build/bitbake.lock /home/hal/build/bitbake.sock 0 None 0
hal 74 0.3 0.4 205772 59148 ? Sl 17:51 0:00 bitbake-server /home/hal/poky/bitbake/bin/bitbake-server decafbad 3 5 /home/hal/build/bitbake-cookerdaemon.log /home/hal/build/bitbake.lock /home/hal/build/bitbake.sock 0 None 0
Edit 6 - ran with -verbose, hangs again, following error after 3 keyboard interrupts:
Keyboard Interrupt, closing down...
Second Keyboard Interrupt, stopping...
Traceback (most recent call last):
File "/home/hal/poky/bitbake/lib/bb/ui/knotty.py", line 644, in main
event = eventHandler.waitEvent(0.25)
File "/home/hal/poky/bitbake/lib/bb/server/process.py", line 685, in waitEvent
self.eventQueueNotify.wait(delay)
File "/usr/lib/python3.6/threading.py", line 551, in wait
signaled = self._cond.wait(timeout)
File "/usr/lib/python3.6/threading.py", line 299, in wait
gotit = waiter.acquire(True, timeout)
KeyboardInterrupt
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/hal/poky/bitbake/lib/bb/ui/knotty.py", line 873, in main
_, error = server.runCommand(["stateShutdown"])
File "/home/hal/poky/bitbake/lib/bb/server/process.py", line 403, in runCommand
if not self.recv.poll(30):
File "/home/hal/poky/bitbake/lib/bb/server/process.py", line 719, in poll
return self.reader.poll(timeout)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 257, in poll
return self._poll(timeout)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 414, in _poll
r = wait([self], timeout)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 911, in wait
ready = selector.select(timeout)
File "/usr/lib/python3.6/selectors.py", line 376, in select
fd_event_list = self._poll.poll(timeout)
KeyboardInterrupt
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/home/hal/poky/bitbake/bin/bitbake", line 37, in <module>
cookerdata.CookerConfiguration()))
File "/home/hal/poky/bitbake/lib/bb/main.py", line 379, in bitbake_main
configParams)
File "/home/hal/poky/bitbake/lib/bb/ui/knotty.py", line 877, in main
state_force_shutdown()
File "/home/hal/poky/bitbake/lib/bb/ui/knotty.py", line 861, in state_force_shutdown
_, error = server.runCommand(["stateForceShutdown"])
File "/home/hal/poky/bitbake/lib/bb/server/process.py", line 403, in runCommand
if not self.recv.poll(30):
File "/home/hal/poky/bitbake/lib/bb/server/process.py", line 719, in poll
return self.reader.poll(timeout)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 257, in poll
return self._poll(timeout)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 414, in _poll
r = wait([self], timeout)
File "/usr/lib/python3.6/multiprocessing/connection.py", line 911, in wait
ready = selector.select(timeout)
File "/usr/lib/python3.6/selectors.py", line 376, in select
fd_event_list = self._poll.poll(timeout)
KeyboardInterrupt
