0

I read a couple of times (e.g. here) that Poky is a reference distribution that contains OpenEmbedded, Bitbake and other things. But I was never sure about the meaning of "contains". Now I checked, and was very surprised to find out that the Poky repository literally contains the content of several other repositories. Among them at least:

It seems as if those repositories were blindly copied together into another repo. But digging further, they weren't blindly copied. In fact, each commit appears to always exist identically in both repos. Example:

me@home:~% cd poky               && git show 8877980c99 > x && cd ..
me@home:~% cd openembedded-core/ && git show 812eb3121e > x && cd ..
me@home:~% diff poky/x openembedded-core/x
1c1
< commit 8877980c99045d53c2465faeb45aa6e81f126708
---
> commit 812eb3121e0aabe4e3de9a8c61b1e62c87f55aa4
6,7d5
<
<     (From OE-Core rev: 812eb3121e0aabe4e3de9a8c61b1e62c87f55aa4)

Except the commit hash and that one line From OE-Core rev: ... the commits are truly identical - even the timestamp matches up to the same second. How does that even work?

Besides, isn't it somehow bad practice to duplicate the content on one repo in another, in the sense that it violates the DRY principle? Is each committer responsible that his commit always lands in both repos? Isn't there a risk that the repos will drift apart? In fact I did spot a few (insignificant) files that differ slightly, like this one:

me@home:~% diff poky/meta-poky/README.poky meta-yocto/meta-poky/README.poky
25c25
< DISTRO = "nodistro") and contains only emulated machine support.
---
> DISTRO = "") and contains only emulated machine support.
Georg P.
  • 2,785
  • 2
  • 27
  • 53
  • Is your *actual* question the bolded text here? Because there are some questions in here that are actively asking for peoples opinions, and that's not actually on-topic here on Stack Overflow. – Lasse V. Karlsen Jan 08 '21 at 14:41
  • I would guess that the answer to "How does that even work?" is that the maintainer of poky has created a script that duplicates all commits into this repository, by replaying the changes or something of that form. If they were simply all added as separate histories into this repository and then merged as unrelated histories, you would see identical commit hashes as well. – Lasse V. Karlsen Jan 08 '21 at 14:43
  • Yeah, the bolded text is what I'm mostly interested in. I thought that there maybe exists some git-magic that I've never heard of (cross-repository cherry-picking or sth like that...). Thinking about it, this is maybe more a git-related question. – Georg P. Jan 08 '21 at 14:48
  • 1
    […Poky is an integration repository (built using a tool called combo-layer)…](https://git.yoctoproject.org/cgit/cgit.cgi/poky/tree/meta-poky/README.poky#n46) combo-layer: https://github.com/dlespiau/poky/blob/master/scripts/combo-layer – phd Jan 08 '21 at 14:53

1 Answers1

3

As noted in comments, the repository uses a Python script called combo-layer to import changes from the other repositories, retaining all their history information.

Although often used with a central server, or even a hosting service like GitHub, git is explicitly designed as a de-centralised versioning system, so this kind of thing is actually pretty easy to do:

  • git remote can add a pointer to any other git repository; there is no identifier that has to match to say they're related
  • git merge has an --allow-unrelated-histories flag that lets you combine two sets of commits that don't have any history in common
  • git cherry-pick and git rebase can recreate the changes in a commit on top of some other history, including timestamp and committer information; there are options for editing the commit message
  • git format-patch and git am allow for exporting and importing a series of patch files in a manner very similar to git cherry-pick or git rebase (originally used to distribute changes by e-mail)

So in your example, something like this has happened:

  1. The latest history from openembedded-core was fetched
  2. The commit 812eb3121e0aabe4e3de9a8c61b1e62c87f55aa4 was found to be new since the last import
  3. A patch file was generated for that commit
  4. The patch was edited to add the line "(From OE-Core rev: 812eb3121e0aabe4e3de9a8c61b1e62c87f55aa4)" to the commit message
  5. The commit was merged into the poky repository, retaining its original committer information

As far as git's concerned, this isn't particularly magic; its happy to apply commits wherever you ask it to.

IMSoP
  • 89,526
  • 13
  • 117
  • 169
  • 1
    Thanks to your pointers I found this [wiki page](https://wiki.yoctoproject.org/wiki/Combo-layer) that explains it nicely. It also has an answer for *Why not use other similar tools (repo, submodules, subtree)?*. – Georg P. Jan 08 '21 at 15:14