potiuk commented on PR #43408: URL: https://github.com/apache/airflow/pull/43408#issuecomment-2440205320
> A dag sync hook or dependence can be created , to make sure both repo sync at same time . I agree to your point there will be a problem with sync staleness , but if we keep the wait parameter value is same for both repo then it will somewhat do the job will less lag . But why not promote submodule (which solves the problem - because in submodule, you can also make sure that you have a single commit, syncing both repos. Say you have * repo a) commit 1 -> commit 2 * repo b) commit 3 -> commit 4 Having a single "umbrella" subrebo and single git-sync allows you to do this: repo c) (umbrella over b) and c) * commit 4: -> links to commit 1 of repo a) and commit 3 of repo b) * commit 5: -> links to commit 3 or repo a) and commit 4 of repo b) That allows for atomic sync of repo c) - being umbrella on top of a) and b) where a) and b) repos are always consistent Is there any problem with that? Would not it be better to explain (and make a PR documenting it) how you can use submodule "umbrella" repo to synchronize multiple repos? Is there any problem with that? Have you looked at the Jagex presentation explaining it and benefits it brings for managing multi-repos? I am afraid that by adding more than one repo in git-sync we are opening up for a number of problems, and basicaly eventually we would have to develop a "submodule" equivalent, because the syncrhronisation problems between several repos will cause various errors - and submodule approach seem to have a solution for that, so it seems better to just describe how you can use it. Any problems with that? -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
