If you really want to keep the binary assets out of the main code
repository, you can use Subversions Externals definitions (see
https://svnbook.red-bean.com/en/1.7/svn.advanced.externals.html) where
you can pull in a file or a folder from some other repository in to
the working copy tree. Then you could have multiple repositories for
the binary assets, retiring (and removing) repositories and switching
the externals definition to point to the new repository.
I have tried external many years ago (10?), I faced tons of issues. I
can't remember exactly what, it was either commiting/update/revert that
needed extra steps , and could be missed out. Was using tortoisesvn.
In Subversion 1.15 (not yet released) there will be an option to skip
the pristine copies, effectively cutting storage requirement in half,
at the expense of more network traffic. When running in this mode, the
client will only have one copy of each file with only a tiny bit of
overhead for configuration.
That's sounds like what I'm looking for, great! When the project size is
in TBs, you do want to try save space when binary assets are not
delta stored.
Thanks Daniel
Robin
On 12/2/2023 7:06 PM, Daniel Sahlberg wrote:
Den lör 2 dec. 2023 kl 05:12 skrev Robin :
Hi, I use SVN for gamedev projects, it works great but our repository
size is growing towards 1TB size. A lot of the files are binary
taking
up bulk of space. For bigger projects in game industry, peforce is
commonly used, and they have a few features which are quite neat.
1. User can specify certain files that don't need to store full
history.
Server keep only X number of rev history cos the older revisions
are not
needed. e.g a artist photoshop file, we don't need full history,
usually only the latest or a few revisions.
This is unfortunately not possible. The history in Subversion is
"immutable", you cannot remove things from the repository. On the
other hand, large repositories is not a problem (except for the
required disk space).
If you really want to keep the binary assets out of the main code
repository, you can use Subversions Externals definitions (see
https://svnbook.red-bean.com/en/1.7/svn.advanced.externals.html) where
you can pull in a file or a folder from some other repository in to
the working copy tree. Then you could have multiple repositories for
the binary assets, retiring (and removing) repositories and switching
the externals definition to point to the new repository.
But that means an old version of the code will effectively be
unbuildable since it points to non-existing binary assets.
I think the cost for some disk space on the server will be a lot lower
than the cost (in hours) for any extra complexity trying to eliminate
old versions.
2. Server don't keep a duplicate copy of those files on client,
client
get 1 copy only. This would be similar to git LFS.
Contrary to Git, in Subversion the client will only store the
"current" version of a file in a working copy (when checking out a
repository). "Current" here means the version you decide to check out,
not necessarily the latest.
In Subversion 1.14 and earlier, you will also get an extra copy called
"pristine version" (hidden in the administrative directory .svn) for
each version of the file you have had in your working copy. If the
user wants to switch to another version, a simple local copy is the
only thing required. The downside is of course the extra storage space
required, in particular when working with a lot of different revisions
of each file. The command svn cleanup can remove unused pristine
copies to save disk space.
In Subversion 1.15 (not yet released) there will be an option to skip
the pristine copies, effectively cutting storage requirement in half,
at the expense of more network traffic. When running in this mode, the
client will only have one copy of each file with only a tiny bit of
overhead for configuration.
If SVN has these features natively, it would be greatest tool.
Otherwise, how would it be possible to make such a setup?
In general, storing large binary files is an area where the design of
Subversion is superior to Git (where you need addons such as LFS).
Robin
Kind regards,
Daniel
