hi,
On Thu Jul 14, 2022 at 14:43 CET, 'Sebastien Binet' via Perkeep wrote:
> hi there,
>
> apologies for the burst of questions.
>
> I am trying to create a new replica for my blobs, on some remote
> storage.
>
> I have the following client config on the "main" blobstorage:
>
> {
> "servers": {
> "localhost": {
> "server": "http://localhost:3179";,
> "auth": "localhost",
> "default": true
> },
> "backup": {
> "server": "https://remote-storage.org";,
> "auth": "userpass:user:pass"
> }
> },
> "identity": "me",
> "ignoredFiles": [
> ".DS_Store"
> ]
> }
>
> after starting the remote-storage's perkeep server, I then tried, on the
> "main" blobstorage:
>
> $> pk sync -src http://localhost:3179/bs -dest
> https://remote-storage.org/bs
>
> but I got the following errors on the remote-storage:
>
> Received blob [sha224-XX1; 76937 bytes]
> Received blob [sha224-XX2; 66332 bytes]
> blobpacked: Packing file sha224-XX3 ...
> blobpacked: Error packing file sha224-XX3: file does not exist
> Received blob [sha224-XXX3; 961 bytes]
>
> according to #681 and #1073
>
> - https://github.com/perkeep/perkeep/issues/681
> - https://github.com/perkeep/perkeep/issues/1073
>
> it seemed like one could get away with first replicating /bs-packed and
> then /bs-loose, but I still got the above errors (although, less of
> those).
>
> is the only way to create a new replicate to scp-copy the blobs?
> (or implement the correct blob-order enumeration? I could try my hand at
> it, if pointed a bit at the correct way)
I have tried the ssh-copy avenue.
AFAICT, I didn't get the packing errors anymore but for some reason,
starting perkeepd on the remote-storage always failed with:
blobpacked: 0 large blobs found in index, 44723 missing from index
sample missing large blob: sha224-0001b1...xxx
sample missing large blob: sha224-0001bf...xxx
sample missing large blob: sha224-000576...xxx
sample missing large blob: sha224-000a2b...xxx
sample missing large blob: sha224-000d89...xxx
sample missing large blob: sha224-001086...xxx
sample missing large blob: sha224-0011d3...xxx
sample missing large blob: sha224-001399...xxx
sample missing large blob: sha224-0014ca...xxx
sample missing large blob: sha224-00153e...xxx
Error: 44723 large blobs missing from index. Please re-start in recovery mode
with -recovery=1
even after restarting with -recovery=1 and/or -reindex (and/or
-recovery=2).
not sure whether that's expected.
my offer of trying to work on a "better" blob-enumeration still stand.
but in the meantime, I'll try to setup the following scaffolding:
- pk list -type=file > list.txt (on localhost)
- for sha224 in list.txt; do pk-get -content $sha224 > filename
- for filename in `ls -1`; do pk-upload remote-storage $filename
(where pk-upload is just pk-put with remote-storage as the end-point and
the following options: "file --filenodes --exiftime")
would that work to extract all files (photos in my case) from
"localhost" and upload them to "remote-storage"?
(albeit, perhaps, in a not-completely-optimal way)
cheers,
-s
--
You received this message because you are subscribed to the Google Groups
"Perkeep" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to [email protected].
To view this discussion on the web visit
https://groups.google.com/d/msgid/perkeep/CLI0EEXOO6CR.2XQR04BB9Y5I9%40zoidberg.
