On Sun Jul 17, 2022 at 16:43 CET, 'Sebastien Binet' via Perkeep wrote: [...]
> > is the only way to create a new replicate to scp-copy the blobs? > > (or implement the correct blob-order enumeration? I could try my hand at > > it, if pointed a bit at the correct way) > > I have tried the ssh-copy avenue. > AFAICT, I didn't get the packing errors anymore but for some reason, > starting perkeepd on the remote-storage always failed with: > > blobpacked: 0 large blobs found in index, 44723 missing from index > sample missing large blob: sha224-0001b1...xxx > sample missing large blob: sha224-0001bf...xxx > sample missing large blob: sha224-000576...xxx > sample missing large blob: sha224-000a2b...xxx > sample missing large blob: sha224-000d89...xxx > sample missing large blob: sha224-001086...xxx > sample missing large blob: sha224-0011d3...xxx > sample missing large blob: sha224-001399...xxx > sample missing large blob: sha224-0014ca...xxx > sample missing large blob: sha224-00153e...xxx > Error: 44723 large blobs missing from index. Please re-start in recovery > mode with -recovery=1 > > even after restarting with -recovery=1 and/or -reindex (and/or > -recovery=2). > > not sure whether that's expected. > > my offer of trying to work on a "better" blob-enumeration still stand. > but in the meantime, I'll try to setup the following scaffolding: > - pk list -type=file > list.txt (on localhost) > - for sha224 in list.txt; do pk-get -content $sha224 > filename > - for filename in `ls -1`; do pk-upload remote-storage $filename > (where pk-upload is just pk-put with remote-storage as the end-point and > the following options: "file --filenodes --exiftime") ok, perhaps unsurprisingly, this didn't pan out: even with a small subsample (~30 images), I get the same error when restarting the "remote-storage" server: Starting perkeepd version master, 2022-07-10-c501f90cd0; Go go1.18.3 (linux/arm64) Starting to listen on http://localhost:3179 blobpacked: checking integrity of packed blobs against index... blobpacked: 0 large blobs found in index, 29 missing from index sample missing large blob: sha224-098b...xxx sample missing large blob: sha224-1185...xxx ... but, then, I tried something else: - on "localhost", keep the same upload scaffolding as described above - on "remote-storage", use the "default" server-config.json (ie: with sqlite instead of postgres) and the "sample missing large blob: ...." errors disappeared. what am I doing wrong? looking at the output of \l in psql, I do see the "perkeep", "pk_<id-lowercase>_blobpacked" and "pk_<id-lowercase>_syncto_index" databases. I have attached the relevant parts of the psql-based server (redacted) config. please let me know what's amiss. cheers, -s -- You received this message because you are subscribed to the Google Groups "Perkeep" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/perkeep/CLI9KTXZB9NK.1HE83SUZ6ZV60%40zoidberg.
pk-psql-config.json
Description: application/json
