It's not simply a zip. I recently went through an incomplete pg incident as
well. I'm not sure why your import is failing, but I do know that much.
Here's a note in slack from our effort to reverse the export. I'm hoping to
explore this a bit more in the next week.
Data frames appear to have the following format:
ceff DTDT SIZE PAYLOAD
Size is probably in bytes? DTDT is frame type, 8-bits, repeated (so BEGIN=1
is encoded as 0101).
File format looks like:
Superblock - starts with: ceff ceff 0200
PG_BEGIN ceff 0101 <64 bit little-Endian size>
PG_METADATA ceff 0909
OBJECT_BEGIN ceff 0303 <64 bit little-Endian size>
(represents first object of a file?)
TYPE_DATA ceff 0505 (represents an object?)
(repeat TYPE_DATA frames until file completed)
TYPE_ATTRS ceff 0606
TYPE_OMAP ceff 0808
OBJECT_END ceff 0404
(repeat above block for all files?)
enum {
TYPE_NONE = 0,
TYPE_PG_BEGIN,
TYPE_PG_END,
TYPE_OBJECT_BEGIN,
TYPE_OBJECT_END,
TYPE_DATA,
TYPE_ATTRS,
TYPE_OMAP_HDR,
TYPE_OMAP,
TYPE_PG_METADATA,
TYPE_POOL_BEGIN,
TYPE_POOL_END,
END_OF_TYPES, //Keep at the end
};
On Jan 14, 2018 1:27 AM, "Brent Kennedy" <[email protected]> wrote:
> I was able to bring a server back online for a short time and perform an
> export of the incomplete PGs I originally posted about last week. The
> export showed the files it was exporting and then dropped them all to a
> PGID.export file. I then SCP’ed the four PGID.export files to a server
> where I had an empty OSD weighted to 0. I stopped that OSD and then tried
> to import all four PGs. I then got the following messages for all four I
> tried:
>
>
>
> finish_remove_pgs 11.720_head removing 11.720
>
> Importing pgid 11.c13
>
> do_import threw exception error buffer::malformed_input: void
> object_stat_sum_t::decode(ceph::buffer::list::iterator&) decode past end
> of struct encoding
>
> Corrupt input for import
>
>
>
>
>
> Command I ran:
>
> ceph-objectstore-tool --op import --data-path /var/lib/ceph/osd/ceph-13
> --journal-path /var/lib/ceph/osd/ceph-13/block --file 11.c13.export
>
>
>
> The files match the space used by PGs on the disk. As noted above, I saw
> it copy the PG to the export file successfully. Both servers are running
> Ubuntu 14 with the newest ceph-objectstore-tool installed via the package
> from here: http://download.ceph.com/debian-luminous/pool/main/c/
> ceph/ceph-test_12.2.2-1trusty_amd64.deb ( cluster is Luminous 12.2.2 .
> Its possible the PGs in question are on the jewel version as I wasn’t able
> to complete the upgrade to luminous on them.
>
>
>
> Am I missing something? Can I just copy the files off the failing server
> via a zip operation locally and then a unzip operation at the destination
> server?
>
>
>
> -Brent
>
> _______________________________________________
> ceph-users mailing list
> [email protected]
> http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
>
>
_______________________________________________
ceph-users mailing list
[email protected]
http://lists.ceph.com/listinfo.cgi/ceph-users-ceph.com
