laskoviymishka commented on code in PR #917:
URL: https://github.com/apache/iceberg-go/pull/917#discussion_r3100145792
##########
io/gocloud/blob.go:
##########
@@ -190,6 +191,24 @@ func createBlobFS(ctx context.Context, bucket
*blob.Bucket, keyExtractor KeyExtr
return &blobFileIO{Bucket: bucket, keyExtractor: keyExtractor, ctx: ctx}
}
+func (bfs *blobFileIO) WalkDir(root string, fn fs.WalkDirFunc) error {
+ parsed, err := url.Parse(root)
+ if err != nil {
+ return fmt.Errorf("invalid URL %s: %w", root, err)
+ }
+
+ walkPath := strings.TrimPrefix(parsed.Path, "/")
+ if walkPath == "" {
+ walkPath = "."
+ }
+
+ prefix := parsed.Scheme + "://" + parsed.Host + "/"
+
+ return fs.WalkDir(bfs.Bucket, walkPath, func(path string, d
fs.DirEntry, err error) error {
+ return fn(prefix+path, d, err)
+ })
+}
Review Comment:
I think here Azure URI userinfo gets dropped
For `abfs://[email protected]/...`, the container sits
in `parsed.User`, not `parsed.Host`. The reconstructed prefix loses it, and
orphan cleanup ends up flagging every Azure file as orphan. Preserve
`parsed.User.String() + "@" + parsed.Host`?
##########
table/orphan_cleanup.go:
##########
@@ -308,62 +307,28 @@ func (t Table) scanFiles(fs iceio.IO, location string,
cfg *orphanCleanupConfig)
return allFiles, totalSize, nil
}
-// getBucket gets the Bucket field from blob storage - absolute minimal
approach
-func getBucketName(fsys iceio.IO) stdfs.FS {
- v := reflect.ValueOf(fsys).Elem() // We know it's a pointer to struct
-
- return v.FieldByName("Bucket").Interface().(stdfs.FS)
-}
+func walkDirectory(fsys iceio.IO, root string, fn func(path string, info
stdfs.FileInfo) error) error {
+ listable, ok := fsys.(iceio.ListableIO)
+ if !ok {
+ return fmt.Errorf("IO implementation %T does not support
directory listing (does not implement ListableIO)", fsys)
Review Comment:
I think it worth to inlcude root in error.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
