krishnakaanchan-png opened a new pull request, #3886: URL: https://github.com/apache/iceberg-python/pull/3886
Closes #3885 # Rationale for this change `_adls` wrote the account name it inferred back into the properties dict it was handed, and the caller at `fsspec.py:515` passes the FileIO's own `self.properties`. So the first ADLS location a `FsspecFileIO` touched pinned `adls.account-name` for the life of that FileIO, and every later location was served a filesystem built for the first account. The `lru_cache` on `(scheme, hostname)` was doing its job. It built a second filesystem for the second hostname. By that point `adls.account-name` was already in the shared dict, so the inference was skipped and the new filesystem got the old account. Resolving into locals instead. The caller's dict is left alone, so the account is worked out again for each location. Precedence is unchanged. An explicit `adls.account-name` still wins, then an account derived from a per account SAS token key, then the hostname. `test_adls_account_name_sas_token_extraction` pins that middle case and still passes. Same change for `adls.sas-token`, which was being written back in the same loop. ## Are these changes tested? Yes. Two new tests. One reads two locations in different accounts from a single FileIO and asserts each filesystem is built for its own account. The other asserts the properties dict is not touched at all. Before, the reproduction in #3885 gave `['accountone', 'accountone']`. Now it gives `['accountone', 'accounttwo']`. The three existing `_adls` account name tests still pass, including the SAS token one, which is the one that pins the precedence order. `make lint` is clean and `make test` passes. ## Are there any user-facing changes? Yes, for anyone reading tables that span more than one storage account through `FsspecFileIO` without setting `adls.account-name`. Those locations were being read from the wrong account and will now go to the right one. Single account setups behave the same as before. A FileIO's `properties` also no longer changes as a side effect of reading a location. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected] --------------------------------------------------------------------- To unsubscribe, e-mail: [email protected] For additional commands, e-mail: [email protected]
