xuang7 commented on issue #4240: URL: https://github.com/apache/texera/issues/4240#issuecomment-4619870243
@Sentiaus Thanks for getting these PRs up. Before going deeper into the implementation across all three PRs, I think it would be helpful to align on the design first, which could make both implementation and review more efficient. After reviewing part of the PRs, we have a few design questions about the current export flow. To summarize the two issues discussed in the PR threads: 1. Frontend-based export flow: https://github.com/apache/texera/pull/5251#discussion_r3352152563 The current design loads the file fully on the frontend before uploading it to Google Drive. This has two caveats: - The file makes an extra round trip through the user’s bandwidth: Texera → browser → Google Drive. - For very large datasets, this could overload the browser memory and the user’s machine. Would it make sense for the frontend to only trigger the export request, and let the backend stream the data directly to Google Drive? 2. Token storage: https://github.com/apache/texera/pull/5250#issuecomment-4614729120 Would storing encrypted Google refresh tokens in the DB be the preferred approach here, or would it be worth considering other options as well? Could you share your thoughts on the current frontend-based approach versus moving export streaming to the backend? It would be helpful to align on the overall direction first before we continue with more detailed review across the PRs. Cc @chenlica @aicam @carloea2, please feel free to add anything I may have missed. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
