gitmodimo opened a new issue, #45592: URL: https://github.com/apache/arrow/issues/45592
### Describe the enhancement requested I am using filter node to remove rows with null values for specific columns. As as result rows are filtered correctly however output schema does not reflect that. Is there a way to enforce "not null" column in schema? In my case the rows are the result of asofjoin. Am I missing another way? Right now I am considering two approaches: 1. Create `filter_not_null` node that filters out null values for selected columns and modifies schema accordingly 2. Modify current `filter` node to detect when expression simplifies to false for specific null fields 3. Combination of 1&2 and modify `filter` node to take additional list of `filter_not_null` columns. I find 1. to be very explicit and somewhat duplicating `filter` node functionality. 2. however seems volatile to change output schema depending on expression. IMO 3. takes best of both What do you think? Maybe is there another way? ### Component(s) C++ -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
