[GitHub] [incubator-bluemarlin] sanjaynirmal opened a new issue #39: Lookalike: spark data pipeline issue

[GitBox]

sanjaynirmal opened a new issue #39:
URL: https://github.com/apache/incubator-bluemarlin/issues/39


   For the latest code that you have released for lookalike, we are facing a 
problem in the last step of data pipeline i.e. tf record generation.
   script: 
https://github.com/apache/incubator-bluemarlin/blob/main/Model/lookalike-model/lookalike_model/pipeline/main_tfrecord_generator.py
   <img width="687" alt="image" 
src="https://user-images.githubusercontent.com/40193781/151826834-42ee6106-4725-4628-a037-b01f8bd69c31.png";>
   
   We are currently running the pipeline for 120 Million AIDs and we get the 
following error on the highlighted line.
   
   <img width="944" alt="image" 
src="https://user-images.githubusercontent.com/40193781/151827090-ac1f7c53-1e18-4dcf-afe1-53aa5a800867.png";>
   
   Kindly help us in running the pipeline for big data.
   
   


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: dev-unsubscr...@bluemarlin.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org