hi all, i've got layered entities in my solr import. it's calling on some transactional data from a MySQL instance. there are two fields that are used to then lookup other information from other tables via their related UIDs, one of which has its own child entity w yet another select statement to grab up more data.
it fetches at about 120/s but processes at ~50-60/s. we currently only have close to 500k records, but it's growing quickly and thus is becoming increasingly painful to make modifications due to the reimport that needs to then occur. i feel like i'd seen some threads regarding commits of new data, master/slave, or solrcloud/sharding that could help in some ways related to this but as of yet can't scrounge them up w my searches (ironic :p). can someone help by pointing me to some good material related to this sort of thing? thanks-
