Recently, it seems my HMM and Bayes checks are no longer working? In mail log, I see:
"HMM-Check has given less than 6 results - using monitoring mode only" I'll include my latest rebuildrun.txt, which looks like it ran successfully. Why is this happening? I'm running ASSP 2.4.7(16004). Also, it seems like if I get this error, it doesn't even perform Bayesian scoring -- basically, spam that was previously being blocked is now being let through... ---rebuildrun.txt--- Jan-28-16 09:05:00 RebuildSpamDB-thread rebuildspamdb-version 7.26 started in ASSP version 2.4.7(16004) Jan-28-16 09:05:00 RebuildSpamDB uses BerkeleyDB for temporary hashes Jan-28-16 09:05:00 RebuildSpamDB uses BerkeleyDB-ENV with 62.50 MByte Jan-28-16 09:05:00 RebuildSpamDB will create a Hidden Markov Model Jan-28-16 09:05:00 RebuildSpamDB will create unicode enabled databases Jan-28-16 09:05:00 RebuildSpamDB will process all words as Sequence of UAX #29 Grapheme Clusters Jan-28-16 09:05:00 RebuildSpamDB will normalize unicode characters Jan-28-16 09:05:00 RebuildSpamDB will use the ASSP_WordStem engine Jan-28-16 09:05:00 ---ASSP Settings--- Jan-28-16 09:05:00 Do Not Collect Messages with RedListed address: Enabled **Messages with RedListed addresses will be removed from the corpus!** Jan-28-16 09:05:00 Do Not Collect RedRe Messages: Enabled **Messages matching the RedRe will be removed from the corpus!** Jan-28-16 09:05:00 Use Subject as Maillog Names: True Jan-28-16 09:05:00 Maxbytes: 4,000 Jan-28-16 09:05:00 RebuildFileTimeLimit: 1 5 Jan-28-16 09:05:00 RebuildFileTimeLimit: files will be moved away from the corpus if their processing takes longer than 5 second(s) Jan-28-16 09:05:00 /data/assp/errors/spam Jan-28-16 09:05:00 File Count: 11 Jan-28-16 09:05:00 Processing... errors/spam with 11 files Jan-28-16 09:05:00 ignore and remove files older than Sep-11-88 10:05:00 in folder errors/spam Jan-28-16 09:05:01 Imported Files for HeloBlackList: 10 Jan-28-16 09:05:01 Imported Files for Bayes/HMM: 10 Jan-28-16 09:05:01 Finished in 1 second(s) Jan-28-16 09:05:01 /data/assp/errors/notspam Jan-28-16 09:05:01 File Count: 1 Jan-28-16 09:05:01 Processing... errors/notspam with 1 files Jan-28-16 09:05:01 ignore and remove files older than Sep-11-88 10:05:01 in folder errors/notspam Jan-28-16 09:05:01 Imported Files for HeloBlackList: 0 Jan-28-16 09:05:01 Imported Files for Bayes/HMM: 0 Jan-28-16 09:05:01 Finished in 1 second(s) Jan-28-16 09:05:01 info: corpusnorm after processing errors/spam and errors/notspam is Spam Weight: 8280 / Not-Spam Weight: 0 => norm: 10.000 Jan-28-16 09:05:01 info: require approx. 6,726 files (3,255,584 words) from folder spam to get the wanted corpusnorm (1.000) Jan-28-16 09:05:01 /data/assp/spam Jan-28-16 09:05:01 File Count: 11,195 Jan-28-16 09:05:01 Processing... spam with 11,195 files Jan-28-16 09:05:01 ignore and remove files older than Dec-28-15 09:05:01 in folder spam Jan-28-16 09:15:31 Removed Old: 5 Jan-28-16 09:15:31 Imported Files for HeloBlackList: 11,190 Jan-28-16 09:15:31 Imported Files for Bayes/HMM: 6,672 Jan-28-16 09:15:31 Finished in 630 second(s) Jan-28-16 09:15:31 info: require approx. all files (3,264,527 words) from folder notspam to get the wanted corpusnorm (1.000) Jan-28-16 09:15:31 /data/assp/notspam Jan-28-16 09:15:31 File Count: 7,009 Jan-28-16 09:15:31 Processing... notspam with 7,009 files Jan-28-16 09:15:31 ignore and remove files older than Dec-28-15 09:15:31 in folder notspam Jan-28-16 09:25:53 Removed Old: 7 Jan-28-16 09:25:53 Imported Files for HeloBlackList: 7,002 Jan-28-16 09:25:53 Imported Files for Bayes/HMM: 6,992 Jan-28-16 09:25:53 Finished in 622 second(s) Jan-28-16 09:25:53 Generating weighted Bayesian tuplets Jan-28-16 09:26:10 populating Spamdb 503166 records - Bayesian check is now disabled Jan-28-16 09:26:23 done - populating Spamdb records - 503166 - Bayesian check is now enabled Jan-28-16 09:26:23 done - Generating weighted Bayesian tuplets Jan-28-16 09:26:23 Bayesian Pairs: 503,166 now in list Jan-28-16 09:26:23 Generating consolidated Hidden-Markov-Model database from 3,772,337 record model Jan-28-16 09:28:22 HMM sequences: 1,848,357 now in list Jan-28-16 09:28:22 generating Spamdb.helo records from 7,502 collected HELO's Jan-28-16 09:28:22 cleaning old Spamdb.helo records Jan-28-16 09:28:22 done - cleaning old Spamdb.helo records Jan-28-16 09:28:22 HELO Blacklist: 4 new, 0 now in list Jan-28-16 09:28:22 Spam Weight : 3,264,527 Jan-28-16 09:28:22 Not-Spam Weight: 3,265,258 Jan-28-16 09:28:22 Corpus norm: 0.9998 - (very good - balanced) Jan-28-16 09:28:22 Corpus confidence: 0.06250000 Jan-28-16 09:28:27 Start populating Hidden Markov Model. HMM-check is disabled for this time! Jan-28-16 09:28:27 start populating Hidden Markov Model with 1,848,357 records! Jan-28-16 09:28:59 Finished populating Hidden Markov Model with 1,848,357 records! Jan-28-16 09:28:59 Finished populating Hidden Markov Model. HMM-check is now enabled again! Jan-28-16 09:28:59 Total processing time: 1,439 second(s) Jan-28-16 09:28:59 Total processing data: 118.85 MByte Jan-28-16 09:28:59 Rebuild processed 14.52 files per second. Jan-28-16 09:28:59 After finishing the Rebuild process, the /data/assp/tmpDB folder contains 791.45 MByte. Jan-28-16 09:28:59 After finishing the Rebuild process, the drive that contains the /data/assp/tmpDB folder has 1.22 GByte free space from total 1.90 GByte. Jan-28-16 09:28:59 building new GripList records and bounce report Jan-28-16 09:28:59 processing Logfile /data/assp/logs/maillog.txt Jan-28-16 09:28:59 processing Logfile /data/assp/logs/16-01-27.maillog.txt Jan-28-16 09:29:01 processing Logfile /data/assp/logs/16-01-26.maillog.txt Jan-28-16 09:29:02 processing Logfile /data/assp/logs/16-01-25.maillog.txt Jan-28-16 09:29:03 processing Logfile /data/assp/logs/16-01-24.maillog.txt Jan-28-16 09:29:03 processing Logfile /data/assp/logs/16-01-23.maillog.txt Jan-28-16 09:29:03 skipping bounce report because 'DoNotCollectBounces' is switched ON Jan-28-16 09:29:03 Uploading Griplist via Direct Connection Jan-28-16 09:29:04 Submitted 6,924 bytes: 0 IPv6 addresses, 768 IPv4 addresses Jan-28-16 09:29:04 Trashlist was saved to /data/assp/trashlist.db -- Dossy Shiobara | "He realized the fastest way to change [email protected] | is to laugh at your own folly -- then you http://panoptic.com/ | can let go and quickly move on." (p. 70) * WordPress * jQuery * MySQL * Security * Business Continuity * ------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140 _______________________________________________ Assp-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/assp-user
