Also: I have BayesAfterHMM blank, and the log doesn't show any Bayes scoring happening (I have DoHMM and DoBayesian both set to "score").
On 1/28/16 10:48 AM, Dossy Shiobara wrote: > Recently, it seems my HMM and Bayes checks are no longer working? In > mail log, I see: > > "HMM-Check has given less than 6 results - using monitoring mode only" > > I'll include my latest rebuildrun.txt, which looks like it ran successfully. > > Why is this happening? I'm running ASSP 2.4.7(16004). Also, it seems > like if I get this error, it doesn't even perform Bayesian scoring -- > basically, spam that was previously being blocked is now being let > through... > > > ---rebuildrun.txt--- > > Jan-28-16 09:05:00 RebuildSpamDB-thread rebuildspamdb-version 7.26 > started in ASSP version 2.4.7(16004) > > Jan-28-16 09:05:00 RebuildSpamDB uses BerkeleyDB for temporary hashes > > Jan-28-16 09:05:00 RebuildSpamDB uses BerkeleyDB-ENV with 62.50 MByte > > Jan-28-16 09:05:00 RebuildSpamDB will create a Hidden Markov Model > > Jan-28-16 09:05:00 RebuildSpamDB will create unicode enabled databases > > Jan-28-16 09:05:00 RebuildSpamDB will process all words as Sequence of > UAX #29 Grapheme Clusters > > Jan-28-16 09:05:00 RebuildSpamDB will normalize unicode characters > > Jan-28-16 09:05:00 RebuildSpamDB will use the ASSP_WordStem engine > > Jan-28-16 09:05:00 ---ASSP Settings--- > Jan-28-16 09:05:00 Do Not Collect Messages with RedListed address: Enabled > **Messages with RedListed addresses will be removed from the corpus!** > > Jan-28-16 09:05:00 Do Not Collect RedRe Messages: Enabled > **Messages matching the RedRe will be removed from the corpus!** > > Jan-28-16 09:05:00 Use Subject as Maillog Names: True > Jan-28-16 09:05:00 Maxbytes: 4,000 > Jan-28-16 09:05:00 RebuildFileTimeLimit: 1 5 > Jan-28-16 09:05:00 RebuildFileTimeLimit: files will be moved away from > the corpus if their processing takes longer than 5 second(s) > > Jan-28-16 09:05:00 /data/assp/errors/spam > Jan-28-16 09:05:00 File Count: 11 > Jan-28-16 09:05:00 Processing... errors/spam with 11 files > Jan-28-16 09:05:00 ignore and remove files older than Sep-11-88 10:05:00 > in folder errors/spam > Jan-28-16 09:05:01 Imported Files for HeloBlackList: 10 > Jan-28-16 09:05:01 Imported Files for Bayes/HMM: 10 > Jan-28-16 09:05:01 Finished in 1 second(s) > > Jan-28-16 09:05:01 /data/assp/errors/notspam > Jan-28-16 09:05:01 File Count: 1 > Jan-28-16 09:05:01 Processing... errors/notspam with 1 files > Jan-28-16 09:05:01 ignore and remove files older than Sep-11-88 10:05:01 > in folder errors/notspam > Jan-28-16 09:05:01 Imported Files for HeloBlackList: 0 > Jan-28-16 09:05:01 Imported Files for Bayes/HMM: 0 > Jan-28-16 09:05:01 Finished in 1 second(s) > Jan-28-16 09:05:01 info: corpusnorm after processing errors/spam and > errors/notspam is Spam Weight: 8280 / Not-Spam Weight: 0 => norm: 10.000 > Jan-28-16 09:05:01 info: require approx. 6,726 files (3,255,584 words) > from folder spam to get the wanted corpusnorm (1.000) > > Jan-28-16 09:05:01 /data/assp/spam > Jan-28-16 09:05:01 File Count: 11,195 > Jan-28-16 09:05:01 Processing... spam with 11,195 files > Jan-28-16 09:05:01 ignore and remove files older than Dec-28-15 09:05:01 > in folder spam > Jan-28-16 09:15:31 Removed Old: 5 > Jan-28-16 09:15:31 Imported Files for HeloBlackList: 11,190 > Jan-28-16 09:15:31 Imported Files for Bayes/HMM: 6,672 > Jan-28-16 09:15:31 Finished in 630 second(s) > Jan-28-16 09:15:31 info: require approx. all files (3,264,527 words) > from folder notspam to get the wanted corpusnorm (1.000) > > Jan-28-16 09:15:31 /data/assp/notspam > Jan-28-16 09:15:31 File Count: 7,009 > Jan-28-16 09:15:31 Processing... notspam with 7,009 files > Jan-28-16 09:15:31 ignore and remove files older than Dec-28-15 09:15:31 > in folder notspam > Jan-28-16 09:25:53 Removed Old: 7 > Jan-28-16 09:25:53 Imported Files for HeloBlackList: 7,002 > Jan-28-16 09:25:53 Imported Files for Bayes/HMM: 6,992 > Jan-28-16 09:25:53 Finished in 622 second(s) > > Jan-28-16 09:25:53 Generating weighted Bayesian tuplets > Jan-28-16 09:26:10 populating Spamdb 503166 records - Bayesian check is > now disabled > Jan-28-16 09:26:23 done - populating Spamdb records - 503166 - Bayesian > check is now enabled > Jan-28-16 09:26:23 done - Generating weighted Bayesian tuplets > > Jan-28-16 09:26:23 Bayesian Pairs: 503,166 now in list > > Jan-28-16 09:26:23 Generating consolidated Hidden-Markov-Model database > from 3,772,337 record model > Jan-28-16 09:28:22 HMM sequences: 1,848,357 now in list > > Jan-28-16 09:28:22 generating Spamdb.helo records from 7,502 collected > HELO's > Jan-28-16 09:28:22 cleaning old Spamdb.helo records > Jan-28-16 09:28:22 done - cleaning old Spamdb.helo records > > Jan-28-16 09:28:22 HELO Blacklist: 4 new, 0 now in list > > Jan-28-16 09:28:22 Spam Weight : 3,264,527 > Jan-28-16 09:28:22 Not-Spam Weight: 3,265,258 > > Jan-28-16 09:28:22 Corpus norm: 0.9998 - (very good - balanced) > Jan-28-16 09:28:22 Corpus confidence: 0.06250000 > > Jan-28-16 09:28:27 Start populating Hidden Markov Model. HMM-check is > disabled for this time! > Jan-28-16 09:28:27 start populating Hidden Markov Model with 1,848,357 > records! > Jan-28-16 09:28:59 Finished populating Hidden Markov Model with > 1,848,357 records! > Jan-28-16 09:28:59 Finished populating Hidden Markov Model. HMM-check is > now enabled again! > > Jan-28-16 09:28:59 Total processing time: 1,439 second(s) > > Jan-28-16 09:28:59 Total processing data: 118.85 MByte > > > Jan-28-16 09:28:59 Rebuild processed 14.52 files per second. > > Jan-28-16 09:28:59 After finishing the Rebuild process, the > /data/assp/tmpDB folder contains 791.45 MByte. > > Jan-28-16 09:28:59 After finishing the Rebuild process, the drive that > contains the /data/assp/tmpDB folder has 1.22 GByte free space from > total 1.90 GByte. > > Jan-28-16 09:28:59 building new GripList records and bounce report > Jan-28-16 09:28:59 processing Logfile /data/assp/logs/maillog.txt > Jan-28-16 09:28:59 processing Logfile /data/assp/logs/16-01-27.maillog.txt > Jan-28-16 09:29:01 processing Logfile /data/assp/logs/16-01-26.maillog.txt > Jan-28-16 09:29:02 processing Logfile /data/assp/logs/16-01-25.maillog.txt > Jan-28-16 09:29:03 processing Logfile /data/assp/logs/16-01-24.maillog.txt > Jan-28-16 09:29:03 processing Logfile /data/assp/logs/16-01-23.maillog.txt > > Jan-28-16 09:29:03 skipping bounce report because 'DoNotCollectBounces' > is switched ON > > Jan-28-16 09:29:03 Uploading Griplist via Direct Connection > Jan-28-16 09:29:04 Submitted 6,924 bytes: 0 IPv6 addresses, 768 IPv4 > addresses > > Jan-28-16 09:29:04 Trashlist was saved to /data/assp/trashlist.db > -- Dossy Shiobara | "He realized the fastest way to change [email protected] | is to laugh at your own folly -- then you http://panoptic.com/ | can let go and quickly move on." (p. 70) * WordPress * jQuery * MySQL * Security * Business Continuity * ------------------------------------------------------------------------------ Site24x7 APM Insight: Get Deep Visibility into Application Performance APM + Mobile APM + RUM: Monitor 3 App instances at just $35/Month Monitor end-to-end web transactions and take corrective actions now Troubleshoot faster and improve end-user experience. Signup Now! http://pubads.g.doubleclick.net/gampad/clk?id=267308311&iu=/4140 _______________________________________________ Assp-user mailing list [email protected] https://lists.sourceforge.net/lists/listinfo/assp-user
