Logging 3000 events per second from c program -
what best options logging 3k events per second c file ? following of options come mind. not able decide robust solution less failure points, higher reliability , less latency.
use messaging server relay events happen
use syslog logging
use unix pipe
use of logging agents fluent send events analysis server
write log file locally , rotate periodically rotate analysis server using rsync
try syslog. no reason make complicated. syslog-ng can local logging through udp, set local syslogd forward through tcp central syslog server. might need run without fsync on central syslog server keep load (but test first), can mitigated forwarding 2 separate machines. gives asynchronous performance locally , enough reliability should never lose events.
another option i've done log events redis, riak or other nosql data store (i don't recommend them complex, event logging right alley). set mirroring redundancy , should able keep way more 3k events per second.
Comments
Post a Comment