Skip to main content

MongoDB Logging to Remote Syslog Server

As per the MongoDB 2.2 release notes, log output for MongoDB can now be redirected to a remote syslog server.

Here is an example configuration.

MongoDB Instance

MongoDB is started as follows (note the extra `syslog` switch):

 $ mongod --dbpath=/data/db/syslog --fork --syslog

The local "/etc/syslog.conf" file (i.e. on the `mongod` instance) is configured to send everything to the syslog server (10.7.100.20):

@10.7.100.20:514

Syslog Server

I ran my Syslog server on Ubuntu 12.04. There are a tonne of links out there describing how to install syslog on Ubuntu - see here. The syslog "facilities" are configured in the server's `/etc/syslog.conf` file (I left this as default):

#################################################################################
#
# First some standard logfiles.  Log by facility.
#
auth,authpriv.*            /var/log/auth.log
*.*;auth,authpriv.none        -/var/log/syslog

#cron.*                /var/log/cron.log
daemon.*            -/var/log/daemon.log
kern.*                -/var/log/kern.log
lpr.*                -/var/log/lpr.log
mail.*                -/var/log/mail.log
user.*                -/var/log/user.log
.....
.....
*.=info;*.=notice;*.=warning;\
        auth,authpriv.none;\
        cron,daemon.none;\
        mail,news.none          -/var/log/messages
#################################################################################

We then need to enable the syslog server to accept remote syslog messages as follows:

more /etc/default/syslogd 
#################################################################################
#
# Top configuration file for syslogd
#
# Full documentation of possible arguments are found in the manpage
# syslogd(8).
#
# For remote UDP logging use SYSLOGD="-r"
#
SYSLOGD="-r"
#################################################################################

Using `tcpdump`, we can see the syslog messages arriving at the syslog server from the `mongod` instance:

#################################################################################
01:27:40.675624 IP 10.7.100.6.55318 > 10.7.100.20.514: SYSLOG user.info, length: 111

    0x0000:  4500 008b 335e 0000 4011 6adc 0a07 6406  E...3^..@.j...d.

    0x0010:  0a07 6414 d816 0202 0077 0a1e 3c31 343e  ..d......w..<14>
    0x0020:  4465 6320 3134 2031 313a 3537 3a31 3720  Dec.14.11:57:17.
    0x0030:  6d61 726b 2d6d 6270 2e6c 6f63 616c 206d  mark-mbp.local.m
    0x0040:  6f6e 676f 642e 3135 3030 315b 3433 3032  ongod.15001[4302
    0x0050:  5d3a 2046 7269 2044 6563 2031 3420 3131  ]:.Fri.Dec.14.11
    0x0060:  3a35 373a 3137 205b 696e 6974 616e 646c  :57:17.[initandl
    0x0070:  6973 7465 6e5d 2072 6563 6f76 6572 2063  isten].recover.c
    0x0080:  6c65 616e 696e 6720 7570 0a              leaning.up.
01:27:40.675703 IP 10.7.100.6.55318 > 10.7.100.20.514: SYSLOG user.info, length: 110
    0x0000:  4500 008a 0d49 0000 4011 90f2 0a07 6406  E....I..@.....d.
    0x0010:  0a07 6414 d816 0202 0076 df4a 3c31 343e  ..d......v.J<14>
    0x0020:  4465 6320 3134 2031 313a 3537 3a31 3720  Dec.14.11:57:17.
    0x0030:  6d61 726b 2d6d 6270 2e6c 6f63 616c 206d  mark-mbp.local.m
    0x0040:  6f6e 676f 642e 3135 3030 315b 3433 3032  ongod.15001[4302
    0x0050:  5d3a 2046 7269 2044 6563 2031 3420 3131  ]:.Fri.Dec.14.11
    0x0060:  3a35 373a 3137 205b 696e 6974 616e 646c  :57:17.[initandl
    0x0070:  6973 7465 6e5d 2072 656d 6f76 654a 6f75  isten].removeJou
    0x0080:  726e 616c 4669 6c65 730a                 rnalFiles.#################################################################################

The logs from the `mongod` instance will typically be located in `/var/logs/messages` on the syslog server:

#################################################################################
Nov 16 01:27:40 10.7.100.6 mark-mbp.local mongod.15001[4302]: Fri Dec 14 11:57:17 [initandlisten] recover create file /data/db/syslog/syslog.ns 16MB 
Nov 16 01:27:40 10.7.100.6 mark-mbp.local mongod.15001[4302]: Fri Dec 14 11:57:17 [initandlisten] recover create file /data/db/syslog/syslog.0 64MB 
Nov 16 01:27:40 10.7.100.6 mark-mbp.local mongod.15001[4302]: Fri Dec 14 11:57:17 [initandlisten] recover cleaning up 
Nov 16 01:27:40 10.7.100.6 mark-mbp.local mongod.15001[4302]: Fri Dec 14 11:57:17 [initandlisten] removeJournalFiles 
Nov 16 01:27:40 10.7.100.6 mark-mbp.local mongod.15001[4302]: Fri Dec 14 11:57:17 [initandlisten] recover done 
Nov 16 01:27:40 10.7.100.6 mark-mbp.local mongod.15001[4302]: Fri Dec 14 11:57:17 [websvr] admin web console waiting for connections on port 16001 
Nov 16 01:27:40 10.7.100.6 mark-mbp.local mongod.15001[4302]: Fri Dec 14 11:57:17 [initandlisten] waiting for connections on port 15001 
#################################################################################
So as you can see, it's quite simple to syslog your logs off MongoDB to a centralised syslog server. If you want to keep an eye on "logging" related MongoDB feature requests and bugs, check out this JIRA link.
Labels:

Comments

Post a Comment

Popular posts from this blog

MongoDB Authori(s|z)ation

Introduction Having answered numerous questions on the new and old authori(s|z)ation within MongoDB, I thought I'd write a short blog post explaining how things work as there seems to be some confusion. What's New Prior to version 2.4 , there was a very basic sense of "Role Based Access Controls" (RBAC) within MongoDB as there were only two roles - read readWrite which is quite limited. For example, if the user has "readWrite", that user is essentially "root" and the user can add/remove users as well as inserting data into the database, i.e. there is no role segregation. Version 2.4 added in the following 3 core roles - userAdmin dbAdmin clusterAdmin with a notable extension such that there are now 4 roles that apply across all databases - readAnyDatabase readWriteAnyDatabase userAdminAnyDatabase dbAdminAnyDatabase This increased RBAC is a significant improvement from a security perspective in MongoDB. It is imp
Post a Comment
Read more

Eurotash GSE

So I'd the pleasure of talking with Chris John Riley, from Eurotrash , on the night of Sunday, July 1st (yes, Chris isn't a football fan so I good-heartedly missed the half-time discussion of Spain's titi-taka brillance). Chris wanted to chat with me about my experience sitting the GIAC GSE exam and lab earlier this year. As always, I thoroughly enjoyed chatting with Chris and I hope it's not too painful listening to me on the podcast . I don't know much but what I would say is stay hydrated, eat as well as you can, prepare Source: http://matchphrases.com/images/large/legends3.jpg properly and have fun!! There's a reason that the GSE has a low pass rate, so going in relaxed makes a huge difference. Enjoy the podcast and if you've any questions on the GSE, just shout :)
Post a Comment
Read more

Being a Support Engineer @ 10gen - Part 1

There's a mis-conception around the role of a "Support Engineer".  As a clue, it's not what Urban Dictionary   says   - A person whose job is to answer calls from customers of a small- to large-sized company...... They are teathered to a their desk all day via phone headset........ phone jockeys usually hate their jobs.......they are are paid well enough..........until they completely burn out, and hate everyone.   and doesn't always involve this - Image Source: http://half-bakedbaker.blogspot.ie/2009/11/cannoli-and-broken-computer.html As you can see  here , there's lots of open roles in  10gen  and more specifically with 10gen, in  Dublin . I thought I'd write this quick blog to explain what Support Engineers actually do and why I joined 10gen as a "Support Engineer". I could be wrong but didn't Google come up with term " Site Reliability Engineer " to do away with the stigma associated with being a
Post a Comment
Read more